Failed Hardware Transcode Attempts Break Subsequent Conversions

[Luke 37054]

It will run on the next execution of the conversion scheduled task.

[Luke 37054]

Hi, has this helped?

[Netfool 18]

@Luke@softworkz  FYI:  Attempts to do transcodes from h.264 to HEVC still fail under the new release (4.5.3.0)

ffmpeg-transcode-b2cbb66c-f8be-4d35-85b4-ea19c72f5082_1.txt

[Luke 37054]

4 hours ago, Netfool said:

@Luke@softworkz  FYI:  Attempts to do transcodes from h.264 to HEVC still fail under the new release (4.5.3.0)

ffmpeg-transcode-b2cbb66c-f8be-4d35-85b4-ea19c72f5082_1.txt 16.84 kB · 0 downloads

Great, thanks for the info.

[Netfool 18]

@Luke@softworkz

FYI:  Attempts to do transcodes from h.264 to HEVC still fail at about 2.6% completion under the new release (4.5.4.0) if the transcode parameter -c:v:0 hevc_mediacodecndk is replaced with -c:v:0 hevc_mediacodecndk -profile:v:0 1 -level:v:0 4096.  The failure appears to force a server restart (or at least a reset) as the logs are roatated, a hardware detection log is produced and a library media scan is initiated.   It also clears the transcode parameter replacement.

Attempting an H.264 to HEVC transcode without parameter replacement initially appears to work, but is too slow to be useful (about 1.6% per hour).

UPDATE:  That software HEVC transcode failed after about an hour and 15 minutes.   These failures also induce the apparent server restart behavior.

So....   HEVC transcodes are still impossible on the Shield. 

 

Edited December 24, 2020 by Netfool

[Luke 37054]

Is your Shield OS software up to date?

[Netfool 18]

On 12/24/2020 at 8:10 PM, Luke said:

Is your Shield OS software up to date?

Yes. The OS is up to date. (I've yet to find where in the Shield UI it shows me the rev number though)

  I did a factory reset on it yesterday.  The net seems to think that's the solution to repeated "...critically low on storage" messages, despite the available space on the expanded internal storage showing 60GB available.  The NVIDIA web site claims that adding a USB stick to REPLACES the 16GB internal flash.   I doubt that because the cache in the old internal storage grows, and somemtimes clearing that helps.   I'm rapidly loosing faith in the Shield OS.   I may be able to free up a mac mini to for the Emby server and just use the Shield for the Emby Client.  I do like the Shield Pro remote much better than the AppleTV remote.

After reinstalling Emby Client & Server on the Shield and restoring from a backup made minutes before the reset, I waited for the library scan to finish and found three instance of each of the libraries on the home page.   Attempting to delete 1 deletes all three (!!), so I guess I need to delete all the libraries and rebuild them from scratch.  Not impressed with the server backup/restore process so far.

[Carlo 4330]

Even when you expand the internal storage with a flash disc it's not a complete replacement as some items will only write to the actual internal space so when that fills you hit a problem.

What I've done is mount new storage as a new disc, installed Emby then changed all the folders you can change for cache, transcoding, meta-data, etc to the new storage device so there is no chance for it to use internal space.

The 16GB model really isn't designed for being a server unless it's a small library or a travel library IMHO.  It also doesn't have the IO throughput a PC has which can be limiting as well.  In many ways it's like a lowend NAS from a performance standpoint.

With that said it does work quite well for many environments but is never going to perform like a full fledge PC with modern GPU and storage.  The Shield TV is a kick ass client platform with the ability to do some light server functionality.

 

[Luke 37054]

Hi, can you try the beta server 4.6.0.21+ and see if that resolves this for you? Thanks !

[Netfool 18]

On 1/28/2021 at 10:32 AM, Luke said:

Hi, can you try the beta server 4.6.0.21+ and see if that resolves this for you? Thanks !

 

On 1/12/2021 at 1:01 PM, cayars said:

The 16GB model really isn't designed for being a server unless it's a small library or a travel library IMHO.  It also doesn't have the IO throughput a PC has which can be limiting as well.  In many ways it's like a lowend NAS from a performance standpoint.

With that said it does work quite well for many environments but is never going to perform like a full fledge PC with modern GPU and storage.  The Shield TV is a kick ass client platform with the ability to do some light server functionality.

 

@Luke @cayars  I have come to the conclusion that the Nvidia Shield Pro is simply not an acceptable server platform for a bunch of reasons:

1.  There's no way to shut down the server from the webUI dashboard, and you can't see the status of running tasks from the Emby Server Shield App.  Between those two issues, there's no safe way to restart the Emby Server without risking massive corruption of the db.  
    
2. Recovering from a backup looses collections and duplicates some libraries complicating recovery from problems caused by #1.
    
3. While the Shield has a GPU, Emby Server on Android can't use it for HEVC encoding.
    
4. The incessant "Shield memory is critically low" messages, even with freshly cleared caches makes its use annoying and confusing for non-technically inclined family members.
    
5. The power button in the upper left corner of the remote shuts down the Shield, including the Emby Server, sometimes causing the same problem as #1.
    
6. Of late after power-fails or other restarts of the Shield, the Emby Server often does not come up until you open the Emby Server Shield App which initiates a restart.  This means you really need a UPS for the Shield.
    
7. There's also been a problem with Emby Server doing spontaneous restarts lately, but I haven't spent any time chasing down that rabbit hole.

All-in-all, it's just not an acceptable server platform.  I agree with @cayars the the Shield is a kick-ass client platform.  The Shield Pro remote blows away the silly AppleTV remote.    At this point my plan is to free-up a mac mini in the next couple of weeks and re-image it with Linux.  I'll use that as the Emby Server platform until I can get around to building a Linux PC with a more modern GPU.

[Carlo 4330]

I think it's fine for light weight installations where you have a couple hundred movies and few dozen shows but it just doesn't have the resources to handle a large system. Not for me anyway.  So tend to agree with you.

With that said a lot of people are using it and not having issues so go figure.

[softworkz 3335]

The Shield is simply not a "transcoding machine" and surely not suitable to do batch conversions to HEVC. 

1 hour ago, Netfool said:

At this point my plan is to free-up a mac mini in the next couple of weeks and re-image it with Linux.  I'll use that as the Emby Server platform

From what I came to understand what you want to do, I'm not sure whether this is a good choice for you either. It will depend on what kind of hardware acceleration will be possible with such a device...

 

1 hour ago, Netfool said:

 There's no way to shut down the server from the webUI dashboard

That's not true. Shutdown is possible, just no restart.

1 hour ago, Netfool said:

you can't see the status of running tasks from the Emby Server Shield App

Correct, there's no server dashboard access at all. The app is simply a launcher for the server.
The problem is that the dashboard UI in general is not optimized for operating with a remote, so even if we would display the web-UI of the server dashboard, you wouldn't have much fun with it.

1 hour ago, Netfool said:

While the Shield has a GPU, Emby Server on Android can't use it for HEVC encoding.

You could try the latest beta. One thing that should be solved is the HEVC profile selection. I have also made some changes to memory allocation, but I can't promise that it will fix the crashes occurring after a while.

 

For all the other points: most of them seem to be related to the HEVC problem.

Which of those problems are still occurring when you stop doing any HEVC conversions?

 

[Netfool 18]

22 hours ago, softworkz said:

For all the other points: most of them seem to be related to the HEVC problem.

Which of those problems are still occurring when you stop doing any HEVC conversions?

Actually I gave up trying HEVC conversion a couple of months ago.  Those other symptoms are recent.

 

22 hours ago, softworkz said:

From what I came to understand what you want to do, I'm not sure whether this is a good choice for you either. It will depend on what kind of hardware acceleration will be possible with such a device...

I agree!  The Mac Mini is just an interim solution until I can get around to building a Linux box with a fully capable GPU.     Right now I just need to get the server running on ANYTHING other than the Shield.  Old Mac Pro's are really cheap right now on eBay because they don't run the current MacOS.  But they do have lots of drive bays and an actual GPU slot.  Wouldn't make a bad Linux box.   The downside to that is that they are big (...and if I remember correctly, have more fan noise than I'd like).  

[Luke 37054]

On 1/28/2021 at 1:32 PM, Luke said:

Hi, can you try the beta server 4.6.0.21+ and see if that resolves this for you? Thanks !

@Netfool can you retry this on this new build and see how it compares? Thanks.

[Netfool 18]

3 hours ago, Luke said:

@Netfool can you retry this on this new build and see how it compares? Thanks.

I'll give it a try, but I think the biggest problems have to do with the Nvidia architecture not the Emby Server itself.

In addition to what I outlined above there's a problem recording content from the HDHR (even the HDHR Extend with transcoding turned on).  It never makes complete recordings dropping at some random point but mostly 7 or 8 minutes before the end of a 30 minute newscast.   Those play fine on the WebUI, but on the Shield if (and only if) you try to skip an ad with a 30sec fwd, then it will play for 15 sec and then stall.   Backing up 5 sec and restarting plays for another 15 seconds before it stalls again.  At first I thought this might be caused by using network storage, but I put a 500GB SSD on USB2 on the shield and pointed the recordings library there.  No Joy.

I'll try Beta 4.6.0.21 and report back here.  

[Luke 37054]

That would be great, thanks !

[Netfool 18]

@Luke I installed 4.6.0.22 beta.  It certainly behaves quite differently from the current release version.  Not in a more useful way, but certainly differently.

I installed it about 3PM yesterday.  The DVR failed to even attempt to record the 5PM news program in the schedule.   The dashboard shows that it is currently recording this evening's version of that show.  I'll report back whether or not the playback works any better under 4.6.0.22.

Attempts to calculate Statistics fail instantly under 4.6.0.22

I attempted a transcode from H264 to HEVC of this file:

Using these settings:

 

That initiated an ffmpeg-remux event that ran for tweleve seconds.

That was followed by ffmpeg-transcode event which logged that it was using the hardware HEVC encoder.  That event appeared to run normally from 13.08 to 13:35 when it started logging "Failed to get input buffer!".   That "conversion" is still running hours later but showing 100% complete in the dashboard.

In the Conversions page is shows:

Manually triggering the Transfers event completes in 0 seconds and appears to do nothing.

Making an SMB connection to the transcode-temp directory in the 500GB SSD in the Shields USB2 slot shows a 250MB mkv file which contains only the soundtrack for the movie.

Clicking the x in the little white dot next to the conversion graph in the dashboard simply restarts the conversion process.   Removing the ill named "download" in the Conversions page appears to stop it.    The terms conversion, download, and transcode still seem to be used interchangeably in the server UI, which I have always found confusing.

Logs attached.

embyserver.txt ffmpeg-remux-f9d291b2-8313-4de4-a602-f6fd23d7860e_1.txt ffmpeg-transcode-c0a1685a-aa11-4d39-ae87-401ff3366673_1.txt

[Luke 37054]

What about the original problem of this topic?

[Netfool 18]

14 hours ago, Luke said:

What about the original problem of this topic?

That was confusion caused by the complexity of the conversion UI, which can be started almost anywhere but if a conversion, aka transcode, aka download fails it gets retried forever, and the dashboard "conversion in process" graph never tells you what's being processed, and to find out you need to read the log.  It's not restricted to hardware HEVC transcodes, which have never worked for me.

I explained all that in my October 5th post in this thread.

[Luke 37054]

Ok thanks for the info.

[Netfool 18]

On 1/30/2021 at 3:45 PM, softworkz said:

Correct, there's no server dashboard access at all. The app is simply a launcher for the server.
The problem is that the dashboard UI in general is not optimized for operating with a remote, so even if we would display the web-UI of the server dashboard, you wouldn't have much fun with it.

I'm not advocating putting the whole dashboard on the Android Server app display, just an indicator that there is a process running that could cause db corruption if the server is stopped from the Android server app.  Right now you need a second device to see that, and training non technical family members to remember that is a big ask.

[Luke 37054]

2 minutes ago, Netfool said:

just an indicator that there is a process running that could cause db corruption if the server is stopped from the Android server app. 

No it won't cause corruption if you click the shutdown button in the android server app. What gave you that idea?

[Netfool 18]

21 minutes ago, Luke said:

No it won't cause corruption if you click the shutdown button in the android server app. What gave you that idea?

On Aug 6th 2020, in the thread titled "Collection Duplicates Itself?"  @cayars said: 'I "think" I've seen it happen when you stop a refresh manually and it was working on things part of a collection'.

...and later that day. "Yes a library scan and resetting the box is worse than just telling Emby to stop the scan. That's like a PC reset.  If you don't have a PC handy an Android phone or tablet can be used to admin the server"'.

...and on the 7th: "You really want to stay away from Shield TV resets if running a server on it.  Instead use a client to restart Emby (gracefully) if that is the objective.  A phone, table or PC will do the trick for this as they expose the admin UI that a TV UI doesn't".

So what you're saying is that a Shield Reset is dangerous, stopping the Emby Server  from the Android server app is always safe even if it's doing a refresh that's working on things that are part of a collection?

[Carlo 4330]

I was talking about turning the unit off or essentially removing power/hard reset.

Shutting things down properly via the GUI will never hurt and is the prefered method of shutdown.

[Netfool 18]

3 minutes ago, cayars said:

Shutting things down properly via the GUI will never hurt and is the prefered method of shutdown.

I understand that, but the GUI isn't available on TV.  The issue is whether shutting the server down from the Emby Server Android app can cause a problem if a library scan is running (particularly while it's working on collections).

As I said a couple of posts ago.  Expecting non-technical family members to find a second device, and bring up the GUI to restart the system is a big ask.  Especially if they have to find a running library scan and stop it manually.

I think what @Luke is saying is that stopping the server from the Emby Server Android app is ALWAYS safe, regardless of what's running.  That can be done from the TV remote.