Jump to content

Failed Hardware Transcode Attempts Break Subsequent Conversions


Netfool
 Share

Recommended Posts

Netfool

Running The Nvidia Shield Emby Server 4.5.1.0 on a 2019 Nvidia Shield Pro.

Attempting to use the Nvidia GPU to do hardware trans-codes has never worked for me.  Ever hopeful, I wanted to test under 4.5.1.0 and discovered a completely different problem.

I attempted to transode a file from H264 to HVEC.   That first conversion failed as before.  I also tried to convert the file to a 4MB/sec version.  Hardware trans-coding of that failed as well.  I did the same for 3 additional files to test whether or not it was a problem with that specific file.  All failed to used the hardware encoder.

However, when I went back and examined the logs, in all cases Emby attempted to convert the first file, NOT the file I asked it to convert.  Conversions were started from the Movie page in some cases and from the library thumbnail for the movie in others.

  DizzyEmoji.png.c55f2ccd81e2b13fc61105322e6aa999.png

A screenshot of the Logs page and relevant logs are attached. (The screenhot to establish the order of the transcode logs).  The Remux log came from playing the last movie to confirm its was not originally shot in 16 by 9 (it was however released letter-boxed) and mostly in monochrome.    It's possible that the initial problem was caused by cancelling the first conversion from the dashboard.  Allowing it to complete software a software trans-code in all cases would have tens of hours.

Screen Shot 2020-10-03 at 1.46.18 PM.png

embyserver.txt ffmpeg-remux-3b027453-a1da-4a8a-811f-c14d291c3ebe_1.txt ffmpeg-transcode-3cb5e847-7345-4bb5-b335-9a1e7e88e3c6_1.txt ffmpeg-transcode-8db60443-f8f7-4cc2-851e-aa53df25e619_1.txt ffmpeg-transcode-bd201aa7-eece-4a32-ae0c-3fe8ae75076a_1.txt ffmpeg-transcode-cc0c53cb-384f-47a0-92d2-8172e76f6c30_1.txt ffmpeg-transcode-d13fff65-36f7-48c7-b57a-9e29c7b38769_1.txt ffmpeg-transcode-d715900d-b7ae-4f42-8d84-a4a8d04c4ca2_1.txt ffmpeg-transcode-da1997bb-81b1-412e-ab21-10b73c21c3a5_1.txt

Link to comment
Share on other sites

Happy2Play

Looks like some adjustments adjustments will need to made.

12:55:35.293 Stream mapping:
12:55:35.293   Stream #0:0 -> #0:0 (h264 (h264_mediacodecndk) -> hevc (hevc_mediacodecndk))
12:55:35.293   Stream #0:1 -> #0:1 (dts (dca) -> aac (native))
12:55:35.293 Press [q] to stop, [?] for help
12:55:36.299 frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=N/A throttle=off speed=   0x    
12:55:36.401 [h264_mediacodecndk @ 0x29d7947800] NdkDec: MediaCodec output format changed: mime: string(video/raw), stride: int32(1280), slice-height: int32(688), color-format: int32(21), image-data: data, crop: Rect(0, 0, 1279, 687), crop-width: int32(1280), crop-height: int32(688), display-width: int32(1280), display-height: int32(688), color-range: int32(2), color-standard: int32(4), color-transfer: int32(3), hdr-static-info: data, width: int32(1280), height: int32(688)}
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: is_rtk: 0 pix_fmt: 23 pixelFormat: 21
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: Mime: video/hevc Size: 1280x688 rc_mode: 1 rc_max_rate: 1680000 rc_buffer_size: 3360000 bit_rate: 200000
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: Framerate: 24000 / 1001
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: Profile: 8 Level: 512
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: aspect: 1 / 1
12:55:36.403 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: mediacodec_name: OMX.Nvidia.h265.encoder
12:55:36.415 [hevc_mediacodecndk @ 0x29d7949600] NdkEnc: Failed to configure encoder; check parameters
12:55:36.417 Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
12:55:36.421 [aac @ 0x29d7987500] Qavg: 13013.449
12:55:36.421 [aac @ 0x29d7987500] 2 frames left in the queue on closing
12:55:36.438 Conversion failed!

 

Link to comment
Share on other sites

Netfool
20 hours ago, Happy2Play said:

Looks like some adjustments adjustments will need to made.

Nothing I can adjust in the dashboard I presume?

Link to comment
Share on other sites

Netfool

"(Dear me!  I'm not certain quite
That even now I've got it right)".

-- Eletelephony
   Laura Elizabeth Richards
  
https://poets.org/poem/eletelephony

 

Additional Information:  It may not be a bug related to cancelled conversions.  Apparently cancelling a conversion only stops the current conversion process and puts the requested conversion back on top of the conversions queue.  To actually cancel the conversion it appears to be necessary to both cancel the conversion on the dashboard and then go to the "Downloads" page and delete the "download". 

I noticed this the morning after I filed the report since the overnight here were a number of "Downloads" that failed.  Often of the the same file, over and over again.  Deleting all of the "downloads" in the queue seemed to stop the process.

The fact that the terms "Transcodes", "Conversions", and "Downloads" are all used interchangeably in different parts of the UI is both very confusing and not at all well documented. 

Cleaning up that language, and having an option to simply delete "downloads" that failed from the queue, as well as having a separate "Downloads, Conversions, and Transcodes" log that indicates a date-stamp, file name and success or failure would greatly simplify understanding what's happening.

Edited by Netfool
Link to comment
Share on other sites

To be honest, HEVC hardware encoders on Android haven't got much attention yet.
I believe it fails because the defaults for profile and level are for AVC rather than HEVC.

You could make the following test:

  • Install the Emby Diagnostics Plugin
    https://mediabrowser.github.io/Emby.DiagnosticsPlugin/
  • In the diagnostic options section you will find two textboxes for doing find/replace within ffmpeg commands
    Enter the following:
    • Find: -c:v:0 hevc_mediacodecndk 
    • Replace: -c:v:0 hevc_mediacodecndk -profile:v:0 1 -level:v:0 4096 
      (corresponds to HEVC Profile 'Main' and Level '4.1')
  • Then retry the conversion

Note: Those settings do not get persisted across restarts

 

Link to comment
Share on other sites

Netfool
22 hours ago, softworkz said:

You could make the following test:

  • Install the Emby Diagnostics Plugin

Thanks, I'll give that a try and report back here.

 

But... to the U/I issues, why is it necessary to manually remove failed conversions from "downloads"?

Edited by Netfool
Link to comment
Share on other sites

20 hours ago, Netfool said:

Thanks, I'll give that a try and report back here.

 

But... to the U/I issues, why is it necessary to manually remove failed conversions from "downloads"?

We are looking into improving it. Thanks.

Link to comment
Share on other sites

  • 3 weeks later...
Netfool
2 hours ago, Luke said:

Hi, were you able to try this?

I got it set up to try but then I got distracted by a server crash I reported in a different thread.

Thanks for the reminder!  I'll give it a try.

I think I'll change the name of this thread too.  It's a bit misleading.  When a transcode fails, it get rescheduled (apparently ad-infinitum) so it's not that subsequent conversions fail, it that they never even get a chance to run without manual intervention. 

Edited by Netfool
Link to comment
Share on other sites

Netfool
On 10/28/2020 at 9:45 PM, Luke said:

Did you try what softworkz suggested?

Yes.   No Joy. 

I made the profile substitution, and rotated the logs and then tried a conversion of a large (26GB) file to HEVC. 

The process ran for about 30 seconds showing reasonable progress in the dashboard, then disappeared.  The system then apparently did a log rotation itself, leaving the file showing in the Conversions page as "Transferring 0.0%":

1655598288_ScreenShot2020-10-30at3_25_38PM.png.a1c06608409cd93ef61cf34ddf121b23.png

I don't see anything significant in the log files.   Would it help if I re-ran the test with debug logging turned on?

If so... does that auto log rotation indicate that I should re-apply the profile change?

embyserver-63739667462.txt ffmpeg-transcode-c6a7d014-82e1-4240-bc92-7e6e75f03f7c_1.txt

Edited by Netfool
Link to comment
Share on other sites

22 minutes ago, Netfool said:

Yes.   No Joy. 

One moment please - first of all, we need to understand that the original problem is fixed by the profile substitution.

HW accelerated encoding to HEVC is working!

Now we have a new and totally different problem.
(I don't have any ideas for that yet)

Edited by softworkz
Link to comment
Share on other sites

Netfool
50 minutes ago, softworkz said:

One moment please - first of all, we need to understand that the original problem is fixed by the profile substitution.

HW accelerated encoding to HEVC is working!

Now we have a new and totally different problem.
(I don't have any ideas for that yet)

It does appear to be using hardware HEVC encoder.       ....until it dies 30 seconds later.

Link to comment
Share on other sites

I understand that the outcome is not much different for you - from 'not working' to 'not working'..

But as we got a totally new situation now, maybe you could do some experiments - e.g. with some more "lightweight" files and see how that goes..?

Actually, you're probably the very first person now, that got HEVC hw encoding working on the Shield (with Emby).

Edited by softworkz
Link to comment
Share on other sites

1 minute ago, Netfool said:

@softworkz I'll give it a try!    Does that auto log rotation indicate that I should re-apply the profile change?

What do you mean - the regular server log? IIRC it rotates when too big and at midnight, but I'm not sure. On startup as well.

The profile change needs to be re-applied only on restart.

Link to comment
Share on other sites

Netfool

@softworkz

Restarted the Shield Pro and tried again.  Zero successes for two tries.

After re-entering the profile change I tried converting "The Ref".

That was a hard fail, showing up in the Conversions page a "failed".  It generated two ffmpeg-transcode logs.

I then tried converting "The Big Sleep".  That ran for about 6 minutes before stopping with nothing notable in the transcode log.  Shows in the Conversions page as "Transferring 0.0%".

When I tried converting "The Ref" the conversions parameters asked for HEVC and 4 MB/sec.   For "The Big Sleep" it was HEVC and original bitrate.

Screenshot attached to help with deciphering which log is which.  (Why the cryptic ids instead of just timestamps?)

378445958_ScreenShot2020-10-31at12_05_59AM.thumb.png.dc3d602d382021219eb7897a595013aa.png

...anything else I should try?

embyserver-63739698721.txt embyserver.txt ffmpeg-transcode-5e3b8d1e-ad86-4bf8-b721-82f88e5a277c_1.txt ffmpeg-transcode-ca344041-e550-4209-a549-fe30f664f143_1.txt ffmpeg-transcode-cac00a8d-25cd-475a-a041-902d3f248449_1.txt

Link to comment
Share on other sites

On 10/31/2020 at 8:12 AM, Netfool said:

...anything else I should try?

I guess I need to go and investigate this more. It will take a while until I get to it.

The only thing you could try meanwhile would be to do the same conversion but with H.264/AVC as target coded. 
This would allow us to rule out other (non-HEVC specific) causes.

Thanks

Link to comment
Share on other sites

Netfool
22 hours ago, softworkz said:

I guess I need to go and investigate this more. It will take a while until I get to it.

The only thing you could try meanwhile would be to do the same conversion but with H.264/AVC as target coded. 
This would allow us to rule out other (non-HEVC specific) causes.

Thanks

Well...  in attempting to do that, I think I may have found a semi-related issue:

If you attempt a conversion that fails, the system retries that conversion ad-infinitum, until it is manually deleted from the Conversions page.

If you manually delete the failed conversion and then attempt to convert a different video, the second conversion immediately fails without ever being attempted.  ( i.e. No ffmpeg-transcode log is created).

It's apparently going to take a server restart before another conversion attempt will even be tried.  I'll restart the next time the server is idle and give it another try.

Edited by Netfool
Link to comment
Share on other sites

On 11/7/2020 at 10:42 PM, Netfool said:

It's apparently going to take a server restart before another conversion attempt will even be tried. 

Let's see what @Luke thinks.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...