DS920+ Constant freezing/crashing on all devices when streaming HEVC4k

[khyle 0]

Hi, 

I've recently built a DS920+ and have started running Emby 4.6.6.0. 

I have been struggling with transcoding I believe from HEVC to any device. I'm running 4 Iron wolfs and 2 Samsung 970 evo plus 500 GB as a cache with an aggregated link. 

I've tried both VAAPI and QuickSync as well as turning off tone mapping. But I do keep getting some freezing while transcoding which after about 30 seconds will and up crashing the instance to the point I need to restart or I cannot play again. 

 

I have attached 2 transcode logs of different attempts and I am running out of ideas.

 

 

 

 

ffmpeg-transcode-eb0ba72e-b2e6-4a57-b81d-c103c2c7a1d2_1.txtffmpeg-transcode-3e7711e4-36df-4a11-b7c1-b81d33d89aa6_1.txt

 

[Carlo 4330]

Hi, I have the same NAS as you.  I'm using WD 14TB drives in min with Samsung 1 GB nvme set to read/write cache using SHR hybrid RAID with btrs file system. I'm using an aggregated link as well.  I also have 8 additional drives direct attached via USB3 pooled and mounted as well as a remote attached storage from a couple Linux servers, Windows Server, WD NASes and storage from my PetaSAN via iSCSI as well.  I also have a couple fuse mounted cloud storage providers just for playing around with.

I can duplicate this only in chromium browsers. Firefox for example plays fine. So if you're using Chrome or other chromium browser try Firefox and see if this makes a difference.
You can also try lowering the bitrate to force 1080p playback as well which I'll show below.

I'll suggest setting up the transcode section like this using Advanced.  I re-order the QuickSync over VAAPI for both encode and decode sections so it will look similar to this:

Click the cog icon in playback and select one of the 1080 bitrate options to force the transcoder to scale the video to 1080 which it may not be doing.  If you are playing back on a 1080 screen this is likely what you would likely want anyway.

Let me know how that works for you.  Here I'll show you a couple things with a likely same result as you at the bottom.

Here's a forced transcode lowering the bitrate of 4K HDR media playing back to Chrome.

(this is from the User Session menu option when you install the Diagnostic Plugin from the catalog)

You can see this is doing a scale to 1920x1080 (2nd blue box).  This will play the whole movie for me with no issues.

Here is the exact same movie but I changed the bitrate to 4K 120 Mbps.

It will start playing but then typically from 30 seconds to 2 minutes will pause itself.  It's impossible to restart on the client or server. I believe this is the same thing you are seeing.

I have no issues with any other client or non Chromium browsers such as Firefox.  So it seems to be a Chrome issue of some kind.

Here you can see I'm already 10+ minutes into the movie playing back in Firefox:

@softworkz can you take a look at this and see if there is anything we can do to help with this even though it doesn't appear to be an Emby issue?

[khyle 0]

HI Cayars, 

I can see a massive difference in Firefox compared to Chromium browsers. 
I have just uploaded logs from my Xbox Series X which has the same issue from the Emby Theatre application. I've added the HEVC codecs to the Xbox but Emby Theatre seems to crash when running 4K video. 
I can play anything up to 1080 without an issue on all other endpoints, just not 4K.

When looking through the dashboard, the video seems to be playing but it has frozen about 10 seconds in. 

ffmpeg-transcode-52511bc6-a221-4fd9-a131-2e0a7abd4345_1.txt

[Happy2Play 8279]

It would appear the hardware is just not up to the task as the conversion fps are just not there, you gradually climb to processing at speed.  So playback exceeds conversion rate.

14:26:34.630 frame=    2 fps=1.5 q=0.0 size=       9kB time=00:00:00.18 bitrate=N/A throttle=off speed=   0x    
14:26:35.116 frame=   18 fps=9.8 q=26.0 size=      46kB time=00:00:00.89 bitrate= 539.7kbits/s throttle=off speed=0.385x    
14:26:35.666 frame=   33 fps= 14 q=26.0 size=    8605kB time=00:00:01.43 bitrate=56484.4kbits/s throttle=off speed=0.53x    
14:26:36.164 frame=   48 fps= 17 q=26.0 size=   17775kB time=00:00:02.13 bitrate=74598.4kbits/s throttle=off speed=0.678x    
14:26:36.739 frame=   65 fps= 19 q=5.0 size=   28163kB time=00:00:02.81 bitrate=87911.0kbits/s throttle=off speed=0.77x    
14:26:37.245 frame=   76 fps= 19 q=5.0 size=   34926kB time=00:00:03.29 bitrate=92165.6kbits/s throttle=off speed=0.792x    

14:27:34.509 frame= 1459 fps= 24 q=7.0 size=  883087kB time=00:01:01.01 bitrate=118921.8kbits/s throttle=off speed=0.994x    
14:27:35.032 frame= 1474 fps= 24 q=7.0 size=  892577kB time=00:01:01.59 bitrate=119072.2kbits/s throttle=off speed=0.995x    
14:27:35.532 frame= 1490 fps= 24 q=4.0 size=  898997kB time=00:01:02.23 bitrate=118691.7kbits/s throttle=off speed=0.997x    
14:27:36.046 frame= 1508 fps= 24 q=4.0 size=  909997kB time=00:01:02.96 bitrate=118735.5kbits/s throttle=off speed=   1x    
14:27:36.521 [segment @ 0x18f48c0] Opening '/var/packages/EmbyServer/var/transcoding-temp/D310E6.m3u8.tmp' for writing
14:27:36.522 SegmentComplete=video:0 Index=20 Start=60.060000 End=63.063000 Duration=3.003000 offset_pts=0 start_pts=60060000 Frames=72 filename=D310E6_20.ts
14:27:36.522 [segment @ 0x18f48c0] Opening '/var/packages/EmbyServer/var/transcoding-temp/D310E6_21.ts.tmp' for writing
14:27:36.525 frame= 1520 fps= 24 q=1.0 size=  920373kB time=00:01:03.51 bitrate=119057.9kbits/s throttle=off speed=   1x    
14:27:37.068 frame= 1535 fps= 24 q=12.0 size=  933022kB time=00:01:04.09 bitrate=119605.6kbits/s throttle=off speed=   1x    
14:27:37.587 frame= 1549 fps= 24 q=9.0 size=  942530kB time=00:01:04.66 bitrate=119745.8kbits/s throttle=off speed=   1x    
14:27:38.060 frame= 1562 fps= 24 q=11.0 size=  950873kB time=00:01:05.27 bitrate=119676.7kbits/s throttle=off speed=   1x    
14:27:38.596 frame= 1577 fps= 24 q=9.0 size=  960001kB time=00:01:05.88 bitrate=119709.0kbits/s throttle=off speed=1.01x    

In the conversion from h265 to h264 there is bandwidth doubling (or client playback set value) to try and maintain quality, so your 64Mbs media is being converted to 120Mbps.

Does lowering the client playback bitrate resolve the issue?

[khyle 0]

Hi Happy2Play, 

I can definitely see a massive improvement when setting to 60Mbps. My guess is the Xbox cannot run anything higher thank 60Mbps as it will just freeze. 

I've attached logs of it at 60Mbps ffmpeg-transcode-ff94eda0-7d1c-4239-8dc6-6debdbf97cdf_1.txt
and 80Mpbs ffmpeg-transcode-71ac02b3-8c69-46fb-862e-23ccd07e52ae_1.txt

 

 

[Carlo 4330]

One of the problems with HEVC to AVC conversion is that HEVC compression/codec is so good it often requires the AVC video to have twice the bitrate to be a similar quality. That's the assumption the transcoder often makes. When you set an upper limit for playback it tells the transcoder the upper limit it's allowed to use, solving the problem for these already high bitrate videos.

Ideally, it would be nice if we could set a limit on the server for transcodes only.  This way for example anything transcoded could be tamed down to 40Mb as an example while still allowing a 60Mb video to direct play if possible.

[bjjones 19]

@cayars Carlo, I'm trying to do some of the same type of diags on some 4K streams. Didn't know about the diagnostics plugin, looks fantastic.

Installed it on mine and my session info looks nothing like yours, what might I be doing wrong to not get the detailed info?

latest Emby 4.6.7.0 on syno dsm7 with the 4.7.0.2 plugin version

 

thanks Robert

[Carlo 4330]

I'm running the blue theme as well as beta versions of the Server.

[Luke 37053]

Are you still having an issue with this?

[Carlo 4330]

@bjjones, what's your current status on this?

[bjjones 19]

@cayars, my status is fine. The OP having the one having trouble, I was just looking into the tools that were available to troubleshot because I was looking into the impacts of moving towards HEVC rather than AVC. It did some odd transcoding going when played on my iPad that it didn't experience on other clients (androidTV) which direct played. I'd be inclined to try out the 4.7.x beta to have the detailed visuals you showed above but I don't know how clean a switch back to stable 4.6.x would be if I needed to revert. Are you also running a beta plugin version as well or is all that info produced out of the beta server pkg vs. the stable ?

[Carlo 4330]

If you have two different machines available setup the beta on the other machine just for your internal testing.  It doesn't need to be setup for remote use, so no port forwarding is needed.

If you run on windows and only have the one machine available you can run both the release and beta on the same server.  Do this:

Stop the current release/production Emby Server (know what ports you are using ie 8096/8920).
Download the portable version of the beta from here: https://emby.media/windows-server.html
You can for example create a folder c:\EmbyServer and then unzip the download to this folder.  The portable version is just the system directory.
Go into the system folder you just unziped and start the Emby Server EXE program.  It will create the other directories, config files, databases.
Run through the configuration so you can get to the Network menu.
Change the local ports to something different not used by your production server.  For example use 8097/8921.
Restart the beta server.
Now start your production server back up.
They both run now using different ports and different directories on your system.

Hope that helps.

[Luke 37053]

@bjjones has this helped?