DS418play Transcoding performances?

[Vesemir7 1]

Hi!,

I'm running Emby 4.6.7.0 (Purchased a one month Premiere option to test full features) on a Synology DS418play (INTEL Celeron J3355 CPU so x64 architecture) with DSM 7.0.1-42218 .

Pretty much everything up to date as this 6th January 2022.

I'm physically distant from my NAS but still connected with a fairly decent internet connection, speed test peaked at 60 Mbps downstream / 10 upstream. It's similar if not a bit better at the source where the NAS is connected.

When playing video content without transcoding (I double check with stats for nerds if it's actually doing it or not), playback is smooth and consistent, when transcoding if I keep checking the hardware stats on my NAS in the background I rarely the CPU going above 30-40% at best. I had already tried increasing the CPU threads at the MAX value through the Emby server config page, but I can't see to push hardware usage any further, what am I missing?

So far, I see transcoding starts for 10bit content, subtitles or (rarely) different container (I understand it refers to the original video file)

Thanks!!

Rob

[Luke 37060]

Hi, is there a problem when playing, or are you just observing resource consumption?

[Vesemir7 1]

Hi Luke,

what I see is I fail to obtain smooth playback at higher resolutions and where I see the bandwidth is not acting as a bottleneck, so I was wondering why the CPU wasn't pushing more.

 

Rob

[Luke 37060]

Can we please look at an example?

Thanks.

[Vesemir7 1]

Here are playback stats for a video that keeps stutter every one and then. Current playback is set at 1080 - 4 Mbps ( please mind that I'm actually able to play it even at 10 Mbps, it just stutters more frequently, I wager to buffer?)

It doesn't even show Transcoding progress anymore, because the process is finished. But when it WAS transcoding during the initial stages of playback, the hardware never went above these levels:

 

 

[Luke 37060]

Can you please provide the information requested in my link above? Thanks !

[Vesemir7 1]

18 minutes ago, Luke said:

Can you please provide the information requested in my link above? Thanks !

ffmpeg-transcodeembyserver.txt-e7a383da-d605-4c36-9b10-b1e52ba55893_1.txt

 

Let's see if this is enough, at this time?

 

[Carlo 4330]

>>>>>>  Processing Plan
Info    Name                                        CanDoInHardware      WillDoInHardware     Reason                                  
Info    VAAPI HD Graphics 500 - H.264 (AVC)      >> True                 True                 Hardware Codec                           
Info    VideoInput                               >> True                 True                 Same adapter (/dev/dri/renderD128), s... 
Info    Scaling                                  >> True                 True                                                          
Info    VideoOutput                              >> True                 True                 Hardware encoder                         
Info    VAAPI HD Graphics 500 - H.264 (AVC)      >> True                 True                 Hardware Codec                           

>>>>>>  Projected Processing Formats
Info    Previous                HW-Context   Format       SW-Format           Size   Next
Info    h264                 >> VAAPI        vaapi_vld    nv12            1920x960 >> scale_vaapi
Info    scale_vaapi          >> VAAPI        vaapi_vld    nv12            1280x640 >> 

Hi, the reason is simple.  Transcoding is handed off to the GPU to do the work.  The CPU doesn't have to do the "muscle" work so it's involved by fetching the stream from disk and making sure the GPU is kept busy.  The CPU is also receiving the data from the GPU and doing all the tasks to stream it. It's not doing more because there isn't anything more it needs to do.

If you want to see it work hard go to the transcode menu option in your Server Dashboard and turn off hardware transcoding.

You may like the picture quality better as you have an older HD Graphics 500 GPU. The newer generation GPU do a little better job quality wise. Before turning if off to test change the first option to Advanced and see if you have an option for QuickSync as well.  If you do set each and every one up like this:

Enable QuickSync and move it up to the top position.  Now try playing again and see if QuickSync is better on your NAS than VAAPI is.
Every CPU/GPU combo and media collection is different so we give you the tools to adjust it best for you system per codec for both Encoders and Decoders as that page will show you.

Let us know what you find.

[Vesemir7 1]

Hey thanks for looking into it!
I tried to switch the decoders/encoders but I can't say I notice much difference to be honest.

If I turn OFF HW transconding I can definitely see the CPU jumping to 98% when the streaming starts, but again I can't say the performances are noticeably different, in terms of the negotiated stream quality and smoothness.

 

Rob
 

[Carlo 4330]

It's going to depend on the codec in the media your playing.  It's been too long since I last looked at this but for some codecs I think there was a 10 to 15% difference in fps while others had no difference.  Softworkz may have made improvements since then as well as he has been working really hard to optimize transcoding as much as possible.

If you have an 4K HDR media or high bitrate 1080 files those would probably be good for a quick test to see if the decode is faster for the codec.  Once this to keep in mind is that encoding will always be h.264 right now.  So once you know which is faster, leave it.  Then a test of H.264 & maybe H.265 (hopefully the only codecs you use) being transcoded let you check quality and speed of the decoder.  Set it then forget it.   Pretty much just playing something back for one minute giving it a quality check then looking at the FPS at the 1 minute mark will do. Try both ways.  If you can tell any difference turn off hardware transcoding then back to on or advanced and it will go back to defaults.

[Vesemir7 1]

If I think I might have found something interesting, observing the different performances and the NAS resource manager during playback.

With HW Transcoding ON, the Quicksync does indeed perform a little better on the VAAPI, in term of how fast the transcoding percentage indicator jump ahead of the playhead (As seen from Emby server dashboard and proven from the stats for nerds on the client) but the playback still struggles to be smooth at 1080 - 2Mbps , sometimes it still fails to break 720 - 2Mb.

I noticed that, regardless of the codec, the process FFMPEG2 as a subprocess of Emby hits hard the storage, both in reads and writes for the transcoding operation, I'm seeing values of 6-7 MBps in read and 5-6 Mbps in writes.

When I turn HW transcoding OFF, since FFMPEG2 is no longer being triggered because the CPU is working hard at 98%, I seem to achieve better performances.

The question now is, if FFMPEG2 acting rogue? It looks like it can be the bottleneck in this situation.

 

 

[Carlo 4330]

That's not correct.  ffmpeg is used either way. The difference is the command line or parameters passed to ffmpeg so it knows to use the CPU or hardware and what codecs and settings to use. 

I would double check your results to make sure it does say transcoding in Stats for Nerds during playback.

Also at the top of the logs (log menu) will be a file starting with ffmpeg in the name for the file you just played.  You could try with CPU and hardware then compare the two files to see if they are both generating the same bitrate and output resolution.

[Vesemir7 1]

Which are the general metrics Emby uses to negotiate the streaming quality? Assuming that from the client "Auto" is selected of course.

What can be done on the server side to tweak the behavior?

[Carlo 4330]

Besides the bitrate limit that can be set on the client or server the reason something would transcode is due to the client not being able to use the media file for one reason or another. Normally if you look at Stats for Nerds during playback or the session on the server it will tell you the reason for the transcode.

[Luke 37060]

@Vesemir7 has this helped? Are these always remote connections?

[Vesemir7 1]

Well the client can be either an android device (tablet) or my windows10 laptop, but the results are not different from each other.

I don't think the hardware (either of the NAS or of my clients) or the bandwidth are constraints here and I don't understand why this struggle at negotiating a higher bitrate, unless there something in the logs that can pinpoint it.

[Carlo 4330]

What does Stats for nerds on the Client show you?

What does the Dashboard show for it while streaming?

Either should tell you why it's transcoding.
You can also look at the ffmpeg log generated as well.

[Vesemir7 1]

It depends on the type of source content and how it's packaged/encoded, I guess.

Often times these days, I'm seeing the reason being about the media bitrate. But I don't see why should I be focusing about WHY it's transcoding rather than HOW / HOW GOOD.

My intent is to have my NAS + Emby serving as my "private Netflix". The embedded Synology media server solution I wasn't happy with and neither I was with Plex. Emby seemed to me a solid option which could also serve for the downloading/offline watching feature, which I use a lot but I'm struggling to have it performing decently.

[Carlo 4330]

Understood, but the format you have the media in dictates if it will play or not on the devices you own. When the NAS is limited in what it can do transcode wise it's even more important to not fall back to transcoding if it can be helped.

If the only reason it's transcoding is due to bitrate limitation you will want to check to make sure you need to have a limit in place.  If not raise the limit so you aren't forcing a transcode when it's not needed.

[Vesemir7 1]

I feel we're going in loops here. I thought the very reason of transcoding even existing was to actually make playable formats which natively can't be played.

And this IS happening. When I have something playing and transcoding on, I can even see from the server console that the transcoding playhead is way ahead of the video playhead, so transcoding performances don't seem to be an issue, in the back-end, at least. My question is WHY I can't seem to negotiate a smooth playback, given the bandwidth is not a constraint (I can stream 4K60 easily).

[Carlo 4330]

Yes it can seem to go in circles when talking in "general" vs a specific file.

Your NAS has a Celeron and QuickSync which will help in most but not all situations.

If you have odd ball formats like 10bit h.264 (Anime) it's not going to like it, etc...
If your getting transcodes due to use of graphical subtitles you can try and replace them with SRT (text based) or other format that won't require burning in.
If you have adequate bandwidth outgoing on the NAS/server side and of course on the receiving side you should make sure there are no limits in place on the server side (user setting and global (network menu at the bottom).

Then on the client side manually change the bitrate rate in the playback settings to the highest setting you can use vs AUTO.  If you test playing anything you've played previously it will use the config from before so you will have to over ride it by changing the quality from the cog menu during play back.

Try those things, then upload a new ffmpeg log file for anything not playing back as expected and we'll look at that specifically!

[Vesemir7 1]

Yeah it seems we are running in circles because I am open to the fact that there might be hardware limitations but Emby just doesn't point them out to me and from my previous log bundle there was no indication about it either. I worked as a senior TSE and had my share of log analysis to investigate for a root cause analysis to paying customers.

I had to prove with hard data that the issue was connectivity, faulty or underperforming hardware, not enough IOPS from the storage, resource starvation of any kind.

I understand the variable of the format, but I don't understand why none of my hardware is pushed to deliver and since it's clearly not a bandwidth issue, I don't know what else to do.

I tried to playback different contents in different formats, bitrates and all. Nothing seems to drastically change this experience and yet no clear cause is shown.

Turning subs on or off doesn't make a difference and I'm tired of struggling to stream 480/720 when in another tab I stream 4K60 from other platforms.

I was also inclined to stay in the paid package because of the downloading / offline content playback on my android tablet (again, the exact experience I would want to have from a private Netflix-like media server, but it's buggy. Subs are not getting downloaded and sometimes download won't just start, whether they are converted or not. This is a battle for another day though, but I don't think, at this stage, I'm inclined to purse it.

[Carlo 4330]

If you like let's discuss specifics. If you upload the logs for specific files that won't play back correctly we can look at those and determine why.

[Vesemir7 1]

Honestly I'm out of time, I gave myself a month to evaluate Emby and I don't feel like renewing it at the moment, only to keep troubleshooting for days/weeks again.

[Carlo 4330]

It's just a matter of staying focused on an actual issue vs others things you have questions about that might relate.  We've tried to answer questions you've asked but these didn't really go to the heart of the issue you had. We only got an opportunity to look at one media file's logs which is not much to go on. We advised you to make some changes to the transcoding options but never got any new logs to look at based on our recommendations to test.

The 418 is a low end NAS that is going to require use of QuickSync for anything that can't be direct played or streamed and there isn't anything we can do about that due to the nature of the hardware being used. Having to use QuickSync is going to semi dictate what can and can't be done but we've only scratched the surface of trying to look at your results.

Carlo