Luke 37093 Posted January 22, 2021 Share Posted January 22, 2021 I'll pull @softworkz in and add him to your PM. 1 Link to comment Share on other sites More sharing options...
Carlo 4330 Posted January 22, 2021 Share Posted January 22, 2021 3 hours ago, mbarylski said: @cayars Well, I tried a few files on the Jellyfish Bitrate Test Files site and those froze too. I'll PM you the transcode log. Looking at your last two logs with JF those are very high bitrate files! jellyfish-120-mbps-4k-uhd-hevc-10bit.mkv jellyfish-400-mbps-4k-uhd-hevc-10bit.mkv I'm downloading them to try them myself. Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 @mbarylski - I had described the effect that you're experiencing in an earlier post a while ago. You were using some mid-range hw acceleration for quite some time and got used to it Then, you upgrade your hwa device to something really powerful, expecting much better performance and fluid operation without limitations But what you're experiencing is just the opposite - possibly even worse than before How can this be? => Well, first of all, you're asking the wrong question: "Why does my RTX-4000 get maxed out?" I mean - why not? What would you expect? Now that you got that ultra-fast GPU, should it just run at 10% of its capabilities? No - of course it runs as fast as possible! And that changes everything! But there's a downside: Now, that the GPU is no longer a limiting factor, it will run as fast until another limit will be reached. This could be storage IO, bus IO, network throughput, or even CPU power: A few years ago, I had upgraded my graphics card and I was excited to check out the benefits of the new GPU, but once I tried, the whole system became almost unusable, with CPU usage at 100%. That was just the opposite of what I had been expecting. What happened? => The new GPU was so fast in doing the video processing (like 350 fps) that the CPU couldn't even keep up with the audio transcoding, maxing out the CPU with it. Back to this case: The three logs that you have posted are all showing a scenario with 4k HEVC decoding to 4k AVC encoding, in each case at about 4.5x speed, which is a realistic value for a single 4k-to-4k live transcoding with this GPU. I don't see anything wrong here. You should activate throttling, which is meant exactly for avoiding such (maxing-out) situations. 1 Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 Ah, one more thing that I might not have made clear enough: Your GPU isn't actually "maxed out" It is running at 4.5x speed, so it can't be the reason why the video doesn't play fluidly Instead, the GPU is operating so fast, that something else in your system is getting (really) "maxed out" I can't say what it is, but that's the reason for the playback issues Link to comment Share on other sites More sharing options...
rbjtech 4283 Posted January 22, 2021 Share Posted January 22, 2021 (edited) 1 hour ago, softworkz said: @mbarylski I don't see anything wrong here. You should activate throttling, which is meant exactly for avoiding such (maxing-out) situations. The OP has and it still freezes. I think it's a combination of the source files being problematic (seen in the logs vs the jellyfish files that are 'clean') and other system factors as you say - possibly I/O or the Network Stack. The 400 Mbit jellyfish is going to kill most systems, but the 120 Mbit should likely be workable on a reasonably modern system ? OP - Can you watch the 'system resources graph' as you play a 40, then 60, then 80, then 120 Mbit jellyfish file - it should look like 'saw teeth' as the system processes at max speed, idle's (waits) and repeats itself until complete. This usually highlights where your 'bottleneck' is from the network, cpu or IO graphs. Edited January 22, 2021 by rbjtech Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 (edited) 1 hour ago, rbjtech said: The OP has and it still freezes. I haven't seen a log file documenting this. We also need to consider that he's doing playback on the same machine in Chrome with the video still being 4k. That part alone takes quite a bit of resources. 1 hour ago, rbjtech said: I think it's a combination of the source files being problematic Even though unlikely, I wouldn't completely rule out the timestamp issues in the source files, but a "combination"? That's not convincing to me.. Anyway - it's quite easy to find out: @mbarylski - Let's do the following test: Run a transcoding playback in a way that resembles the reported issue After 30s, go to Emby's transcoding-temp folder, find the subfolder that is currently used for writing transcoded segments Make a copy of that folder to a location of your choice Stop playback The data in transcoding-temp will be deleted, but you still have the copy Now we need to concatenate the segments to a single file and try to play. PS: Don't play the m3u in that folder from the file system. This doesn't work like HLS.. Edited January 22, 2021 by softworkz Link to comment Share on other sites More sharing options...
Carlo 4330 Posted January 22, 2021 Share Posted January 22, 2021 So I downloaded those 2 JF test files. If I try and play them back in a browser even when transcoding on the server I get pausing. If I try and force transcode from Android ATV I get "Too many errors giving up". @ebr sent ATV log around 6:45 am EST user Carlo Using Luke's new Exo in the Android test app, the transcodes play butter smooth. Butter smooth transcoding to Theater and Roku as well (even 400 Mb version). So that begs the question, what client are you using when testing transcoding and how is it connected? Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 (edited) 5 minutes ago, cayars said: If I try and play them back in a browser even when transcoding on the server I get pausing. If I try and force transcode from Android ATV I get "Too many errors giving up". @ebr sent ATV log around 6:45 am EST user Carlo Using Luke's new Exo in the Android test app, the transcodes play butter smooth. Butter smooth transcoding to Theater and Roku as well (even 400 Mb version). As always - nothing can be said about those results without seeing the logs... 5 minutes ago, cayars said: So that begs the question, what client are you using when testing transcoding and how is it connected? He's using Chrome on the local machine (same as Emby Server) Edited January 22, 2021 by softworkz Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 (edited) 7 minutes ago, cayars said: So that begs the question, what client are you using when testing transcoding and how is it connected? ... 1 hour ago, softworkz said: We also need to consider that he's doing playback on the same machine in Chrome with the video still being 4k. That part alone takes quite a bit of resources. Edited January 22, 2021 by softworkz Link to comment Share on other sites More sharing options...
MBSki 1020 Posted January 22, 2021 Author Share Posted January 22, 2021 8 hours ago, cayars said: Looked at the logs from the JF files and they looked clean with no errors. When you say froze it just stops playing or are you doing any ff/rw type things? What clients have you tested in? It just stops playing and instead of seeing the pause button, I see the play button. When I click the play button again nothing happens. I've only tried in the browser (Edge-Chrome) so far. I'll try on Xbox later. 1 Link to comment Share on other sites More sharing options...
MBSki 1020 Posted January 22, 2021 Author Share Posted January 22, 2021 Ok, thanks all for your input. I did some testing on the Shield TV Pro connected to 4k TV with the Emby Theater app and manually changed to 1080p/5 Mb. That one played really smooth. I also tested on Xbox One connected to 1080p TV and that played too. Even my videos with errors played, although the errors did present themselves throughout the video. Watching the performance monitor during the playback CPU stays low and GPU peaks at about 80 and keeps a decent buffer size of at least 15 minutes, but continues to increase. A few of my disks get close to capacity although I'm using StableBit with duplication which is supposed to use duplicates from a different disk if there's too much activity. I suppose the bus could be saturated though. So, the new exo does seem to make a difference. But also, playing from a separate device appears to have made a difference as well. Carlo, what device were you playing from when the jellyfish video froze? @softworkz @cayars I'll send you the transcoding logs. FYI, colors were muted as expected since tonemapping hasn't arrived yet. Seemed to be more vibrant in the browser on my PC yesterday. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted January 22, 2021 Share Posted January 22, 2021 I tried to do what I think you were doing and that was open Chrome on the Server itself and playback transcoded video in the browser. On my server it's just to much for it and it's all jerky and can sort of freeze. But if I just let the server "serve" and not be a client then everything worked well even the 400 Mb jellyfin file. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted January 22, 2021 Share Posted January 22, 2021 Unless you have a powerhouse server with lots of IO it likely just overwhelmed the machine like softworkz suggested. It does on my machine! Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 22, 2021 Share Posted January 22, 2021 This situation is kind of weird and difficult to understand at first sight. Actually, there wouldn't be any problem for that system to do both, server-side transcoding and browser-playback at the same time. There are plenty of resources available for this. And there's also no limit of resources being hit, which would require to use smaller video resolutions for playback ---- in theory! The problem is not about limited resources, the problem is that both operations (server-transcoding and client-playback) are competing against each other. And the advantage lies at the side of the GPU (operating on the PCI-bus, etc). This is a problem that Emby is supposed to manage in some way. And that way is called "Throttling" ( @mbarylski - you still haven't enabled it!) Our throttling isn't perfect. We have no "smooth throttling". Instead it works just like an oven or a refrigerator: there's a thermostat that is switching it on of off based on a value on a linear scale and stabilized by a hysteresis range. The on/off throttling doesn't matter for the server operation, as the server is usually a bit ahead of time. But local playback (video decoding) happens in real-time, and when another process suddenly shoots up CPU usage to 100%, the local playback may easily get interrupted. 2 Link to comment Share on other sites More sharing options...
Carlo 4330 Posted January 22, 2021 Share Posted January 22, 2021 Great explanation softworkz! 1 Link to comment Share on other sites More sharing options...
rbjtech 4283 Posted January 23, 2021 Share Posted January 23, 2021 (edited) 13 hours ago, softworkz said: The on/off throttling doesn't matter for the server operation, as the server is usually a bit ahead of time. But local playback (video decoding) happens in real-time, and when another process suddenly shoots up CPU usage to 100%, the local playback may easily get interrupted. Thanks for the info @softworkz - I've often thought why does the transcoding throttle not produce more frequent but smaller chunks to regulate multiple transcodes but I guess that is more difficult to implement than it sounds as presumably you can't change the process once it's started haha. Is any sort of process priority placed on the ffmpeg transcoding processes to limit it's possible interruption with other system processes ? Just curious. Thanks. Edited January 23, 2021 by rbjtech Link to comment Share on other sites More sharing options...
pwhodges 1532 Posted January 23, 2021 Share Posted January 23, 2021 Maybe one could juggle with processor affinity to reduce the impact of full-speed transcoding when it's not required, at least to ensure that one core is available for other work. Paul Link to comment Share on other sites More sharing options...
MBSki 1020 Posted January 23, 2021 Author Share Posted January 23, 2021 Yea, REALLY appreciate the explanation @softworkz. Seems strange to me that it can't handle playback on the server, but not a big deal since I've got plenty of other devices to watch on. Btw, I did test throttling as well, but it doesn't help when playing back on the server. It does help in general though so I've got it enabled now. 1 Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 25, 2021 Share Posted January 25, 2021 On 1/23/2021 at 10:16 AM, rbjtech said: I've often thought why does the transcoding throttle not produce more frequent but smaller chunks to regulate multiple transcodes Look at ThrottleHysteresis in encoding.xml... The default is 8 seconds. I wouldn't go below 2 seconds. On 1/23/2021 at 10:16 AM, rbjtech said: Is any sort of process priority placed on the ffmpeg transcoding processes to limit it's possible interruption with other system processes ? No, and I wouldn't recommend to do that. It doesn't have the effect that you would expect. 1 Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 25, 2021 Share Posted January 25, 2021 PS: You need to set ThrottlingMethod to ByStreamBufferSize Link to comment Share on other sites More sharing options...
softworkz 3338 Posted January 25, 2021 Share Posted January 25, 2021 1 minute ago, softworkz said: PS: You need to set ThrottlingMethod to ByStreamBufferSize @mbarylski - That might also help in your case, but only once the transcoding buffer limit has been reached. 1 Link to comment Share on other sites More sharing options...
MBSki 1020 Posted February 2, 2021 Author Share Posted February 2, 2021 @cayars You mentioned earlier in this thread that you typically re-encode your 4k videos. What methods/tools do you use to retain the exact same quality? I'm considering adding that step to my process, but I don't want to lose any quality in the conversion. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted February 2, 2021 Share Posted February 2, 2021 No such thing as same quality if you transcode/convert it but you can keep the quality very good so you really can't notice. I reencode everything but keep 4K HDR as rips as well for local use. I add default AAC 2 channel audio to everything so I never have to transcode because of audio. I've been using ffmpeg AV Batch right now in mass to convert everything to HEVC as well. 1 Link to comment Share on other sites More sharing options...
RanmaCanada 345 Posted February 3, 2021 Share Posted February 3, 2021 2 hours ago, cayars said: No such thing as same quality if you transcode/convert it but you can keep the quality very good so you really can't notice. I reencode everything but keep 4K HDR as rips as well for local use. I add default AAC 2 channel audio to everything so I never have to transcode because of audio. I've been using ffmpeg AV Batch right now in mass to convert everything to HEVC as well. You should try OPUS instead as it has better fidelity You can easily get a 5.1 track at 384kb with very little if any discernible quality loss. Link to comment Share on other sites More sharing options...
Carlo 4330 Posted February 3, 2021 Share Posted February 3, 2021 AAC is a universal format that works on every device I know of. I limit it to 2 channel only at a max of 256KB. I'll copy over other English tracks but the 2 channel track is for compatibility. This allows all clients to have an audio track that doesn't require transcoding. 1 Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now