sentest 0 Posted January 4, 2015 Share Posted January 4, 2015 Hi there i was wondering if anyone could help implement Transcoding Using segments. This is how plex media server does it and this is why cpu usage on plex is alot less when transcoding Link to comment Share on other sites More sharing options...
techywarrior 688 Posted January 4, 2015 Share Posted January 4, 2015 Do you mean that you want transcoding to break the file into lots of small segments? (100 10mb files vs. 1 1gb file) If so then MBS already does that for any client that supports HLS. Link to comment Share on other sites More sharing options...
sentest 0 Posted January 4, 2015 Author Share Posted January 4, 2015 No thats standard HLS streaming anyway. What i am on about it segmenting the transcode request so it transcodes each segment as needed not constantly trying to transcode on the fly Link to comment Share on other sites More sharing options...
sentest 0 Posted January 4, 2015 Author Share Posted January 4, 2015 If you go look at Plex when its transcoding and inspect the transcoder you will see it transcodes for about 15 seconds then stops and then a minute later transcodes again for 15 seconds.because at the moment i have it running on a 8 core server and with 3 people transcoding it was sh**ting its self Link to comment Share on other sites More sharing options...
techywarrior 688 Posted January 4, 2015 Share Posted January 4, 2015 No thats standard HLS streaming anyway. Yea, just sounded like what you were asking for. What i am on about it segmenting the transcode request so it transcodes each segment as needed not constantly trying to transcode on the fly Wouldn't the CPU load end up being the same though. You can't do it exactly when needed because then slower CPUs would cause constant buffering. You'd need to transcode ahead X segments. I guess the benefit would be when you end up not watching the entire movie/show but I don't see how else it would be beneficial. (Note: I have no more influence on this then you so I'm not suggesting it not be worked on) Link to comment Share on other sites More sharing options...
Suspense 2 Posted January 4, 2015 Share Posted January 4, 2015 Yea, just sounded like what you were asking for. Wouldn't the CPU load end up being the same though. You can't do it exactly when needed because then slower CPUs would cause constant buffering. You'd need to transcode ahead X segments. I guess the benefit would be when you end up not watching the entire movie/show but I don't see how else it would be beneficial. (Note: I have no more influence on this then you so I'm not suggesting it not be worked on) The benefit of segmented transcodes is the ability to transcode between sessions. Example of how mediabrowser works now; User 1 begins a transcode job, ffmpeg is spawned and use about 40-50% cpu until the full duration of the content is transcoded User 2 begins a transcode job same way as user 1, cpu is now almost maxed User 3 begins a transcode job, and everyone begins buffering Now if this was done segmented, no buffering would accour User 1 begins a transcode job, ffmpeg is spawned and transcodes a segment of 120 seconds, this is most likely done in 20-30 seconds User 2 begins a transcode job, ffmpeg is spawned and transcodes a segment of 120 seconds. User 1 has already transcoded 120 seconds, currently cpu is only occupied by user 2 User 3 begins a transcode job while user 2's job is active. CPU is not maxed because 1 user isnt using any cpu, and user 2 and 3 are the only one currently transcoding. If a 4th user comes in and wants to transcode, the segments of user 2 and 3 is most likely several seconds ahead of where the user is, and thus user 2 and 3s transcode can be throttled to make user 4 recieve part of his segment. Makes sense? Link to comment Share on other sites More sharing options...
techywarrior 688 Posted January 4, 2015 Share Posted January 4, 2015 If you go look at Plex when its transcoding and inspect the transcoder you will see it transcodes for about 15 seconds then stops and then a minute later transcodes again for 15 seconds. because at the moment i have it running on a 8 core server and with 3 people transcoding it was sh**ting its self I don't personally have Plex to see how it does it but except for the case when you don't end up watching some of the segments this actually sounds worse to me. If you watch the entire video anyway then Plex's method would end up using more CPU time and reduce the overall idle time of the CPU. It's sort of the opposite of Intel's power saving strategy of late. They initially just powered down when able but their newer CPUs all boost CPU in order to finish work sooner and go into a low power state faster. Same for MS in Win8. They chose to batch processes into groups to only ramp up the CPU once instead of 10 times of less individual power. On a different note. My older dual core CPU can transcode two BD streams at the same time without stuttering. Surprised your 8 core server is having trouble with 3. Also, I believe that ffmpeg will use as much power as it can (each instance I think is limited to 2 cores tho) to transcode as fast as possible so it may not show any CPU usage difference between 3 streams and 5 streams on your server (just each ffmpeg process would be transcoding at slightly less FPS). Link to comment Share on other sites More sharing options...
Beardyname 195 Posted January 5, 2015 Share Posted January 5, 2015 (edited) The benefit of segmented transcodes is the ability to transcode between sessions. Example of how mediabrowser works now; User 1 begins a transcode job, ffmpeg is spawned and use about 40-50% cpu until the full duration of the content is transcoded User 2 begins a transcode job same way as user 1, cpu is now almost maxed User 3 begins a transcode job, and everyone begins buffering Now if this was done segmented, no buffering would accour User 1 begins a transcode job, ffmpeg is spawned and transcodes a segment of 120 seconds, this is most likely done in 20-30 seconds User 2 begins a transcode job, ffmpeg is spawned and transcodes a segment of 120 seconds. User 1 has already transcoded 120 seconds, currently cpu is only occupied by user 2 User 3 begins a transcode job while user 2's job is active. CPU is not maxed because 1 user isnt using any cpu, and user 2 and 3 are the only one currently transcoding. If a 4th user comes in and wants to transcode, the segments of user 2 and 3 is most likely several seconds ahead of where the user is, and thus user 2 and 3s transcode can be throttled to make user 4 recieve part of his segment. Makes sense? actually this is not true, I remembered something about a blog post about this a while back and here it is : http://mediabrowser.tv/community/index.php?/blog/1/entry-104-media-browser-305366-released/ so unless everyone starts playback at the same time it "should" not be an issue, there might be some more goodies if this where to be implemented but saying it maxes out the trans-coding until everything is done is actually wrong *edit* I should point out that if this for some reason this does not work for you then it should be looked into, since i have no idea on how they throttle the process. Also not sure what the post means about "may be able" etc but this could perhaps be expanded upon? Edited January 5, 2015 by Beardyname Link to comment Share on other sites More sharing options...
Suspense 2 Posted January 8, 2015 Share Posted January 8, 2015 actually this is not true, I remembered something about a blog post about this a while back and here it is : http://mediabrowser.tv/community/index.php?/blog/1/entry-104-media-browser-305366-released/ so unless everyone starts playback at the same time it "should" not be an issue, there might be some more goodies if this where to be implemented but saying it maxes out the trans-coding until everything is done is actually wrong *edit* I should point out that if this for some reason this does not work for you then it should be looked into, since i have no idea on how they throttle the process. Also not sure what the post means about "may be able" etc but this could perhaps be expanded upon? This is exactly what i was looking for, i expected this to not be implemented because in my initial tests, simply 3 clients transcoding on a Xeon E5 caused buffering and a maxed cpu, while 10 transcoding clients on plex is of absolutely no issue, with less then 50% cpu use. Is this throttling something i would need to actually enable for it to work? Regards Link to comment Share on other sites More sharing options...
JeremyFr79 228 Posted January 8, 2015 Share Posted January 8, 2015 I think you may have something else going on, I have a pair of older Xeon X5420's in my server and it can transcode 4 1080P streams with 0 issues. Link to comment Share on other sites More sharing options...
Suspense 2 Posted January 8, 2015 Share Posted January 8, 2015 (edited) I think you may have something else going on, I have a pair of older Xeon X5420's in my server and it can transcode 4 1080P streams with 0 issues. It depends alot on bitrate, filesize etc. CPU usage is cpu usage, and i can see the ffmpeg process eating a lot for a long time, without any throttling, which apparently its supposed to do. EDIT; I took a look at this again, and the throttling is actually happening i can see. Process is reduced to 10-14%, sometimes jumps to 20. Its still not as great as plex tho, it seems the throttle is set to just about 12-15% cpu usage, no matter how far ahead it is. It would be great to have this tweaked a bit more, i have seen maybe 3 minutes, but transcoded 15 or so, could definatly throttle it more, and let it spike for quick processing instead of maintaining. Plex is currently using 0.2% cpu on a few throttled transcodes, when it needs to transcode itll spike to 30-40%, and complete a segment of 4 or so minutes, then throttle back down to 0.2%, allows for far more transcodes Regards Edited January 8, 2015 by Suspense Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now