softworkz 3966 Posted April 8, 2019 Posted April 8, 2019 I'd like to a address a question here that is coming up regularly and sometimes getting into the focus of user desire as a must-have feature with apparent benefits. Obviously it's an evident conception that some graphics acceleration hardware should be able to accelerate graphics related operations like image extraction. But in this case that doesn't quite apply. The short answer to the subject question is:It wouldn't make a lot of sense. That answer is not really convincing, so let's get into depth.. Most of you will have noticed that we've taken great effort in the area of hardware acceleration for real-time media conversion (transcoding), so from a technical perspective it's not that we couldn't apply that to image extraction. But it doesn't fit well because image extraction is quite a different process.So, what's different here compared to transcoding? - Quite a few things: No Video Encoder RequiredIn case you didn't know: In transcoding situations it's always the Encoder doing the hard part of the job. Situations differ, but to name some figures, you may imagine something like from 90%<>10% to 70%<>30% for Encoder<>Decoder processing requirements.In case of image extraction, it's just about a few hundred still images (jpg) to be saved - as opposed to thousands of video frames having inter-frame dependencies, so no video encoding needs to be doneNo video encoder also means, that there's no 70, 80,90% percent of the process that could be hardware accelerated.. There's no hardware acceleration for encoding jpg imagesThe encoding of jpg images will always be done by the CPU.(Intel and Nvidia have an MJPEG encoder but that would require an additional processing step that might just slow down things altogether). Decoding could be done by HardwareRight. That's the only part that could in fact be performed by using hardware decoders.But the simple question here is: Why? What would be the benefit? Getting the Job done faster?We still have the output jpg encoder running on CPU resources which is also limited to a single thread.So, that is the limiting factor actually - no matter how fast the GPU might be. => Would hardly be faster (at least not in magnitudes that would be of interest - e.g. 5x, 10x, 100x). Saving Resources?When the GPU does the decoding, then there's less processing for the CPU, but the CPU will also have some additional load to handle:Copying video stream data to GPU memoryA CPU based decoder can read the stream from a memory-mapped file without copying Copying decoded uncompressed(!) frames back to system memoryUncompressed video frames means huge amounts of dataWe need to copy those back to system memory in full size uncompressed, even though we just want to create rather small images from it.=> Would not save resources (significantly) Emby High-Speed ExtractionQuite recently we've introduced a new mechanism for image extraction. This procedure requires only a minimal amount of frames to be decoded, many of them being keyframes.Previously we had to decode every single frame of a video for thumbnail extraction. With an extraction interval of 10s and a framerate of 25 fps that means we had to decode 250 frames for a single thumbnail image.Now, we're seeking to the position, decode 0-20 frames for accuracy (from keyframe to desired position), might be about 10 frames average. That means, with the new image extraction we need to decode only 4% of all video frames instead of 100%.=> There isn't much work left to do for a hw decoder. Acceleration PotentialTest's have shown that using hw acceleration with the new extraction method can achieve performance gains ranging from 1.0 (same speed) to 2.0 (double speed) only.But that usually at the cost of high consumption of system resources (CPU for JPG encoding)And that leads to the question: Why?There's no hurry and no need to get the extraction done a bit faster. Instead, this is meant to be a Background Operation that shouldn't interrupt your Emby Server's availability (and also not occupy your GPU resources). MaintenanceAt the time of writing, Emby supports 8 different hardware acceleration variants each supporting a bunch of decoders all with different specifics.Making sure that all those are working with image extraction would be a huge effort for no practical benefit at all. 3
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now