Subtitle extraction process performance - mkv optimisation - osd freez

[Gecko 62]

Hi Luke, softworkz,

I created this topic to give my feedback about the subtitle extraction process that Emby runs when on-the-fly extraction is activated.

I ran into this old issue yesterday and dig a little deeper this time. This is what I discovered and wanted to share with all of you guys.

 

In fact, when running the iOS app with subrip choosen, Emby tries to extract embedded subtitles and send it to the player like you know. But for some mkv, the subtitles are not located in the beginning of the mkv file. So, I though, well maybe Emby reads the whole file to extract them. Depending on the file size and network/server load, this may time out as shown in the logs attached (embyserver-timeout.txt - ffmpeg-timeout.txt).

The problem is that, during this time, iOS player is unresponsive like in the old thread above. (@Luke, is there any improvement possible for such case?)

System.TimeoutException: System.TimeoutException: Operation timed out after 600000ms
       at Emby.Server.MediaEncoding.Subtitles.SubtitleEncoder.ExtractTextSubtitleInternal(String inputPath, Boolean isAudio, String inputSubtitleCodec, MediaStream subtitleStream, SubtitleMediaTypes outputCodec, String outputPath, Nullable`1 startTime, Nullable`1 endTime, Boolean preserveOriginalTimestamps, CancellationToken cancellationToken)
    Source: Emby.Server.MediaEncoding
    TargetSite: Void MoveNext()
    
2023-01-31 10:17:34.464 Error App: Error getting subtitles
    *** Error Report ***
    Version: 4.7.11.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3

 

I tested again after running mkvcleaner which optimized the file for streaming by moving all the necessary informations in the start of the container (MKVToolNix seems to call that "tags" and "cues"). After refreshing the file in emby and started playback again in iOS app with the same settings, the whole subtitle extraction process only took 3 seconds, during which, the osd was unresponsive like before (timer stayed at 0:00 for example and jumped to 0:03 when subtitles arrived). (logs *-ok attached)

2023-01-31 10:47:20.033 Info SubtitleEncoder: ProcessRun 'ffmpeg-subtitle_extract' Process exited with code 0 - Succeeded
2023-01-31 10:47:20.033 Info SubtitleEncoder: ffmpeg subtitle extraction completed for file:/media/tv/Billy the Kid (2022)/Season 1/Billy.the.Kid.2022.S01E05.MULTi.1080p.WEB.H264-AVON.mkv to /config/cache/temp/3291ef0789f847be9bb3d233a1ca59dc.srt
2023-01-31 10:47:20.047 Info SubtitleEncoder: Subtitle extraction took 2,908ms for file:/media/tv/Billy the Kid (2022)/Season 1/Billy.the.Kid.2022.S01E05.MULTi.1080p.WEB.H264-AVON.mkv
2023-01-31 10:47:20.048 Info Server: http/1.1 Response 206 to host2. Time: 2911ms. http://emby:8096/emby/Videos/52474/03babf2032a8c25dcd320da3203cb95a/Subtitles/4/0/Stream.srt

 

> All this means that ffmpeg depends on how the container stores the subtitles or at least some sort of index that help reaching them.
> The way iOS app (and maybe others ?) connects to the server does not allow multiple requests at the same time (except for the actual .ts chunks?)
> I recall that when I tried jellyfin (but my files where on a nas over smb at that time), a small text was displayed on the osd saying that subtitles was extracting in the background. It seems that's the same process for emby except that Emby's osd is not usable during that time.

 

As a side note:

I also tried to disable on-the-fly extraction, but then, the file does not start when transcoding, at least on 4.7.11. I assume, again, that's because the CPU is requested to handle the subtitle burn-in and it needs to also process the whole file and does not have the same timeout or timeouts at all or something like that. I did not dig deeper for this one. What's sure about it, is that on the latest beta, activating the GPU for the subtitle processing (and the new "Prefer Direct Conversion") renders the subtitles kind of immediately on screen and does not introduce osd freezes, optimized mkv or not. But not everyone may not benefit from that...

Note2: All my tests have been done direct connecting to http port of emby server from my phone (V.4.7.11 for stable and V.4.8.0.21 for the beta, v2.2.5 for iOS).

 

Hope this helps !

Gecko

 

 

 

embyserver-timeout.txt ffmpeg-transcode-31cebcab-2348-4233-9919-a503990b6215_1-timeout.txt embyserver-ok.txt ffmpeg-transcode-c761694d-b310-4d4f-ad40-7bac8cf223ba_1-ok.txt

[Luke 37267]

Hi, thanks for sharing your results.

Quote

The problem is that, during this time, iOS player is unresponsive like in the old thread above. (@Luke, is there any improvement possible for such case?)

Yes, improvements to this are coming.