[FIXED] mkvs with embedded fonts

[Protected 86]

EDIT: Fixed in vanilla as of 4.9.*!

Some users have complained that in mkv files with ass subtitles and bundled fonts (I've checked, the fonts are definitely there), in other words anime, the web player doesn't render subtitles using the correct font, but rather with a default font or fallback. This isn't a huge problem for regular subtitles but it can be whenever text is set to appear in a specific position to anotate something or cover something up, or when composition between multiple text elements is involved.

I'm trying to figure out why this is; I've confirmed that it happens in every web browser and when accessing locally and remotely, on Windows and Linux. I've looked at the javascript for htmlvideoplayer where it's instantiating subtitlesoctopus. Based on what I (don't) see, I wanted to confirm - is the video player currently at all set up to render subtitles using bundled mkv fonts, without transcoding (burning)?

Emby's predecessor on my server was a solution I developed myself in 2016, and it could do this by passing to libjass a map of font names and associated URLs on instantiation. If Emby currently can't do this I'm thinking I might solve the problem using a plugin with two API endpoints, one to generate the list of fonts and the other to serve as an access point to extract and serve each font on the fly, and fiddle with the video player a bit.

Edited April 21, 2025 by Protected

[Protected 86]

I see PlaybackInfo already has all the fonts (attachments) in the list of streams in its results (ultimately from ffprobe in MediaProbeManager). This should be doable with a single endpoint that behaves like the subtitle download endpoint, but for a font attachment. I'll look into it in a bit.

[Protected 86]

EDIT: Fixed in vanilla as of 4.9.*!

I hacked it like this for now. This is probably missing a million checks server-side and could be made more efficient, but the SubtitleService has a lot of complexity I don't need. Provided I haven't missed some existing way to get this to work, I would appreciate if this got a proper fix in vanilla at some point.

htmlvideoplayer/plugin.js

//define "./../emby-apiclient/connectionmanager.js" ... _connectionmanager
//...
    self.currentSubtitlesOctopus = new responses({
        //...
        fonts: function(track) {
            var fonts = [];
            let apiClient = _connectionmanager.default.currentApiClient();
            for (let stream of mediaSource.MediaStreams) {
                if (stream.Type == "Attachment") {
                    fonts.push(apiClient.getUrl("Videos/" + mediaSource.ItemId + "/Font/" + stream.Index + "/Stream", {SetFilename: true}));
                }
            }
            return fonts
        }(track),
        //...
    })

 

MatroskaFontService.cs

Edited April 21, 2025 by Protected

[Luke 42077]

Hi, can you provide a copy of the video for testing? Thanks.

[Protected 86]

Here's a crop of the video used for the above screenshots with no audio.

test.mkv

[Protected 86]

In case anyone else wants to use this temporary fix, here's a version of MatroskaFontService tweaked to work on linux and with large fonts (.net 5 or higher required). The previous version doesn't run well on linux.

MatroskaFontService.cs

[Luke 42077]

Yea the challenge with this is that for those with slower disk i/o, the length of time to extract these fonts. And then also the renderer will have to be looked at in terms of how it reacts to those font downloads potentially taking a long time.

[Protected 86]

You could cache the fonts when a request is made to play the video and prior to actually playing it? The file is known at the time after all. It can even be done via plugin with another endpoint.

I suspect the renderer will truck along and change the font when it arrives, based on my experience, but that might be wrong.

[Luke 42077]

Quote

I suspect the renderer will truck along and change the font when it arrives, based on my experience, but that might be wrong.

Hopefully yes. But that is the important part.

[Protected 86]

Our experience so far has been good. We are using a remote server located in a different country!

I wouldn't mind if it delayed the start of the video until it received the fonts but I don't think it does that. I don't think the sync suffers though. It wouldn't make sense for them to implement it like that.

[Protected 86]

I moved this plugin to a git repository to make it easier to keep track of changes.

https://github.com/Protected/EmbyMatroskaFonts/blob/master/MatroskaFontService.cs

There were some improvements since the attached version, so make sure you use this one: A default font was added so fonts can be served from the endpoint even if they're missing from the mkv. A bug was fixed that was preventing some fonts from being served correctly. And font names are now read from within the fonts themselves in order to improve matching with subtitle styles.

Edited September 18, 2024 by Protected

[Protected 86]

EDIT: Fixed in vanilla as of 4.9.*!

Alright, after one of my users had a remote connection experience so slow it took a significant amount of time for the subtitles to appear (since they had to wait for the fonts to download), I modified the code to pre-download all the fonts and cache them client side.

This goes in HtmlVideoPlayer.play, in an anoymous function that's already there:

return async function(tracks, item, mediaSource) {
	let apiClient = _connectionmanager.default.currentApiClient();
    self._fonts = [];
    let fontPromises = [];
    
    for (let stream of mediaSource.MediaStreams) {
        if (stream.Type == "Attachment") {
            fontPromises.push(fetch(apiClient.getUrl("Videos/" + mediaSource.ItemId + "/Font/" + stream.Index + "/Stream"))
                .then((response) => response.blob())
                .then((blob) => { self._fonts.push(URL.createObjectURL(blob)) })
            );
        }
    }                        
    await Promise.all(fontPromises);
  
    ... rest of the function ...

So when instantiating SubtitlesOctopus, you just need:

    ...
    fonts: function(track) {
        let fonts = self._fonts;
        if (!fonts.length) {
            ... the default that was already there ...
        }
        return fonts
    }(track)
    ...

This ensure subtitles can always render immediately on playback.

Edited April 21, 2025 by Protected
Cleared up code

[Gremory17 15]

El 10/03/2024 a las 15:35, Protected dijo:

Muy bien, después de que uno de mis usuarios tuvo una experiencia de conexión remota tan lenta que tomó una cantidad significativa de tiempo para que aparecieran los subtítulos (ya que tuvieron que esperar a que se descargaran las fuentes), modifiqué el código para descargar previamente todas las fuentes y almacenarlas en caché del lado del cliente.

Esto va en HtmlVideoPlayer.play, en una función anónima que ya está allí:

Entonces, al crear una instancia de SubtitlesOctopus, solo necesitas:

Esto garantiza que los subtítulos siempre podrán reproducirse inmediatamente durante la reproducción.

Estoy teniendo el mismo problema con emby porque no es capaz de reproducir mi fuente personalizada en el video mkv únicamente en el navegador ocurre esta falla, en la aplicación del teléfono si lo detecta correctamente la fuente ".ttf"

 

Edited January 12, 2025 by Gremory17

[Protected 86]

Did you try the changes here or are you just requesting the feature in vanilla?