HLS (streaming video) supports subtitles?

[BC101 31]

According to the IETF draft of HLS (HTTP Live Streaming) -- http://tools.ietf.org/html/draft-pantos-http-live-streaming-12 -- subtitles are a supported attribute , however the only supported format is webvtt.

The good news -- according to http://en.wikipedia.org/wiki/SubRip#WebVTT -- is the format is based on .srt and is very similar.

 

If this isn't already supported I guess this is a feature request (and should be moved).

Is this supported currently?

[bloodtaker 71]

I know the web client has it setup to have subtitles turned on or off as well as different audio as well. It uses the .srt format.

[gcw07 347]

This is the reason why we transcode with subtitles often. So we can burn in the subtitles to the video itself.

[BC101 31]

What i'm proposing is not burning them in, but using the webvtt subtitle format (converting .srt to it is trivial) and streaming those in segments just as you would the video (..segment0.ts, ..segment1.ts, etc) if anyone wants to know what the HLS format looks like, see these examples:

 

 

https://devimages.apple.com.edgekey.net/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8

 

If you load that up in notepad, find:

 

#EXTM3U


#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 1",AUTOSELECT=YES,DEFAULT=YES
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="bipbop_audio",LANGUAGE="eng",NAME="BipBop Audio 2",AUTOSELECT=NO,DEFAULT=NO,URI="alternate_audio_aac_sinewave/prog_index.m3u8"


#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="en",URI="subtitles/eng/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="en",URI="subtitles/eng_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="fr",URI="subtitles/fra/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Français (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="fr",URI="subtitles/fra_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="es",URI="subtitles/spa/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="Español (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="es",URI="subtitles/spa_forced/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語",DEFAULT=YES,AUTOSELECT=YES,FORCED=NO,LANGUAGE="ja",URI="subtitles/jpn/prog_index.m3u8"
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="日本語 (Forced)",DEFAULT=NO,AUTOSELECT=NO,FORCED=YES,LANGUAGE="ja",URI="subtitles/jpn_forced/prog_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=263851,CODECS="mp4a.40.2, avc1.4d400d",RESOLUTION=416x234,AUDIO="bipbop_audio",SUBTITLES="subs"
gear1/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=28451,CODECS="avc1.4d400d",URI="gear1/iframe_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=577610,CODECS="mp4a.40.2, avc1.4d401e",RESOLUTION=640x360,AUDIO="bipbop_audio",SUBTITLES="subs"
gear2/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=181534,CODECS="avc1.4d401e",URI="gear2/iframe_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=915905,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=960x540,AUDIO="bipbop_audio",SUBTITLES="subs"
gear3/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=297056,CODECS="avc1.4d401f",URI="gear3/iframe_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=1030138,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1280x720,AUDIO="bipbop_audio",SUBTITLES="subs"
gear4/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=339492,CODECS="avc1.4d401f",URI="gear4/iframe_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=1924009,CODECS="mp4a.40.2, avc1.4d401f",RESOLUTION=1920x1080,AUDIO="bipbop_audio",SUBTITLES="subs"
gear5/prog_index.m3u8
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=669554,CODECS="avc1.4d401f",URI="gear5/iframe_index.m3u8"


#EXT-X-STREAM-INF:BANDWIDTH=41457,CODECS="mp4a.40.2",AUDIO="bipbop_audio",SUBTITLES="subs"
gear0/prog_index.m3u8

Then look at the subtitle stream file:

https://devimages.apple.com.edgekey.net/streaming/examples/bipbop_16x9/subtitles/eng/prog_index.m3u8

#EXTM3U
#EXT-X-TARGETDURATION:30
#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-PLAYLIST-TYPE:VOD
#EXTINF:30, 
fileSequence0.webvtt
#EXTINF:30, 
fileSequence1.webvtt
#EXTINF:30, 
fileSequence2.webvtt
#EXTINF:30, 
fileSequence3.webvtt
#EXTINF:30, 
fileSequence4.webvtt
#EXTINF:30, 

You see every 30 seconds the client requests the latest "chunk" so the client is doing proper streaming.

Edited January 8, 2014 by BC101

[Tikuf 663]

Since we don't currently use srt  directly (we convert on the fly to .ass) and we write the m3u8 on the fly it is not as trivial as you may think we have a few changes in the works with subs. But for now burn in is our best option as not all clients support webvtt and without ffmpeg support it would have to be a custom solution. Subtitle support is still maturing in ffmpeg but right now there are gaping holes this is what we can encode to >

 

 ASS (Advanced SubStation Alpha) subtitle

 DVB subtitles (codec dvb_subtitle)

 DVD subtitles (codec dvd_subtitle)

 3GPP Timed Text subtitle

 SubRip subtitle with embedded timing

 SSA (SubStation Alpha) subtitle

 SubRip subtitle

 DivX subtitles (XSUB)

Edited January 8, 2014 by Tikuf

[BC101 31]

Great information, thanks!

 

Sorry, what I was saying is that it's trivial to convert .srt to .webvtt.

So I take it what you support is what ffmpeg supports natively, no more, no less.

So if ffmpeg supported outputing segments with webvtt, then it would be trivial to implement into MB3 ?

Edited January 8, 2014 by BC101

[BC101 31]

According to 

"After this commit (and the preceding commits) HLS WebVTT subtitles

should work properly (ticket #2833)." He (Anssi Hannula, Thanks!)

has WebVTT working in FFMPEG: https://github.com/anssih/FFmpeg

[BC101 31]

https://github.com/anssih/FFmpeg/commits/hls-patchset-wip-2 Very good progress on the HLS and webvtt subtitle support, looks like it might be found in official branch soon.

[Tikuf 663]

Yup I have seen that still it will be a bit before it filters and we can support it. There are changes coming to subs in the near future.

[BC101 31]

Sounds good. I'm not going to put considerable effort getting .srt's working via a side-loading method in that case. ffmpeg will support it officially soon and I'm sure it will be in the MB3 api shortly after 

[Luke 42077]

This is now supported in the next dev build.

 

It's only supported with the new hls api and not the old one. you also have to add a param

 

SubtitleMethod=hls

 

because the default method is burning them in.