Closed Captions / Subtitles

[csadoian 34]

I'm having a problem with captions on different Roku devices.

I have two Roku devices.  The first is an older Roku 3, model 4200X.  It is running software version 11.50, build 4312.  The second is a Roku Express 4K+, model 3941X2.  It is running Software version 11.5.0, build 4315.  Both devices say their software is up to date.

I recorded a program from my OTA CBS station.  The video is 1080i MPEG2, Audio is English AC3 5.1.  The subtitle track is listed as "Und (Default EIA_608), Closed Captions 1, Default=yes, Forced=no, external=no.

When I play this program on my Roku 3, I get captions.  When I play this program on my Roku Express 4K+ I do not.  I fiddled with the caption settings on the Roku Express, and it made no difference at all.

However, i DO get captions on the Roku Express if I play a recording that has EXTERNAL subtitles, such as a .srt file.  It just doesn't seem to be able to play embedded subtitles.

I find it strange that an older Roku can handle the captions, while the newer one is having problems.  You'd think it would be the other way around.

My Emby server is version 4.7.11.0, running on Windows 10.

If you need server logs I can provide them.

Chuck

[ebr 14925]

 

Hi there, let's look at an example. Please attach the information requested in how to report a media playback issue. Thanks!

 

[csadoian 34]

Ok, I think I got all the logs for this.  What I did:

1)  Shut down and then restarted Emby Server

2)  Enabled debug on both Rokus

3)  Played about a minute of the recording I referenced in the original message on my Roku Express LT.  No captions were displayed, but they were enabled on the Emby app on the Roku.

4)  Played about a minute of the same recording using my Roku 3.  Captions were visible while playing on the Roku 3.

5)  Shut down the Emby server.  Captured all logs produced, zipped them and attached them to this post.

6)  At 14:05 EST on my Roku 3 I told Emby to send the debug log.

7)  At 14:06 EST on my Roku Express LT I told Emby to send the 2nd debug log.

Let me know if you need anything else.

Chuck

Edit:  Forgot to mention the logged in Emby user was "Chuck"

 

emby_logs.zip

Edited March 10, 2023 by csadoian

[ebr 14925]

Hi.  The Roku logs I found are actually at 17:05 but neither one of them had any playback in them.  Did you send them right after the playback?

The older Roku is transcoding on the server which is then extracting the closed captions and sending them on as a subtitle stream which the app is able to display.  The newer one is direct streaming which is just passing the video along unchanged with the closed captions embedded but it appears the player is not able to display them that way.

Try two things on the one that isn't working:

1) Try hitting the "*" key during playback and messing with the closed caption options in the Roku itself.  Can you get them to display?

2) Try the "Playback Correction" option under the cog menu during playback and see if the subs start showing up.  Try it more than once if necessary.

[csadoian 34]

Yeah, brain fart on the Roku log times .... it was 14:05 my time (PST, West Coast) and I forgot to actually convert the time to EST. 

I'm pretty sure I did try the "*" key several times on the new Roku, but I'll double check again along with looking at the "Playback Correction" option.

I may not have sent the Roku logs right away, I had turned debug on but at first I wasn't aware I actually had to tell the app to send the log.  There is a possibility I exited the app before I sent the logs (the Rokus are in two different parts of the house, so I was having to run between rooms to conduct the tests).  I will try to send another set of logs and let you know.

 

[csadoian 34]

Additional information ... on the newer Roku if I force transcode by using the "Playback Correction" option I get the embedded subtitles.

I've attached two pictures with stats for nerds ... one is when Emby is direct play, the other (with subtitles visible) is when Emby is transcoding.  Also attached the server logs and I sent the Roku log at 12:00pm EST (9:00am PST).  The user this time is "Nancy".

 

emby_logs.zip

Edited March 11, 2023 by csadoian

[ebr 14925]

@softworkzis it apparent to you why the cc are being lost in the direct stream example?  Logs are above and, when direct streaming, it doesn't appear we are extracting the cc to a sub stream but they're getting stripped in the video output it appears.

[softworkz 3338]

1 hour ago, ebr said:

@softworkzis it apparent to you why the cc are being lost in the direct stream example?  Logs are above and, when direct streaming, it doesn't appear we are extracting the cc to a sub stream but they're getting stripped in the video output it appears.

I don't know. We never came to explore and test these things like it was intended. It's all in an unfinished state.

Probably the data doesn't survive the deinterlacing but I can't say for sure. Also the options to configure alternative delivery of cc haven't been activated yet.

Edited March 11, 2023 by softworkz

[ebr 14925]

2 hours ago, softworkz said:

Probably the data doesn't survive the deinterlacing

The transcoded (deinterlaced) one does work.  It is the stream-copied one that is losing the CC - at least in a way that the Roku can see and display them.  On the one that works, you can see that we are extracting the CC and outputting via VTT.  That step is not there for the stream copy.

[softworkz 3338]

 

14 minutes ago, ebr said:

The transcoded (deinterlaced) one does work.  It is the stream-copied one that is losing the CC - at least in a way that the Roku can see and display them.  On the one that works, you can see that we are extracting the CC and outputting via VTT.

Thanks for the clarification. I thought it was about in-stream closed caption. 

In case of transcoding, the closed captions are split-off (splitcc) before deinterlacing (yadif) and encoded as WebBTT subtitles:

But when remuxing only, it's not possible to split-off the closed captions, because that requires decoding of the video stream as CC are part of the video stream - they aren't available as a stream of their own, so in case of remuxing, the CC data will be definitely preserved in the video stream, but we cannot provide them as WebVTT subtitles. For this case (when the client can process in-stream cc), we'll need to force transcoding.

[speechles 1920]

Quote

Roku supports EIA-608 closed caption data (analog TV format) encapsulated within a EIA-708 container (digital TV) in an H.264 elementary stream and HEVC. EIA-608 captions are delivered as part of the video stream itself.

https://developer.roku.com/docs/developer-program/media-playback/closed-caption.md
It is only for streams still broadcast in MPEG2 that this will be a problem. Don't force transcoding for all codecs please. Thanks.

 

Edited March 12, 2023 by speechles

[csadoian 34]

I did notice something today ... when watching the OTA station that I made the recording of (mentioned in my first post) on the newer Roku (Roku Express LT) LIVE, Emby transcodes the video to the Roku, and the embedded CC is displayed.  However, when playing A RECORDING made off the same OTA station Emby will direct stream it, and the embedded CC is not displayed.

I am using a HD Homerun EXTEND with NO transcoding set on the device.

Why does Emby transcode the Live TV but is able to direct stream the recording later on?

By they way, now that I know what's going on it's not a big deal to force Emby to transcode the recorded program, if I want my embedded CC to show up.

[ebr 14925]

17 hours ago, csadoian said:

Why does Emby transcode the Live TV but is able to direct stream the recording later on?

We try to direct play whenever possible.  Clearly, this shouldn't be so the app will have to somehow adjust to that.

Can you provide two ffmpeg logs?  One from playing the live channel (properly showing the CC) and then one from playing a recording from that channel (and not showing the CC but having them selected).  Don't try to correct it so they show, just let them not show and provide that log.

Thanks.

[csadoian 34]

Ok, attached are the logs.

1.   Played a recording made from the OTA channel (Dr. Phil)

2.   Played the same OTA channel live

3.   Played more of the recording

I played the recording again because I noticed when I played the live channel the captions were turned off for some reason (on the Emby app on the Roku) so I turned them back on mid-stream during #2, then played the recording again, just in case that made any difference.  I DID NOT try to correct it by forcing a transcode while playing the recordings.

Emby_Logs.zip

[Luke 37099]

On 3/13/2023 at 12:48 PM, csadoian said:

Ok, attached are the logs.

1.   Played a recording made from the OTA channel (Dr. Phil)

2.   Played the same OTA channel live

3.   Played more of the recording

I played the recording again because I noticed when I played the live channel the captions were turned off for some reason (on the Emby app on the Roku) so I turned them back on mid-stream during #2, then played the recording again, just in case that made any difference.  I DID NOT try to correct it by forcing a transcode while playing the recordings.

Emby_Logs.zip 107.3 kB · 1 download

And did the closed captions show?

[csadoian 34]

30 minutes ago, Luke said:

And did the closed captions show?

@Luke

No, that was the point of the test.  Embedded captions don't show up when played on a Roku UNLESS the video is transcoded.  My newer Roku (Roku Express 4K+) apparently can handle direct streaming while my old Roku (a Roku 3) cannot, at least as far as Emby is concerned.  It's strange, because they both appear to run the same versions of the Roku OS, although I assume the hardware must be different.

In any case, the conclusion is that on a Roku, if the media is direct play the closed captions do not display.  If the media is transcoded, closed captions DO display.  Note that when the media is being transcoded,  Emby server reports "Converting subtitles directly into video"

Note this is only true with EMBEDDED captions.  If I am playing media that has an external subtitle file (i.e. *.srt) the caption appears regardless of the Roku player I use and whether or not it is direct play or transcoded.

 

[speechles 1920]

The Roku 3 is transcoding the video (Roku 3 does not support MPEG-2) and it can play these subtitles properly because they can be pulled from the video stream as it is being processed and repackaged. The transcoded video has burned the subtitles into the video stream and they are no longer EIA-608/708.

The Roku Express 4K is direct stream copy the video. It cannot play these subtitles properly because they are not being pulled from the video stream because it is not being processed, it is just being repackaged. The subtitles are expected to be played by the Roku directly when rendering which the Roku does not support MPEG2 with EIA-608/708 subtitles so it will not do this.

When repackaging it is not decoding the video stream. The subtitles cannot be sidecar because no decode is done. The client must decode the subtitles the same as if it were Direct Playing. EIA-608/708 subtitles are embedded into the video stream.

The problem is the Roku when Direct Playing/Streaming cannot decode EIA-608 subtitles from an MPEG-2 stream. It can only do this when it is an H264 stream.

 

Since the majority of Live TV is still using MPEG2 with AC3/AAC and EIA-608 subtitles embedded within cannot Direct Play on the Roku inside MPEG-2 we must adapt the capabilities profile of the Roku client or the server must know this when dealing with the Roku and adapt itself.

@softworkz@LukeThis will continue to be a problem because Roku isn't supporting MPEG2 with EIA-608/708. How to proceed?

Edited April 4, 2023 by speechles

[softworkz 3338]

36 minutes ago, speechles said:

@softworkz@LukeThis will continue to be a problem because Roku isn't supporting MPEG2 with EIA-608/708. How to proceed?

The new subtitle features we have, provide detailed control over closed caption handling. You can disable in-stream delivery (= clients decodes EIA-608 from the video stream) generally and choose whether to either burn-in or deliver them as HLS subtitles. In the latter two cases, there's a lot of control available over the appearance and even the content.

Unfortunately, this has been postponed until after 4.8.

Independent of that though: Yes, the Roku Profile needs to be updated for this case, so the server will know that it needs to transcode and burn-1-in instead.

Edited April 4, 2023 by softworkz

[csadoian 34]

By the way, the workaround I have been using (on the Roku Express 4K+) is to start playing the program, then go over to the gear icon and choose "Attempt Playback Correction (Transcode)".  However, I have to do this twice in a row in order for Emby to decide to transcode the stream.  Once the stream is transcoded I can used embedded captions.   It would be nice if I didn't have to do it twice to get it to transcode, but I can live with it for now.

Thanks to all you that keep Emby running and adding great new features, I've been a premier member since 2017 and never looked back!  I still find it amazing that the devs actually take the time to interact with us in the community forums.  Before I chose Emby I tested Plex at the same time, but never got any support at all when I asked for help in their community forums, so I abandoned Plex and went all in on Emby.

[ebr 14925]

9 hours ago, softworkz said:

Yes, the Roku Profile needs to be updated for this case, so the server will know that it needs to transcode and burn-1-in instead.

The current Roku beta has an adjustment to try and correct this.  @csadoianlet me know if you'd like to try it out.

[csadoian 34]

2 hours ago, ebr said:

The current Roku beta has an adjustment to try and correct this.  @csadoianlet me know if you'd like to try it out.

@ebrYes, I would like to try it out.

 

[ebr 14925]

Okay I added you to the PM thread that contains a link to the beta.  Please let me know if it helps.

Thanks.