Audio Track Selection with Roku Ultra 4670x

[roaku 797]

I recently installed Emby on my Synology DS218J to potentially migrate away from its built in Video Station/Media Server offering because Emby offered some features that were either non-existent or only available through Synology's abandoned VS Roku client. And using Media Server's UPNP/DLNA client with Roku Media Player is not a fast or fun experience.

The main features I'm after, beyond fast browsing, are being able to select the audio track for play back for multi-track mp4s and presenting special features and alternate versions of a title for viewing and playback integrated within the movie details view.

I'm happy with the UI overall and the special features/alternate movie versions integration.

However...

Audio track selection doesn't always match what plays back on my Roku Ultra 4670x v9.3.0 build 4194-46

I have an example mp4 (sourced from Bluray -> makeMKV -> Handbrake) with three audio tracks:

1st - DTS-HD MA 5.1 Default (passed through from source)
2nd - AAC 5.1
3rd - AAC stereo (commentary)

When I select the 1st track, it (usually) starts playback with *something* transcoded to Dolby Digital 5.1.

When I select the 2nd track, it seems to be right. That is I get a Dolby Digital 5.1 signal, which I assume is a result of transcoding the AAC 5.1 (I have the decode Multi-channel AAC option checked in the Emby Rokuapp)

When I select the 3rd track, it consistently starts playback with the 1st track (DTS-HD is passed through and decoded as legacy DTS as expected for Roku).

No selection can get the 3rd track to actually play.

The 1st track will also sometimes pass through the DTS if I haven't made any audio track selections recently.

All three audio tracks play back fine through the web app, although ffmpeg pops up as a process in Synology's resource monitor with each selection, so I assume it's transcoding something my laptop's built ind audio card can comprehend.

I've experimented with other mp4s with AC3 5.1 as the default track and AAC for the additional. The behavior is the same for the AAC tracks and I've yet to get a commentary track to play back at all (is it maybe because they're usually stereo/mono?).

I'll also note that I have no issues playing back the Default tracks (DTS gets passed through, the various DDs are detected correctly) of these mp4s through Roku Media Player.

Is my experience unique here? Any advice?

--------------

Debug excerpt for when the 1st/Default track is the selection in the audio track selector. It looks like it errors out on the AudioCodec instead of passing the DTS through, which it has no problem doing when the 3rd track is selected (even though the actual 3rd track is AAC):

2020-11-01 23:14:54.131 Info App: User policy for everyone. EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
2020-11-01 23:14:54.136 Info HttpServer: HTTP Response 200 to 192.168.1.143. Time: 38ms. http://192.168.1.139:8096/emby/Items/249/PlaybackInfo?UserId=d2524bfa915c4d8d9aa45ba8b16771e5&starttimeticks=00000000&audiostreamindex=1&isplayback=true&maxstreamingbitrate=35000000&subtitlestreamindex=-1&mediasourceid=9b9d4697f423962824dc1cd243ce2e9c. ConnectionId: null
2020-11-01 23:14:54.554 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/Videos/249/index.bif?width=320&mediaSourceId=9b9d4697f423962824dc1cd243ce2e9c. Connection=close, Host=192.168.1.139:8096, User-Agent=Roku/DVP-9.30 (469.30E04194A)
2020-11-01 23:14:54.586 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/videos/249/master.m3u8?DeviceId=44494a7a-ac64-5f72-8315-a70bc297bde1&MediaSourceId=9b9d4697f423962824dc1cd243ce2e9c&PlaySessionId=b6a43a29573c4670a490881152b16bd3&VideoCodec=h264,mpeg1video,mpeg2video,hevc&AudioCodec=ac3,aac,mp2,mp3,eac3,flac,opus,vorbis,lpcm&VideoBitrate=34616000&AudioBitrate=384000&MaxFramerate=60&MaxWidth=3840&MaxHeight=2160&AudioStreamIndex=1&SubtitleMethod=Encode&TranscodingMaxAudioChannels=6&SegmentContainer=ts&SegmentLength=3&MinSegments=1&BreakOnNonKeyFrames=True&ManifestSubtitles=vtt&h264-maxrefframes=16&h264-videobitdepth=8&h264-profile=high,main,baseline,constrainedbaseline&h264-level=51&aac-audiochannels=2&eac3-audiochannels=6&ac3-audiochannels=6&flac-audiochannels=6&lpcm-audiochannels=6&mp3-audiochannels=2&mp2-audiochannels=2&vorbis-audiochannels=6&opus-audiochannels=6&TranscodeReasons=AudioCodecNotSupported.

Debug excerpt for when the 2nd track is selected. This time it complains about AudioChannels, but that might be normal since we expect it to send DD over the wire instead of the AAC:

020-11-01 23:31:28.093 Info HttpServer: HTTP Response 200 to 192.168.1.143. Time: 30ms. http://192.168.1.139:8096/emby/Items/249/PlaybackInfo?UserId=d2524bfa915c4d8d9aa45ba8b16771e5&starttimeticks=00000000&audiostreamindex=2&isplayback=true&maxstreamingbitrate=35000000&subtitlestreamindex=-1&mediasourceid=9b9d4697f423962824dc1cd243ce2e9c. ConnectionId: null
2020-11-01 23:31:28.465 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/Videos/249/index.bif?width=320&mediaSourceId=9b9d4697f423962824dc1cd243ce2e9c. Connection=close, Host=192.168.1.139:8096, User-Agent=Roku/DVP-9.30 (469.30E04194A)
2020-11-01 23:31:28.487 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/videos/249/master.m3u8?DeviceId=44494a7a-ac64-5f72-8315-a70bc297bde1&MediaSourceId=9b9d4697f423962824dc1cd243ce2e9c&PlaySessionId=e8cf6a4d16c44551b74deba832d6da31&VideoCodec=h264,mpeg1video,mpeg2video,hevc&AudioCodec=ac3,aac,mp2,mp3,eac3,flac,opus,vorbis,lpcm&VideoBitrate=34360000&AudioBitrate=640000&MaxFramerate=60&MaxWidth=3840&MaxHeight=2160&AudioStreamIndex=2&SubtitleMethod=Encode&TranscodingMaxAudioChannels=6&SegmentContainer=ts&SegmentLength=3&MinSegments=1&BreakOnNonKeyFrames=True&ManifestSubtitles=vtt&h264-maxrefframes=16&h264-videobitdepth=8&h264-profile=high,main,baseline,constrainedbaseline&h264-level=51&aac-audiochannels=2&eac3-audiochannels=6&ac3-audiochannels=6&flac-audiochannels=6&lpcm-audiochannels=6&mp3-audiochannels=2&mp2-audiochannels=2&vorbis-audiochannels=6&opus-audiochannels=6&TranscodeReasons=AudioChannelsNotSupported.

 

Debug excerpt for when the 3rd track is selected. Now, it starts passing through the DTS with no transcoding even though the audiostreamindex is 3, not 1 (I assume these are 1 based indexes...).

2020-11-01 23:26:13.972 Info HttpServer: HTTP Response 200 to 192.168.1.143. Time: 51ms. http://192.168.1.139:8096/emby/Items/249/PlaybackInfo?UserId=d2524bfa915c4d8d9aa45ba8b16771e5&starttimeticks=00000000&audiostreamindex=3&isplayback=true&maxstreamingbitrate=35000000&subtitlestreamindex=-1&mediasourceid=be88cd6d0ce1b1147537708ac92374a0. ConnectionId: null
2020-11-01 23:26:14.365 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/Videos/249/stream.mp4?static=true&mediaSourceId=be88cd6d0ce1b1147537708ac92374a0. Host=192.168.1.139:8096, User-Agent=Roku/DVP-9.30 (469.30E04194A), Accept=*/*
2020-11-01 23:26:14.366 Info HttpServer: HTTP GET http://192.168.1.139:8096/emby/Videos/249/index.bif?width=320&mediaSourceId=be88cd6d0ce1b1147537708ac92374a0. Connection=close, Host=192.168.1.139:8096, User-Agent=Roku/DVP-9.30 (469.30E04194A)
2020-11-01 23:26:14.511 Info HttpServer: HTTP POST http://192.168.1.139:8096/emby/Sessions/Playing. UserAgent: Roku/DVP-9.30 (469.30E04194A)
2020-11-01 23:26:14.526 Info SessionManager: Playback start reported by app Roku SG 4.0.4 playing The Movie. Started at 0 ms

Unrelated to the audio selection, another thing I noticed is that when I hit the Roku home button while playing back a video that's being transcoded, the Emby server doesn't seem to shut down the ffmpeg process. I ended up with 6 or so ffmpeg processes and Synology choking before I caught on. I can work around it now that I know, but it'd be nice not to have to think about how I end playback.

Edited November 2, 2020 by roaku

[ebr 14959]

Hi.  Can you please try the beta version of the app and see if that works correctly?

[roaku 797]

Sure. Just added the beta version now.

[roaku 797]

I just had the same experience with the beta version. Selecting the 3rd track results in playback of the 1st track.

I should also mention that this title has multiple versions. The Video dropdown selector seems to be working correctly, but I suppose it could be throwing off the audio selector, since the audio options are a function of the video selection.

I've also done some further experimentation, so I'll add that info now.

I reencoded the aac audio tracks to ac3, while copying the video and DTS track over as is. This new mp4 demonstrates the same behavior as the original, so I don't think AAC is the culprit.

I'm beginning to think that there's just a simple mismatch between the audio track the Roku client thinks it's asking for and what the server responds with.

[speechles 1929]

I can test against this. It might be the name and the Roku is being clumsy and choosing the first AAC track it finds. I can see if we can smarten that up a bit see if there is anything we can do in that area.

Make sure none of the tracks are marked default or forced as the Roku may try to always play that track when you do this and direct play. When you choose Playback Correction and change audio tracks afterwards it should work because ffmpeg becomes involved.

It is only during Direct Play that the Roku is in full control and not Emby. This means the Roku can make a mistake and we have to provide better clues for the Roku to get it correct than we presently are doing.

Edited November 2, 2020 by speechles

[roaku 797]

Thanks for the tips and added context.

I'll use them to do some further experimentation later today.

Quote

Make sure none of the tracks are marked default

Thanks for mentioning this.

XMedia Recode seemed to automatically set *all* the audio tracks in my reencoded mp4 to Default (at least, Emby sees them that way in the UI), so I'll see if I can find a setting in XMR to turn that behavior off and retry with an mp4 that is only DTS and AC3 tracks.

Edited November 2, 2020 by roaku

[ebr 14959]

1 hour ago, speechles said:

It might be the name

The selection should be based on index number so the name shouldn't matter.

[ebr 14959]

59 minutes ago, roaku said:

XMedia Recode seemed to automatically set *all* the audio tracks in my reencoded mp4 to Default

I'm pretty sure this is the problem.  I seem to recall seeing this before with the Roku player.  If multiple tracks are set to default, it gets confused.

[roaku 797]

41 minutes ago, ebr said:

I'm pretty sure this is the problem.  I seem to recall seeing this before with the Roku player.  If multiple tracks are set to default, it gets confused.

Just to be clear, only my experimental re-encoded mp4 has multiple defaults. My originals do not have multiple defaults, but do have the problem..

[roaku 797]

I've reencoded an mp4 with *only* the commentary track in AAC format. This plays back and hits my receiver as PCM stereo. Yay.

So, it does seem to come down to the track selection mechanism breaking down somewhere along the way between Emby and Roku, not the tracks themselves.

 

[roaku 797]

 

On 11/2/2020 at 11:37 AM, speechles said:

I can test against this. It might be the name and the Roku is being clumsy and choosing the first AAC track it finds. I can see if we can smarten that up a bit see if there is anything we can do in that area.

Make sure none of the tracks are marked default or forced as the Roku may try to always play that track when you do this and direct play. When you choose Playback Correction and change audio tracks afterwards it should work because ffmpeg becomes involved.

It is only during Direct Play that the Roku is in full control and not Emby. This means the Roku can make a mistake and we have to provide better clues for the Roku to get it correct than we presently are doing.

I'm curious if you've had a chance to do any testing on this yet?

I finally noticed that there's a lot of debug info that can be turned on in the Roku app, so I was able to confirm through the on screen display and stats for [curious people] that Emby Roku has the correct audio track selected during playback in all cases and that the Roku is receiving the DTS track due to Direct Playback. All of which you described.

 

I've also confirmed that Attempt Playback Correction (took me awhile to find that option) causes Roku to actually play back the desired audio track. So, I do have a workaround for when I want to listen to the commentary tracks (yay). Although, transcoding is pretty stressful on my DS218j, so if at all possible, I'd like to avoid it, especially since Roku has the tendency to zombify the ffmpeg processes.

 

Finally, I noticed that the DTS-HD track is missing bitrate metadata. That is, xmedia recode thinks the birate is 0 and Emby Roku displays ????? in the stats pane. Could this be the reason for the AudioCodecNotSupported error when the DTS-HD track is actually selected?

Edit:
I was able to rule out the ???? bitrate for DTS-HD by re-ripping with the standard DTS track from the same source. This DTS track has a defined bitrate but the AudioCodecNotSupported error is still triggered.

 

Edited November 5, 2020 by roaku

[ebr 14959]

Hi.  You ensured that the item doesn't have multiple tracks selected as "default"?

Can you provide a sample for testing?

[roaku 797]

I've confirmed that Emby Roku sees only the DTS-HD track as Default.

The attached video file demonstrates the issue I'm having.

It should have the following audio tracks in order:

1st - DTS-HD MA 5.1 Default (passed through from source)
2nd - AAC 5.1
3rd - AAC stereo (commentary)

Edited November 5, 2020 by roaku

[ebr 14959]

Thanks for the sample.  The Roku player cannot see the different tracks in this item which is why we cannot switch them when direct playing (which we only do on this item in the case of the third track).  You can confirm this by pressing the "*" key on your remote during playback and observe that audio track selection is greyed out.

This may be a limitation of the mp4 container on the Roku.  If so, we need to be sure we react properly to that.  @speechles can you investigate?

[speechles 1929]

1st - DTS-HD MA 5.1 Default (passed through from source)
2nd - AAC 5.1
3rd - AAC stereo (commentary)

The issue is the setup on the Roku does not believe it can pass-through the DTS. So it has to transcode the DTS-HD core to AC3 (Dolby).

The issue will be the 2 AAC tracks. The Roku when direct playing will not be able to play the commentary track. This is because the Roku is clumsy when direct playing. We cannot over ride what happens when it direct plays.

When you use playback correction and the DTS-HD track is chosen this will cause the CPU to jump because the DTS codec is more complex to decode and processor heavy to convert. If you choose the AAC 5.1 track it will be much less processing. The 5.1 AAC might also be transcoded because by default we choose to do this in settings otherwise 5.1 AAC will become 2.0 AAC. By default the 5.1 AAC should be converted on the Emby server to 5.1 AC3 (Dolby) and keep surround.

I can investigate and see where the differences are and if there is anything we can do when we know this will happen.

@roaku Can you show me the media information for that file as shown on the web app detail screen? I want to copy that exact structure you have with my own streams in the same type of container and build my own (rather than ask you for a sample) that will replicate the exact scenario you are doing.

Edited November 5, 2020 by speechles

[ebr 14959]

3 minutes ago, speechles said:

I can investigate and see where the differences are and if there is anything we can do when we know this will happen

Please see my post above.  I've already confirmed the Roku does not support the secondary audio in this item.  I suspect it is due to the container but need you to investigate that.

[speechles 1929]

9 minutes ago, ebr said:

Please see my post above.  I've already confirmed the Roku does not support the secondary audio in this item.  I suspect it is due to the container but need you to investigate that.

 

I need to see the media information to replicate his exact scenario. I do not want to upset what we have and create another problem where now things are not direct play that should. This is surgery and we need to be so careful not to kill the patient. I can investigate and see where we can possibly try some things and hopefully get better results.

@roaku If you can provide media information (and Roku device model number and Roku firmware version) that would be most excellent. Then we can party on dudes! ( I am watching that new bill and ted movie.. haw)

 

@ebr I can replicate on a few items I have with multiple audio streams where it isn't choosing the correct audio track. Once you choose playback correction it works. But this is because that first track is marked as default. Because of that to get that second track it must remux. If that first track isn't marked as default we can play and select that second audio track just fine while direct playing. That is what we are not doing because we prefer direct play always and only check if secondary audio is supported. That is the part needs some attention I believe. I can start there. If secondary audio is supported it should only be supported if there are no default/forced tracks marked.

Edited November 5, 2020 by speechles

[roaku 797]

Your description matches what I'm experiencing. And now that I know about the playback correction option, I'm able to hear the commentary track when needed.

It looks like I would have a functional workflow now, except for this:

29 minutes ago, speechles said:

The issue is the setup on the Roku does not believe it can pass-through the DTS. So it has to transcode the DTS-HD core to AC3 (Dolby).

So, my Roku and AVR both support DTS. My receiver can handle the latest DTS codecs, but Roku is limited to the backward compatible core DTS track embedded in the fancier flavors.

But, my Roku can and does play (technically pass through) standard DTS when it receives a DTS flavored signal. I'm currently using a bare bones custom Roku app I wrote to stream these files based on a DLNA url, and it has no problem delivering the stream to my receiver. This app doesn't worry about what the Roku says it can or can't do, and is not capable of falling back to any kind of transcoding back end. The result is DTS hitting my receiver.

Even the Emby Roku app will stream with that DTS signal when it accidentally triggers Direct Play attempting to play back the AAC stereo commentary track.

My questions at this point are:

- Why does the Emby Roku app decide it has to do something when it sees DTS when it would clearly be better off doing nothing (Direct Streaming)?

- Are Roku's answers about its capability unreliable? Is the Emby Roku app misunderstanding the response?

Edited November 5, 2020 by roaku

[roaku 797]

32 minutes ago, speechles said:

Can you show me the media information for that file as shown on the web app detail screen? I want to copy that exact structure you have with my own streams in the same type of container and build my own (rather than ask you for a sample) that will replicate the exact scenario you are doing.
 

Is this what you mean?

Please let me know if you're talking about a different view.

Edited November 5, 2020 by roaku
scrolled down, saw the audio profile details

[speechles 1929]

Press HOME twice on your Roku remote. On the Roku homescreen go to HDMI AUDIO inside settings and make sure not to select AUTO. Choose "DTS, Dolby" or "DTS, Dolby+". AUTO may not always enable DTS. Choose DTS, Dolby+ if your equipment can support E-AC3 otherwise choose DTS, Dolby. After that make sure the AUDIO setting is set to Auto and NOT stereo. Then once playing an item press * and make sure that VOLUME MODE is set to OFF. Otherwise you will only get stereo if VOLUME MODE is anything other than OFF. Emby on Roku respects all the internal Roku video/audio settings.

.. and about the media information I need.. I meant at the bottom of the detail view in the web app. What the streams are composed of exactly. Not just based off the names in the app. But the entire media information for that file.

I am going to test all this logic and make sure we aren't doing anything incorrectly. I am sure we are not but the devil is in the details. Another glance over everything and checking doesn't hurt it confirms. I can do this.

Edited November 5, 2020 by speechles

[roaku 797]

@speechles

I've attached the more detailed audio info to my previous post.

----------------

Ok, so my options under 'Audio - HDMI' are 'Auto detect' (checked) or 'PCM-Stereo'.

My options under 'Audio - Audio mode' are 'Auto (DTHD, DD+, DTS-HD)' (checked) or 'Stereo'.

 

I had the 9.3 Roku software until today, when 9.4 became available for me. The upgrade didn't fix or break anything related to this issue.

[roaku 797]

Hmm...so I went with mp4 back when I made these files because it seemed that that container was the friendliest for Roku.

But looking at their current documentation, it looks like they offer more features with mkv at this point:

https://developer.roku.com/docs/specs/media/streaming-specifications.md#!

 

And more specifically, DTS isn't in the list of supported formats under mp4 for some fun reason, even though Roku definitely will pass it through if you ask nicely:

•  

  Quote

   

  • MP4, MOV, M4V

    • supported video codecs: AVC, HEVC
    • supported audio codecs: AAC, AC3, EAC3, AC4, ALAC, MP3, PCM

  • MKV, WebM

    • supported video codecs: AVC, HEVC, VP8, VP9
    • supported audio codecs: AAC, AC3, EAC3, Dolby Atmos, ALAC, DTS, DTSE, FLAC, MP3, OPUS, PCM, VORBIS

   

 

I guess I'll be trying out mkv with this video now.

Edited November 5, 2020 by roaku

[Happy2Play 8356]

Doesn't adding DTS violate the specification of the mp4 format?

Google

Quote

DTS is not officially supported in a mp4 container. Yes, it can be done and some software players can see and decode the DTS track, but the Roku will not direct play it. ... MKV is the way to go if you want DTS with direct play.Dec 17, 2013

 

Edited November 5, 2020 by Happy2Play

[roaku 797]

It certainly could.

I just did it in the various encoding apps and none of them ever complained and Roku happily played them back, so I never had a reason to ask that question.

I've got ffmpeg generating an mkv from my problem mp4 as I type, and I'm perfectly happy if that solves my problem.

I have no deep and abiding love for the mp4 container.

 

[Happy2Play 8356]

All you should have to do process your mp4 in MKVToolNix GUI.