Direct Stream HEVC on Apple Devices?

[K22R8CT 25]

Most of my HEVC mp4s transcode to H264 when played on Apple devices because they're hev1 (which they don't support) and not hvc1 (which they do.)

If I "fix" the files with ffmpeg they'll direct play:

ffmpeg -i 'test.mp4' -c: copy -tag:v hvc1 'test.hvc1.mp4'

 

But couldn't emby do that on-the-fly i.e. Direct Stream?

It takes a lot less processor (the command above runs ~500x on my system) than transcoding.

My knowledge of codecs and containers is limited, so there may be something I'm missing.

 

[K22R8CT 25]

Apparently if I convert the file to mkv emby will direct stream to mp4 applying the required hvc1 tag.)

So HEVC mp4's are transcoded but HEVC mkv -> mp4s direct streamed!

Is that a simple oversight (not many apple users) or some rationale behind it?

[visproduction 315]

Related: - hvc1 appears to be a choice made by the video encoder that made the current video.  It seems that ffmpeg command or remuxing the video with a program like AVIdemux probably fixes this issue for Apple playback.
===

https://stackoverflow.com/questions/63468587/what-hevc-codec-tag-to-use-with-fmp4-hvc1-or-hev1

Quote

hvc1' stores all parameter sets inside the MP4 container below the sample description boxes. 'hev1' stores all parameter sets in band (inside the HEVC stream).

This doc at Apple appears to state that hvc1 is preferred by Apple products.
https://webstore.iec.ch/preview/info_isoiec14496-15{ed5.0}en.pdf
 

Quote

1.10. You SHOULD use video formats in which the parameter sets are stored in the sample descriptions, rather than the samples. (That is, use 'avc1', 'hvc1', or 'dvh1' rather than 'avc3', 'hev1', or 'dvhe'.)

 

[Luke 42080]

3 hours ago, K22R8CT said:

Apparently if I convert the file to mkv emby will direct stream to mp4 applying the required hvc1 tag.)

So HEVC mp4's are transcoded but HEVC mkv -> mp4s direct streamed!

Is that a simple oversight (not many apple users) or some rationale behind it?

It has nothing to do with the original container. It depends on how the hevc is encoded as suggested above.

[K22R8CT 25]

32 minutes ago, Luke said:

It has nothing to do with the original container. It depends on how the hevc is encoded as suggested above.

Now I'm confused.

Here's the ffmpeg command used to direct-stream the mkv, from the logs. Note the vtag

/bin/ffmpeg -loglevel +timing -y -print_graphs_file "/config/logs/ffmpeg-directstream-b1810f44-76c3-47da-a31e-56e7b7292122_1graph.txt" -copyts -start_at_zero -f matroska,webm -noaccurate_seek -c:v:0 hevc -i "/media/Movies/My Movie.mkv" -map 0:0 -map 0:1 -sn -c:v:0 copy -bsf:v:0 hevc_mp4toannexb -c:a:0 copy -metadata:s:a:0 language=eng -disposition:a:0 default -copypriorss:a:0 0 -max_delay 5000000 -avoid_negative_ts disabled -f hls -map_metadata -1 -map_chapters -1 -vtag hvc1 -start_number 0 -hls_time 00:00:06.000 -hls_list_size 0 -hls_segment_filename "/cache/transcoding-temp/07A111/07A111_%d.m4s" -hls_segment_type fmp4 -hls_fmp4_init_filename "07A111_init.mp4" -hls_flags +temp_file+split_by_time -hls_playlist_type vod "/cache/transcoding-temp/07A111/07A111.m3u8"

Why can't emby do something similar for hev1 mp4s – converting to hvc1 on the fly, which requires very little processing power?

As I said in the first post, the same mp4 that required transcoding direct-played once I applied the hvc1 tag (which ran on the CPU at 500x so it wasn't re-encoded.)

Edited July 10, 2023 by K22R8CT

[Luke 42080]

We'd have to look at a specific example:

How to Report a Problem

Thanks.

[K22R8CT 25]

I found a post from '18 where you all discussed it, apparently it's intentional:

On 12/14/2018 at 5:35 PM, ebr said:

Yeah, it is missing the codec tag we are looking for.

Right now we require this tag because, otherwise, the content may not play at all (if it is the incorrect format).  So we err on the side of transcoding when we don't have the tag of hvc1.

 

I couldn't find a resolution but hoping the restriction might not be necessary:

Edited July 10, 2023 by K22R8CT

[K22R8CT 25]

17 hours ago, Luke said:

We'd have to look at a specific example:

How to Report a Problem

Before I go through all that, is the post (above) by ebr correct, that by design emby will never "direct stream" hev1 to hcv1?

If so, then it's working as intended, no need for logs.

 

[ebr 16185]

56 minutes ago, K22R8CT said:

that by design emby will never "direct stream" hev1 to hcv1?

No, that post is from five years ago when we first discovered this quirk with Apple.  We have since employed some techniques to work around it but they are not foolproof.

[K22R8CT 25]

48 minutes ago, ebr said:

No, that post is from five years ago when we first discovered this quirk with Apple.  We have since employed some techniques to work around it but they are not foolproof.

That's great, I'll run a test and upload the logs ASAP.

[K22R8CT 25]

Test Results

The original video (a) is an hev1 mp4.

I generated an hvc1 version (b) with the following command:

ffmpeg -i '<show>.mp4' -c:v copy -c:a copy -tag:v hvc1 '<show>.hvc1.mp4'

and an mkv version (c) with the following command:

ffmpeg -i '<show>.mp4' -c:v copy -c:a copy '<show>.mkv.mkv'

Client was the latest Safari/MacOS

The embyserver log shows the following sequence:

• Play (a) - transcoded
• Play (b) - direct play
• Play (c) - direct stream

ffmpeg logs for (a) and (c) are included.

I repeated the test with the latest beta, same result (logs attached.)

emby_test.zip embybeta_test.zip

[Luke 42080]

HI, what do you mean by sequence? Was that three different tests?

[K22R8CT 25]

I restarted emby to clear the log and played them sequentially in the order listed – the log should reflect that.

Does that answer the question?

[Luke 42080]

HI, yes you are right. There is an opportunity to improve the frequency of when we direct play. We are looking into it. Thanks.