How can media be "direct streaming" and also transcoding?

[cmacfarlane93 10]

I noticed with one of the media files in my library I get poor performance when it first starts playing. I checked my server dashboard and noticed that it was "direct streaming" but also transcoding. I thought direct played files don't get transcoded?

[neik 873]

It isn't actually transcoding it is just remuxing.

The labelling is a bit misleading.

Edited June 26, 2020 by neik

[Happy2Play 9780]

Why do you think to is Transcoding?  Is it do the stats being displayed.  The Direct Stream is just putting your video and audio tracks into a supported container.

[cmacfarlane93 10]

Just now, Happy2Play said:

Why do you think to is Transcoding?  Is it do the stats being displayed.  The Direct Stream is just putting your video and audio tracks into a supported container.

What made me think transcoding was happening is the dashboard referencing transcoding position and transcoding buffer. Also my CPU usage spikes when this file is played.

Is there anything I can do to not make the CPU spike? I was watching a 1080p h264 mkv file. I tried both chrome and firefox and both make this CPU spike.

[Happy2Play 9780]

Only have media in a format that is supported by the client/app.  But all the remux is doing is recreating the existing file in a new container.

I don't believe any browser natively supports MKV containers, that is why the item is being copied/remuxed into a different container.

[cmacfarlane93 10]

15 minutes ago, Happy2Play said:

Only have media in a format that is supported by the client/app.  But all the remux is doing is recreating the existing file in a new container.

I don't believe any browser natively supports MKV containers, that is why the item is being copied/remuxed into a different container.

Would upgrading the GPU on my server help reduce the strain on the CPU?

[Happy2Play 9780]

16 minutes ago, cmacfarlane93 said:

Would upgrading the GPU on my server help reduce the strain on the CPU?

Possibly, what system is this one that a remux is puttling a lot of strain on the system.

[cmacfarlane93 10]

1 hour ago, Happy2Play said:

Possibly, what system is this one that a remux is puttling a lot of strain on the system.

Roku Stick and Browsers seem to be triggering the CPU spike.

Emby server specs:

GPU: GeForce GT 720

CPU: Ryzen 5 2600

RAM: 16GB Quad Channel

OS: Ubuntu Desktop

 

[Carlo 4561]

What do you consider a "spike"?

GPU upgrade isn't going to help a lot for this because it's not a CPU intensive thing to rewrite the file in a new container format.  If you want to make an upgrade that would benefit your system think about an SSD drive if you don't have one.  Doesn't need to be big or expensive.  But basically an SSD drive that can hold temp and transcode files will save writing to HDD which is faster.

[cmacfarlane93 10]

16 minutes ago, cayars said:

What do you consider a "spike"?

GPU upgrade isn't going to help a lot for this because it's not a CPU intensive thing to rewrite the file in a new container format.  If you want to make an upgrade that would benefit your system think about an SSD drive if you don't have one.  Doesn't need to be big or expensive.  But basically an SSD drive that can hold temp and transcode files will save writing to HDD which is faster.

I have emby installed onto an SSD where the OS resides so I think the default transcoding area would be on SSD as well? I'm not sure because the media itself is stored on HDD. 

Also I see multiple cores hit 100% load when I am streaming said media in my original post. 

[Carlo 4561]

They hit 100% but don't stay there correct?  If so that's just OS overhead likely from the quick processing of the spawning ffmpeg, feeding it an input file and having it generate an output file.  That process goes on over and over during the playback.

From your DASHBOARD page on EmbyServer scroll down to the bottom and you'll see some of the different paths listed.

[cmacfarlane93 10]

9 minutes ago, cayars said:

They hit 100% but don't stay there correct?  If so that's just OS overhead likely from the quick processing of the spawning ffmpeg, feeding it an input file and having it generate an output file.  That process goes on over and over during the playback.

From your DASHBOARD page on EmbyServer scroll down to the bottom and you'll see some of the different paths listed.

They stay at 100% until it's finished transcoding

[Carlo 4561]

If you drill down on the CPU is it ffmpeg hogging them?

[cmacfarlane93 10]

43 minutes ago, cayars said:

If you drill down on the CPU is it ffmpeg hogging them?

Yes with emby running my sitting at around 10% cpu usage per thread. I can watch my system monitor and the second I start the transcoding I see the cpu usage spikes. I also observe the second I stop playing the cpu spikes back down to about 10% per thread. I am certain it's the transcoding causing the spikes.

Edited June 27, 2020 by cmacfarlane93

[Carlo 4561]

Drill down in performance monitor and know for sure exactly what process is doing it.

[cmacfarlane93 10]

9 minutes ago, cayars said:

Drill down in performance monitor and know for sure exactly what process is doing it.

What do you mean? How would I find that out?

[Carlo 4561]

You can also use Task Manager as well. I "expanded" the Windows Command Processor in this example to see what was running under it.  This btw, is not from Emby but just showing how to look at it.

 

Edited June 27, 2020 by cayars

[speechles 2055]

If you disabled "throttle" for transcoding this is expected. It wants to finish ASAP. 

The reason it shows transcoding is because it is. When an item goes into a new container it spawns ffmpeg. The video and audio streams within your container can be copied. But they cannot be played in the original container. So in this case transcoding is used to directstream your copied audio and video streams into an HLS/m3u8 manifest with ts slices. It is basically a remux as mentioned earlier but is transcoding to get there. Any time ffmpeg is invoked is transcoding something: container or codec.

[Carlo 4561]

That is very confusing speechles as it is NOT transcoding but simply changing the package.  These are different things.

Transcoding is not just a package change but changing and actual stream inside the file.  That could be video or audio being rewritten.  If you are not changing the audio and video but simply copying those streams into a different package it's a remux and not a transcode.

[cmacfarlane93 10]

3 minutes ago, speechles said:

If you disabled "throttle" for transcoding this is expected. It wants to finish ASAP. 

The reason it shows transcoding is because it is. When an item goes into a new container it spawns ffmpeg. The video and audio streams within your container can be copied. But they cannot be played in the original container. So in this case transcoding is used to directstream your copied audio and video streams into an HLS/m3u8 manifest with ts slices. It is basically a remux as mentioned earlier but is transcoding to get there. Any time ffmpeg is invoked is transcoding something: container or codec.

This is extremely informative thank you very much for posting.

So if I was to enable hardware throttling should I expect reduced performance? Example: Less frames per second during playback.

[speechles 2055]

6 minutes ago, cmacfarlane93 said:

This is extremely informative thank you very much for posting.

So if I was to enable hardware throttling should I expect reduced performance? Example: Less frames per second during playback.

No it will have a point where it stops transcoding instead of running through the entire file in one go. Throttle will introduce pauses until the client gets caught up to where it needs more data. It might introduce several pauses that a client might intercept as a problem (like Roku on firmware version 9.3). This is entirely experimental my advice to you and you are your best detective as you are the closest to the problem.

To get more transcoding speed you can adjust cores, but this won't affect some transcoding methods as they max out on cores. This is why Auto is the default on cores. But you can experiment. This is why it allows change.

Edited June 27, 2020 by speechles

[speechles 2055]

Cayars, transcoding is any changes done to a file. It does not mean just the codecs are changed. Anything that causes ffmpeg to process the file is transcoding. Point blank. There is transcoding info created by the server inside the session info during a fetch of /sessions. What part is hard to understand? The only time transcoding info is not created by the server is during direct play. Point blank. Black and white. There is no grey area.

Direct Play = Direct

Direct Stream, Remux, Transcoding = Transcoding

The above is ffmpeg terminology.

 

As far as network direct play Emby rarely uses this because it cannot convey resume points and watched states correctly as the files are transported "outside" Emby. Emby uses network directstream for most devices and acts as the middleman serving an http streaming endpoint Emby created. This is where the confusion lies. Is it network or ffmpeg in relation to play method?

 

 

Edited June 27, 2020 by speechles

[Carlo 4561]

Sorry but that is not correct. Transcoding is the direct digital-to-digital conversion of one encoding to another.  If you are not re-encoding a stream you are not transcoding (hence the name). Transcoding refers to the streams inside the file and not the file or package itself.

Transcoding = Decoding and then Encoding a stream

Remux means the data has been copied (often simply to a new container like mkv) without any changes. 

Remux = Repackage but the encoding didn't change.

In Emby talk:

Direct Play - file used as is with no changes

Direct Streaming - A Remux is taking place to repackage the stream.

Transcoding - A stream in the file is not suitable for the player and needs to be decoded and re-encoded.  This could be due to bitrates, codecs or subtitles that are not compatible for instance.

* While re-encoding audio tracks is technically a transcode in Emby "Transcoding" is usually reserved for video re-encoding to distinguish it from audio as audio conversion is pretty easy on the CPU/GPU compared to video.

* During a transcode the package might be changed as well to use a streaming format.

Reasons for Transcoding are covered in this knowledge base article: 

https://support.emby.media/support/solutions/articles/44001159897-transcoding

This knowledge base article discusses the difference of direct play, direct stream and transcoding:

https://support.emby.media/support/solutions/articles/44001920144-direct-play-vs-direct-streaming-vs-transcoding

Edited June 27, 2020 by cayars

[speechles 2055]

https://blog.stackpath.com/transcoding/

 

@cayars You know I love you. Cmon. But presently having done work on the stats for nerds side of the Roku app. It has always exposed "Transcoding Info" to users if present. Recently the server also adopted doing this too, as it is complete honesty. Now technically a remux is still TRANS-remuxing. So while we can ditch the "coding" side as obviously that confuses and means "codec" but we keep the "Trans" side and it doesn't mean anything deragatory.

 

But you can see why we cannot simply call it "TRANS" in todays society right? It is not politically correct anymore. It is mean.

(full disclosure: I have a son who wishes to become a girl who dresses trans right now and takes hormones. This is not acceptable term to use ever again.)

Emby should never make light of religious, human rights, or any other issue or it fails as a leader.

Lets say this anytime you see "Transcoding" type information on the stats for nerds it will not be direct playing. It will say in the play method what the proper play method is. Anytime you see transcoding info it could be for a directstream(both audio/video copied), remux(only video copied, audio is coverted), or transcode(both audio and video converted). The above is honestly how the web app terminology comes from. The Roku simply adopts these terms and follows the web app as if it is the preacher at the church of Emby. All apps do similar. There is only 1 band leader and it is the web app (on all its various platforms). Then Android TV is a close second. So forth and so on and etc.

You honestly have the terms REMUX and DIRECTSTREAM backwards on what they actually do in the wiki. I am not joking. This is honestly what is causing confusion. A remux always has converted audio. A direct stream always copies both audio and video streams.

I am not trying to confuse you. I am trying to elucidate you. Remember "Come, come, elucidate your thoughts" from the Eliza program back forever ago.. Elucidation is key to understanding.

Edited June 27, 2020 by speechles

[Carlo 4561]

Your blog post completely agrees with what I said.  This sums it up "Transcoding and encoding are often and incorrectly used interchangeably; encoding is the original compression of data and transcoding is the process of decoding, reformatting, and re-encoding data."

Like I said to transcode you decode the stream and re-encode it.  Nowhere does that blob post call a package change a transcode and they go out of their way to call it what it is a "transmux".  A transmux in Emby speak is a "direct stream" with the exception of re-encoding audio.

That's funny with the "TRANS" thing.  Didn't even think of that and being PC. LOL

Emby also incorrectly labels some "Direct Streaming" when in fact it's transcoded.  In "Emby speak" it's the video that determines "stream" vs "transcoded" to differentiate between the two and it's done to make it easy to know how CPU BOUND the process is.  Even when an audio track is transcoded it's very easy on the CPU compared to video so Emby uses the video to determine the difference.  That's how a true transcode can still be marked in Emby as a "direct stream" even that it's technically not.  I'm personally not a fan of this but it is what it is and I understand why they choose to label it this way.  With that said I'm not sure if Emby Server still does this or not concerning only the audio conversion calling it a direct stream.

In Emby speak "Trans" is a transcode and a Transmux is a "direct stream".