Jump to content

Make sure I understand size requirements for the transcoding directory


markkundinger

Recommended Posts

markkundinger

It seems to me that as of right now, if I want to be on the safe size, I need to dedicate a hefty amount of space to my transcoding-temp directory for emby:

 

I should have enough space available to store the full size of the source file for whatever show(s) are being watched at that moment, plus a small buffer.

 

In other words, if there are three 4K remuxes playing at the same time, 80GB each,  and they're all transcoding at once, then I need at least 240GB of transcoding space available.

 

No idea what the requirement would be for live tv, I don't have that feature.

 

I say this from observation when my Apple TVth gen is a client.  It seems to be able to direct stream a Blu-ray or DVD rip except it doesn't like the mkv format.  The resulting transcode file seems to roughly match the file size of the original movie. And then it's deleted when I stop watching the show.

 

Also, it seems that if I run out of disk space, the whole movie (and sometimes the server) just stops with an out of disk space condition, there's no auto-pruning while the show is playing.

 

Does this mean that transcoding to RAM is a bad idea unless I have tremendous amounts of RAM?

 

So in other words, if I'm running in a virtual machine, I can't use the default folder with my piddly small storage dedicated to that VM.  I need to specify something with more serious space capability.

 

Did I get that right?

Edited by markkundinger
Link to comment
Share on other sites

As happy2play said this is correct, but we plan to improve this in the future to be more adaptive to low storage situations.

Link to comment
Share on other sites

markkundinger

That would be good. I'm a Linux noob so when the whole system stopped working when out of space it took me a while to figure out what was going on.

 

In the plus side it let me enforce that my kids could only watch for 15 minutes before bedtime

  • Like 1
Link to comment
Share on other sites

  • 2 years later...
caffeineshock

is this still true?

right now im transcoding into a ram disk that is 20gb in size and this is sufficient right now. but in the future more ppl will join my server and im afraid i will need to upgrade my ram or switch to a ssd for the transcoding if this is still the case. but i need to know it before i run into an issue due to the size requerements of the transcoding

Edited by caffeineshock
Link to comment
Share on other sites

rbjtech
4 minutes ago, caffeineshock said:

is this still true?

right now im transcoding into a ram disk that is 20gb in size and this is sufficient right now. but in the future more ppl will join my server and im afraid i will need to upgrade my ram or switch to a ssd for the transcoding if this is still the case. but i need to know it before i run into an issue due to the size requerements of the transcoding

A RAM disk will not give you any (noticeable) performance improvements over a SATA SSD yet alone an NVME SSD.    

Size is impossible to say - with 1-2 streams transcoding - 20Gb is probably ok, for 10+ streams transcoding - it's probably not.

NVME's are cheap for the performance/storage you are getting - a lot cheaper than the equivalent DDR4/5 RAM anyway - and ultimately is more than sufficient for the emby transcoding requirement.

Link to comment
Share on other sites

caffeineshock
2 minutes ago, rbjtech said:

A RAM disk will not give you any (noticeable) performance improvements over a SATA SSD yet alone an NVME SSD.    

Size is impossible to say - with 1-2 streams transcoding - 20Gb is probably ok, for 10+ streams transcoding - it's probably not.

NVME's are cheap for the performance/storage you are getting - a lot cheaper than the equivalent DDR4/5 RAM anyway - and ultimately is more than sufficient for the emby transcoding requirement.

i know that ramdisk wont give me this. but ram forgets the data after a power loss so i dont need to encrypt this :)  dont question my decisions please 

so the transoce requirement is still the same? allright. time for more ssd then

Edited by caffeineshock
Link to comment
Share on other sites

rbjtech
3 minutes ago, caffeineshock said:

i know that ramdisk wont give me this. but ram forgets the data after a power loss so i dont need to encrypt this :)  dont question my decisions please 

so the transoce requirement is still the same? allright. time for more ssd then

Understand that answers given here are to help the community - not just you.

  • Like 1
Link to comment
Share on other sites

caffeineshock
5 minutes ago, rbjtech said:

Understand that answers given here are to help the community - not just you.

theres never been a question about ram transcode speeds or similar. only about size... teaching me about "ram does not give you transcode speed" is going full offtopic 😅 sorry
please understand this

so you do not know if its still the same as several years ago? how much space will i need if for example we take lord of the rings UHD HDR that is ~120GB in size and i want to stream it in, lets say, 10mbit fhd

will i need 10mbit*(runtime in seconds)? 
or will it autodelete the files and only keep the transcoded media that is needed on the client for buffering?

Edited by caffeineshock
typo
Link to comment
Share on other sites

If the server detects low disk space then it will clean up transcoding segments on the fly as you go in order to conserve space.

Link to comment
Share on other sites

caffeineshock
8 hours ago, Luke said:

If the server detects low disk space then it will clean up transcoding segments on the fly as you go in order to conserve space.

cool

that means, for example, if i watch movie with the size of 100 gb and i transcode it while playing it so that the resulting size would be 10gb (just an example) 5gb space would be sufficient because emby would transcode the first 5gb and then clean up the older segments "first in first out" so that the transcoded movie will never be completely in the temp. transcoding directory?

is this right?

Link to comment
Share on other sites

3 hours ago, caffeineshock said:

cool

that means, for example, if i watch movie with the size of 100 gb and i transcode it while playing it so that the resulting size would be 10gb (just an example) 5gb space would be sufficient because emby would transcode the first 5gb and then clean up the older segments "first in first out" so that the transcoded movie will never be completely in the temp. transcoding directory?

is this right?

Correct, although if there is enough free space, then the whole thing will be there.

  • Like 1
Link to comment
Share on other sites

caffeineshock
13 hours ago, Luke said:

Correct, although if there is enough free space, then the whole thing will be there.

yeah sure ofc. if there is enough space
nice. so in this case i can stay on my ram-disk for the transcoding, since only a small part of the file will be stored there. and after its buffered or played it gets removed

thanks ❤

  • Thanks 1
Link to comment
Share on other sites

  • 2 months later...
MrMackey

Greetings

i was planning to transcode to ram to avoid having to write to the ssd.
So I created a ramdisk with limited size.
In this test I took 1GB to make it faster but I also tried it with 10GB with the same result.
I understood that emby automatically creates space again and deletes the .ts files when the memory is full.
But unfortunately this is not the case with me and the stream stops after a short time when the limit is reached. The .ts files are only deleted when the stream is closed.

ramdisk.png.6c281bfd973d4ae2bf3772b0591987fb.png

Does anyone know what I am doing wrong?

ffmpeg-transcode-f197df58-3776-46df-a79c-b582f9b4deb6_1.txt

Edited by MrMackey
Link to comment
Share on other sites

12 hours ago, MrMackey said:

Greetings

i was planning to transcode to ram to avoid having to write to the ssd.
So I created a ramdisk with limited size.
In this test I took 1GB to make it faster but I also tried it with 10GB with the same result.
I understood that emby automatically creates space again and deletes the .ts files when the memory is full.
But unfortunately this is not the case with me and the stream stops after a short time when the limit is reached. The .ts files are only deleted when the stream is closed.

ramdisk.png.6c281bfd973d4ae2bf3772b0591987fb.png

Does anyone know what I am doing wrong?

ffmpeg-transcode-f197df58-3776-46df-a79c-b582f9b4deb6_1.txt 67.38 kB · 0 downloads

Hi, please attach the main emby server log as well. Thanks.

Link to comment
Share on other sites

MrMackey

@Luke

Hey
Sure, I had already deleted the logs so I did the test again with a 2gb ramdisk.

To better understand what I did:
I have in Emby Docker in Extra Parameters settings: --mount type=tmpfs,destination=/tmp,tmpfs-size=2000000000

262295506_unraidembydocker.thumb.png.43b2866ae24a8dc307687c87107bd53e.png

And in Emby in Transcode settings:
/tmp

1860731356_embytranscoding.png.4993cb9e28a36c151a15993ccad1b79d.png

embyserver.txtffmpeg-transcode-7a38281a-8905-490a-a751-5de7cf3b6668_1.txt

2GB tmp.png

Link to comment
Share on other sites

Happy2Play

A guess but seems to be a mismatch in paths here.

2022-09-22 04:34:31.516 Info Main: Transcoding temporary files path: /config/transcoding-temp

Eventhough "/tmp/transcoding-temp/F8B2F0/F8B2F0_%d.ts" path is being used.

Link to comment
Share on other sites

  • 2 weeks later...

@MrMackey can you please temporarily enable debug logging under the logs section in the server, then repeat the test and attach the debug server log? Thanks !

Link to comment
Share on other sites

Happy2Play

@Luke I am still confused as to why there is the mismatch unless it is just something with this platform as these should be same path.

2022-10-10 00:59:29.043 Info Main: Transcoding temporary files path: /config/transcoding-temp

/tmp/transcoding-temp/A9D49B/A9D49B.m3u8

Link to comment
Share on other sites

44 minutes ago, Happy2Play said:

@Luke I am still confused as to why there is the mismatch unless it is just something with this platform as these should be same path.

2022-10-10 00:59:29.043 Info Main: Transcoding temporary files path: /config/transcoding-temp

/tmp/transcoding-temp/A9D49B/A9D49B.m3u8

It's just a logging mistake that needs to be looked at. the logger is printing the default path.

Link to comment
Share on other sites

  • 1 month later...

I have a Radian RMS-200 (basically an 8 GB NVMe on a PCIe card) in my Emby server that I mount and use as my transcode directory. If I tune my Live TV (via HDHR 4K) to one of the 4K stations and let it play via Emby Theater, I can watch the RMS mount point fill up to 100% and the stream will eventually freeze. Stopping the Live TV stream will empty the transcoding directory, at which point I can restart playing the channel, and it continues to work. 

 

embyserver.txt

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...