Transcoding temporary path

[casaper 4]

Hello

The box my Emby server runs on (Odroid H2+) has actually quite fast storage with a nvme SSD, and currently I have the transcoding temporary path on that (seperate btrfs subvolume). 
So far so good.

But this implies two general problems:

1. The SSD might be affected with wear levelling from this kind of use
2. The board has 32GB of ram

So I presume this could all be done in ram.

So either

- Emby has some hidden config option that lets me configure it to not store these temp files on disk?
- I create a ramdisk for this if it doesn't (I presume it doesn't)

If Emby could be configured that would of course be the best way to do such a thing, since it would be way more flexible and the choice of size wouldn't be an issue (or at least not to me, since you guys code the thing :wink:).

But I would be surprised if Emby really had such a config option/feature.

So it'll be a Ramdisk then, and that is actually not hard to do at all.
There is only two questions I have regarding this:

A: What would be a reasonable size for that ram disk?
B: What is Emby's behaviour when that disk runs full?

Currently I found that subvolume where my transcoding temp resides on to run up to approx. 3GB full. So could I manage to have for ex. a 4GB ramdisk and could expect things to run smoothly?

I presume due to the fact that this ramdisk can be expected to run even faster than the nvme drive, some cuts in space might be more tollerable, right?

TIA for any answers.

Regards

[Happy2Play 9780]

There are currently no options in Emby to use Ram.

Emby currently maintains the entire Transcode/Stream in the folder so it is hard to estimate a size as you could have multiple item playing.  So if you have 4 5Gb files being played you would need about 20Gb of space.  A server restart will clear the folder or if the media is properly stopped in the app/client it will clear that media.

Once you run out of space I believe the stream crashes and logs the disk space error.

[casaper 4]

Hello

Thx for that super speedy answer!

I was just fighting with that quote feature of that forums' editor....
 

Quote

There are currently no options in Emby to use Ram.

Which is not such a surprise, since most boxes Emby server runs on will probably not even closely have enough for that.

Quote

So if you have 4 5Gb files being played you would need about 20Gb of space.

My server will rarely serve more than 2 clients at the time, so there I could decide on a tolerable compromise.  What I noticed though, is that it also stores the whole film when "direct" streaming is at work. Because actually most of the time that is what is happening, since Emby is most of the time only serving me via LAN and rarely over a symmetric FTTH link, so even remotely it often still ends up direct streaming.

 

Quote

Once you run out of space I believe the stream crashes and logs the disk space error.

Bummer. That practically renders that approach with the ram disk impossible. I wasn't aware that Emby will simply transcode and store the whole film. I had the illusion it'd kind of did it like one would cache streaming, so as much as doable ahead and as much as tolerable backwards. ¯\＿(ツ)＿/¯

Well anyway. I put ashes on that idea and entomb it.

Thanks anyway for the quick answer.

Edited July 28, 2020 by casaper

[Sammy 790]

I recently had a transcode.tmp file that was 169Gb so there's that. This should really never happen but it does. The point being is you're going to need a lot of RAM..

I think m.2 SSD's are pretty robust on read-write capacity before they "wear out". Doesn't RAM suffer the same problem over time too? I use a 1Tb m.2 that is also my boot drive but I've considered moving these to another m.2 drive. My MoBo supports two so why not?!

[casaper 4]

15 minutes ago, Sammy said:

I recently had a transcode.tmp file that was 169Gb so there's that. This should really never happen but it does. The point being is you're going to need a lot of RAM..

 

But that seems more like a bug, unless you were transcoding from a 8K film or so?

Quote

I think m.2 SSD's are pretty robust on read-write capacity before they "wear out".

I guess you're probably right. And additionally it'd not drive me to bankruptcy, even if it wears out.

Quote

Doesn't RAM suffer the same problem over time too?

I think I heard of electron migration or something like that? I was a terrible student at physics , but someone once told me something like that. But AFAIR that is something completely different from the wearing off of a NAND cell. If I remember correctly that will not happen until I will bring the thing to a tec museum in a decade or two.. But dear physicists, bash me if that was rubbish.
I never heard that more or less intensive use of RAM had any effect on its life expectancy.

Quote

I use a 1Tb m.2 that is also my boot drive but I've considered moving these to another m.2 drive. My MoBo supports two so why not?!

Yea, I'm kinda in the Scrooge McDuck mode actually . So I should really stop worrying about my m.2 drives wear off when I just bought a totally overpowered board for the actual use...

 

Edited July 28, 2020 by casaper

[Happy2Play 9780]

2 minutes ago, casaper said:

16 minutes ago, Sammy said:

I recently had a transcode.tmp file that was 169Gb so there's that. This should really never happen but it does. The point being is you're going to need a lot of RAM..

 

But that seems more like a bug, unless you were transcoding from a 8K film or so?

If I had to guess I would say Run Away Live TV session.

[Sammy 790]

16 minutes ago, Happy2Play said:

If I had to guess I would say Run Away Live TV session.

 

 

[sfatula 207]

You could just hook up an old hard drive you might have laying around for the transcoding. I do many transcodes at once, and it doesn't even remotely stress the iron. Very cheap too (possibly $0). No need for memory or SSD really. 

[casaper 4]

39 minutes ago, sfatula said:

You could just hook up an old hard drive you might have laying around for the transcoding. I do many transcodes at once, and it doesn't even remotely stress the iron. Very cheap too (possibly $0). No need for memory or SSD really. 

Sure your perfect Morse code abilities help you with punching the data in. Another sense of SMR if you like ...

[sfatula 207]

My system has over 32TB of data now, not going to use SSDs for obvious reasons. Video access is sequential, pretty minimal requirements. 

[casaper 4]

4 hours ago, sfatula said:

My system has over 32TB of data now, not going to use SSDs for obvious reasons. Video access is sequential, pretty minimal requirements. 

Of course, I hope it is sequential, if you have to punch it all in by hand...

[Q-Droid 989]

15 hours ago, casaper said:

Hello

The box my Emby server runs on (Odroid H2+) has actually quite fast storage with a nvme SSD, and currently I have the transcoding temporary path on that (seperate btrfs subvolume). 
So far so good.

But this implies two general problems:

1. The SSD might be affected with wear levelling from this kind of use
2. The board has 32GB of ram

 

BTRFS is a CoW filesystem and naturally "kind" to SSD. Run a periodic trim (weekly?) and the SSD will outlast the rest of your gear.

[Sammy 790]

12 minutes ago, Q-Droid said:

BTRFS is a CoW filesystem and naturally "kind" to SSD. Run a periodic trim (weekly?) and the SSD will outlast the rest of your gear.

Is this Linux specific or can this be done in Windows too? How to do it?

[Q-Droid 989]

5 minutes ago, Sammy said:

Is this Linux specific or can this be done in Windows too? How to do it?

What is, BTRFS? I think there are BTRFS drivers for Windows but I have no real knowledge about them. Keep in mind BTRFS is not a high performance FS, there is some overhead built in that's handled in the kernel. As a bolt-on in a foreign OS those layers might become noticeable. And there's no guarantee the Windows drivers support all features in the spec. YMMV.