transcode path smart watch drive space

[alturismo 37]

hi,

made some tests with putting the transcode location to a ramdisk, after some investigation it looks like emby has no smart view on fillrate from transcode location.

behaviour now is, disk runs full, stream stops, return to main and restart stream will clean the "old" files ... but current stream will stay stalled.

 

behaviour wished, watch directory and clean old files to only keep a fillrate between 80 - 90 % ... (like other media servers currently do)

prolly something for @softworkz even i know he is busy

 

tested with stable and beta branch, ramdisk 1, 2, 4 GB, always the same result

here a sample of a stalled stream while /tmp is a tmpfs ramdisk as note, black screen, end ... same for local files and live streams.

[rbjtech 4304]

Using a Ramdisk for the transcode area does not make a great deal of sense as emby needs to 'keep' the current stream available incase you need to rewind, pause or otherwise re-use any part of the previous transcode.  Only once the stream is stopped does it remove all the files as you have experienced.

Why not use an SSD for the transcode area, or if you are concerned with wearing, then a fast HDD will also perform well ?

 

 

 

 

[Luke 37151]

We plan to improve this in future updates. Thanks for reporting.

[alturismo 37]

On 2/3/2021 at 1:03 PM, rbjtech said:

Using a Ramdisk for the transcode area does not make a great deal of sense as emby needs to 'keep' the current stream available incase you need to rewind, pause or otherwise re-use any part of the previous transcode.  Only once the stream is stopped does it remove all the files as you have experienced.

Why not use an SSD for the transcode area, or if you are concerned with wearing, then a fast HDD will also perform well ?

i dont think we should discuss if it makes sence to keep GB wise of "timeshift" on a disk, either ssd or ramdisk, i personally use nvme drives as note and i wouldnt like to waste a ssd for transcoding space cause a app doesnt handle it properly ... well, not properly in my point of view.

and as transcoding should be more or less a remote play feature only anyway (at home you really should use direct play) i dont think it makes sence at all to keep transcoded data from start on, but thats not the point. the point is, as soon the disk runs full (any disk, hdd, ssd, ramdisk, whatever) emby will "stop" and doesnt handle it properly (clean "older" files as space is needed).

as its a HLS procedure anyway its may also possible to add a switch (keep xx or let it run), but hey, its a report about a not ideal behaviour and may a suggestion, by starting a discussion about using a ramdisk or not is not helpful or even straight forward.

in this case we can discuss, when is the end ? 10GB, 100GB, 1TB, 10TB ? the problem will be the same, disk full, stream stalls.

 

@Luke thanks for the info

here a sample part from the ffmpeg line from my personal livestream hls server for my ios device, each segment 4 seconds, keep last 10 segments and delete them afterwards.

-f hls -hls_time 4 -hls_list_size 10 -hls_flags delete_segments

 

[rbjtech 4304]

I don't disagree with your logic - we need this functionality - or as in your ffmpeg command, simply keep a small time buffer available, the storage is then not relevant.

However, the point I was making is your use case (using a Ramdisk) makes this an essential requirement - so until the feature is implemented, you'll need to use SSD/HDD instead.

 

 

 

Edited February 7, 2021 by rbjtech

[Carlo 4330]

This is harder than it sound for a few reason I'll briefly mention.

No throttling?  The purpose behind this is to transcode the media as fast as possible and get done all the transcoding so a person can FF/RW quickly and easily plus free up the HW to handle another transcode as soon as possible.  This type of functionality couldn't be used with a Ramdisk.

Doing a RW to something already seen but removed from the transcode folder would mean having to regenerate everything again from that spot wiping what you have.  Take for example if the base media is a TS file which doesn't have ATOMs or timing codes in it.

This last one to me is the "killer".  Say you're watching a sporting event live (TS stream) and want to rewind back a bit to see a play.  If it's gone from the transcode folder it could prove really tricky to find that spot since the base stream has no timing or index.

With that said Luke has already commented this is something he's looking to improve, but it's not as simple as many people think it would be.

 

Edited February 9, 2021 by cayars

[Luke 37151]

Hi, we will be looking at improving this in future updates to clean up older segments in the event of low disk space. Thanks for the feedback.