Beta Version 3.0.5666.4 Transcode

[plazma 13]

First let me explain I run emby server on a vms as a result disk access is not great, to work around this I was running a 1.5gb ram drive for the transcode folder and then running a batch script every hour that ensures anything older than 4 hours is gone.

 

Here is my issue the last two beta updates yesterday and todays, emby is throwing out 100's on small 1mb files the data in total size that's being created is talking over 4 times the space it should for the 2mbps im transcoding as.

 

Due to this im now unable to use emby, before I could hang 2 x 720p 2mbps and at least a 420kbps active clients at the same time and stream all day long using my setup no issue.

 

Im sure when I set this up I got one transcode file per transcode, not 100's of 1mb ish files.

 

playback does work for one single client but the ram disk gets hit so hard and fast it doesn't last long.

 

its not a permissions issue as its creating and working with the files just fine.

 

And no its not the ram disk, I put it back to the local vms disk and its just the same.

 

Others may likely have this issue and have not noticed, surely one transcode should equal one file... logs attached.

transcode-77b4d16f-0d1f-4271-8372-49aef796dba7.txt

Edited July 10, 2015 by plazma

[Luke 42080]

we're using the HLS streaming protocol. HLS and MPEG Dash are the way the streaming industry is headed. That's how they both work. They segment the video into small little pieces. Many of our apps have been doing it this way for a long time, but the feature is just now finally coming to the desktop web interface. Unfortunately, we can't really provide an option to go back to the old way because the old way was not reliable in Internet Explorer. I suppose we could have the option for Chrome, but I'm not really sure what the demand for that would be.

[plazma 13]

umm that's not good.....

 

I cant afford to throw 8gb of ram for a transcode temp and the vms disk speed can keep up with multiple users, I can afford to build another dedicated server for emby.

 

its a real shame, that to make ie work (why on earth anyone uses ie) everything else suffers. surely if chrome works properly with single file encodes it would be easy to detect the client as chome and not enter eat all your disk space mode.

 

Question, what if instead I swap my monitor script to run once every 10 or 5 minutes purging only the files older than 10 minutes, as I take once the new file is create the old or older files are no longer used?

 

Just testing the 10 minute forced clear down of files now...

Edited July 10, 2015 by plazma

[Luke 42080]

It's not just for IE. This method is superior for every browser because now we can move away from webm and back to h264. That's going to allow for multi-threaded encoding, as well as stream copy which will sometimes eliminate video encoding completely. To be honest, there really is a long list of benefits here. It's a big win for us.

 

You know that you can change the transcoding temporary directory, right? That is probably your best bet. The server will clean up those files automatically so i wouldn't attempt to write any scripts to do that.

[plazma 13]

Im fully aware that I can change the transcode folder, if you read my first post I save ive changed this to a ram disk and back.

 

Changing it again, to what, the local VMS hard drive, which ive already stated doesn't have the bandwidth to cope with multi users (hence why moving to a ram disk).

 

The server does correctly clean those files, problem is it only cleans down when a user stops viewing / finishes.

 

Whilst I do appreciate my 10 minute clear down will mean pause can no longer be used or to any great extent (not a problem for me), I may adjust it to half an hour, its just I want to see what the minimum size I can now get away with for transcode.

 

If it breaks wind back (skip backwards) again no problem for me and its a feature that's never really used, its play or skip forward.

 

Interestingly tried a skip backward and despite the file being gone, it worked it out and played fine for the short time I tested.

 

Next I tried pausing something for 15 minutes (so files were all gone/deleted relative to my time code). And as expected it hung as soon as the buffer was depleted.

 

This leads me to conclude my forced clear down could be made so it breaks nothing... possibly an additional option could be added to the transcode page to do the same and ask for a time to keep files for, default would be 0 until user quits playback. The if the play pause button was adjusted to pause (as it does) but when play was used it would (if the above setting was not 0) the stream a new as if you were using resume.

 

Im happy losing or greatly reducing my ability pause (or working around with a pause close and resume).

 

thanks for you help, I know its not exactly kosher but ive a workable solution now, im going to test to see if im right and its only the above two features im loosing, but so far so good, if anyone is using a similar setup I can tell you the difference between using a native drive for transcode and a ram drive is like night and day, in cases where the local disk has poor bandwidth or is already saturated (my case the drive does a lot more than host the VM).

 

But for me the benefits outweigh the negatives, it allows me to screw the transcode folder down to something much much smaller and keep it in ram and away from the spinning rust.

 

If anyone's interested im happy to share the script.

 

With the addition of disabling client side spinning rust cache, I can keep the transcodes in ram all the way from the server to the screen.

 

Im glad it worked and I can continue to use emby again.

 

As a side note ive a few things im working on (along side emby) for windows only platforms to value add things (like the ram drive transcode tweak) ive been slowly collecting and building the a cache of scripts and programs to do things that are not currently possible that I ran into. I wont spoil the surprise and im plan to make a demo video for Luke (admins) to view first just to demo / explain a few of the things to make sure they are happy for me to share and if so let me know where and how.

 

Im sure a few things may ultimate be built into emby, I know on one thing its already been discussed, but if its not or while were waiting its a solution it fills a few gaps where there is not currently a solution.

 

Like I said I don't want to share just yet as I don't want to step on anyone's toes

Edited July 11, 2015 by plazma