Transcoding to a Ram Drive

[Malock 20]

Ok... @@Luke is going to be able to decide if it`a bug related thing or new feature request, or I`m pushing it too far...

 

I will try to keep this as short as possible.

 

Since I`ve been using Media servers that do transcoding on the fly, I found that "it" was the most Straining process on the system.

 

After burning a couple HDD;s and SSD`s from the delirious transcoding life they had and from precipitated Hard disks wear, bottleneck because of intensive multiple transcoding process, Heavy I/O tasks etc.

 

I thought that the best way to circumvent the problems was to transcode over a Ram Disk. Because RAM is able of years of reads and writes without any physical wear. And as a big plus the results where an immediate boost in all aspect of server operations. I/O operations in particular.

 

Now, the problem or the feature request.

In many Software that I have tried I could Use a batch file like this.

Echo off
:Start
Timeout 60
call c:\windows\ClearEncode.cmd
goto start

To run every minute another batch like this. that verify if ffmpeg.exe is running.

Echo off
tasklist /FI "IMAGENAME eq ffmpeg.exe" 2>NUL | find /I /N "ffmpeg.exe">NUL 
if %ERRORLEVEL%==1 goto :EOF
E:
cd\transcoding-temp
setlocal
call :DateToMinutes %date:~-4% %date:~-10,2% %date:~-7,2% %time:~0,2% %time:~3,2% NowMins
for /f "delims=" %%a in ('dir * /a-d /b') do call :CheckMins "%%a" "%%~ta"
goto :EOF
:CheckMins
set File=%1
set TimeStamp=%2
call :DateToMinutes %timestamp:~7,4% %timestamp:~1,2% %timestamp:~4,2% %timestamp:~12,2% %timestamp:~15,2%%timestamp:~18,1% FileMins
set /a MinsOld=%NowMins%-%FileMins%
if %MinsOld% gtr 10 del %file%
goto :EOF
:DateToMinutes
setlocal
set yy=%1&set mm=%2&set dd=%3&set hh=%4&set nn=%5
if 1%yy% LSS 200 if 1%yy% LSS 170 (set yy=20%yy%) else (set yy=19%yy%)
set /a dd=100%dd%%%100,mm=100%mm%%%100
set /a z=14-mm,z/=12,y=yy+4800-z,m=mm+12*z-3,j=153*m+2
set /a j=j/5+dd+y*365+y/4-y/100+y/400-2472633
if 1%hh% LSS 20 set hh=0%hh%
if /i {%nn:~2,1%} EQU {p} if "%hh%" NEQ "12" set hh=1%hh%&set/a hh-=88
if /i {%nn:~2,1%} EQU {a} if "%hh%" EQU "12" set hh=00
if /i {%nn:~2,1%} GEQ {a} set nn=%nn:~0,2%
set /a hh=100%hh%%%100,nn=100%nn%%%100,j=j*1440+hh*60+nn
Endlocal&set %6=%j%&goto :EOF

And if it is running, Deletes all files older than 10 minutes in the transcoding-temp directory. Every minutes, if it is not running, it waits for another minute and start over.

 

Making a server with 16 gig of ram able to transcode 10 times more movies simultaneously, using the same amount of Ram disk space.

 

At this point everything works like a charm, with CPU Throttle and the ram drive, the machine barely works 10% of what it was before :)

 

The problem encountered comes from the fact that, if a client scroll back more than ten minutes in the movie, something weird happens.

 

From my quick observations.

 

On the server side.ffmpeg seem to loose it`s throttling abilities but picks up the transcoding to the place intended and continues at warp 10 speed. Maxing CPU resources.

 

On the Client side the player is stuck on an image and cannot go back to main, Closing the web page fixes everything, server and client side. You can then go back and continue with no harm.

 

What i think is occurring. Is simply that Emby tries to go back to an already encoded file that does not exist anymore instead of restarting Encoding from that point, like other Software do..

 

The Question. Would It be hard to implement that in Emby ? Can I modify this myself ?

 

Or even more audacious... Could a RAM disk friendly version be made ???

Edited December 11, 2018 by Malock

[ebr 14913]

This isn't really about a RAMDisk per se - but, rather, just operating with limited transcode temp space.

 

We do have it on our radar to be able to set some sort of limit that the server would self-manage.  However, this has not even entered preliminary design stage yet so I cannot give you an idea of when it might be a reality.

 

Thanks.

[nyplayer 90]

@@Malock

 

That is a good Idea... I think I will try it with ImDisk.

[Malock 20]

 

This isn't really about a RAMDisk per se - but, rather, just operating with limited transcode temp space.

 

We do have it on our radar to be able to set some sort of limit that the server would self-manage.  However, this has not even entered preliminary design stage yet so I cannot give you an idea of when it might be a reality.

 

Thanks.

 

Exact... But my concern where primordially regarding this aspect of the problem... no ? If not why would I delete files a soon as they are streamed ??? If not to recover their space asap...

 

Imo the simplest solution to this. Is to delete files as soon as delivered and restart encoding from requested point.

 

Honestly "insert name here" is doing it like that. I don't want to sound harsh, far from my thoughts... Seriously Emby`s the one...

Edited December 8, 2018 by Malock

[Malock 20]

@@Malock

 

That is a good Idea... I think I will try it with ImDisk.

 

Go with Softperfect Ram disk, It enables you to create large drives.

[nyplayer 90]

Go with Softperfect Ram disk, It enables you to create large drives.

 

Thanks using Softperfect Ram Disk along with your code.

Edited December 8, 2018 by nyplayer

[Malock 20]

Thanks using Softperfect Ram Disk along with your code.

 

The line that contains

 

if %MinsOld% gtr 10 del %file%

 

Where the "10" is... the age in minutes

Edited December 8, 2018 by Malock

[Malock 20]

I modified the original batch to make it start deleting files only if there is less than 500 meg left of space on the drive. Also it will delete files older than 45 minutes only.

@[member="Echo"] off
tasklist /FI "IMAGENAME eq ffmpeg.exe" 2>NUL | find /I /N "ffmpeg.exe">NUL 
if %ERRORLEVEL%==1 goto :EOF
for /f "usebackq delims== tokens=2" %%x in (`wmic logicaldisk where "DeviceID='E:'" get FreeSpace /format:value`) do set FreeSpace=%%x
If %FreeSpace% gtr 500000000 goto :EOF
E:
cd\transcoding-temp
setlocal
call :DateToMinutes %date:~-4% %date:~-10,2% %date:~-7,2% %time:~0,2% %time:~3,2% NowMins
for /f "delims=" %%a in ('dir * /a-d /b') do call :CheckMins "%%a" "%%~ta"
goto :EOF
:CheckMins
set File=%1
set TimeStamp=%2
call :DateToMinutes %timestamp:~7,4% %timestamp:~1,2% %timestamp:~4,2% %timestamp:~12,2% %timestamp:~15,2%%timestamp:~18,1% FileMins
set /a MinsOld=%NowMins%-%FileMins%
if %MinsOld% gtr 45 del %file%
goto :EOF
:DateToMinutes
setlocal
set yy=%1&set mm=%2&set dd=%3&set hh=%4&set nn=%5
if 1%yy% LSS 200 if 1%yy% LSS 170 (set yy=20%yy%) else (set yy=19%yy%)
set /a dd=100%dd%%%100,mm=100%mm%%%100
set /a z=14-mm,z/=12,y=yy+4800-z,m=mm+12*z-3,j=153*m+2
set /a j=j/5+dd+y*365+y/4-y/100+y/400-2472633
if 1%hh% LSS 20 set hh=0%hh%
if /i {%nn:~2,1%} EQU {p} if "%hh%" NEQ "12" set hh=1%hh%&set/a hh-=88
if /i {%nn:~2,1%} EQU {a} if "%hh%" EQU "12" set hh=00
if /i {%nn:~2,1%} GEQ {a} set nn=%nn:~0,2%
set /a hh=100%hh%%%100,nn=100%nn%%%100,j=j*1440+hh*60+nn
Endlocal&set %6=%j%&goto :EOF

[nyplayer 90]

 

I modified the original batch to make it start deleting files only if there is less than 500 meg left of space on the drive. Also it will delete files older than 45 minutes only.

@[member="Echo"] off
tasklist /FI "IMAGENAME eq ffmpeg.exe" 2>NUL | find /I /N "ffmpeg.exe">NUL 
if %ERRORLEVEL%==1 goto :EOF
for /f "usebackq delims== tokens=2" %%x in (`wmic logicaldisk where "DeviceID='E:'" get FreeSpace /format:value`) do set FreeSpace=%%x
If %FreeSpace% gtr 500000000 goto :EOF
E:
cd\transcoding-temp
setlocal
call :DateToMinutes %date:~-4% %date:~-10,2% %date:~-7,2% %time:~0,2% %time:~3,2% NowMins
for /f "delims=" %%a in ('dir * /a-d /b') do call :CheckMins "%%a" "%%~ta"
goto :EOF
:CheckMins
set File=%1
set TimeStamp=%2
call :DateToMinutes %timestamp:~7,4% %timestamp:~1,2% %timestamp:~4,2% %timestamp:~12,2% %timestamp:~15,2%%timestamp:~18,1% FileMins
set /a MinsOld=%NowMins%-%FileMins%
if %MinsOld% gtr 45 del %file%
goto :EOF
:DateToMinutes
setlocal
set yy=%1&set mm=%2&set dd=%3&set hh=%4&set nn=%5
if 1%yy% LSS 200 if 1%yy% LSS 170 (set yy=20%yy%) else (set yy=19%yy%)
set /a dd=100%dd%%%100,mm=100%mm%%%100
set /a z=14-mm,z/=12,y=yy+4800-z,m=mm+12*z-3,j=153*m+2
set /a j=j/5+dd+y*365+y/4-y/100+y/400-2472633
if 1%hh% LSS 20 set hh=0%hh%
if /i {%nn:~2,1%} EQU {p} if "%hh%" NEQ "12" set hh=1%hh%&set/a hh-=88
if /i {%nn:~2,1%} EQU {a} if "%hh%" EQU "12" set hh=00
if /i {%nn:~2,1%} GEQ {a} set nn=%nn:~0,2%
set /a hh=100%hh%%%100,nn=100%nn%%%100,j=j*1440+hh*60+nn
Endlocal&set %6=%j%&goto :EOF

 

Thanks going to give it a try.