Transcoding temp file overload and log files

[Scott D 62]

After returning from a weeklong vacation, I was stunned to find a 500G SSD Drive filed with temp files almost to max capacity.  In the past I have been able to avoid this situation by restarting Emby every morning.  Every morning the temp files only occupy 4-15 gig, but left unattended it tends to run away quickly.

I believe I have stumbled on a possible problem, but have not done much testing.  What I have found is, when a client pauses a video that has been transcoded, the temp files do not get cleared/deleted when the video is stopped/ended.  If played from start to finish without a pause, temp files go away.

While I would love to assist in resolving the ongoing problem with the ever expanding transcoding temp file folder, I have a problem in trying to provide information for testing.

Today, I had the transcode file problem once again.  I pulled the server and ffmpeg log files.  I started to look through the files to see if there was something obvious, and once again I came upon the problem with log files; they are HUGE!

Is it really necessary to have logging of:  “Info HttpServer: HTTP GET…” and “Info HttpServer: HTTP Response…”  every few seconds?   This particular info line contains not only the server IP address, but also the client IP address.

I have set “Rotate Log File” to every 6 hours, but the log file continues to exceed 50 meg per file.

Suggestions:

1.  Redact or remove any IP address information from log files. 

2.  Cut back on the amount of log information being recorded. 

3. Provide checkbox options for the level of logging.

4. Provide an option to turn logging off completely. 

5. Split "playback" logging to a log file separate from the server log file.

 

 

[Luke 37064]

Hi, what Emby app were you playing from? How did you stop playback?

[Luke 37064]

@Scott D ?

[Scott D 62]

Playback was on a Roku.  I have several versions both local and at a summer cabin (remote).  The problem occurs on all devices.

I find several gig of temp files on a daily basis.  I have worked around by simply restarting the server.  However, when I am away, the problem can become a problem.

As stated above, the playback was stopped in a few ways:

1.  Allowed to play to completion.

2. Using the <- on the top row of the roku remote.

3. Paused using the roku remote and then resumed at a later time.  Did not exit the playback, simply paused for a period of time.

 

Edited July 9, 2020 by Scott D

[Carlo 4330]

On Roku due to it's OS and less than flexible way things have to be done, you need to be careful to back up out of Live TV.

One of the BEST THINGS you can do on Roku is Enable 'Are You Still Watching?' prompt.

That will prompt the person watching every few shows to answer a prompt to make sure they are still there and watching.  Otherwise it will disconnect the stream/use and save the server from using the tuner and disc space.

BTW, we are looking to improve this but Roku doesn't make it easy.

[Scott D 62]

This happening not only with Live TV, but with media files on the server.  I use the Live TV option on rare occasions.  When I do, It will fill the drive quite quickly.   But once the server is restarted, huge amounts (100's of gig) of space are freed up.

Majority of the time, the temp files are created from files on the server.  This happens on a daily basis.

 

Edit - Seems in the transcoding-temp folder Live TV is creating a few files.   

First is a named as such 4c194ea4327646cea8ba1a803aa50e32.ts  This file gets HUGE.  34 Gig as I look at it now and growing.

And the others are named as such F85BF5_*.ts  There are over 10,000 of these files.  Most around 500K but with 10,000 of them, it fills quickly.

All of this for a single Live TV stream.

Edited July 9, 2020 by Scott D

[Carlo 4330]

You should look at your logs and/or activity to see what is causing this.  Find the person and device and see what they are doing.

Typically when I personally see this it's a Live TV session from a Roku and the person is exiting Live TV wrong and my server never knows the person has stopped watching.  That or they put TV on at night and fall asleep.

Once you know where it's coming from we can help you with suggestions.

[speechles 1917]

3. Paused using the roku remote and then resumed at a later time.  Did not exit the playback, simply paused for a period of time.

How long was it paused? If you stay paused for a significant time and have throttle disabled then its going to stay at real time transcoding to your hard drive. Then when you unpause the real time is however far ahead. It stays that far ahead. Is this what is causing the problem? If you stay paused for 3 hours the transcoding goes on for those 3 hours with no throttle. Then when you come back to unpause and watch it already has 3+ hours transcoded and keeps going. Pause is a two-way sword. Cuts both ways.

If you stay paused on a RokuTV after a certain period(22-30 minutes depends on resolution of the stream) it will unpause. The RokuSTB/Stick do not do automatic unpause.

Edited July 10, 2020 by speechles