Jump to content

Massive transcoding temp files


andersoc27

Recommended Posts

kevinf63

Also experiencing this particularly on Samsung Smart TVs were someone would simply put the TV into standby mode/turn it off rather than backing out of the TV stream or exiting the app.

 

Might be worth looking at how timeouts are implemented and model them after SAT>IP design: if the client is not responding, the unicast stream is stopped and the frontend is released.

This "RTSP session timeout" value is set to 60 seconds by default on my most SAT>IP certified devices.

By specification, each SAT>IP Client (in this case... Emby Server is technically considered the client even if it is a middle-man/distributing) is required to send an "alive" signal every 30 seconds in order to keep the connection open, if it fails to meet those demands, the connection shuts shop.

When the connection is shut, the transcoding temp should then be cleared as usual.

 

Expanding on this, it appears despite my Emby server performing "direct-streams" of my HDHomeRun Quattro and TeleStar Digibit R1 SAT>IP, I'm still building up large transcoding files.

Edited by kevinf63
Link to post
Share on other sites
  • 2 weeks later...

Using Emby Server for Windows 4.1.1.0.

 

Transcoding temp directory filling up, *.ts files still in there for a weeks (!).

Is it expected behavior ?

Link to post
Share on other sites
andersoc27

Using Emby Server for Windows 4.1.1.0.

 

Transcoding temp directory filling up, *.ts files still in there for a weeks (!).

Is it expected behavior ?

Until the guys put a more sustainable fix in place - restarting server is the best thing to do I have found. deleting the temp file obviously works but a fresh server restart also cleans temp files.

Link to post
Share on other sites

Until the guys put a more sustainable fix in place - restarting server is the best thing to do I have found. deleting the temp file obviously works but a fresh server restart also cleans temp files.

 

Thanx for comments to all.

No problem, will schedule something like this to clean up:

 

SET log_file="logs/helper_cleanup_%RANDOM%.log"
...
echo "======= Clean-up log files =======" >> %log_file%
forfiles /p logs /m *.log /d -7 /c "cmd /c del /S @PATH" >> %log_file%
 
echo "======= Clean-up EMBY transcoding files =======" >> %log_file%
forfiles /p H:\TEMP\Emby-Server\transcoding-temp\ /m *.ts /d -1 /c "cmd /c del /S @PATH" >> %log_file%
 
Link to post
Share on other sites

Hi there @@Masterz, we are looking into this, thanks.

 

I've just been remotely watching a film, which is just over 2 hours long and ~12GB for a 1080p video file.  It was being transcoded and played at 8Mbps through Chrome on my laptop.

 

Had about 30 minutes left of the film, and it just went blank.  Turns out that Emby had been storing loads of ~3MB / 2 second long files since I started playing the film, and it stopped because there was only 88KB left on the drive.  Keeping the last 5-10 minutes may be understandable if the aim was to cater for people potentially rewinding a little, but other than that, there is just no logical reason for it

 

 

It has been about 15 minutes now since I closed all sessions and the files are still there, so I have no choice but to restart the Emby server.  Also, just to be clear, this was not a Live TV stream... just a .MKV file sitting on my hard drive.

 

5cf11a837740d_Annotation20190531130712.p

 

Link to post
Share on other sites

I've just been remotely watching a film, which is just over 2 hours long and ~12GB for a 1080p video file.  It was being transcoded and played at 8Mbps through Chrome on my laptop.

 

Had about 30 minutes left of the film, and it just went blank.  Turns out that Emby had been storing loads of ~3MB / 2 second long files since I started playing the film, and it stopped because there was only 88KB left on the drive.  Keeping the last 5-10 minutes may be understandable if the aim was to cater for people potentially rewinding a little, but other than that, there is just no logical reason for it

 

 

It has been about 15 minutes now since I closed all sessions and the files are still there, so I have no choice but to restart the Emby server.  Also, just to be clear, this was not a Live TV stream... just a .MKV file sitting on my hard drive.

 

5cf11a837740d_Annotation20190531130712.p

 

We plan to make this more efficient in the future but I think you will still need to provide a transcoding temp location with more than just a few GB of free space for an Emby system to operate well.

 

Thanks.

Link to post
Share on other sites

We plan to make this more efficient in the future but I think you will still need to provide a transcoding temp location with more than just a few GB of free space for an Emby system to operate well.

 

Thanks.

 

Haha!  I had been wondering who would make that point  :P I have been playing around with RAM Drives and so I currently have 20GB allocated for the cache, database and transcoding folders for both Emby and Plex.  Whilst I could easily switch back to normal storage for Emby, my hand should not be forced when there isn't actually anything wrong with my design/implementation.

 

There may be some things which you can make "more efficient" but I have also proven in this thread that there are aspects of this which are just broken (namely how pressing "stop" on the Dashboard doesn't actually stop anything).  I got the impression it was just localised to Live TV, but this now seems to be across the board :(

Link to post
Share on other sites

I do about a hundred gigs a day in my temp folder doesn't bother me:)

 

I wouldn't be bothered either if I wasn't liking how fast RAM disk made my Plex and Emby :P

Link to post
Share on other sites

Lol get yourself a m.2 512gb Nvme. That's my dedicated transcode Drive. But I do understand what you're saying

 

I've got a m.2 256GB SSD that has only Windows 10 Pro, Emby, and MCEBuddy installed and nothing else... so, mostly free space.  More than once, I've been unable to login to the server from a remote client because the SSD was completely filled up due to a Live TV transcoding session that didn't terminate when it should have.  Having a 512GB disk means only that the problem would have taken approximately twice as long to manifest.  And m.2, SSD, NVME are also all irrrelevant to the problem being discussed — speed has nothing to do with it.

Link to post
Share on other sites
Jason'sEmby

Talking with ebr right now on another post about the issue of live TV not terminating. Hopefully they will have a solution soon:)

Link to post
Share on other sites
  • 3 weeks later...
spoonchild

Not at this time but managing this is on our list of things we'd like to do.

This would be an awesome feature. I picked up emby for it's live TV. My problem is I run a 32GB ramdisk for transcodes. Normally this is enough, but when direct playing, after about 3-4 hours I have to stop/play. Would be nice to have it split the transcode into 5 minute, or a configurable size file so if I want tons of small files for the transcode, I could have less time lost off the total. something like LiveTV history 60 minutes, size of history files 5 minutes default, configurable down to 1m. With 6 tuners, I'm looking at moving off the ramdisk just for emby. The other option would be give a "Live TV" streaming directory so I can keep my movies and everything in that faster smaller drive, and then have LIve TV on a slower spinning disk.

Link to post
Share on other sites

I know this topic has many other posts, but they appear to be getting fairly old.  So to keep up with current situation, decided to make a new post.  If you feel it necessary, please relocate as needed.

 

Edit - Thanks for relocating.  Just for clarity purposes; this is happening when attempting to play a media file on the server, and not from a LIVE TV stream. 

 

Hardware – Windows 10 Home

500Gb (320 GB Free) SSD for transcoding

Emby Server – 4.1.1.0

Emby Internet streaming bitrate limit – 3.0 (Mbps)

Internet Speed 300/30 cable

 

Remote Client Hardware – Samsung S7

Android Version  8.0.0

Emby Android Mobile 3.0.48

Utilizing Cellular Network for remote access.

Quality set to 480p – 720kbps

 

Test Media File info

See attached S01E01 and S01E02 jpg’s.

 

 

I have read many of the posts online about the transcoding-temp files running away and/or filling entire hard drives rendering the server useless.  I have had this problem for quite some time and have worked around the problem by doing a daily restart of the server.  Some days the problem is only a few Gb’s of data being left over.  Other times, the director is filled to the rim and on the verge of creating problems.

 

Due to an upcoming trip that will take me away from my daily restarts, I decided to see if I can determine the problem.  Is it an incorrect method of exiting an app?  Is it caused by not stopping the media correctly?    Can it be duplicated on demand?

 

So here is what I did today:

  1. Shut down the server to clear all files (transcode-temp files, logs, activity).
  2. Restart server and allow to finish startup process.
  3. Disconnect cellphone from the WiFi network and establish on Cellular network.
  4. Start Emby app on cellphone and navigate to desired show (S01E01).
  5. Start playing the show.  Once the show started, I paused the playback.  Watched the transcoding on the dashboard and waited until completed.  Once completed, jumped to the final 2 minutes of the show.  Allowed it to play to completion.
  6. Allowed Auto-play next episode to begin playing S01E02.
  7. Once the playback of E02 started allowed it to play for several seconds before exiting (backing out).
  8. Used the BACK button to exit all the way out to the cellphone home screen.
  9. Allowed server activity to stop and then shut-down the server.
  10. Checked the transcoding-temp file.  Found all remaining transcode files (approx.. 110 MB).

 

I have been able to reproduce the same results 3 times.  I hope this helps in some way.  If you need more information, let me know.

embyserver-63696617615.txt

ffmpeg-transcode-9f9d6206-1ee9-4689-85cd-3bdb6c035c80_1.txt

ffmpeg-transcode-ad8d35e7-7b41-47ed-9502-596cfe94ef1c_1.txt

ffmpeg-transcode-d0ccebc0-2505-4454-ad0f-5f4631611f0a_1.txt

hardware_detection-63696616911.txt

post-80485-0-12355300-1561049471_thumb.jpg

post-80485-0-82940900-1561049490_thumb.jpg

post-80485-0-88441500-1561049505_thumb.jpg

post-80485-0-41707200-1561049518_thumb.jpg

post-80485-0-16734800-1561049535_thumb.jpg

post-80485-0-84665800-1561049548_thumb.jpg

Edited by Scott D
Link to post
Share on other sites

The issue of the server not always closing live streams after stopping will be resolved in Emby Server 4.2. If you need immediate relief the fixes will be on the beta channel by tomorrow. Thanks.

  • Like 2
Link to post
Share on other sites

The issue of the server not always closing live streams after stopping will be resolved in Emby Server 4.2. If you need immediate relief the fixes will be on the beta channel by tomorrow. Thanks.

 

I'm currently running v4.2.0.21, so if I ran the test I detailed here, everything should be fine?

Link to post
Share on other sites
  • 3 months later...

Running 4.2.1.0 and a couple of days ago I found my 512 SSD almost full. Checked and found the problem was almost 400 GB in the temp trans-coding file. the hang up is from watching recorded tv shows not live streaming. These shows are usually deleted every day after they are watched.  Today I checked and once again from watching this mornings recorded local news shows it had around 14 GB in transcode file restarted emby server and they were then cleared. This just started recently. Does this server now have to be restarted to clear this folder? Seems like this happened a few years ago also for a while

Link to post
Share on other sites

Running 4.2.1.0 and a couple of days ago I found my 512 SSD almost full. Checked and found the problem was almost 400 GB in the temp trans-coding file. the hang up is from watching recorded tv shows not live streaming. These shows are usually deleted every day after they are watched.  Today I checked and once again from watching this mornings recorded local news shows it had around 14 GB in transcode file restarted emby server and they were then cleared. This just started recently. Does this server now have to be restarted to clear this folder? Seems like this happened a few years ago also for a while

 

Hi, what Emby app were you playing with?

Link to post
Share on other sites
  • 8 months later...
jacotec

Sorry for capturing that old thread, but today my monitoring sent me a warning that my Emby VM is running low on disk space. Wondering why (I have all media files on a mounted external source) I've checked the /dev/sda1 contents and found a 195GB(!!!) .ts temporary transcoding file in /tmp.

Needed to wipe the file and reboot the server.

I'm running the current 4.4.3.0 build on Ubuntu 16.04 and I'm wondering how this could happen ...

Link to post
Share on other sites
Dan_Austin

#1 cause is a Roku that someone hit home without exiting the livetv channel first.

I've also see it happen with an Android TV app crash, but that one was limited to
a specific beta and wasn't easy to trigger.

You can check the dashboard while it is happening to see if it might be a client
that has just been left on, but if it is one of the above, there will likely be no
evidence in the dashboard.

Link to post
Share on other sites
cayars

Look through the activity section on the Dashboard to see who started playing back Live TV might help.

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...