Jump to content

Live TV Pause Buffer fills HD and crashes server


dr_faulkner

Recommended Posts

Mbedford
On 07/11/2020 at 02:49, cayars said:

Hi, Curious, what directory is filling up on you?
Is it an Emby or WinTV folder?
Have you tried setting up a batch/script to delete older files automatically to help with this problem?

Hi, 

There are 2 issues here that needs addressing, my circumstances wouldn't be any different to others.

Live TV buffer filling up hard disk

The live TV buffer directory D:\WinTV 10 capture\Pause Buffer as set by Hauppauge WInTV (see screenshot below how its configured) which Emby uses in the background. This fills up the harddisk, then all Live TV in Emby stops working. Sometimes people when they are watching live TV and have finished, press 'Stop' sometimes they just turn of the TV screen so it keeps playing in the background. If you are able simply introduce the 'Are you still watching?' feature for live TV, this could go some way to helping.

wintvbuffer.PNG.bda847cde97da7949bc5c7a8eab41b05.PNG

Recorded TV filling up hard disk

The recorded TV directory D:\Recorded TV keeps getting larger as more shows are recorded over time. With Windows Mediacenter it used to automatically remove the OLDEST recording to free up space as required. A similar option could be done for Emby.

Looking forwards to the dev's to find solutions here.

Have you tried setting up a batch/script to delete older files automatically to help with this problem? < No as this is a end user hack, not fixing the root issues.

Edited by Mbedford
Link to comment
Share on other sites

The first issue isn't with Emby but with Win TV.  You likely will need to setup a script to delete this win tv buffer content older than 12 hours or something like that and have it run automatically twice a day.

The second issue isn't a problem at all.  Emby DOES NOT EVER automatically delete content on it's own.  You can delete content after you watch it or archive it to other drives.  You can setup series or shows to only keep X amount of episodes as well.  This one is completely on the admin for not keeping up either by adding more storage, archiving or deleting content.  Emby could add an option like this (off by default) but if you want something like this I'd suggest making a Feature request for it here (see if it exists already first) https://emby.media/community/index.php?/forum/98-feature-requests/

Again if you wanted older content removed, you can script this outside of Emby as well.

Link to comment
Share on other sites

FordGT90Concept

My two cents:
1) Yes, it's still an issue, even with scripts clearing the folders daily.
2) If there's an open stream, Emby Server ignores it's daily restart.  Restarting Emby Server tends to clean up both folders because Emby Server automatically purges all the temp files it used.  Perpetual open stream -> ignores daily restart-> 30+ hours of recording -> fills drive -> stuff starts crashing -> restart Emby Server -> fixed until it happens agian.

I can never trace the open streams back to a specific client because by the time I notice it happens, it's full panic mode.  More stuff runs on the server than just Emby.

Edited by FordGT90Concept
Link to comment
Share on other sites

Happy2Play
9 minutes ago, FordGT90Concept said:

My two cents:
1) Yes, it's still an issue, even with scripts clearing the folders daily.
2) If there's an open stream, Emby Server ignores it's daily restart.  Restarting Emby Server tends to clean up both folders because Emby Server automatically purges all the temp files it used.  Perpetual open stream -> ignores daily restart-> 30+ hours of recording -> fills drive -> stuff starts crashing -> restart Emby Server -> fixed until it happens agian.

I can never trace the open streams back to a specific client because by the time I notice it happens, it's full panic mode.  More stuff runs on the server than just Emby.

OT:  But you could use Disk Space plugin and setup notifications as a specific amount of free space.  But a 30+ hour stream, I would ensure "Are you Still Watching" is enabled on all clients.

  • Like 1
Link to comment
Share on other sites

Just an FYI, in the new version of Live TV you likely won't need Win TV anymore which will completely solve you first issue.

As Happy2Play mentioned, you should figure out what clients are streaming constantly like this and enable "are you still watching" to stop this madness. :)

Link to comment
Share on other sites

Mbedford
On 09/11/2020 at 12:05, cayars said:

Just an FYI, in the new version of Live TV you likely won't need Win TV anymore which will completely solve you first issue.

New version of Live TV since when? tell me more? do I need to reconfigure something to use this?

 

On 09/11/2020 at 12:05, cayars said:

As Happy2Play mentioned, you should figure out what clients are streaming constantly like this and enable "are you still watching" to stop this madness. :)

I have this option 'Enable 'Are You Still Watching?' prompt' enabled already.

On 09/11/2020 at 09:26, cayars said:

The first issue isn't with Emby but with Win TV.  You likely will need to setup a script to delete this win tv buffer content older than 12 hours or something like that and have it run automatically twice a day.

Hopefully cayars comment above about a new version of Live TV this wont be an issue anymore, so will focus on my Recorded TV folder. I've just done this, everything older than 1 month gets deleted with this cmd run by a scheduled task 9pm every night:

forfiles -p "E:\Recorded TV" -s -m *.* /D -30 /C "cmd /c del @path"

Link to comment
Share on other sites

You should be able to change that to 1 day or even every 12 hours.

The new version of Live TV is still currently in development but I just  wanted to let you know about it.

Link to comment
Share on other sites

Happy2Play
12 minutes ago, Mbedford said:

I have this option 'Enable 'Are You Still Watching?' prompt' enabled already.

So you are saying you know what client is causing the runaway transcoding and this option is enable on that client?  This is not a global server setting.

Link to comment
Share on other sites

Mbedford
34 minutes ago, cayars said:

Which client is it Mbedford?

Just to be clear here, we are talking here about the 'Enable 'Are You Still Watching?' prompt to stop the WinTV10 Live TV buffer folder getting HUGE and filling up the harddisk.

The client is Emby theatre application running on the same PC and Emby server is running.

I have just checked and the  'Enable 'Are You Still Watching?' prompt is enabled on the both the Emby server settings page and also on the client, which is Emby theatre application.

Link to comment
Share on other sites

There is no Are you still watching prompt on the server.  This is a client only settings that MUST be turned on for each and every problem client.

Don't confuse your personal web settings in the menu as server settings.  Those are just your settings for web use.

Link to comment
Share on other sites

FordGT90Concept

It just happened again to me.  WinTV pause buffer was full, 0 bytes free on

C:

 and no one could log in to Emby Server because of it.   After I cleared the pause buffer, I tried restarting Emby Server and it still wouldn't permit log in so I restarted the entire machine.  Here's all the logs on the server.  I have no idea what device/time/program caused WinTV to run forever.

embylogs.zip

Link to comment
Share on other sites

You should be able to see what clients are playing what at any time on the Web Dashboard.

It should not be hard to spot the offending client this way without ever looking at a log file. Just casual observation of what device is playing what.

Link to comment
Share on other sites

dr_faulkner
22 minutes ago, cayars said:

You should be able to see what clients are playing what at any time on the Web Dashboard.

It should not be hard to spot the offending client this way without ever looking at a log file. Just casual observation of what device is playing what.

Hi Cayars,

I updated Emby Server to the latest version as you suggested but this issue is not fixed - As I write this Emby has written 68 Gb and is still writing to a single file in transcoding-temp (file appears to have a GUID for a name)

I am using a HD Home Run.

There are no clients currently active - if I manage server in browser it tells me the last active client (Emby Theatre running on a PC) started at 7:30pm yesterday and closed 10:30pm (which is correct).

Interestingly, the only thing I watched yesterday was a recording of the news that went for 30 minutes and then stopped. The transcoding-temp file date time is when I started watching this 30 minute recording. Why it is still writing to this temp file 12 hours later is beyond comprehension.

So Emby Server must know  nothing is playing and no clients are active but keeps writing pointlessly to this temp file regardless.

I can't reproduce this with the Web Client btw. It appears to be an Emby Theatre-related issue.

Link to comment
Share on other sites

If you like I can do a remote session with you via TeamViewer and we can try a few things to see if we can stop this, help fix it and/or try to eliminate the issue.

First thing is likely to see if we can force it to happen.  If you got some free time today it would be a perfect time as I can spend some time with you on this while I'm watching football. :)

Send me a PM if you'd like to do this.

Carlo

Link to comment
Share on other sites

Got your PM and responded with instructions.  Just wanted to ask a question.

Got a question about Theater and Live TV.  When done playing back Live TV are you backing out or just closing the app?  If you are just closing the app try backing out gracefully first.

No idea if that's the problem but just thought I'd mention it.

Carlo

Link to comment
Share on other sites

dr_faulkner
4 minutes ago, cayars said:

If you like I can do a remote session with you via TeamViewer and we can try a few things to see if we can stop this, help fix it and/or try to eliminate the issue.

First thing is likely to see if we can force it to happen.  If you got some free time today it would be a perfect time as I can spend some time with you on this while I'm watching football. :)

Send me a PM if you'd like to do this.

Carlo

Yes am keen to get this fixed - but I can't run emby theatre on my desktop pc (remember you reinstalled it, it ran once successfully, but then hangs when you try to run again)

Fixing this will make it easier to fix the other issue

Link to comment
Share on other sites

Ahh, ok, we can take a look at that and see what's up with Theater not running on your desktop as well.  It might take a couple support sessions to get you fixed up since it's getting late for you but we're here to help you and will do what we can.

Carlo

Link to comment
Share on other sites

dr_faulkner
6 minutes ago, cayars said:

Ahh, ok, we can take a look at that and see what's up with Theater not running on your desktop as well.  It might take a couple support sessions to get you fixed up since it's getting late for you but we're here to help you and will do what we can.

Carlo

its not getting late for me - 8am here. If I can get ET working on my laptop I can get a reproducible case going now that I know even playing a recording can cause the issue. Am free in 45 mins if thats ok?

Edited by dr_faulkner
Link to comment
Share on other sites

Sorry had you confused with another person I was working with and it was 11pm for him.  Send me TeamViewer credentials like before and I'll remote in whenever you want.

Link to comment
Share on other sites

FordGT90Concept
8 hours ago, cayars said:

You should be able to see what clients are playing what at any time on the Web Dashboard.

It should not be hard to spot the offending client this way without ever looking at a log file. Just casual observation of what device is playing what.

Did you miss the part where I said 

Quote

no one could log in to Emby Server because of it

Emby Server needs to put a stop to it long before it reaches 0 bytes free.  All bets are off when there's literally no room to page file.

 

I have no idea what client did it nor which machine.  I don't even know what day it happened on.  I just know that it's a threat to my server's stability and I do have a process that runs every night to purge the files.

Edited by FordGT90Concept
Link to comment
Share on other sites

6 minutes ago, FordGT90Concept said:

Did you miss the part where I said 

Emby Server needs to put a stop to it long before it reaches 0 bytes free.  All bets are off when there's literally no room to page file.

 

I have no idea what client did it nor which machine.  I don't even know what day it happened on.  I just know that it's a threat to my server's stability and I do have a process that runs every night to purge the files.

No I didn't miss it.  What I'm saying is to be proactive and take note of what's playing to figure out what client is the culprit BEFORE it uses up all the disc space.

It would be a hundred times better if Emby made the "Are you still watching" prompt server controlled vs client controlled! @Luke

Link to comment
Share on other sites

  • 1 year later...
FordGT90Concept

This is still a problem.  As the exception showed, Emby crashed because C drive was full.  Because C was back to normal by the time I looked at it (12 hours later), presumably it was the WinTV Buffer that filled it.  When Emby crashed, WinTV detected it had no one to send data to, so it purged all the files empying C again.

1035.zip

Edited by FordGT90Concept
Link to comment
Share on other sites

Software needs proper space to do it's work and of course if you pause something that is ongoing the back queue will get larger and larger.

The easy "fix" is to move the pause in wintv to a disk with much more space available.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...