ServerWMC Not Deleting Streams No Longer in Use

[theniteow1 25]

Hello,

 

I've been noticing an issue lately that I haven't been able to resolve.  For some reason ServerWMC is not deleting .TS files after the tuner is no longer watching them.  I am unable to delete the .TS streams because ServerWMC has them locked, so they are being used in some fashion.  However, it is clear they are not being actively watched.  Eventually it will eat up all the hard drive.  I've run into this scenario a couple of times now.  The only way to fix it, is to close down ServerWMC, and then delete the files.  Is there some reason ServerWMC is not actively doing this?  (Logs posted)

 

ServerWMC.log

ServerWMC_20160103.log

[krustyreturns 191]

Are you saying this happens consistently, or occasionally?  Any change you can give me the steps to reproduce it?  When it happens, do you know if the ts file is growing in size?

[theniteow1 25]

Are you saying this happens consistently, or occasionally?  Any change you can give me the steps to reproduce it?  When it happens, do you know if the ts file is growing in size?

 

It seems to be happening consistently.  Although, I'm not on my server regularly checking the folder.  However, I've noticed them for a month or so.  They don't seem to grow in size.  Once the channel is not viewed anymore.  It stays stationary, they are just never removed.

[krustyreturns 191]

What client are you using for Emby?

[theniteow1 25]

What client are you using for Emby?

 

I'm using the FireTV App, latest version.  I just closed down ServerWMC and deleted the files.  Will keep better watch what happens tonight.

[krustyreturns 191]

I so far can't reproduce it with the web client.  If you can give me some steps to make it happen that would be great.

[theniteow1 25]

I so far can't reproduce it with the web client.  If you can give me some steps to make it happen that would be great.

 

It just happened again tonight.  We were watching a channel 9.1 and then switched it to channel 7.1.  It didn't delete the .TS file from channel 9.1.  Tried deleting it manually, and it says the same thing that ServerWMC has locked the file.  Here are the logs.  I'll experiment with the web client as well, and see if the behavior is the same as well.

ServerWMC.1.log

[krustyreturns 191]

I can't find where you tried to open a live stream in that log.  If you are running swmc as a service I need the service log file instead.

 

Playing around with it some more, I have found something.  In the mode you are running in serverwmc opens the file twice - it opens the file for writing the ts data to disk AND it also opens the file for reading through the http service.  It seems the closing of the file by the http service is sometimes significantly delayed relative to emby sending a close stream command.  So when the remux (the file writer) is shutdown the ts file delete fails because the file is still open for reading.  Later when the reading stream is closed the file gets left behind on disk.  

 

I can make this happen from time to time, but each time it does I can delete the ts file after the read file is eventually closed.  Since you are saying that you can't delete the file until swmc is restarted it must not be the same thing you are seeing, but at least this is a problem I can fix now.  

[theniteow1 25]

I can't find where you tried to open a live stream in that log.  If you are running swmc as a service I need the service log file instead.

 

Playing around with it some more, I have found something.  In the mode you are running in serverwmc opens the file twice - it opens the file for writing the ts data to disk AND it also opens the file for reading through the http service.  It seems the closing of the file by the http service is sometimes significantly delayed relative to emby sending a close stream command.  So when the remux (the file writer) is shutdown the ts file delete fails because the file is still open for reading.  Later when the reading stream is closed the file gets left behind on disk.  

 

I can make this happen from time to time, but each time it does I can delete the ts file after the read file is eventually closed.  Since you are saying that you can't delete the file until swmc is restarted it must not be the same thing you are seeing, but at least this is a problem I can fix now.  

 

I just tried it using the web client, and it closes out and deletes successfully every time.  I'm not running as SWMC service (do you recommend doing that?)  Yes, I'm using HTTP as the method to delivery to the FireTV, as this has been the most reliable method, and allows me to "Direct" stream without transcoding.  Here is the latest log.  Thanks for your help with this and your dedication.

ServerWMC.log

[theniteow1 25]

Hi @@krustyreturns was just checking to see if you need anything from me on this?  Thanks.

[krustyreturns 191]

I really didn't remember how we left. it.  I guess the only remaining question is why the stream is never closed with the fire tv.  If you can generate a clean log where the stream is started and stopped from the ftv but the ts file remains locked I can look in the file and see if a close stream command was ever issued from emby.

[theniteow1 25]

I really didn't remember how we left. it.  I guess the only remaining question is why the stream is never closed with the fire tv.  If you can generate a clean log where the stream is started and stopped from the ftv but the ts file remains locked I can look in the file and see if a close stream command was ever issued from emby.

 

Hi, yes, that's what the above log is.  I can create a newer one today.  I won't be able to do it until this evening however.

[krustyreturns 191]

I was hoping for something clean, the log you posted contains multiple openings and closings so its a bit of a slog to determine if I have a close for every open.  But yeah, I can do it, just need to overcome my laziness and the beckoning of fallout 4.  

 

I think you have seen this tread - since you posted on the child thread he points to in his last post:

 

http://emby.media/community/index.php?/topic/29149-live-tv-streams-die-randomly/

 

In it, I went through the work of tracing it all through his lengthy log.  I assume I'll get the same result with yours, especially since you don't see this problem with the web client. 

[theniteow1 25]

Hey @@krustyreturns...  Sorry to interrupt you playing Fallout 4.. lol.. here is a clean log for you.  After watching the football game today, and switching the channel, the old stream remained once again.  I can't delete the stream with out closing out ServerWMC first.

ServerWMC.log

[krustyreturns 191]

Thanks for that.  That does clearly show that its not Emby's fault since it set the close stream, I don't why the ts is getting locked but I guess I'll have to figure that out (luckily I am sharing the ps4 with my son so I'll have some F4 downtime  ).    

 

I don't have a fire tv, but on my nexus7 I can't change the channel without first closing the channel I am watching (it also never seems to have this problem)  Is that not the case for the FTV?  If you can do a channel switch, what happens if you first stop what you are watching and then start the new channel?  (Did I ask the already?)

Edited January 10, 2016 by krustyreturns

[theniteow1 25]

Thanks for that.  That does clearly show that its not Emby's fault since it set the close stream, I don't why the ts is getting locked but I guess I'll have to figure that out (luckily I am sharing the ps4 with my son so I'll have some F4 downtime  ).    

 

I don't have a fire tv, but on my nexus7 I can't change the channel without first closing the channel I am watching (it also never seems to have this problem)  Is that not the case for the FTV?  If you can do a channel switch, what happens if you first stop what you are watching and then start the new channel?  (Did I ask the already?)

 

Haha, I'm sure your son will enjoy that!  That's a good question.  I usually don't "stop" the stream.  I will go to the mini-guide, and select another channel.  However, I've had it happen on something where I didn't ever go to the mini-guide as well.

[theniteow1 25]

Just an observation, it seems like the longer the stream is open, the more likely chance this will occur.  I've been playing around here for the last 30 minutes, and if I switch channels frequently, I haven't been able to duplicate the problem.  Not sure what significance that has. 

[krustyreturns 191]

Maybe that's why I can duplicate it.  I'll try that.

[krustyreturns 191]

@@theniteow1, I think I have a handle on what the problem is.  Working on a fix now.

[krustyreturns 191]

@@theniteow1

 

I leased a preview update for this today in swmc (1226).  I think it works but I have trouble triggering the bug in the first place.  If you can, try it out and let me know if it works.  If it doesn't then please post your swmc log.  

Edited January 11, 2016 by krustyreturns

[theniteow1 25]

@@theniteow1

 

I leased a preview update for this today in swmc (1226).  I think it works but I have trouble triggering the bug in the first place.  If you can, try it out and let me know if it works.  If it doesn't then please post your swmc log.  

 

Thanks @@krustyreturns.  I'll give it a try tonight.

[theniteow1 25]

Hi @@krustyreturns.  This has seemed to have made the problem better!!  Streams are deleting now when they are no longer in use.  However today, I had a weird thing occur.  I had channel that kept on streaming even though it wasn't being viewed.  What was different was it actually showed the channel still being streamed in ServerWMC.  I hadn't seen that before.  It was the Emby^MCSERVER^7 stream.  It looks like it was started last night, and for some reason was never closed out, and just continued today until I reset the tuner.  I've also set a stream's to not continue any longer than 4 hours, so I'm not sure why this would have been allowed to go longer that.  Here are the logs, and a screen shot of what I saw.

 

 

Thanks for the good work you are doing!

ServerWMC.log

ServerWMC_20160111.log

[krustyreturns 191]

Having the stream file deleted after it is closed is what this thread was about, so I am going to assume that problem is solved.  Right?

 

Having streams not being closed by Emby (i.e. serverwmc and the tuner are left streaming data to emby - even though the stream has been closed on the client) is the problem in the this thread:

http://emby.media/community/index.php?/topic/29500-not-closing-live-tv-streams/

 

which was spawned by this thread (see toward the end):

http://emby.media/community/index.php?/topic/29149-live-tv-streams-die-randomly/

 

I checked your logs and there is never a close stream command being sent for stream id 7 (thanks for letting me know which one to look for - saved a lot of time).  I don't know if the problem is Emby not sending a close stream command OR whether it's a problem with the emby client, but since the web clients always seems to work I assume its the later.  Maybe @@Luke will comment.

Edited January 12, 2016 by krustyreturns

[theniteow1 25]

 

Having the stream file deleted after it is closed is what this thread was about, so I am going to assume that problem is solved.  Right?

 

Having streams not being closed by Emby (i.e. serverwmc and the tuner are left streaming data to emby - even though the stream has been closed on the client) is the problem in the this thread:

http://emby.media/community/index.php?/topic/29500-not-closing-live-tv-streams/

 

which was spawned by this thread (see toward the end):

http://emby.media/community/index.php?/topic/29149-live-tv-streams-die-randomly/

 

I checked your logs and there is never a close stream command being sent for stream id 7 (thanks for letting me know which one to look for - saved a lot of time).  I don't know if the problem is Emby not sending a close stream command OR whether it's a problem with the emby client, but since the web clients always seems to work I assume its the later.  Maybe @@Luke will comment.

 

Thanks @@krustyreturns.  Yes, you are correct, the original problem of streams not being deleted after it's closed is fixed, so I will mark this thread as answered.  Just wanted to make sure you aware of this other issue, in case it was related in the way.  I will visit those other threads.  Although I tried to duplicate the problem last night and I couldn't.  I'm wondering if it has something to do with the logic of the timer for "Prime time" viewing.  I couldn't duplicate the problem at all last night, but yet today I have another stream stuck from this morning.  I set the prime time viewing from 10:00pm - 10:00am.

Edited January 13, 2016 by theniteow1

[krustyreturns 191]

Yeah, good point.  The prime-time view window should be a way to combat streams being left open by emby, since outside this time window streams will be forced closed if they have run longer then the time you configure.    I'm confused... Are you saying it is working just the opposite?  Can you post logs for this?

 

Just curious: why do you have the settings set so that streams are never auto-closed late at night?   That's the opposite of the way I would expect.