Recording ended early

[Luke 37069]

20 minutes ago, BoomerGamer62 said:

@LukeThanks for taking a look at this.  Ive completed testing.  I did 10 one-hour recordings. Ten were with the command string exactly as it shows in the Emby log, and ten recordings with the modified command string.  The same URL/channel was used each time, at the same time of day.  The results were that the unmodified command string failed two times, but the modified command string did not fail at all.  I did not see any difference in performance or system resource use on the server.

Therefore, I think it would benefit everyone to change the ffmpeg command string in Emby.  My suggestion is to replace

 -reconnect_streamed 1 -reconnect_delay_max 2

with

-reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 127 -reconnect_on_http_error 5xx,4xx

Ive attached the bat file I tested with for reference (input URL is edited).

These changes are based on the article I quoted earlier (which mentioned you need -reconnect 1 so I included that too).  Hope you will consider making this change in a future release.  If I can help further let me know.

Did you actually experience an http error to necessitate that param, or are you just throwing it in because you think it might happen?

[BoomerGamer62 18]

50 minutes ago, Luke said:

Did you actually experience an http error to necessitate that param, or are you just throwing it in because you think it might happen?

Yes I did, it was a 503 (server not ready). Also have seen http errors in a couple of past Emby recording logs.  With my changes, the recording was able to continue.  Mostly I received -10053 errors, which is an I/O issue, but it kept right on going.

[BoomerGamer62 18]

@Luke, by the way, I also did one very long test, to see just how far a recording can go before stopped.  At the 4 hour mark, the http error 410 (GONE) started being thrown, which meant that my provider ended the stream totally.  Even with the http error command, ffmeg was smart enough to not continue and stopped.

[iiiJoe 55]

1 hour ago, BoomerGamer62 said:

@LukeThanks for taking a look at this.  Ive completed testing.  I did 10 one-hour recordings. Ten were with the command string exactly as it shows in the Emby log, and ten recordings with the modified command string.  The same URL/channel was used each time, at the same time of day.  The results were that the unmodified command string failed two times, but the modified command string did not fail at all.  I did not see any difference in performance or system resource use on the server.

Therefore, I think it would benefit everyone to change the ffmpeg command string in Emby.  My suggestion is to replace

 -reconnect_streamed 1 -reconnect_delay_max 2

with

-reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 127 -reconnect_on_http_error 5xx,4xx

Ive attached the bat file I tested with for reference (input URL is edited).

These changes are based on the article I quoted earlier (which mentioned you need -reconnect 1 so I included that too).  Hope you will consider making this change in a future release.  If I can help further let me know.

@Lukethis could be a game changer for those of us trying to record m3u’s. Any idea when this could possibly be implemented?

[iiiJoe 55]

9 minutes ago, iiiJoe said:

@Lukethis could be a game changer for those of us trying to record m3u’s. Any idea when this could possibly be implemented?

Not sure if this is possible but would you consider making this user adjustable?

[Luke 37069]

I'm going to put these changes into the next beta build. 

[iiiJoe 55]

1 hour ago, BoomerGamer62 said:

@iiiJoeaccording to what Luke said, there unfortunately isnt a way to change the reconnect_delay_max value in Emby.  Im doing testing now, and I hope to convince everyone at Emby to change the command string to keep the recording alive longer.

If you want to test it outside of Emby, I would set up a fresh folder with nothing else in it so you dont accidentally affect anything.  Then copy the ffmpeg.exe file into that (its in the system folder where your Emby system is, like the logs are).  Ive attached a sample bat file you can run to test it, but you will have to edit it to replace the sample URL I have (URLSTREAM.m3u8).  I put a pause in the file so you can see the results in the command window.  Once you edit the bat file, just run it.

By the way, one suggestion that no one has mentioned yet:  check the speed of you internet connection (I use speedtest.net), and if you use a VPN, check that the load on the VPN isnt maxed out at 100%.  A slow connection can cause timeouts and I/O errors and lead to your recording failing.

Hope this helps.

simple.bat 190 B · 0 downloads

@BoomerGamer62ty so much for this! Will be my first time trying to do work with a bat file so I may hit you up for some advice after I try it out, if you wouldn’t mind. Also, checking connection speed is a great idea and essential to ferreting out failed recordings and dropped streams as it can be a primary driver for these issues. I have experimented many different ways to nail down failed recording causation. Different channels, different times, different computers and different providers. I’ve even documented results with error codes using two different providers on multiple pc’s simultaneously and alternatively. I’m currently operating ember server on three different pc’s with two providers. I actually found that a VPN caused failed recordings to occur more often, at least for me. My connection is fast and stable but this obviously won’t be the case for everyone. 

[iiiJoe 55]

39 minutes ago, Luke said:

I'm going to put these changes into the next beta build. 

[BoomerGamer62 18]

1 hour ago, iiiJoe said:

@BoomerGamer62ty so much for this! Will be my first time trying to do work with a bat file so I may hit you up for some advice after I try it out, if you wouldn’t mind. Also, checking connection speed is a great idea and essential to ferreting out failed recordings and dropped streams as it can be a primary driver for these issues. I have experimented many different ways to nail down failed recording causation. Different channels, different times, different computers and different providers. I’ve even documented results with error codes using two different providers on multiple pc’s simultaneously and alternatively. I’m currently operating ember server on three different pc’s with two providers. I actually found that a VPN caused failed recordings to occur more often, at least for me. My connection is fast and stable but this obviously won’t be the case for everyone. 

Happy to help out, if I can.  One last tip.  If you want to see which recordings failed, go to the log folder in Windows explorer.  Under advanced options, make sure File Contents is checked.  In the search area put OPERATION NOT PERMITTED.  You will get a list of all the log files where the recording failed. I do this regularly to clean up my library and throw away recordings that didnt complete.  Nothing worse than watching something with the end missing!

I also noticed that many streaming providers specifically say they do NOT support recording (including the one I use, sadly).  This is probably because they dont want to deal with the complaints of their streams dropping out.  So Im looking forwarded to these changes happening and perhaps recordings will stay "alive" longer.

[BoomerGamer62 18]

2 hours ago, Luke said:

I'm going to put these changes into the next beta build. 

@Luke Great news!  Thanks for the fast response.  Be happy to help test the beta if I can.  Not sure how to get to it though, whenever it's ready.  Is there a topic or help you can point to on that?

[BoomerGamer62 18]

I have been researching more about recordings failing. In addition to the ffmpeg option improvements, Ive tried different protocols (TCP/UDP) and settings, and recording outside of Emby altogether.  I get some improvements but not a 100% success rate.  Let's face it, IPTV stream is not dependable enough for recording.  Most reviewers say as much.

So I thought maybe we can approach this from another angle.  Would it be possible to have a check to see whether the recording finished early?  If it did finish early, just restart recording.  Id rather have a few seconds missing from a recording than not have the end of it at all.  Having it in two pieces is a hassle, but again better than nothing.  So for example, if the schedule for the program Im recording says to record from 6:00pm to 6:30pm, and the recording finished at 6:18pm, kick off another recording from now to 6:30pm

Any chance of this? (fingers crossed)

[iiiJoe 55]

20 minutes ago, BoomerGamer62 said:

I have been researching more about recordings failing. In addition to the ffmpeg option improvements, Ive tried different protocols (TCP/UDP) and settings, and recording outside of Emby altogether.  I get some improvements but not a 100% success rate.  Let's face it, IPTV stream is not dependable enough for recording.  Most reviewers say as much.

So I thought maybe we can approach this from another angle.  Would it be possible to have a check to see whether the recording finished early?  If it did finish early, just restart recording.  Id rather have a few seconds missing from a recording than not have the end of it at all.  Having it in two pieces is a hassle, but again better than nothing.  So for example, if the schedule for the program Im recording says to record from 6:00pm to 6:30pm, and the recording finished at 6:18pm, kick off another recording from now to 6:30pm

Any chance of this? (fingers crossed)

I’ve actually wondered about this myself. Losing a few minutes or even more of a recording would not bother me, especially when recording a basketball or football game. This would be a much better alternative to losing most if not all. However, I would estimate that 60%-70% of my recordings are successful on most channels even with the current 2 second reconnection limit. I’d be very interested to see how a higher value would impact successful recordings. If you’re recording a daily or weekly 1 hr program and the stream is consistently being interrupted, this is more of a provider issue (assuming solid internet connection ofc), imho. But, I do think it’s very smart to approach this from a couple of angles. If Emby, could recognize when a recording stopped prematurely and then restart, this could be VERY useful. What about a plug in that would perform this task? Possible?

[iiiJoe 55]

Also, @BoomerGamer62@luke, could the max reconnection value be increased beyond the 127 mentioned in the article? Could it be doubled, tripled or raised even higher?

Edited May 29, 2023 by iiiJoe

[Luke 37069]

19 minutes ago, iiiJoe said:

Also, @BoomerGamer62@luke, could the max reconnection value be increased beyond the 127 mentioned in the article? Could it be doubled, tripled or raised even higher?

That sounds a little extreme, no?

[iiiJoe 55]

17 minutes ago, Luke said:

That sounds a little extreme, no?

Well, speaking from experience lol, the pain of a failed recording 10, 15 or 30 min in easily surpasses the pain of losing minutes and getting most of it. 

[iiiJoe 55]

@Luke, why not just make this user adjustable so everyone can decide how much they're willing to lose?

[Luke 37069]

13 minutes ago, iiiJoe said:

@Luke, why not just make this user adjustable so everyone can decide how much they're willing to lose?

Yes more options to control it are certainly possible.

[iiiJoe 55]

7 minutes ago, Luke said:

Yes more options to control it are certainly possible.

Would be a game changer, imho

[BoomerGamer62 18]

1 hour ago, Luke said:

That sounds a little extreme, no?

I agree with @Luke, I dont think a higher reconnect value will do any good, since it wont handle issues when its the provider dropping the connection.  That's why I suggested Emby just restart the recording (or at least have an option to).  I think that would benefit everyone.

[iiiJoe 55]

15 hours ago, BoomerGamer62 said:

I agree with @Luke, I dont think a higher reconnect value will do any good, since it wont handle issues when its the provider dropping the connection.  That's why I suggested Emby just restart the recording (or at least have an option to).  I think that would benefit everyone.

I can def see where you’re coming from but I might prefer extending the reconnection delay to have one succinct(at least as much as possible) recording as opposed to multiple individual recordings of the same, for instance, 1 hr program. I’m still of the mindset that the user is in the best position to determine how much of a delay they’re willing to live with. If their recordings are too choppy and they have a sorry provider they can simply try another one until they find success. I’ve had to do this myself. Also, increasing the max delay seems like a relatively straight forward adjustment, programming wise, as opposed to perhaps a plug in that would re-start failed recordings, which I am totally for, btw. @Lukedoes the development of the plug-ins normally come from the Emby team?

[Luke 37069]

1 hour ago, iiiJoe said:

I can def see where you’re coming from but I might prefer extending the reconnection delay to have one succinct(at least as much as possible) recording as opposed to multiple individual recordings of the same, for instance, 1 hr program. I’m still of the mindset that the user is in the best position to determine how much of a delay they’re willing to live with. If their recordings are too choppy and they have a sorry provider they can simply try another one until they find success. I’ve had to do this myself. Also, increasing the max delay seems like a relatively straight forward adjustment, programming wise, as opposed to perhaps a plug in that would re-start failed recordings, which I am totally for, btw. @Lukedoes the development of the plug-ins normally come from the Emby team?

What plugins?

[iiiJoe 55]

4 minutes ago, Luke said:

What plugins?

 

[Luke 37069]

3 minutes ago, iiiJoe said:

 

Which one are you asking about?

[iiiJoe 55]

Just now, Luke said:

Which one are you asking about?

Any of them, actually. Do they normally come from outside sources? From internal Emby devs? Or, maybe a mixture of both?

[Luke 37069]

2 minutes ago, iiiJoe said:

Any of them, actually. Do they normally come from outside sources? From internal Emby devs? Or, maybe a mixture of both?

Both, but all of the ones that are bundled with Emby Server are built by us.