Simultanious scheduled recordings

[Spaceboy 2494]

Not really - occasionally you will get an error - say your modem gets a forced firmware update - as my provider does occasionally (every month or so) - this can take over 5 mins before things settle down and full connectivity is restored

 

But irrespective of that - if the time period of the recording still has outstanding time (once emby comes back online) then it should try again

 

Some logic would need to be added - to say try every minute or so - and say give up after x trys - so the server is not spamming itself etc etc

agreed, I would say try once a minute for the remainder of the schedule

[maegibbons 1267]

I agree totally with @@Spaceboy and @@PenkethBoy .

 

We have told you that this does not work so please fix it.

 

If its within the scheduled recording window it should retry to a -1 or-2 file etc.

 

It actually should be a sliding scale of backoff. I.e. retry once every 10 secs for 1st minute, every 30 secs for next 2, and then every minute until end of recording slot.

 

Krs

 

Mark

 

Sent from my SM-N976B using Tapatalk

[maegibbons 1267]

Five minutes is a long time though, isn't it?

Are you trying to suggest that if he had pulled it out after two that the result would have been different?

 

Because thats not true either.

 

Krs

 

Mark

 

Sent from my SM-N976B using Tapatalk

[softworkz 3337]

I have just detailed the retry behavior here: https://emby.media/community/index.php?/topic/78465-retries-for-livetv-iptv/?p=861261

 

 

I'm not sure whether I agree to what is being asked for above.

 

When you pull out the antenna when using WMC, it also stops recording after some time and it also doesn't restart recording to record a part 2 or part 3.

 

IMO, what needs to be corrected is that an interrupted recording must not be marked as successful recording. Instead it should be considered as failed and the recording should be re-scheduled.

 

A recording where a significant part is missing (and even 1 minute can be quite significant) is unsuccessful IMO and it can be very frustrating to watch such recordings.

[softworkz 3337]

Are you trying to suggest that if he had pulled it out after two that the result would have been different?

 

If it would be plugged back in after 1 min - yes.

[softworkz 3337]

We can talk about retry timings - that's not the point from our side to dictate strictly.

 

Main point is that we don't want to return to a situation where retries are supposed to "fix" bad IPTV providers.

[Spaceboy 2494]

I have just detailed the retry behavior here: https://emby.media/community/index.php?/topic/78465-retries-for-livetv-iptv/?p=861261

 

 

I'm not sure whether I agree to what is being asked for above.

 

When you pull out the antenna when using WMC, it also stops recording after some time and it also doesn't restart recording to record a part 2 or part 3.

 

IMO, what needs to be corrected is that an interrupted recording must not be marked as successful recording. Instead it should be considered as failed and the recording should be re-scheduled.

 

A recording where a significant part is missing (and even 1 minute can be quite significant) is unsuccessful IMO and it can be very frustrating to watch such recordings.

well what you are saying conflicts with how Luke described it. We aren’t asking for anything other than to understand the current intended behaviour so we can show it doesnt work as Luke suggested. But you’ve confirmed that Edited March 25, 2020 by Spaceboy

[softworkz 3337]

well what you are saying conflicts with how Luke described it. We aren’t asking for anything other than to understand the current intended behaviour so we can show it doesnt work as Luke suggested. But you’ve confirmed that

 

Where I said 'should' I was talking about the future. 

The timeouts I mentioned in the other thread are describing the current situation.

Edited March 25, 2020 by softworkz

[Spaceboy 2494]

Where I said 'should' I was talking about the future.

The timeouts I mentioned in the other thread are describing the current situation.

ok so you are saying if I repeat the test with a gap of less than a minute I should see a second file created? I think you are wrong but I will try it

 

Also I disagree with your approach to this. If I’m recording a live sport event I want the recording to resume whatever the break. There isn’t a later showing to reschedule. Yes it might be annoying to watch but it’s better than nothing.

[softworkz 3337]

ok so you are saying if I repeat the test with a gap of less than a minute I should see a second file created? I think you are wrong but I will try it

No, but it should continue the existing recording.

 

Also I disagree with your approach to this. If I’m recording a live sport event I want the recording to resume whatever the break. There isn’t a later showing to reschedule. Yes it might be annoying to watch but it’s better than nothing.

I admit that this is a valid point. I promise to think about it

[Spaceboy 2494]

No, but it should continue the existing recording.

 

 

I admit that this is a valid point. I promise to think about it

Ok I will test.

 

And thanks, please do. It’s not a regular issue, my provider is generally very good. But when it does it’s quite disappointing [emoji3]

[maegibbons 1267]

 

 

Also I disagree with your approach to this. If I’m recording a live sport event I want the recording to resume whatever the break. There isn’t a later showing to reschedule. Yes it might be annoying to watch but it’s better than nothing.

Just to add my endorsement to this. By all means mark it as failed (in some manner) but recording retry (whatever interval is static or sliding scale backoff) SHOULD ALWAYS happen within the recoding slot. Yes, having breaks in the recording may not ideal BUT it is better than nothing. Losing a minute or two out of a 3 hour "Tour De France" stage or Football game is not a killer!!

 

Krs

 

Mark

 

Sent from my SM-N976B using Tapatalk

[revengineer 125]

Adding my name to this. With my new server setup I did 3 recordings so far on v4.4.3. Two of them failed minutes into the recording with "SharedHttpPipelineSource: Live Stream ended." I agree with Mark, during a recording slots, retries should be continuous.

[Luke 37084]

We'd have to go over an example. Thanks.

[Spaceboy 2494]

We'd have to go over an example. Thanks.

We don’t need to. Softworkz already confirmed your understanding of how this works is not correct. You should speak with him

[Carlo 4330]

What is now being asked for is the ability to infinity keep trying a recording until the time finish time is done.  In other words if a sporting match is on from 2pm to 5pm and something happens at 3pm the system will try again at 3:01, 3:02, 3:03, etc up to 5pm (plus padding).

It should do this regardless if it starts on time or not.  I could for example have 3 recordings going and the earliest finishes at 3:30 so as soon as that frees up the recording starts but should be marked at incomplete.  If communication is lost at any time the system keeps trying every minute to reestablish the connection.

Nothing worse than watching 3/4 of a sporting match and not have the end (most important part usually).

[Fannybaws 0]

Tossing my hat into the ring on this too.

Only recently rebuilt my Emby server (Win 10 2004, 16Gb RAM 16vCPUs ) inside ESXi and provisioned a bunch of storage.

I too am seeing the same issues as reported elsewhere with 4.4.3.0.

Keen to know if there is anything in the pipeline in the form of a fix for this or is the suggestion that "get a better IPTV provider" is the mantra?

[dcol 165]

1 hour ago, Fannybaws said:

Tossing my hat into the ring on this too.

Only recently rebuilt my Emby server (Win 10 2004, 16Gb RAM 16vCPUs ) inside ESXi and provisioned a bunch of storage.

I too am seeing the same issues as reported elsewhere with 4.4.3.0.

Keen to know if there is anything in the pipeline in the form of a fix for this or is the suggestion that "get a better IPTV provider" is the mantra?

Try updating to 4.5.0.14 and see if that helps. I personally find long term streams are risky from IPTV providers. Rarely do I have issues with one hour shows. Usually if there is an issue it will pop up in the first couple minutes, but that is rare these days.

Another note worth mentioning is that I have a propietary tool that tests a stream for consistency. When testing 5 IPTV providers multiple times for a 24 hour continuous stream, only one went the distance, the others ended within the first two hours. So I have to assume that some IPTV providers may have a limit on continuous streaming.

Edited July 17, 2020 by dcol

[revengineer 125]

@dcol We all understand that IPTV can be flaky; it's not emby, it's the network, vpn, or the service). Emby just need to handle this robustly by keep retrying unlimited during the recording window. What @cayars describes above (the unlimited retry) does not seems to be happening. I will need to watch for this and save the log next time.

[softworkz 3337]

It is important to understand that there are two different levels of retry required:

1. Continuation Retry at the Stream Level

When we are in the process of receiving a stream (either continuos http or HLS segment stream) and it gets interrupted, that can be caused by a number of different things.

In some cases, the streaming can be recovered within a short timespan (max 1 to 3 min). This is already covered by our streaming implementation. But at some point it's getting unlikely that the connection can continue and then we must stop it and consider the recording as finished (yet incomplete).

The worst part:: the recording files are becoming invalid and will eventually cause problems when we would just blindly write into those files whatever we're receiving. Some of you might remember the problems we had (1-2 years ago) where you couldn't do any seeking in such recordings and/or playback ran into an endless loop.- That means for the recording that it needs to be ended and shut down.at this time and we cannot append to that recording later.

The above is roughly what we're having right now.

 

2. Retry Recording by the Recording Scheduler

Due to the reasons above, we cannot keep a recording active for a longer amount of time when we can't receive the stream. Also, we cannot hammer the provider with requests because we might get banned and loose other streams as well - that's why we need adaptive retry intervals (for both, 1 and 2).

This topi 2. simply means that the Recording Scheduler needs to regularly (or better: in intervals) retry to start a recording again (saved to a separate recording file).

 

Summing up

1 is status quo

2 is what's planned

 

Best regards,
softworkz

[softworkz 3337]

On 7/16/2020 at 7:07 PM, cayars said:

It should do this regardless if it starts on time or not.  I could for example have 3 recordings going and the earliest finishes at 3:30 so as soon as that frees up the recording starts but should be marked at incomplete.  If communication is lost at any time the system keeps trying every minute to reestablish the connection.

It's not the best example because conflicts between recordings are supposed to be handled right at the moment they are scheduled and conflicts between watching and recording will (eventually) be a matter of user interaction. When a recording gets cancelled this way, it will be cancelled and there won't be any attempts to record like half of it. 

Hence, the retry described as '2' will only apply to technical and environmental reception problems but not to a case like cayars has described.
(there might be other way, though, to make sure that a recording doesn't get stopped or intrerrupted.
 

[Carlo 4330]

I'll disagree.  For the simple fact that Emby really never knows how many tuners are available if using IPTV or network tuners. A person in the house could be using LiveChannels or an IPTV software using up one of your "tuner" slots.  Emby would have no idea in this case that it's down a tuner so the logical thing to do is start recording as soon as it can and then mark the file as partial.

Of course we have no such status but we should.  Something that got recorded partial could be rescheduled again in the hopes of getting a full recording when we get to that point.

[revengineer 125]

@softworkz Thank you for the explanation and path forward. I agree that the recording file must remain valid. I do not understand this well enough to judge why the file can be kept open until the valid stream starts again without writing junk to it in the middle. But if the file needs to be closed, the having the recording scheduler open a new one is the next best option, and this is not yet happening. Of course, multiple files are inconvenient, so I hope that emby will merge the different parts at the end.

[softworkz 3337]

11 hours ago, revengineer said:

@softworkz Thank you for the explanation and path forward. I agree that the recording file must remain valid. I do not understand this well enough to judge why the file can be kept open until the valid stream starts again without writing junk to it in the middle. But if the file needs to be closed, the having the recording scheduler open a new one is the next best option, and this is not yet happening. Of course, multiple files are inconvenient, so I hope that emby will merge the different parts at the end.

I don't think that merging is a good idea. Besides technical issues which I can't go into details about right now, there's also the question whether one would really want that.

When you would merge several parts together to a continuous video, you would sometimes not even notice that there's a jump. It depends on the cut points and the watchers degree of attention whether he/she would even notice that some part has been skipped. That could go as far as changing the meaning and the perception of the plot.

I could imagine that clients might be able to automatically proceed to play the next segment of a disrupted recording, similar to the "auto-play next episode" feature - maybe.

But in general, an interruption of the network stream should be an exceptional and rare situation. It can happen when there's network or power outage locally or at the provider, but when it happens frequently or you would end up with several segments for a recording, then it's time to switch the provider, or check whether your password might have been compromised or you're exceeding your simultaneous streams limit.

That's not much different to when you would receive terrestrial TV with an antenna and reception is often bad. What would you do? Fix the anntenna/cabling or look for a software that combines the "good fragments" to a continuous video?

[softworkz 3337]

20 hours ago, cayars said:

 Something that got recorded partial could be rescheduled again in the hopes of getting a full recording when we get to that point.

Yes - a good DVR should be able to do that.

20 hours ago, cayars said:

I'll disagree.  For the simple fact that Emby really never knows how many tuners are available if using IPTV or network tuners. A person in the house could be using LiveChannels or an IPTV software using up one of your "tuner" slots.  Emby would have no idea in this case that it's down a tuner so the logical thing to do is start recording as soon as it can and then mark the file as partial.

I can't talk about possible future Emby features, but we can drive a bit into theory and look at the implications of DVR operation and scheduling of recordings.

The example you are giving above is actually hitting THE one important decision point which opens up two different worlds of possible DVR operation modes. That point of decision can be expressed as the question:

Is the DVR guaranteed to have exclusive access to the resources it is given (configured for)?

The answer to that question has severe consequences for the behavior of the DVR.
In your example above, the answer would be 'No'. You could turn that into a 'Yes' by not giving the provider credentials to anybody for direct use, or by configuring the number of streams for the DVR to use to a number that is guaranteed(!) to be available at any time. 

And those are the possible modes of operation depending on the answer to the above question:

1. YES - Intelligent resource allocation and predictive scheduling
(DVR can rely on having exclusive access)

This will allow a DVR to:

• Perform reliable and uninterrupted recordings
• Guarantee that a recording will actually be performed
  (when the recording is configured to disallow being cancelled by live watching requests)
• Reliably indicate possible scheduling conflicts (with other recordings) right at the time when the user schedules the recording
• Automatically resolve recording conflicts 
• Inform a user that is starting to watch a live channel about possible conflicts between live watching and scheduled recordings, similar to:
  You won't be able to watch this program to the end unless you cancel the recording xyz

2. NO - Trial and Error resource allocation and best effort scheduling
(DVR will never know which resources are available. It will only ever know that those devices are unavailable that it is using itself)

• None of the above things will be possible
• It can't make any calculations and predictions, so it "can't plan for the future"
• Then it depends on the provider's behavior regarding stream over-usage
  • Does it reject the request for an additional stream
    or
  • Does it end a currently running stream instead?
• That can often lead to stream-limit ping-pong
  Those retries that you are always asking for can easily screw-up everything. When there are more stream consumers than allowed and all are constantly re-trying, it may always kick out one of the other streams which would start its own retryx loop

20 hours ago, cayars said:

I'll disagree.

Back to your disagreement...Right now we do not have a DVR scheduler like I'm describing above, but assuming we would have something like it at some time, then you probably wouldn't want the "NO" configuration. You would of course be able to do so, but you would need to accept and live with the consequences. Might be OK for some and not OK for others.