Completed Show Intro Skip Option

[rbjtech 4316]

A 'Skip Intro' (single click) button like Netflix or Prime would be perfect !  - but maybe there is something we can do in the meantime to make use of all the fantastic work @chef has done.

Inserting into the existing chapters should, in theory, be relatively easy ?

[chef 3749]

1 hour ago, Cheesegeezer said:

All the clients are driven from one source now and passed thru electron depending on what device they are on.  So this would be a fairly straightforward process to implement to all clients.

The player just needs to watch for a Intro Start Chapter tag from chapters(as you suggest) and then show the button until the intro end chapter tag has passed.

But if you plan on backing out then clicking the chapter for intro end, it's probably quicker to just hit skip on the remote 2 or 3 times and watch the last 10 secs of an intro.

Really need the admins on board for this to work and be a polished feature.  I mean @chef has done all the heavy lifting and hard work here for them.

It was @samuelqwe who figured out the magic. All I did was translate and integrate.

 

But something like this could also be done without any need for implementation form upstairs (I'm just sayin'):

 

Edited September 3, 2021 by chef

[chef 3749]

@rbjtech I have tried implementing the task together and running title detection on each season as they are encoded.

The difference so far, is we are holding the sequence data in memory (instead of writing it to the db) until the entire season in processed. This might be faster.

The only thing to be weary of is if the task errors, or the user ends the task, we lose the currently processing seasons worth of data (encodings and all).

Restarting the task would begin encoding it from episode 1 of the list season. 

[rbjtech 4316]

16 minutes ago, chef said:

@rbjtech I have tried implementing the task together and running title detection on each season as they are encoded.

The difference so far, is we are holding the sequence data in memory (instead of writing it to the db) until the entire season in processed. This might be faster.

The only thing to be weary of is if the task errors, or the user ends the task, we lose the currently processing seasons worth of data (encodings and all).

Restarting the task would begin encoding it from episode 1 of the list season. 

ok - I think that is fine personally - as long as it's stops gracefully, albeit throwing away all transactions for that season up to a previous 'save point' - then I see no issues with this. 

My system just finished with the 5 shows (170 episodes) - so I'm just going to look at what it did. 

Immediate impressions are that it is a LOT faster, query on the Plugin is now instant, dB has gone down from the 17Meg to 3Meg on the 2nd run (then finished immediately).  Detection memory usage is a fraction of what it was before and CPU during detection seemed to be much lower too.

So it's all looking very positive ! 

Just going to look at the logs .. and if ok .. increase the number of shows to 10 to see if it's scaling well.

Edited September 3, 2021 by rbjtech

[samuelqwe 313]

1 hour ago, chef said:

It was @samuelqwe who figured out the magic. All I did was translate and integrate.

Even I didn’t come up with this solution, I  only took the relevant parts from https://github.com/VictorBitca/matcher and adapted the code for Python, then @chef translated it to C# and created the plugin.

It’s really a group effort and probably would not be where it is today without the above code from GitHub, as well as all the work from everyone else who contributed.

[rbjtech 4316]

Looking at the logs - if any Season has a 'false' for detection (HasSequence=0) - then on repeating the detection task, it re-detects the entire Season again.

If I manually set the detection to 'True' - then it skips the season/show.

If the Intro was not found on a scan, then it is not going to be found a second time around ?

I note in the dB, that you have a 'Confirmed' and 'Processed' field - I'm not sure on the logic, but I would have thought that once you have processed an item - regardless of the result, you would not want to process it again ?

edit - so yep, if I set all to True - then dB for 170 episodes is now 28K .. 

Edited September 3, 2021 by rbjtech

[chef 3749]

14 minutes ago, rbjtech said:

Looking at the logs - if any Season has a 'false' for detection (HasSequence=0) - then on repeating the detection task, it re-detects the entire Season again.

If I manually set the detection to 'True' - then it skips the season/show.

If the Intro was not found on a scan, then it is not going to be found a second time around ?

I note in the dB, that you have a 'Confirmed' and 'Processed' field - I'm not sure on the logic, but I would have thought that once you have processed an item - regardless of the result, you would not want to process it again ?

There should be a condition that looks at the amount of episodes we have in the db, and the amount of episodes in the library. They shouldn't be scanned again unless there are more episodes in the library (a new item was added).

The conditions must be wrong and I'll take a look.

The new column,  'processed', isnt used yet, but I thought it could also be used to stop rescans in the future.

I'll look at that condition and see why it isn't met.

Brb

[rbjtech 4316]

Perfect - it's looking really good @chef 

I've just doubled the workload to 10 shows (in additional to the 5 it already has fully processed, but it should ignore those now). 

[chef 3749]

This is what I meant to put in the code:

                    if(dbEpisodes.Count() == episodeQuery.TotalRecordCount && dbEpisodes.All(item => item.Processed))
                    {
                        Log.Info($"{series.Name} S:{season.IndexNumber} have no new episodes to scan.");
                        continue;
                    }

 

[Micael456 43]

1 hour ago, rbjtech said:

ok - I think that is fine personally - as long as it's stops gracefully, albeit throwing away all transactions for that season up to a previous 'save point' - then I see no issues with this. 

 

For what it's worth, I agree here. It's pretty common for running tasks to lose data if terminated, so wouldn't be too out of place here either.

[chef 3749]

2.0.2.6

• Handles marking the db entries as processed. If we have scanned the season in it's entirety, we won't try rescan episodes again. Unless there are new episodes added to the season. Then we'll try and attempt sequence detection  using the new episode.

IntroSkip_v2.0.2.6.zip

 

Probably have to remove the db file one last time.

[rbjtech 4316]

You are a coding machine ! 

Trying the new version now .. 

[rbjtech 4316]

It's looking good now - a live tail of the log is showing it skipping the episodes it has already done .. nice 

[chef 3749]

2 minutes ago, rbjtech said:

It's looking good now - a live tail of the log is showing it skipping the episodes it has already done .. nice 

[rbjtech 4316]

btw - I'm not sure the 'Detection took X seconds' is adding much info to the log - maybe if you added which title it was referring to ?  Or maybe remove it ?

[chef 3749]

1 minute ago, rbjtech said:

btw - I'm not sure the 'Detection took X seconds' is adding much info to the log - maybe if you added which title it was referring to ?  Or maybe remove it ?

Will do. It was kinda to help know how long these fingerprint comparisons where taking. But, having more info there is much more helpful.

[chef 3749]

@rbjtech do you have a GitHub? 

 

 

----

I ran this over night, and it seemed to be working pretty well.

Reading the logs, I could see, a season being ignored, then later a newer episode of a show got added, which was caught as a new item, encoded and the title sequence was found automatically. The next time the task ran, it switched back to ignoring the season.  Pretty great! 

 

The only thing I'm not sure about is the vacuum in the fingerprint task. Since it has to run again to clear out the db file. Might not be a big deal, but I could try and put it at the end of the sequence task. Something tells me that is where it should be. Unsure.

 

If it works for other people, I would like to switch all the log entries to 'debug'. Then we won't see them unless debug is turned on. It's just that with debug turned on in the logs, they get pretty big.

 

Then we should look at adding an experimental switch which could edit chapters... Maybe.

 

Oh! We should totally tap into embys ItemAdded event, so we can run right away!

Edited September 4, 2021 by chef

[chef 3749]

We are now tied into the ItemAdded event

        private async void LibraryManager_ItemAdded(object sender, ItemChangeEventArgs e)
        {
            var fingerprint = TaskManager.ScheduledTasks.FirstOrDefault(task => task.Name == "Episode Audio Fingerprinting");
            if(fingerprint.State != TaskState.Running)
            {
                try
                {
                    await fingerprint.ScheduledTask.Execute(new CancellationToken(), new Progress<double>()).ConfigureAwait(false);
                }
                catch { }
            }
        }

 

[rbjtech 4316]

the vacuum task - What I did is cheat the system a bit by running the task on an interval - every 2 hrs, but limit the task to 1 hr.  This then auto vacuum's the dB every 2 hrs .. 

This is why I was enquiring around doing 1) FP for a Season 2) Detect for a Season 3) Vacuum a Season - then move onto the next.  BUT I don't think we need to do this anymore as the query time / memory usage seems significantly better this time. 

I'm not sure if it would run as well if I let the dB build for say 30K episodes (without a vacuum) but the schedule 'hack' works around this without any further changes.

Log the items in Debug mode is a great idea - but I would keep the 'Season Complete' in the main log ?

On the edit chapters - YES ! - I looked at some of the Tick conversations yesterday actually, and there appears to be a C++ TickConversion library which should make things a little easier.

The existing chapters are easy to get, so just inserting the IntroStart and IntroEnd (converted to ticks) and then a re-order and write back 'should' be all that is required here. (he says)

On the Github - no, I'm not a coder, I'm an infrastructure network/security guy - but do dabble in scripting but I'm not very good at it - haha.. 

[Painkiller8818 203]

I also wanted to try this plugin and installed it (windows)

 

i saw it needs a new ffmpeg so i downloaded the latest compiled version from here: https://github.com/BtbN/FFmpeg-Builds/releases

i replaced the ffmpeg.exe and ffdetect.exe with the new ones and restarted the server, but after that i got the "no stable streams" warning, so i reverted to the old files.

I think windows users have to wait as just replacing the files seems not to work
 

[rbjtech 4316]

1 hour ago, Painkiller8818 said:

I also wanted to try this plugin and installed it (windows)

 

i saw it needs a new ffmpeg so i downloaded the latest compiled version from here: https://github.com/BtbN/FFmpeg-Builds/releases

i replaced the ffmpeg.exe and ffdetect.exe with the new ones and restarted the server, but after that i got the "no stable streams" warning, so i reverted to the old files.

I think windows users have to wait as just replacing the files seems not to work
 

The latest Beta (4.7.0.9) contains the required ffmpeg as part of the update process.  It's needs to be an 'emby' ffmpeg as it has many enhancements over the standard ffmpeg.

At the moment, I would suggest testing on a separate beta instance (run on another port) with a limited size TV test library.   

The plugin will not harm anything - but it has the potential to grab all the resource as it is a very intensive task to chromeprint all the episodes and then detect the Intro sequence of each one.

The chromaprint technology has been 'working' for a couple of weeks with a very high success rate (amazing work from @samuelqwe & @chef ), the work happening now is how to 'control' the task and make it scalable - ie make it ready for the masses to test without breaking systems .. 

Edited September 4, 2021 by rbjtech

[chef 3749]

5 hours ago, rbjtech said:

the vacuum task - What I did is cheat the system a bit by running the task on an interval - every 2 hrs, but limit the task to 1 hr.  This then auto vacuum's the dB every 2 hrs .. 

This is why I was enquiring around doing 1) FP for a Season 2) Detect for a Season 3) Vacuum a Season - then move onto the next.  BUT I don't think we need to do this anymore as the query time / memory usage seems significantly better this time. 

I'm not sure if it would run as well if I let the dB build for say 30K episodes (without a vacuum) but the schedule 'hack' works around this without any further changes.

Log the items in Debug mode is a great idea - but I would keep the 'Season Complete' in the main log ?

On the edit chapters - YES ! - I looked at some of the Tick conversations yesterday actually, and there appears to be a C++ TickConversion library which should make things a little easier.

The existing chapters are easy to get, so just inserting the IntroStart and IntroEnd (converted to ticks) and then a re-order and write back 'should' be all that is required here. (he says)

On the Github - no, I'm not a coder, I'm an infrastructure network/security guy - but do dabble in scripting but I'm not very good at it - haha.. 

Let's put the vaccum in a separate task, and have it run at the end of the sequence detection. It can be hidden, and it will run automatically. It will also run on a schedule, to keep things tidy. But it won't run at the same time as our other two tasks... Just to be safe.

[GiGo 103]

11 hours ago, rbjtech said:

the work happening now is how to 'control' the task and make it scalable - ie make it ready for the masses to test without breaking systems .. 

This very much needs to be done! Last time I tried to run it it consumed 100% resources on my NAS. 

This will be a great addition to the emby plugin library! Can't wait, is there any idea when this will be released as a working plugin? 

[crusher11 858]

It's not currently possible to release as a working plugin.

[rbjtech 4316]

I hope I'm not speaking out of place for @chef & @samuelqwe but I believe we are at the following stages for the implementation.

Note ETA's are just that, estimates, these guys are not full time Developers for emby, but progress has been rapid since the first proof of concept only a couple of weeks ago. 

 

#	Item	Description	Stage	Enhancements	Needs Core Updates ?	ETA
1	FingerPrint(Chroma)	Creates a template based on a percentage of the Audio runtime.	Ready	More efficient use of dB but good enough for release when used with scheduler	No	Complete
2	Detection	Detects the 'repeating Audio' vs other episodes and calculates the IntroStart and IntroEnd	Ready	Possibly more accuracy but certainly good enough at the moment with 90-95%	No	Complete
3	Schedule	Due to the Detection needing the FP database, a schedule needs to be run to ensure the data is available.	In Progress	Working now but needs manual config	No	~ 1 Week
4	Scaling	Depending on library size, these tasks can take days, even weeks - so it's important it's broken into manageable chunks.	In Progress		No	~ 1 Week
5	Manual Skip	The IntroStart/IntroEnd can be added into the existing chapter points without needing Core development work	Design		No	~ 2 Weeks
6	IntroSkip - Manual	Detection of an 'IntroStart/IntroEnd (in emby Chapters?) should bring a 'Skip Intro?' button to the front of the playback screen.	Concept		Yes	?
7	IntroSkip - Auto	Assuming accuracy is good enough, then #6 could also be automatic if given the option.	Concept		Yes	?

 

Fingers crossed the Core development aspects deliver on this as well but given the interest, I believe this is reasonably high on the list but of course @Luke and @ebr will need to comment on that. 

edit - @Abobader - That's an interesting view when using Rich Text from an Excel table, I though it would use a scrollbar like it did when I use the editor ..  ?

Edited September 5, 2021 by rbjtech