Completed Show Intro Skip Option

[sydlexius 240]

4 minutes ago, datanet said:

They only compare within a season so it's not a factorial calculation of all the episodes...just a factorial of the number of episodes within a season.

I'm aware of that.  And as I mentioned in my message, there are 130 episodes in season 1 of Mr. Rogers'.  I have some shows like "Smarter Everyday" that have hundreds of episodes in a single season.

[sydlexius 240]

9 minutes ago, datanet said:

They only compare within a season so it's not a factorial calculation of all the episodes...just a factorial of the number of episodes within a season.

Something else to be aware of is that big comparisons like this seem to need a TON of memory.  Here's a snapshot of my "top" screen.  Notice the Swap + reserved RAM that EmbyServer requires.  I've had the container go OOM on me several times during the scan.

Edited January 30, 2022 by sydlexius
fixed screenshot...again.

[datanet 120]

1 hour ago, sydlexius said:

I'm aware of that.  And as I mentioned in my message, there are 130 episodes in season 1 of Mr. Rogers'.  I have some shows like "Smarter Everyday" that have hundreds of episodes in a single season.

My apologies, I did overlook that you pointed out there were 130 episodes in a single season.  But I still do not believe they compare each episode to every other episode in the season.  I know they've discussed the logic they're using in previous posts (and I'm too tired at 2:50am to try to search for it) but I am fairly confident that's not the behavior I've witnessed while monitoring my disk i/o during an initial scan.  I very well may be wrong though.

The memory utilization is peculiar though.  I'm not sure I've ever seen or heard of Emby consuming anywhere near that much memory.  What version of the intro skip dll do you have installed? Also, what is your typical baseline memory utilization after a restart of Emby? 

[WahrerNemesis 0]

Does the plugin have a "Skip Intro"-Button? (because it doesn't appear)

And is the bug already known that on the Apple TV when Auto Skip is activated, the player is closed and the series continues to run in the background?

Edited January 30, 2022 by WahrerNemesis

[chef 3745]

7 hours ago, sydlexius said:

Something else to be aware of is that big comparisons like this seem to need a TON of memory.  Here's a snapshot of my "top" screen.  Notice the Swap + reserved RAM that EmbyServer requires.  I've had the container go OOM on me several times during the scan.

That is something we need to fix. 

The more a season has, the bigger the memory will grow. It starts off only using minimal memory, but as we memoize fingerprint data, it will get bigger.

When scanning seasons, we can clear the fingerprint data from memory as we move through them.

You definitely do not want to load an entire series worth of fingerprint data at once, in memory. That is a bad idea.

Mr Rogers Neighborhood causes an interesting issue because the season is pretty much the entire series.

It is a case we haven't come across yet. 

We'll have to handle shows like this differently.

In the meantime, it would be best to add our good friend Mr. Rodgers to the exclude list for the plugin. 

 

 

Edited January 30, 2022 by chef

[catzerino 0]

I just stumbled upon this plugin and this is one of the main reasons I didn't fully convert to emby.

2 Problems I am having right now:

Correlated console error:

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'includes')
    at configurationpage?name=AutoSkipConfigurationPageJS&v=4.6.7.0:103

and second the following error I am getting when trying to start the Episode Title Sequence Detection:

The resulting log:

Spoiler

2022-01-30 14:14:44.744 Info Intro Skip: FINGERPRINT: The Rising of the Shield Hero - S:1 - E:25 complete - 62 ms.
2022-01-30 14:14:44.744 Info Intro Skip: FINGERPRINT: Chromaprint Task Complete
2022-01-30 14:14:44.773 Info Intro Skip: DETECTION: Beginning Title Sequence Task
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.791 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.792 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.792 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.793 Warn Intro Skip: Input string was not in a correct format.

 

[chef 3745]

6 minutes ago, catzerino said:

I just stumbled upon this plugin and this is one of the main reasons I didn't fully convert to emby.

2 Problems I am having right now:

Correlated console error:

Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'includes')
    at configurationpage?name=AutoSkipConfigurationPageJS&v=4.6.7.0:103

and second the following error I am getting when trying to start the Episode Title Sequence Detection:

The resulting log:

  Hide contents

2022-01-30 14:14:44.744 Info Intro Skip: FINGERPRINT: The Rising of the Shield Hero - S:1 - E:25 complete - 62 ms.
2022-01-30 14:14:44.744 Info Intro Skip: FINGERPRINT: Chromaprint Task Complete
2022-01-30 14:14:44.773 Info Intro Skip: DETECTION: Beginning Title Sequence Task
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.789 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.790 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.791 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.792 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.792 Warn Intro Skip: Input string was not in a correct format.
2022-01-30 14:14:44.793 Warn Intro Skip: Input string was not in a correct format.

 

Hi! Welcome! 

We have some fixes for the user list.

I am about to post another update dll shortly.

Can you pm me a full log please.

Are you running one of the compatible version of emby?

 

 

Edited January 30, 2022 by chef

[chef 3745]

4 hours ago, WahrerNemesis said:

Does the plugin have a "Skip Intro"-Button? (because it doesn't appear)

And is the bug already known that on the Apple TV when Auto Skip is activated, the player is closed and the series continues to run in the background?

A button is coming. It is in the works!

For now, we have an auto skip feature which you can turn on if you want to. We also incorporate in to chapters.

 

[chef 3745]

Version 0.0.1.83

IntroSkip_v0.0.1.83.zip

 

This version has taken into consideration some of the bugs that have been mentioned on this thread, as well as some we have found while testing.

This shouldn't need anymore then clearing of the browser cache, to load the latest javascript.

We'll start working on  the chapter code next. Very Exciting!  

If something comes up, let us know so we can fix it!  

TODO's:

• Series that have all their episodes in season 1 need to be handled differently. How? I don't know yet. 
• When Chromaprint is not available in ffmpeg, IntroSkip needs to better alerts the logs, make sure the alert is shown in the web config, and stop iterating the library.

 

 

If you have TV Shows in your library that place entire series into one season (ex. Mr.Rogers Neighborhood) they need to be added to the exclude list in the "Advanced" settings of the plugin, until they can be handled properly. Thank you @sydlexius for bring this to our attention.

 

The one thing that didn't end up changing was Auto Skip.  We tried a bunch of different way to handle Frame Rate Switching, and monitoring the session, but it was  buggy. Sorry @datanet 

 

Auto skip is a large undertaking in of itself. It is a very cool feature.

When Intro Skip has it's next major released, we can extract Auto Skip into its own plugin.  Auto Skip will be open source.

So anyone who wants to make a pull request can do it, and we can work together to make it really awesome, and work smoothly with all devices steaming all sorts of media.

 

Looking forward to feedback.

 

 

 

 

[Gecko 61]

Hello Chef,

Should I worried about this error message ? (I'm still on the previous version, not the one you just released). It seems that it wanted to run something but the video preview thumbnail was already running in the background and may have blocked it? In the logs, I have a " ---> Busy: database is locked" in the trace at the exact same time.

 

Also, I fully analyzed my tv show episodes, it took something like 24 hours to complete the fingerprints (my bottleneck was indeed the gigabit between Emby and the nas) and 24 hours to do the intro detections (using only ⅔ of my cpu cores but each at 100%). 

Quick questions :

- Where are the fingerprints located when created ? Are they deleted ? If yes, when ?
- If I migrate Emby, what do I need to copy in order to avoid repeating the full process ? I asked to create a new chapter entry in the settings but besides that, I don't know what's save and where for each episode in the end. 
- And lastly, why the process does not process one season completely (fp -> I/o detect -> thumbnail)  at a time instead of all fingerprints then all intro detection then ... I would have save a lot of time by chaining the process in background like FP working for tv show #2 while intro detection was running for tv show #1 and so on...

 

I also quickly compare for some tv shows with short intro, Plex does not bother tagging them while your plugin tag them properly and skip it. Wonderful!  

[Spegeli 18]

@chef would be nice if we can get a "Remove All Series" or a "Clear List" Option under Advanced (for the exclude from scans function).

 

The double entry (when multiple folders for the same serie) in the list seems to be fixed.

//Edit: but when i See it right, in the execlude list, they still appear double. 

And when i see it right, excluded items no longer show up in the list unter Intros (except they alrdy got skip datas). 

nice

 

tomorrow / later, i will test the rest of the new version  

Edited January 31, 2022 by Spegeli

[chef 3745]

1 hour ago, Spegeli said:

@chef would be nice if we can get a "Remove All Series" or a "Clear List" Option under Advanced (for the exclude from scans function).

 

The double entry (when multiple folders for the same serie) in the list seems to be fixed.

//Edit: but when i See it right, in the execlude list, they still appear double. 

And when i see it right, excluded items no longer show up in the list unter Intros (except they alrdy got skip datas). 

nice

 

tomorrow / later, i will test the rest of the new version  

Yes. A remove all button on the exclude list has actually been brought to my attention earlier today.

I will make sure that a button exists, and create a release shortly.

 

I'll try to fix duplicates in the series list. 

There are some side effects of grabbing one of a duplicate series. Mainly, each entry has it's own ID, and we might request the wrong one. 

 

 

Edited January 31, 2022 by chef

[pcorderog205 7]

When will it be available on linux servers?

Edited January 31, 2022 by pcorderog205

[samuelqwe 313]

22 minutes ago, pcorderog205 said:

When will it be available on linux servers?

Linux servers were one of the first platforms to be compatible with the plugin actually

So if you’re on Linux, you can likely already try the plugin.

Latest version is found here: 

 

[chef 3745]

7 hours ago, nSGecko said:

Hello Chef,

Should I worried about this error message ? (I'm still on the previous version, not the one you just released). It seems that it wanted to run something but the video preview thumbnail was already running in the background and may have blocked it? In the logs, I have a " ---> Busy: database is locked" in the trace at the exact same time.

 

Also, I fully analyzed my tv show episodes, it took something like 24 hours to complete the fingerprints (my bottleneck was indeed the gigabit between Emby and the nas) and 24 hours to do the intro detections (using only ⅔ of my cpu cores but each at 100%). 

Quick questions :

- Where are the fingerprints located when created ? Are they deleted ? If yes, when ?
- If I migrate Emby, what do I need to copy in order to avoid repeating the full process ? I asked to create a new chapter entry in the settings but besides that, I don't know what's save and where for each episode in the end. 
- And lastly, why the process does not process one season completely (fp -> I/o detect -> thumbnail)  at a time instead of all fingerprints then all intro detection then ... I would have save a lot of time by chaining the process in background like FP working for tv show #2 while intro detection was running for tv show #1 and so on...

 

I also quickly compare for some tv shows with short intro, Plex does not bother tagging them while your plugin tag them properly and skip it. Wonderful!  

Hi! Thanks for taking the time to get this up and running.

Quote

Where are the fingerprints located when created ? Are they deleted ? If yes, when ?:

The fingerprint Binary files are saved in the "Plugins/data/intro_encoding" folder. No they are not deleted unless you reset the season in the configuration, and select the button to do so

 

 

Quote

If I migrate Emby, what do I need to copy in order to avoid repeating the full process ? I asked to create a new chapter entry in the settings but besides that, I don't know what's save and where for each episode in the end. :

 If you migrate Emby to a new installation, and it changes the library.db in a way that it creates new InternalIds for your media, then (currently) the process will have to be run again.

However this is not the way that it will stay. We are currently working on integrating specifically with embys chapter API. That means that we will save the data in a way that migrating to a new install  will be "super easy - barely and inconvenience", while also keeping the data proprietary to Emby. 

This is also why we decided to keep the fingerprint files. That means that any beta user who processed their library already will be ahead of the game as far as having compiled the data required for future releases.

 

Quote

And lastly, why the process does not process one season completely (fp -> I/o detect -> thumbnail)  at a time instead of all fingerprints then all intro detection then ... I would have save a lot of time by chaining the process in background like FP working for tv show #2 while intro detection was running for tv show #1 and so on...

 

The reason the processes have to be run in succession is why you got that SQLite error posted above.  

While we are writing to the database table during fingerprinting/Detection, it gets a write lock on it.

If the other process tries to access it, we'll throw that error.

In fact it is a bug, and the user should  only see it in the log. I will fix that before we put up the next releases with some fixes. Thank you for bring that to my attention. 

 

Another thing to keep in mind is that there are some power users out there that have massive libraries, and fingerprinting alone could take a very long time. It is expensive on the I/O, but,  so is the Detection Task. Flipping back and forth between the two methods, and using FFMPEG as many times as we do, started to get complicated. 

Breaking the tasks into two allowed us to keep the code simple, and also the process simple as well. 

In the future we will not be using that database, and instead we'll be using the Chapter.db/API. It will makes things a lot better, no SQLite errors.  

 

Quote

I also quickly compare for some tv shows with short intro, Plex does not bother tagging them while your plugin tag them properly and skip it. Wonderful!  

Thanks so much for the feedback! That's really great to hear. Sounds like you're happy so far with how the plugin has come along.

Edited January 31, 2022 by chef

[Mickroz 2]

Hi all, im not seeing the messages?

Did i forgot the activate something?

All the scheduled tasks ran, but im not seeing any option to skip any title or end sequence during play?

[samuelqwe 313]

17 minutes ago, Mickroz said:

Hi all, im not seeing the messages?

Did i forgot the activate something?

All the scheduled tasks ran, but im not seeing any option to skip any title or end sequence during play?

There is no intro skip button yet. Though, it will come eventually.

For now, the plugin can only skip the intros automatically and this must be enabled in the settings of the plugin.

[rbjtech 4222]

33 minutes ago, Mickroz said:

Hi all, im not seeing the messages?

Did i forgot the activate something?

All the scheduled tasks ran, but im not seeing any option to skip any title or end sequence during play?

It's a work in progress - but this wiki should help you out :

https://github.com/chefbennyj1/Emby.IntroSkip.Release/wiki

[Mickroz 2]

13 minutes ago, samuelqwe said:

There is no intro skip button yet. Though, it will come eventually.

For now, the plugin can only skip the intros automatically and this must be enabled in the settings of the plugin.

Ah thanks, was beginning to think i missed something in the wiki.

[Mickroz 2]

6 minutes ago, rbjtech said:

It's a work in progress - but this wiki should help you out :

https://github.com/chefbennyj1/Emby.IntroSkip.Release/wiki

I could activate chapters then and skip to the next chapter?

[rbjtech 4222]

19 minutes ago, Mickroz said:

I could activate chapters then and skip to the next chapter?

If you want only manual control - then at this time, yes, that is the only way to skip.  The button is in progress, but we need core Dev development to accommodate the changes we need.

But the Auto-Skip function works well - and will automatically skip the 'Intro' and jump to the next episode when the 'End Credits' roll - this can be set per user, so maybe setup a test user and have a play.

[stevehoffman203 16]

Just saw the updated version.  Are there any special steps that I need to take once I copy in the new .dll?

[chef 3745]

4 hours ago, stevehoffman203 said:

Just saw the updated version.  Are there any special steps that I need to take once I copy in the new .dll?

Should be okay with only clearing browser cache.

 

There is going to be one more update with some fixes. I'll post it tonight. 

[Tangsgod 22]

Great!

[Gecko 61]

On 1/31/2022 at 6:18 AM, chef said:

A very long and detailed response from @chef about how the plugin works

Thanks for your response, I better understand the choice you made. In fact, there should be a master database somewhere that keep all those intros and endings timings so that it's computed once (for all). That would be the dream. Isn't there something like that out there, like a opensubittle for chapters ?

And just for the record, I migrated Emby to another server yesterday and by simply copying titlesequence.db and library.db to the new server (after restoring the last backup of the source server of course), everything was working. Piece of Cake!