Completed Show Intro Skip Option

[rbjtech 4270]

9 minutes ago, crusher11 said:

I'm still on 0.0.1.9. Kinda lost touch with this.

Is it worth updating at this point? What steps do I need to take?

lol - we are @ 0.0.1.89 now - so one or two changes have been made .. 

Let me speak to @chef and we'll get the latest beta on here.

You'll need to just delete the plugin and replace with the new DLL and restart emby - and as your version is so old, I think your best bet is to delete the titlesequence.db file as well and start as if it was new.

 

[crusher11 851]

Yeah, I know there were a couple of updates that required a few extra steps, and then I lost track of which updates they were and what those steps were and just kinda forgot about it.

[chef 3745]

I will post a new version with some fixes here right after work.

Sorry for the wait.

 

Crusher good to see you here to check out some updates.

[tmurd87 8]

Definity a must have for any home server! Thanks for this

[seanbuff 840]

9 hours ago, rbjtech said:

Just a minor cosmetic thing that otherwise spoils an awesome looking screen design

Hey IntroSkip team, I finally got around to giving this a go starting at v0.0.1.83 and wow what an awesome plugin! Appreciate the work you all have put into getting it to this point.

Just on the topic of "minor cosmetic" - for those of us who aren't using the default light theme and a different accent colour. This text was a little confusing to me at first... I was looking everywhere for some "Green" 

Even the text itself seems to suggest that a user must make changes on this screen in order for the tool to work - at least that's what I thought when I used it for the first time.

Functionally I have nothing else to comment on as it all works perfectly for me, great work guys! Look forward to the updates

Edited February 9, 2022 by seanbuff
clarity

[sydlexius 241]

12 minutes ago, seanbuff said:

Hey IntroSkip team, I finally got around to giving this a go starting at v0.0.1.83 and wow what an awesome plugin! Appreciate the work you all have put into getting it to this point.

Just on the topic of "minor cosmetic" - for those of us who aren't using the default light theme and a different accent colour. This text was a little confusing to me at first... I was looking everywhere for some "Green" 

Even the text itself seems to suggest that a user must make changes on this screen in order for the tool to work - at least that's what I thought when I used it for the first time.

Functionally I have nothing else to comment on as it all works perfectly for me, great work guys! Look forward to the updates

FYI, this bug was fixed in a subsequent.  Please upgrade to v0.1.87, which can be found here.

[datanet 120]

Since we're discussing minor cosmetic issues...here's another...

I'm running 1.87.

[chef 3745]

 

0.0.1.90

• Cosmetic fixes - Empty logo images show broken img link, Button text is lost in dark UI on dashboard.
• Globalization\Localization in the Plugin config UI (Did what we could using what emby already offered for localization)
• Apple TV message bug... fixed?? Most likely... We currently don't have an apple TV to make sure. 
• 'HasChromaprint' endpoint logic using 'muxer' command args instead of 'version' to find the data

Please clear browsing data  

Beta Released on the Wiki Github page under releases:

https://github.com/chefbennyj1/Emby.IntroSkip.Release/releases/download/beta_intro_skip_0.0.1.90/IntroSkip_v0.0.1.90.zip

 

future maintenance for V2 beta:

• When a series is found to have only one season with the entire series in it:

1. Automatically add it to Exclude list (easiest), 
2. Or split the series up into pseudo seasons for detection (most likely easy... just needs more time to code, and test).

 

• Possibly have to look at Apple TV messaging again if the current fix (which looks at both Client and Device name being an "Apple" product) doesn't work. 

 

Edited February 10, 2022 by chef

[my95z34 3]

I've been watching this thread for some time now, but finally got around to installing it.  I have quite a large library, somewhere in the neighborhood of 25TB between TV and Movies.  (not sure exactly how much for each tho, lol)  I've been adding shows slowly, a few at a time.  But, occasionally it'll completely tank my Emby instance.  I'm running Emby in Unraid on a pretty beefy system.  It has Dual  Xeon E5-2660 v3's (20 cores, 40 threads) and 64 gigs of DDR4.  Normal system usage has my ram sitting at roughly 30% throughout the day.  (Many docker apps, and 2 vm's) but when running a scan it just EATS ram.  Currently sitting at 32 gigs and climbing rapidly.  Once the usage on the entire system gets to about 90-95% my other dockers start to crash too.  I generally have to kill the Emby instance to get it to respond again.

 

I believe it has something to do with shows that have a large number of episodes in a single season, but I'm not entirely sure.  Any input would be appreciated!

(up to 36 gigs now, lmao)

[chef 3745]

2 minutes ago, my95z34 said:

I've been watching this thread for some time now, but finally got around to installing it.  I have quite a large library, somewhere in the neighborhood of 25TB between TV and Movies.  (not sure exactly how much for each tho, lol)  I've been adding shows slowly, a few at a time.  But, occasionally it'll completely tank my Emby instance.  I'm running Emby in Unraid on a pretty beefy system.  It has Dual  Xeon E5-2660 v3's (20 cores, 40 threads) and 64 gigs of DDR4.  Normal system usage has my ram sitting at roughly 30% throughout the day.  (Many docker apps, and 2 vm's) but when running a scan it just EATS ram.  Currently sitting at 32 gigs and climbing rapidly.  Once the usage on the entire system gets to about 90-95% my other dockers start to crash too.  I generally have to kill the Emby instance to get it to respond again.

 

I believe it has something to do with shows that have a large number of episodes in a single season, but I'm not entirely sure.  Any input would be appreciated!

(up to 36 gigs now, lmao)

Hi!

Are you currently fingerprinting your tv collection, or are you on to detection?

[my95z34 3]

2 minutes ago, chef said:

Hi!

Are you currently fingerprinting your tv collection, or are you on to detection?

Sorry, should have been more clear.  It's detection now.

[chef 3745]

Just now, my95z34 said:

Sorry, should have been more clear.  It's detection now.

Perhaps you could pm me some logs. 

You mentioned you were adding series a bit at a time. Smart!

Out of curiosity, how many episodes are in the largest of your seasons? When you start to see resource spikes?

 

[my95z34 3]

3 minutes ago, chef said:

Perhaps you could pm me some logs. 

You mentioned you were adding series a bit at a time. Smart!

Out of curiosity, how many episodes are in the largest of your seasons? When you start to see resource spikes?

 

So, I was actually pouring through the recent log.  It seems that the detection gets progressively slower.  It'll start at ~1000ms.  But when I just killed the Emby instance, it was up to ~32,000ms.  Not sure if that's a cause, or a symptom, lol.

 

Just to clarify, in the advanced screen, the stuff in the dropdown list is the stuff being scanned, correct?  And the scrolling list towards the bottom is the stuff being skipped?  The language is a little ambiguous.

 

One show I had to set up to not scan is Boruto: Naruto Next Generations.  It's listed as 1 season with 229 episodes.

 

When it tanked this time it was scanning Detroiters, Dexter, and DC Legends of Tomorrow.  None of which have very large seasons.  I had it set to scan 4 series at a time, but have now lowered it to just 1 to see how it does.

 

I'll shoot you a PM real quick.

[gihayes 34]

Hi My95z34,

I've been following the progress of the plugin an and am looking forward to using it. When will it be available to us using the current version of Emby?

[crusher11 851]

So what do I need to do with my super-old version to update?

[chef 3745]

10 minutes ago, my95z34 said:

 

One show I had to set up to not scan is Boruto: Naruto Next Generations.  It's listed as 1 season with 229 episodes.

 

This right here.

This is what we have dubbed the "Mr. Rogers Neighborhood" problem.

 Mr. Rogers had an entire series worth of shows in only one season.

This is going to cause paging issues because we hold results in memory, and would be impossible to run detection on, in the current state of the plugin.

O(229^2) = system crash!

 

We are going to have to do a record check before attempting to process series, and make sure we are not about to process a series lime this.

From there we'll have to create pseudo seasons in order to get through the series.

 

If you leave that series in the exclude list (for now), and make sure that your concurrent detection settings (under advanced settings) is set to 4 (or on your big machine, you could override by typing a value in, no higher then 6). 

See if you are able to detect more of your library.

Any other series that have more then 25 episodes in one season shoukd be left in the exclude list, untill we can write appropriate code to handle them.

I'll look through the logs (thank you for that), and I will start figuring out the Mr  Rogers problem right away.

[my95z34 3]

27 minutes ago, my95z34 said:

Just to clarify, in the advanced screen, the stuff in the dropdown list is the stuff being scanned, correct?  And the scrolling list towards the bottom is the stuff being skipped?  The language is a little ambiguous.

 

4 minutes ago, chef said:

This right here.

This is what we have dubbed the "Mr. Rogers Neighborhood" problem.

 Mr. Rogers had an entire series worth of shows in only one season.

This is going to cause paging issues because we hold results in memory, and would be impossible to run detection on, in the current state of the plugin.

O(229^2) = system crash!

 

We are going to have to do a record check before attempting to process series, and make sure we are not about to process a series lime this.

From there we'll have to create pseudo seasons in order to get through the series.

 

If you leave that series in the exclude list (for now), and make sure that your concurrent detection settings (under advanced settings) is set to 4 (or on your big machine, you could override by typing a value in, no higher then 6). 

See if you are able to detect more of your library.

Any other series that have more then 25 episodes in one season shoukd be left in the exclude list, untill we can write appropriate code to handle them.

I'll look through the logs (thank you for that), and I will start figuring out the Mr  Rogers problem right away.

Could you confirm my understanding in the settings?

 

Also, yeah I've been avoiding those types of shows, but still getting this excessive memory usage.  Out of curiosity, is it storing all data from the current scan in memory until the scan finishes?  Or does it just store the data from whatever series/season it's currently working on?

[sydlexius 241]

2 minutes ago, chef said:

This right here.

This is what we have dubbed the "Mr. Rogers Neighborhood" problem.

 Mr. Rogers had an entire series worth of shows in only one season.

This is going to cause paging issues because we hold results in memory, and would be impossible to run detection on, in the current state of the plugin.

O(229^2) = system crash!

 

We are going to have to do a record check before attempting to process series, and make sure we are not about to process a series lime this.

From there we'll have to create pseudo seasons in order to get through the series.

 

If you leave that series in the exclude list (for now), and make sure that your concurrent detection settings (under advanced settings) is set to 4 (or on your big machine, you could override by typing a value in, no higher then 6). 

See if you are able to detect more of your library.

Any other series that have more then 25 episodes in one season shoukd be left in the exclude list, untill we can write appropriate code to handle them.

I'll look through the logs (thank you for that), and I will start figuring out the Mr  Rogers problem right away.

@my95z34It looks like we have similar systems (Though I'm rolling with dual E5-2630L v4's to reduce power consumption), and like you I've had my Unraid server kill off docker containers due to OOM.  This is even after adding 32GB swap to my 64GB of RAM!

@chef Your recommendation to keep it under 25 is fairly conservative, but a safe approach.  I've had other kids shows that have 50-70 episodes go through without trashing my server...I think it's because the episodes are only 6-10 minutes in length, so the size of the fingerprint data is most likely much smaller than a typical 20-24 minute show.

[my95z34 3]

5 minutes ago, sydlexius said:

@my95z34It looks like we have similar systems (Though I'm rolling with dual E5-2630L v4's to reduce power consumption), and like you I've had my Unraid server kill off docker containers due to OOM.  This is even after adding 32GB swap to my 64GB of RAM!

@chef Your recommendation to keep it under 25 is fairly conservative, but a safe approach.  I've had other kids shows that have 50-70 episodes go through without trashing my server...I think it's because the episodes are only 6-10 minutes in length, so the size of the fingerprint data is most likely much smaller than a typical 20-24 minute show.

Glad to know I'm not the only one, lol.

 

//edit// I didn't even think about swap space.  I just installed and set up that plugin, just in case. lol

Edited February 10, 2022 by my95z34

[chef 3745]

1 minute ago, my95z34 said:

 

Could you confirm my understanding in the settings?

 

Also, yeah I've been avoiding those types of shows, but still getting this excessive memory usage.  Out of curiosity, is it storing all data from the current scan in memory until the scan finishes?  Or does it just store the data from whatever series/season it's currently working on?

Right the items in the drop-down are being scanned. 

Series added to the list will be ignored.

We will store data per season. After the season has been processed the memory is freed for garbage collection, and the data is saved to the database.

We do however (although it is about to change as we integrate with emby) store every comparison we make (which is every episode compared to every episode... Minus itself), in memory. We also gradually chunk out the finger print binaries in to memory as well.

 

 

 

[chef 3745]

16 minutes ago, sydlexius said:

@my95z34It looks like we have similar systems (Though I'm rolling with dual E5-2630L v4's to reduce power consumption), and like you I've had my Unraid server kill off docker containers due to OOM.  This is even after adding 32GB swap to my 64GB of RAM!

@chef Your recommendation to keep it under 25 is fairly conservative, but a safe approach.  I've had other kids shows that have 50-70 episodes go through without trashing my server...I think it's because the episodes are only 6-10 minutes in length, so the size of the fingerprint data is most likely much smaller than a typical 20-24 minute show.

It is most definitely the fingerprint data (like you say) that is filling the memory.

That seems to be where we need to free up space during detection. I guess it would be  easiest to flush the entire result cache, and fingerprints every 25 episodes. 

 

 

 

[sydlexius 241]

1 minute ago, chef said:

It is most definitely the fingerprint data (like you say) that is filling the memory.

That seems to be where we need to free up space during detection. I guess it would be  easiest to flush the entire result cache, and fingerprints every 25 episodes. 

 

 

 

That seems like it would only be useful if you implement comparing a max of 25 eps in a (virtual) season.

[chef 3745]

1 hour ago, crusher11 said:

So what do I need to do with my super-old version to update?

Hey crusher11, remove the db, load the plugin, and restart.

I don't think there is much else.

[chef 3745]

29 minutes ago, sydlexius said:

That seems like it would only be useful if you implement comparing a max of 25 eps in a (virtual) season.

Right, and only do this if the season itself has a record count larger then  X number of episodes.

Edit: Grab 25 episodes, which result in a possible 625 results. Then grab the next 25, and so on.

Edited February 10, 2022 by chef

[crusher11 851]

10 minutes ago, chef said:

Hey crusher11, remove the db, load the plugin, and restart.

I don't think there is much else.

What DB do I remove? Do I need to shut down Emby to do that?

I, too, have a lot of shows that would be well over 25 episodes per season. Weekly wrestling at 50+ episodes per year, sports, that sort of thing.