Completed Show Intro Skip Option

[diogee 17]

So a somewhat interesting fact, it loads on my android devices completely fine now. Takes maybe about 30 seconds to load the list and maybe about 10-15 to change tv show selection but it loads and is somewhat useable
But all chromium based browsers on any of my desktops still crash due to memory errors lol Was hoping the double processing fix would stop that but sadly not

Firefox still reliably loads the list and still in about 30 seconds as well but no more double processing taking that to 60 seconds! 

For those mentioning how fast it is with ~600 shows, yeah, I realized just how fast it can be during my testing! I experienced this for myself just recently, I was just so used to it crawling along since I first installed it ages ago that it blew me away how fast it was the moment I tried seeing what would happen if I removed one of my libraries!

But 600 is a very big gap from my 2700 lol
The issues I am having don't affect people until excessively large amounts of shows come into play! If I had to guess, I'd say somewhere between 1500 to 2000+ shows is where performance starts to really shit itself

Edited February 11, 2022 by diogee

[Painkiller8818 203]

2 minutes ago, diogee said:

So a somewhat interesting fact, it loads on my android devices completely fine now. Takes maybe about 30 seconds to load the list and maybe about 10-15 to change tv show selection but it loads and is pretty useable but all chromium based browsers on any of my desktops still crash due to memory errors lol Was hoping the double processing fix would stop that but sadly not

Firefox still reliably loads the list and still in about 30 seconds as well but no more double processing taking that to 60 seconds! 

For those mentioning how fast it is with ~600 shows, yeah, I realized just how fast it can be during my testing! I experienced this for myself just recently, I was just so used to it crawling along since I first installed it ages ago that it blew me away how fast it was the moment I tried seeing what would happen if I removed one of my libraries!

But 600 is a very big gap from my 2700 lol
The issues I am having don't affect people until excessively large amounts of shows come into play! If I had to guess, I'd say somewhere between 1500 to 2000+ shows is where performance starts to really shit itself

there is one more test you can do on firefox just for fun to check something.

Can you create a new Profile in Firefox, select it and log into your emby server and try it again? I am very interested if this would speed up things (you don't need to delete your existing profile) 

[chef 3746]

Hey check this out! I've done it.

We went from making consecutive request for each sequence in the database (a lot), to only making 2 requests  ... where is the {mind-blown} emoji?... ... nope that's not it...oh well....

 

This is how the page loads now:

1. One call to get the episode objects that correlate to the sequences
2. The Series object they all belong to

BOOM!

[diogee 17]

8 minutes ago, Painkiller8818 said:

Can you create a new Profile in Firefox, select it and log into your emby server and try it again? I am very interested if this would speed up things (you don't need to delete your existing profile) 

Good point, I've been doing this with Brave/Vivaldi but hadn't tried with firefox!
The results are the same, around 30 seconds for it to load even in a fresh profile

 

2 minutes ago, chef said:

where is the {mind-blown} emoji?

!!! (It's like exploding head or something?)

Edited February 11, 2022 by diogee

[chef 3746]

@diogee would you want to try this version? See how the loading times are now that we don't have to  wait for consecutive request.

 

 

I don't think I broke anything in the UI with the refactor... 

 

 

This is how the code should be written. When we were writing the plugin initially, we were trying to get stuff working, but now I can see there is some areas that need refactoring. 

 

Edited February 12, 2022 by chef

[chef 3746]

I wonder if it is possible to send back multiple images when doing thumb nail extraction, instead of requesting each one, we could try requesting them all at once (there it is).

Maybe... I don't want to break our shiny new toy. 

Edited February 11, 2022 by chef

[diogee 17]

16 minutes ago, chef said:

@diogee would you want to try this version? See how the loading times are now that we don't have to  wait for consecutive request.

 

Oh it's improved a bit! Around about 25 seconds to load the list now lol

Which to be fair, 25 seconds to load a list of 2700 tv shows, not so terrible (honestly I'm not expecting miracles there, I kind of even expected some kind of processing wait for my sized list in general, definitely wouldn't expect it to be instant!), it just becomes a problem when you are trying to add/remove things from exclusions really as each time you add/remove, the list refreshes and takes the full load time to do so again.
Which also to be fair, this probably isn't something that will be done often given that at this point I'm really only trying to exclude those "large amounts of episodes per season" stuff that seems to be causing problems (can't remember the term you dubbed for it a few pages back )

The Activity page doesn't really have the same issue, once that list is loaded it stays loaded even while switching shows, the only time that page specifically is a problem, is when I want to change something in some other tab. Ie, i'd have to wait for the tv show list to load before changing to chapter markers or advanced etc

I don't know if chromium just hates me in general or what but those browsers still oom crash

Edited February 11, 2022 by diogee

[rbjtech 4289]

10 minutes ago, diogee said:

Oh it's improved a bit! Around about 25 seconds to load the list now lol

Which to be fair, 25 seconds to load a list of 2700 tv shows, not so terrible (honestly I'm not expecting miracles there, I kind of even expected some kind of processing wait for my sized list in general, definitely wouldn't expect it to be instant!), it just becomes a problem when you are trying to add/remove things from exclusions really as each time you add/remove, the list refreshes and takes the full load time to do so again.
Which also to be fair, this probably isn't something that will be done often given that at this point I'm really only trying to exclude those "large amounts of episodes per season" stuff that seems to be causing problems (can't remember the term you dubbed for it a few pages back )

The Activity page doesn't really have the same issue, once that list is loaded it stays loaded even while switching shows, the only time that page specifically is a problem, is when I want to change something in some other tab. Ie, i'd have to wait for the tv show list to load before changing to chapter markers or advanced etc

I don't know if chromium just hates me in general or what but those browsers still oom crash

btw - if you KNOW the Series Id of the items you want excluded, then you can just add them manually in the XML ..

..\plugin\configurations\IntroSkip.xml

<IgnoredList>
    <long>709906</long>
    <long>709907</long>
    <long>709813</long>
    <long>1257225</long>
    <long>1257142</long>
  </IgnoredList>

as examples ..

Then restart the plugin.

 

[rbjtech 4289]

16 minutes ago, chef said:

I wonder if it is possible to send back multiple images when doing thumb nail extraction, instead of requesting each one, we could try requesting them all at once (there it is).

Maybe... I don't want to break our shiny new toy. 

Leave it for v3 @chef - we gotta have some wizzy bits left for that release .. 

[Painkiller8818 203]

any chance we get emby to use http2 protocol? This would also speed things up and send multiple requests in one

[diogee 17]

24 minutes ago, rbjtech said:

btw - if you KNOW the Series Id of the items you want excluded, then you can just add them manually in the XML ..

 

See, this would have been the smart thing to do... Hell instead of spending 2 weeks on trying to work out the problems myself, a sensible person would probably have come to the forums for quick little workarounds like these Didn't even think of checking the xml file to see what I could play with... I'm just going to blame the long work days and lack of sleep!

As for chromium, I think it's hitting some tab memory limit wall at 4GB, thought that limit was larger but it's something I'll look into some more

Edit: So it's definitely a 4GB limit per tab in chromium still from what I can gather which is actually kind of funny given that firefox peaks at about 2GB to load the full list
Garbage collection or something must be a bit more aggressive in firefox maybe? I don't know but for now I'm just going to exclude all chromium browsers from further testing unless I have a reason to, if I need to fiddle with IntroSkip i'll just load up firefox haha

Edited February 11, 2022 by diogee

[sydlexius 243]

6 minutes ago, Painkiller8818 said:

any chance we get emby to use http2 protocol? This would also speed things up and send multiple requests in one

With few exceptions, Emby has supported HTTP/2 since v4.4.  Make sure you're using HTTPS for your connections.

[rbjtech 4289]

39 minutes ago, chef said:

would you want to try this version? See how the loading times are now that we don't have to  wait for consecutive request.

 

.. so for me .94 is noticeably faster on PC (~1 sec now, previously was ~2 sec) but on Android it's still about ~5 secs.   I'm fine with that. 

If I choose a different tab in the plugin, then I appear to re-load the item list again - maybe this list can be cached rather than re-loaded each time ?   This would I guess make it bearable after the initial wait for a huge show list ?

[Painkiller8818 203]

4 minutes ago, sydlexius said:

With few exceptions, Emby has supported HTTP/2 since v4.4.  Make sure you're using HTTPS for your connections.

 

Ok got it, so only while testing i am connecting to the internal ip directly and checked the protocol

[slevin7 54]

I updatet to version 0.0.1.94.
But the Problem is, I See the "circle of wait forever" again and have this error "undefined is not an object (evaluating 'sequences[0].SeriesId')"
But what also surprises me a little, I have quite a lot of entries twice in the list. now, however, I do not know if the problem is not already somewhere else.
 

[chef 3746]

1 hour ago, slevin7 said:

I updatet to version 0.0.1.94.
But the Problem is, I See the "circle of wait forever" again and have this error "undefined is not an object (evaluating 'sequences[0].SeriesId')"
But what also surprises me a little, I have quite a lot of entries twice in the list. now, however, I do not know if the problem is not already somewhere else.
 

Okay, that is definitely a refactoring bug. I can fix that. 

You remembered to clear browsing data?

[slevin7 54]

13 minutes ago, chef said:

You remembered to clear browsing data?

I even used another browser 

[chef 3746]

2 hours ago, slevin7 said:

I updatet to version 0.0.1.94.
But the Problem is, I See the "circle of wait forever" again and have this error "undefined is not an object (evaluating 'sequences[0].SeriesId')"
But what also surprises me a little, I have quite a lot of entries twice in the list. now, however, I do not know if the problem is not already somewhere else.
 

I found it hard to reproduce this error.

I believe this happens when a series from the library has not been processed. 

The unprocessed series, is not in the exclude list, it just has yet to be processed.

 

Please try this and see if the issue is fixed.

IntroSkip_v0.0.1.95.zip

 

With regards to duplicate entries in the series select box. I haven't been able to reproduce it yet. But I am still trying.

 

[slevin7 54]

thank you, with the new version it works again. 
The duplicate entries, to be honest I never paid attention to when they started or if they were always there.

[chef 3746]

5 hours ago, rbjtech said:

.. so for me .94 is noticeably faster on PC (~1 sec now, previously was ~2 sec) but on Android it's still about ~5 secs.   I'm fine with that. 

If I choose a different tab in the plugin, then I appear to re-load the item list again - maybe this list can be cached rather than re-loaded each time ?   This would I guess make it bearable after the initial wait for a huge show list ?

If I scope the list of series globally, that should work. On it!

[diogee 17]

Totally going to be that guy again here.... I'm just trying to see where things can break on large libraries at this point
It's not possible to load the tv show list if too many items excluded

I have absolutely no idea where the limit is for that but I sure as hell hit it when I hit the button to add all series to the scan exclusion list!
No big deal as I just took the easy way out and reset the plugin, and I think in close to every use case this wouldn't happen unless people only want a handful of series scanned in a large library for whatever reason, so I'd say this is just more of a heads up really

The actual error I get is a 414 Request URI too long error. Peek at the console from edge shows:

Firefox actually specifically shows the error code as 414 request too long




I'm also noticing duplicates in my list but I've never actually paid attention to the list itself either so I'm also pretty useless in working out when they started occurring! I'll try a few of the older versions and see if I can spot which version it starts happening in

Edit: Now I can't reproduce it either so I think in my case it must have been cache shenanigans and one of those moments where I thought I had cleared the cache but didn't or only cleared one browsers cache but not the others while testing one of the updates last night I'll keep an eye on it and mention it if it occurs again

 

Edit 2: Magic number to start hitting the 414 error is somewhere between 900-1000 items in the exclusion list, pretty sure that's going to be pretty rare so maybe just a warning if the "add all series" will result in more than 500 shows in the exclusion list or something? I don't know, I feel like this one isn't something to focus dev time on really

Edited February 12, 2022 by diogee

[chef 3746]

6 hours ago, diogee said:

Totally going to be that guy again here.... I'm just trying to see where things can break on large libraries at this point
It's not possible to load the tv show list if too many items excluded

I have absolutely no idea where the limit is for that but I sure as hell hit it when I hit the button to add all series to the scan exclusion list!
No big deal as I just took the easy way out and reset the plugin, and I think in close to every use case this wouldn't happen unless people only want a handful of series scanned in a large library for whatever reason, so I'd say this is just more of a heads up really

The actual error I get is a 414 Request URI too long error. Peek at the console from edge shows:

Firefox actually specifically shows the error code as 414 request too long




I'm also noticing duplicates in my list but I've never actually paid attention to the list itself either so I'm also pretty useless in working out when they started occurring! I'll try a few of the older versions and see if I can spot which version it starts happening in

Edit: Now I can't reproduce it either so I think in my case it must have been cache shenanigans and one of those moments where I thought I had cleared the cache but didn't or only cleared one browsers cache but not the others while testing one of the updates last night I'll keep an eye on it and mention it if it occurs again

 

Edit 2: Magic number to start hitting the 414 error is somewhere between 900-1000 items in the exclusion list, pretty sure that's going to be pretty rare so maybe just a warning if the "add all series" will result in more than 500 shows in the exclusion list or something? I don't know, I feel like this one isn't something to focus dev time on really

I was wondering if we might see something like this happen.

We'll have to take a look and see if we can exclude items by location on big libraries.

 

[Josephus 15]

Will you consider manually batch import of time spots in a certain way, like json import or csv, thus enabling me to fetch the already measured result provided by Netflix according to their private API (which you can capture and get the result).

Data from Netflix is just like this: 

"skipMarkers":{"credit":{"start":0,"end":0},"recap":{"start":0,"end":0}

And I wish I can import it in batch, with my scraper or something, instead of editing the database directly to avoid possible failures.

Edited February 12, 2022 by Josephus

[fillidill 153]

1 minute ago, Josephus said:

Will you consider manually batch import of time spots in a certain way, like json import or csv, thus enabling me to fetch the already measured result provided by Netflix according to their private API (which you can capture and get the result).

Data from Netflix is just like this: 

"skipMarkers":{"credit":{"start":0,"end":0},"recap":{"start":0,"end":0}

And I wish I can import it in batch, with my scraper or something, instead of editing the database directly to avoid possible failures.

Making it possible to save this as well to the media folder for backups or quickly re-adding media would also be a great feature.

[Josephus 15]

Just now, fillidill said:

Making it possible to save this as well to the media folder for backups or quickly re-adding media would also be a great feature.

Acturally, I am working on scraping metadata on the almost whole Netflix on my local server. And I am almost trying to dierctly write certain data directly into the database if there is to be no official implementation. 

An API of importation is also of great help.