Plugin - Chapter Editor (ChapterApi)

[TeamB 2352]

22 minutes ago, rbjtech said:

However, trying to save these to the Theme Service,  if I try and re-use a json (lets say s1e02-04) - and just change the episode number - it won;t let me do that.   It just retains the last change.   Is it checking to see if the MD5/FP exists already ?

you can not upload the same chromaprint data, it will as you point out use the md5 to check and just give you the already existing entry. This is to stop duplicate items in the system.

if the chromaprint is exactly the same there is no point in uploading it anyway, the existing one will detect just fine.

To simulate how the Theme Service DB lookup would work with your 5 intro info items, zip them all up in the same zip and submit that to the detection job form. It should them use all the 5 to detect.

[TeamB 2352]

New Version : 1.3.0.1

 - bunch of bug fixes
 - some more sorting
 - added auto processing of newly added episodes for intro detection
 - add schedule to reload remote Intro Infro data
 - some other stuff

New detection results info available for each job item showing which was the best match and all the stats.

 

[neik 837]

Just got un unhandled exception which restarted my server.
I think the trigger was adding a new tv show.

Quote

2022-09-02 08:08:41.547 Info ChapterApi - LibraryMonitor: ProcessItemsTimerCallback
2022-09-02 08:08:41.675 Error GlobalExceptionHandler: UnhandledException
	*** Error Report ***
	Version: 4.8.0.8
	Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
	Operating system: Linux version 5.15.0-43-generic (buildd@lcy02-amd64-076) (gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #46-Ubuntu SMP
	Framework: .NET 6.0.8
	OS/Process: x64/x64
	Runtime: opt/emby-server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: /var/lib/emby
	Application path: /opt/emby-server/system
	System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
	   at ChapterApi.LibraryMonitor.GroupItems(List`1 item_ids)
	   at ChapterApi.LibraryMonitor.ProcessItemsTimerCallback(Object state)
	   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
	--- End of stack trace from previous location ---
	   at System.Threading.TimerQueueTimer.Fire(Boolean isThreadPool)
	   at System.Threading.TimerQueue.FireNextTimers()
	Source: ChapterApi
	TargetSite: System.Collections.Generic.Dictionary`2[System.String,System.Collections.Generic.List`1[MediaBrowser.Controller.Entities.TV.Episode]] GroupItems(System.Collections.Generic.List`1[System.Int64])
	

 

unhandled_f14249c4-cce3-4baa-bb87-943135b13e99.txt

[TeamB 2352]

25 minutes ago, neik said:

which restarted my server

I will look into the error.

I dont think it is possible for a plugin to fully crash the server.

[TeamB 2352]

17 hours ago, neik said:

Just got un unhandled exception which restarted my server.
I think the trigger was adding a new tv show.

 

unhandled_f14249c4-cce3-4baa-bb87-943135b13e99.txt 768 B · 1 dowload

I have added some more null checking and a little logging to the attached build, If you could try that it would help.

I also added a on metadata refresh trigger to the new/added trigger so you should just be able to refresh the metadata on an item to have it trigger a new detection job.

Also try removing and adding an episode just to check that also.

If it is still causing issues PM me your server log.

 

 

Edited September 2, 2022 by TeamB

[rbjtech 4266]

Some nice updates ! - Thanks @TeamB ..  

 

[neik 837]

11 minutes ago, TeamB said:

I have added some more null checking and a little logging to the attached build, If you could try that it would help.

I also added a on metadata refresh trigger to the new/added trigger so you should just be able to refresh the metadata on an item to have it trigger a new detection job.

Also try removing and adding an episode just to check that also.

If it is still causing issues PM me your server log.

 

ChapterApi.zip 55.85 kB · 0 downloads

Seems to have fixed it.
What I did:

1. Removed the tv show I added before
2. Ran a library scan to get everything related to it out of Emby
3. Readded the tv show and ran a library scan again to get it into Emby

No issues so far.

[TeamB 2352]

1 hour ago, neik said:

Seems to have fixed it.
What I did:

1. Removed the tv show I added before
2. Ran a library scan to get everything related to it out of Emby
3. Readded the tv show and ran a library scan again to get it into Emby

No issues so far.

to see it working check the jobs list and make sure a new detection job was created for the newly added episodes.

[neik 837]

6 minutes ago, TeamB said:

to see it working check the jobs list and make sure a new detection job was created for the newly added episodes.

Not quite sure what you mean but I can see the following in the Emby logs:

2022-09-02 10:11:01.204 Info ChapterApi - LibraryMonitor: Episode Added : 933056

And a bit later on:

2022-09-02 10:13:25.481 Info ChapterApi - LibraryMonitor: ProcessItemsTimerCallback
2022-09-02 10:13:25.482 Info ChapterApi - LibraryMonitor: Grouping Items : 933056
2022-09-02 10:13:25.489 Debug ChapterApi - LibraryMonitor: No intro data for series : tt3696476

So, for me it seems like it detected the new episode and tried to process it but couldn't due to missing fingerprint (which is OK).

[TeamB 2352]

1 hour ago, neik said:

Not quite sure what you mean but I can see the following in the Emby logs:

2022-09-02 10:11:01.204 Info ChapterApi - LibraryMonitor: Episode Added : 933056

And a bit later on:

2022-09-02 10:13:25.481 Info ChapterApi - LibraryMonitor: ProcessItemsTimerCallback
2022-09-02 10:13:25.482 Info ChapterApi - LibraryMonitor: Grouping Items : 933056
2022-09-02 10:13:25.489 Debug ChapterApi - LibraryMonitor: No intro data for series : tt3696476

So, for me it seems like it detected the new episode and tried to process it but couldn't due to missing fingerprint (which is OK).

yep you are correct, that is what I expected and how it should work.

[neik 837]

One more thing:
Sometimes when I try to download the JSON file it open up in the browser instead of downloading it.
Couldn't find a real pattern yet but it is reproducable for me with Baywatch (1989) S1E1.

Wiil send you the Emby log via PM.

[TeamB 2352]

23 minutes ago, neik said:

Sometimes when I try to download the JSON file it open up in the browser instead of downloading it.
Couldn't find a real pattern yet but it is reproducable for me with Baywatch (1989) S1E1.

When the JSON is loaded in the browser is the JSON data correct?
What browser are you using?
The server just returns a file content type so it is up to the browser what it does with the data. You could try a different browser to see how that works and see if we can narrow it down.

I could not see anything obvious in the log, it looks like the extract worked correctly and the data was returned with a response of 200 wich is a http success response.

Edited September 2, 2022 by TeamB

[neik 837]

On 8/30/2022 at 11:52 AM, rbjtech said:

Agreed - and I found out just that yesterday - GoT S1 needs multiple FP's. !     Not sure why we didn't realise that during the Introskip development - no wonder it never matched them all ..

...

So it's basically this - manually using JSON's (selecting the appropriate JSON for each episode) - I can correctly identify GoT in millseconds as you say using -

GoT s1e01.json

GoT s1e02-04.json

GoT s1e05-08.json

GoT s1e09.json

GoT s1e10.json

 

However, trying to save these to the Theme Service,  if I try and re-use a json (lets say s1e02-04) - and just change the episode number - it won;t let me do that.   It just retains the last change.   Is it checking to see if the MD5/FP exists already ?

The only way to upload these (which is what I want to do) is to upload 10 unique FP's ?

...

Perfect identification (HBO stripped out as you say) .. using the above jsons 

Afirm, just used the JSONs in the webservice and let them run through my GoT S1 and is exactly as you said.
This sharing idea is getting me really excited.
If this keeps going and people contribute then it will be pretty easy to identify intros once the database has a considerable amount of intros in it.

[rbjtech 4266]

2 minutes ago, neik said:

Afirm, just used the JSONs in the webservice and let them run through my GoT S1 and is exactly as you said.
This sharing idea is getting me really excited.
If this keeps going and people contribute then it will be pretty easy to identify intros once the database has a considerable amount of intros in it.

Yea I'm adding new Series as I get them.

I'm working on a script to import all my previously detected IntroSkip Plugin TV Series - but the issue is making sure they are 100% accurate before doing so.  It needs a bit of thought.

The great thing about this 'method' is it also has the potential to be used for so much more - for example 'Previously on..' scenes - it would be just as easy to identify those as Intro's for example .. imagine those being marked (for skipping) as well ..

[neik 837]

17 minutes ago, TeamB said:

When the JSON is loaded in the browser is the JSON data correct?
What browser are you using?
The server just returns a file content type so it is up to the browser what it does with the data. You could try a different browser to see how that works and see if we can narrow it down.

I could not see anything obvious in the log, it looks like the extract worked correctly and the data was returned with a response of 200 wich is a http success response.

Yes, the data is correct as I can use it if I save it manually via Notepad++ for instance.
Chrome 104, latest updates are pending currently. With Edge it is the same.

My idea of it being due to long intro doesn't seem correct, just did some testing on another tv show and can't reproduce there with intro >= 2min.
Currently it seems to happen randomly.

[TeamB 2352]

5 minutes ago, neik said:

Yes, the data is correct as I can use it if I save it manually via Notepad++ for instance.
Chrome 104, latest updates are pending currently. With Edge it is the same.

My idea of it being due to long intro doesn't seem correct, just did some testing on another tv show and can't reproduce there with intro >= 2min.
Currently it seems to happen randomly.

I have tested on firefox and edge, while firefox does use its internal json viewer to show the data you just have right click and say save to save the data.

With edge it prompts for download. Both browsers are consistent for me. I will try to run some more tests tomorrow to see if i can reproduce what you are seeing.

[rbjtech 4266]

2 minutes ago, TeamB said:

I have tested on firefox and edge, while firefox does use its internal json viewer to show the data you just have right click and say save to save the data.

With edge it prompts for download. Both browsers are consistent for me. I will try to run some more tests tomorrow to see if i can reproduce what you are seeing.

This did happen once for me a couple of days ago (on edge) - but always saves to file now.  I'll report it if I see it happen again. 

[TeamB 2352]

On 9/2/2022 at 11:27 PM, neik said:

Yes, the data is correct as I can use it if I save it manually via Notepad++ for instance.
Chrome 104, latest updates are pending currently. With Edge it is the same.

My idea of it being due to long intro doesn't seem correct, just did some testing on another tv show and can't reproduce there with intro >= 2min.
Currently it seems to happen randomly.

I tested on Firefox Chrome and Edge. About 40 to 50 downloads of the extracted intro data, I could not reproduce the issue.

However I did make a change, instead of

"text/html; charset=UTF-8" 

I now use

"application/json"

which is more correct and I was using this originally but for some reason some browsers were showing the json in their internal json viewer (firefox) so I added

"Content-Disposition": "attachment; filename="filename.json"

and that looks like it forces the browser to trigger the download to file action.

 

Edited September 5, 2022 by TeamB

[pünktchen 1258]

@Luke i've added a second pair of intro skip markers to an episode with this plugin, because the core feature only detected something that is similar to "previously" but it missed the real "intro". Unfortunately the player osd only shows one time the skip intro button. Are you aware of this?

@TeamB the season summary shows only one intro marker pair, but the episode view shows both. Can you make all intro marker pairs available for the season view. Maybe with the same highlight color for the same episode. It would also help to identify doubled intro marker pairs that i've created by accident. The same would be handy with a red highlight color for the case that a episode has only the start or only the end marker.

[TeamB 2352]

the plugin uses the first intro start and end markers it finds, this is the ones closest to the start. the reason for this is from what I understand the first skip into area will be used by the clients, and I guess this means one one skip intro is valid, the first one.

having two intro areas (two start and end markers) I am not sure is valid or is intended to be valid. I guess it could be valid but I am not sure how the core emby systems want this to be managed so at the moment the second intro area (start and end pairs) are ignored.

[pünktchen 1258]

The player uses the second one btw.

[rbjtech 4266]

So this is interesting.

When the core first implemented the 'markers' this was one of the very first things I tested - ie would it recognise more than one set of markers - and my findings were that it only recognised the first.

I'll test this again - maybe things have changed.

If they have - then this actually opens up a few areas for further investigation - one being Commercial skipping.

When we spoke to the core team on this, they have also setup the markers to be expandable down the line - so they 'could' easily add a 'PreviouslyOnStart/PreviouslyOnEnd' marker etc - but without client support this is of course not going to achieve much.

There are also options for CreditsStart (which is valid and used now by the Android ATV client) but CreditsEnd is not used - but is just as valid because sometimes there is content after the credits have finished for example.

The whole 'marker' concept needs a little forward thinking imo but the core have only implemented the bare essentials ...

 

[Luke 37068]

18 hours ago, pünktchen said:

@Luke i've added a second pair of intro skip markers to an episode with this plugin, because the core feature only detected something that is similar to "previously" but it missed the real "intro". Unfortunately the player osd only shows one time the skip intro button. Are you aware of this?

It only shows one time, yes.

[pünktchen 1258]

2 hours ago, Luke said:

It only shows one time, yes.

Okay, maybe i should have asked is this intentional?

[rbjtech 4266]

On 24/09/2022 at 21:46, pünktchen said:

Okay, maybe i should have asked is this intentional?

On the Plugin the view was that if the markers were incorrect and you had auto-skip on, then because auto-skip only fire ONCE per episode, you could rewind and watch the missed part.  if it kept using the markers each time, then you could never watch the content that got accidently skipped (short of turning off auto-skip ..)   

That was our rationale - and I suspect the Core's as well but of course hopefully they will confirm.