Completed Show Intro Skip Option

[rbjtech 4348]

4 minutes ago, Sammy said:

How'd you parse the data to put into a spreadsheet to do the pivot table? What are you pivot table parameters? (Screenshot of the parameters?)

Emby Reports plugin, select Episodes..

 

Export the data to Excel - you can do all of it, or just select the RunTime ..

In Excel - select Runtime column and then create a PivotTable and sum using 'Count', finally select the Pivot output and click on Recommend Graphs to show the graph.  

 

[Sammy 739]

I didn't have it installed already. This run of fingerprinting started about 8 or 9 hours ago and is set at 20 minutes. I'll have to either kill it and start over or I'll have to wait..

[chef 3750]

5 hours ago, Banquero said:

Buenos dias, se puede probar este plugin, aunque sea en beta? En ese caso, donde se encuentra la carpeta de plugins de Emby para DSM7? Muchas gracias...

Hola Banquero, Actualmente estamos esperando que Emby agregue algunas características a su producto para que podamos ejecutar este complemento de forma nativa en la aplicación. Cuando Emby agregue las funciones, podrá utilizar este Synology.

 si esto no se tradujo muy bien.

 

It will run on Synology in the future.

Edited August 21, 2021 by chef

[chef 3750]

56 minutes ago, rbjtech said:

So based on the above - I believe (for me) a 40 minute cut off is too soon to 'return 20' and you would scanning a lot of 40-45 min shows with unnecessary scan time - imho.

 

Right! I'll use those same values of runtime from the reports plugin to best calculate encode time. 

That's great! Thank you!

Edited August 21, 2021 by chef

[Banquero 31]

1 hour ago, chef said:

Hola Banquero, Actualmente estamos esperando que Emby agregue algunas características a su producto para que podamos ejecutar este complemento de forma nativa en la aplicación. Cuando Emby agregue las funciones, podrá utilizar este Synology.

 si esto no se tradujo muy bien.

 

It will run on Synology in the future.

Muchas gracias @chef  Espero deseoso que Emby agregue dichas características. Saludos... 

[Micael456 43]

1 hour ago, rbjtech said:

Export the data to Excel - you can do all of it, or just select the RunTime ..

In Excel - select Runtime column and then create a PivotTable and sum using 'Count', finally select the Pivot output and click on Recommend Graphs to show the graph.  

Sorry @rbjtech, this is where you've lost me. PivotCharts/Tables always melt my brain. I don't have the option to do both colums *and* count of, only one or the other (and then the count is obviously of the wrong thing). Any idea what step I'm missing?

I've also attached my report as that might be easier than explaining excel!

ReportExport.xls

[chef 3750]

• Added  "AudioFingerprintDurationMatchException" -  This gets thrown if the encoding durations don't match between two episodes of the same season in Title Sequence scan.
• Removed the encoding duration options from UI - Encoding durations are now calculated based the on the average runtime of episodes in the season and are extended based on 14 minute intervals of runtime. (This will probably change to a percentage of the average runtime - that could be configured by the user).
• Fine tuning/editable table values in the UI  (Has Sequence - bool,  Start time , End Time values can be tuned and saved to the data base)

IntroSkip_v2.rar

 

The changes to the duration calculation in the fingerprinting task may re-encode some seasons again. 

Edited August 21, 2021 by chef

[rbjtech 4348]

1 hour ago, Micael456 said:

Sorry @rbjtech, this is where you've lost me. PivotCharts/Tables always melt my brain. I don't have the option to do both colums *and* count of, only one or the other (and then the count is obviously of the wrong thing). Any idea what step I'm missing?

I've also attached my report as that might be easier than explaining excel!

eportExpor.xls 40.49 kB · 2 downloads

#1 - you've only select the first 100 records  so ensure you select 'All' from the export ..

In Excel - Drag 'Runtime' to Rows, then Drag Runtime again into Values.  Change the Value to 'Count' instead of 'Sum'

In your attached example - you then get what is shown below ..

ie on this 100 episode example, 31 of them are 44 minutes long.  Once you do this for your entire TV library (episodes) then you'll get some meaningful stats.  

Row Labels	Count of Runtime
8	1
20	2
22	6
23	1
26	4
39	1
42	6
43	25
44	31
45	14
46	2
47	2
49	1
84	1
86	1
87	1
89	1
(blank)
Grand Total	100

[Sammy 739]

31 minutes ago, chef said:

• Added  "AudioFingerprintDurationMatchException" -  This gets thrown if the encoding durations don't match between two episodes of the same season in Title Sequence scan.
• Removed the encoding duration options from UI - Encoding durations are now calculated based the on the average runtime of episodes in the season and are extended based on 14 minute intervals of runtime. (This will probably change to a percentage of the average runtime - that could be configured by the user).
• Fine tuning/editable table values in the UI  (Has Sequence - bool,  Start time , End Time values can be tuned and saved to the data base)

IntroSkip_v2.rar 40.62 kB · 2 downloads

 

The changes to the duration calculation in the fingerprinting task may re-encode some seasons again. 

I suppose I should kill the current scan and give this a whirl..

[Micael456 43]

@rbjtech- cheers, I did think it processed a bit too quickly the first time .

I have a much more sensible looking chart / table now. Only ~8000 total entries in mine, but here's the stats (report with numbers also attached):

Weirdly enough I'm getting some funny runtimes (2 minute episodes?) when I pivot-table this, so guess I've got a few bad entries.

 

EmbyReportsRuntimes.xlsx

[chef 3750]

2 hours ago, Micael456 said:

@rbjtech- cheers, I did think it processed a bit too quickly the first time .

I have a much more sensible looking chart / table now. Only ~8000 total entries in mine, but here's the stats (report with numbers also attached):

Weirdly enough I'm getting some funny runtimes (2 minute episodes?) when I pivot-table this, so guess I've got a few bad entries.

 

EmbyReportsRuntimes.xlsx 214.4 kB · 0 downloads

I wonder if because the runtime data is a nullable long integer, the data could be returning null, which might cause some strange results in the chart. 

[Micael456 43]

23 minutes ago, chef said:

I wonder if because the runtime data is a nullable long integer, the data could be returning null, which might cause some strange results in the chart. 

I do have some blanks, not sure when Emby calculates runtime but looking at the blank list I'm guessing when it first plays?

 

As for the 2-9 minutes, they seem to all be either specials (Season "0" clips) or early episodes of The Guild (though I imagine other shortform web series might have that issue).

Might be worth coding some really low numbers in there, or skipping sub 5 minutes entirely lol.

 

Edit: In fact, all of my sub 10 minutes are either The Guild, The Guild Specials, or Bewitched Specials.

Edited August 21, 2021 by Micael456

[labrjbcmb 1]

4 hours ago, chef said:

• Added  "AudioFingerprintDurationMatchException" -  This gets thrown if the encoding durations don't match between two episodes of the same season in Title Sequence scan.
• Removed the encoding duration options from UI - Encoding durations are now calculated based the on the average runtime of episodes in the season and are extended based on 14 minute intervals of runtime. (This will probably change to a percentage of the average runtime - that could be configured by the user).
• Fine tuning/editable table values in the UI  (Has Sequence - bool,  Start time , End Time values can be tuned and saved to the data base)

IntroSkip_v2.rar 40.62 kB · 6 downloads

 

The changes to the duration calculation in the fingerprinting task may re-encode some seasons again. 

Hi! I'm very interested in this plugin - thank you so much for creating it! I installed it and it detected the intros but no option to skip comes up when I play a video?

[Micael456 43]

6 minutes ago, labrjbcmb said:

Hi! I'm very interested in this plugin - thank you so much for creating it! I installed it and it detected the intros but no option to skip comes up when I play a video?

This is still in a proof of concept stage. Once we (well, chef really) can get the plugin reliably detecting intros, we'll have to wait for Luke to add the actual skip functionality in (or expose it to plugins).

[chef 3750]

I'm running a scan from the beginning again with the new duration code. 

Im hoping for some Agent Carter results this time. If that happens... that's awesome!

 

[STR8 69]

Thought I would just share some of my results so far as I was going to wait until the process finished scanning but at the rate my Terramaster is going it could be another 24 to 48 hours LOL!

I'm getting good results for game of thrones and agent carter... but some are True and some are False but I'm thinking that is because it's not a 100% complete because the show 11.22.63 with Franco was doing the same some True/False but now it's showing all episodes as True! 

Also I'm using the plugin that Chef posted I believe on 8/18 or 8/19 before he made changes and add the most 2 recent plugins.

Thanks for the hard work on this everyone it's greatly appreciated.

 

 

 

[chef 3750]

5 minutes ago, STR8 said:

Thought I would just share some of my results so far as I was going to wait until the process finished scanning but at the rate my Terramaster is going it could be another 24 to 48 hours LOL!

I'm getting good results for game of thrones and agent carter... but some are True and some are False but I'm thinking that is because it's not a 100% complete because the show 11.22.63 with Franco was doing the same some True/False but now it's showing all episodes as True! 

Also I'm using the plugin that Chef posted I believe on 8/18 or 8/19 before he made changes and add the most 2 recent plugins.

Thanks for the hard work on this everyone it's greatly appreciated.

 

 

 

Thank you for taking the time do do this! That awesome! Positive results might make this whole things a reality!

 

You're correct that results may be set false for longer as the episodes attempt to match with each other. 

 

 

What would be interesting in the future is to compare results with each other as a community and see just how similar the timestamps are. Some people think each users encodings are different enough to create competing results, and therefore a mass database is impossible. However, I'm not totally sold on that idea yet. Ive seen some results posted that are too close to tell.  

Even if we created scheduled a task (people could opt into) to take the fingerprint of the exact title sequence of a series and host that as an online resource to compare users results with would make this plugin very fast.

[STR8 69]

I wish I could help out more but I don't code.

But I am more then willing to test and share results or any other simple task I could help with.

[chef 3750]

1 hour ago, STR8 said:

I wish I could help out more but I don't code.

But I am more then willing to test and share results or any other simple task I could help with.

Do you run a Linux machine?

[STR8 69]

Yes - As the abbreviation of TerraMaster Operating System, TOS is a Linux-based platform operating system specially developed for TNAS storage device. 

[chef 3750]

1 hour ago, STR8 said:

Yes - As the abbreviation of TerraMaster Operating System, TOS is a Linux-based platform operating system specially developed for TNAS storage device. 

You'll have to side load the plugin, but it'll work on your computer. The emby Linux distro has the capability built in, it's just windows which is a bit behind.

 

There is a bunch of number crunching that happens, so best to run the tasks over night, when it won't effect any needed resources

[STR8 69]

Couple of questions 

1. Where would I get the plugin or can you post it here?

2. I'm a little over 70% on the skip intro scan so should I run it after the scan is finished or can I run it at the same time?

3. Might be a silly question but when you say - You'll have to side load the plugin, but it'll work on your computer. (I have the Emby server installed on the Terramaster nas not on my Dell desktop) So I will install the plugin into the Emby server on the Terramaster just like I do all the others through the file manager correct?

 

[rbjtech 4348]

9 hours ago, chef said:

What would be interesting in the future is to compare results with each other as a community and see just how similar the timestamps are. Some people think each users encodings are different enough to create competing results, and therefore a mass database is impossible. However, I'm not totally sold on that idea yet. Ive seen some results posted that are too close to tell.  

Even if we created scheduled a task (people could opt into) to take the fingerprint of the exact title sequence of a series and host that as an online resource to compare users results with would make this plugin very fast.

For the development of the plugin - I think this is a good idea to capture all the exceptions - but once released, other than some sort of hash match, there would be no way to identify 'your' version of the episode in question ? 

You only have to look at the subtitle scene to understand the magnitude of the problem - as an example, on just opensubtitles, there are 35 English options for GoT s01e01 !  No doubt, some of these are duplicates - but how would you know unless you looked at them all ?  Another example of a failed on-line resource for this impossible task is chapters - no-body keeps a reliable and up to date chapter database as you face the same challenges - what version do you have ? 

I think the best way to achieve this is to use the Plugin to identify the Intro to the best of it's ability - if it's still wrong, then make it easy to modify.  This is a one off task, so it's not like you need to maintain it once the show has been correctly identified.

I guess there could be an option within the Emby GUI itself to Tag the Start and End of an Intro to update/override it manually - but I suspect that would be some way down the line.

Should have some time today to test V2 

[chef 3750]

4 hours ago, rbjtech said:

For the development of the plugin - I think this is a good idea to capture all the exceptions - but once released, other than some sort of hash match, there would be no way to identify 'your' version of the episode in question ? 

We save the TMDB ID.

An audio fingerprint encoding of just the title sequence at PCM_S16le are all pretty close to the same, or useable for matching. 

We could host those fingerprint arrays on an online resource.

 

Edited August 22, 2021 by chef

[chef 3750]

9 hours ago, STR8 said:

Couple of questions 

1. Where would I get the plugin or can you post it here?

2. I'm a little over 70% on the skip intro scan so should I run it after the scan is finished or can I run it at the same time?

3. Might be a silly question but when you say - You'll have to side load the plugin, but it'll work on your computer. (I have the Emby server installed on the Terramaster nas not on my Dell desktop) So I will install the plugin into the Emby server on the Terramaster just like I do all the others through the file manager correct?

 

Yes, that looks to be the correct folder.  A restart of the Emby service should load the plugin. 

@samuelqwe does that folder look right to you?

Edited August 22, 2021 by chef