Completed Show Intro Skip Option

[chef 3750]

30 minutes ago, CBers said:

I assume these are the initial steps required?

When I open the plugin config page, it defaults to a TV show that has no seasons currently.



What is the function of the Series field? Shouldn't the first option in the list be "All Shows"?

Also, shouldn't the field be "Show Name", rather than "Series"?

The default "Maximum parallel series to process at once" value is 2, but the text below the field says default is 4.



I've deleted the schedule tasks triggers, as suggested earlier in this thread - once all completed, I will put them back.

So what order do the tasks need to run in, as is on the Scheduled Tasks page?



Thanks.
 

This is great, and much appreciated. 

We had originally defaulted to 4 but it was too much for low powered NAS machine to run 4 at once so we changed it to 2.

We have to update the UI. Thanks for catching that.

 

The first drop down will allow you to choose a series, and the second will allow you to choose a season of that series to populate the list full of episodes.

Does that not make sense when it is the first time looking at the UI? 

 

It sounds like we might need a help tab... Or something, to explain how to get results. We take it for granted because we have been working with the system for about a year, but it would make sense that a user looking at this for the time might not understand the steps to take, and what to do.

 

[CBers 6806]

Just now, chef said:

The first drop down will allow you to choose a series, and the second will allow you to choose a season of that series to populate the list full of episodes.

Does that not make sense when it is the first time looking at the UI?

No, not really.

I expected the process to only work against the selected TV show.

Also, what order does the processing work, as it seems to have started half-way though the list of TV Show, rather than A-Z?
 

2021-09-15 12:40:23.255 Info Intro Skip: Chroma-print database contains 0 items.
2021-09-15 12:40:23.267 Info Intro Skip: Father Brown (2013) - Season 9 chromaprint profile is up to date.
2021-09-15 12:40:24.798 Debug Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: Extracting chunks from binary chroma-print.
2021-09-15 12:40:24.799 Debug Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: .bin file removed.
2021-09-15 12:40:24.799 Info Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: Saving.
2021-09-15 12:40:24.804 Info Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0 complete - 1 seconds.
2021-09-15 12:40:25.332 Debug Intro Skip: Silent Witness - S:24 - E:4: Extracting chunks from binary chroma-print.
2021-09-15 12:40:25.336 Debug Intro Skip: Silent Witness - S:24 - E:4: .bin file removed.
2021-09-15 12:40:25.336 Info Intro Skip: Silent Witness - S:24 - E:4: Saving.
2021-09-15 12:40:25.337 Info Intro Skip: Silent Witness - S:24 - E:4 complete - 2 seconds.
2021-09-15 12:40:26.146 Debug Intro Skip: The Expanse - S:5 - E:0: Extracting chunks from binary chroma-print.
2021-09-15 12:40:26.147 Debug Intro Skip: The Expanse - S:5 - E:0: .bin file removed.
2021-09-15 12:40:26.147 Info Intro Skip: The Expanse - S:5 - E:0: Saving.
2021-09-15 12:40:26.147 Info Intro Skip: The Expanse - S:5 - E:0 complete - 1 seconds.
2021-09-15 12:40:38.319 Debug Intro Skip: Manifest - S:3 - E:1: Extracting chunks from binary chroma-print.
2021-09-15 12:40:38.327 Debug Intro Skip: Manifest - S:3 - E:1: .bin file removed.
2021-09-15 12:40:38.327 Info Intro Skip: Manifest - S:3 - E:1: Saving.
2021-09-15 12:40:38.329 Info Intro Skip: Manifest - S:3 - E:1 complete - 15 seconds.

I have "Maximum parallel series to process at once" set to 4.
 

[Cheesegeezer 3089]

2 minutes ago, chef said:

This is great, and much appreciated. 

We had originally defaulted to 4 but it was too much for low powered NAS machine to run 4 at once so we changed it to 2.

We have to update the UI. Thanks for catching that.

 

The first drop down will allow you to choose a series, and the second will allow you to choose a season of that series to populate the list full of episodes.

Does that not make sense when it is the first time looking at the UI? 

 

It sounds like we might need a help tab... Or something, to explain how to get results. We take it for granted because we have been working with the system for about a year, but it would make sense that a user looking at this for the time might not understand the steps to take, and what to do.

 

Morning Treacle!!

We could just create a Help button and link to this buddy, all the hard work is done then and we can just edit this without having to keep releasing updates.

 

[neik 838]

1 minute ago, CBers said:

Also, what order does the processing work, as it seems to have started half-way though the list of TV Show, rather than A-Z?

My fingerprinting started from Z-A with 1 concurrent fingerprinting only.

[chef 3750]

3 minutes ago, CBers said:

No, not really.

I expected the process to only work against the selected TV show.

Also, what order does the processing work, as it seems to have started half-way though the list of TV Show, rather than A-Z?
 

2021-09-15 12:40:23.255 Info Intro Skip: Chroma-print database contains 0 items.
2021-09-15 12:40:23.267 Info Intro Skip: Father Brown (2013) - Season 9 chromaprint profile is up to date.
2021-09-15 12:40:24.798 Debug Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: Extracting chunks from binary chroma-print.
2021-09-15 12:40:24.799 Debug Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: .bin file removed.
2021-09-15 12:40:24.799 Info Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0: Saving.
2021-09-15 12:40:24.804 Info Intro Skip: The Falcon and the Winter Soldier - S:1 - E:0 complete - 1 seconds.
2021-09-15 12:40:25.332 Debug Intro Skip: Silent Witness - S:24 - E:4: Extracting chunks from binary chroma-print.
2021-09-15 12:40:25.336 Debug Intro Skip: Silent Witness - S:24 - E:4: .bin file removed.
2021-09-15 12:40:25.336 Info Intro Skip: Silent Witness - S:24 - E:4: Saving.
2021-09-15 12:40:25.337 Info Intro Skip: Silent Witness - S:24 - E:4 complete - 2 seconds.
2021-09-15 12:40:26.146 Debug Intro Skip: The Expanse - S:5 - E:0: Extracting chunks from binary chroma-print.
2021-09-15 12:40:26.147 Debug Intro Skip: The Expanse - S:5 - E:0: .bin file removed.
2021-09-15 12:40:26.147 Info Intro Skip: The Expanse - S:5 - E:0: Saving.
2021-09-15 12:40:26.147 Info Intro Skip: The Expanse - S:5 - E:0 complete - 1 seconds.
2021-09-15 12:40:38.319 Debug Intro Skip: Manifest - S:3 - E:1: Extracting chunks from binary chroma-print.
2021-09-15 12:40:38.327 Debug Intro Skip: Manifest - S:3 - E:1: .bin file removed.
2021-09-15 12:40:38.327 Info Intro Skip: Manifest - S:3 - E:1: Saving.
2021-09-15 12:40:38.329 Info Intro Skip: Manifest - S:3 - E:1 complete - 15 seconds.

I have "Maximum parallel series to process at once" set to 4.
 

Currently the task will not take a sort order.

When dealing with TPL (Task Parallel Library) it doesn't care which order the items are taken from a list, it only cares that it eventually takes all of them (in your case 4 at a time).

 

We definitely need to implement a way to choose which series to scan. 

I can put together a tab which allows series selection. This is probably one of the most requested, and important things to add.

 

@samuelqwe you had an idea yesterday for series selection, what was it again?

[chef 3750]

9 minutes ago, neik said:

My fingerprinting started from Z-A with 1 concurrent fingerprinting only.

What number is set in the advanced settings? It should be taking at least 2. 

Thanks @neik!

 

Taking items Z-A is only a coincidence. 

Edited September 15, 2021 by chef

[Cheesegeezer 3089]

4 minutes ago, chef said:

We definitely need to implement a way to choose which series to scan. 

I can put together a tab which allows series selection. This is probably one of the most requested, and important things to add.

 

@samuelqwe you had an idea yesterday for series selection, what was it again?

Also can we add TV Show/Folder to omit.  Could put this on another popout dialogue with an open button on the main config page.

Here's Sam's post

 

[Cheesegeezer 3089]

7 minutes ago, chef said:

Currently the task will not take a sort order.

When dealing with TPL (Task Parallel Library) it doesn't care which order the items are taken from a list, it only cares that it eventually takes all of them (in your case 4 at a time).

I agree, it's not important what order it goes, because the whole library needs to be scanned anyway.  it needs to be done in the most efficient way and ordering it may cause performance issues.

[Cheesegeezer 3089]

@chef  I just threw these in.... What are your thoughts?  The setup info button links to the Instructions page on the forum.

The Exclude folder or TV Shows is not wired up yet and I should leave you to do your wonders with HTML and JS lol - Coz I suck at it.

[Micael456 43]

1 hour ago, Cheesegeezer said:

Very interested to get all the information you have during this process if you don't mind.

Happy to turn on debug and provide any logs, just don't think there's much in there. I'm running Ubuntu, is there anything I can enable at the OS level to monitor the embyserver process and provide more useful stats?

[chef 3750]

15 minutes ago, Cheesegeezer said:

@chef  I just threw these in.... What are your thoughts?  The setup info button links to the Instructions page on the forum.

The Exclude folder or TV Shows is not wired up yet and I should leave you to do your wonders with HTML and JS lol - Coz I suck at it.

Nice!

What about creating tabs at the top of the page. I know how to do that. 

The new layout would be similar to auto organize, where the first tab would be the series selection/editing tab.  The second tab could be advanced settings.

Then we'd have an entire page to handle/add settings and we wouldn't be limited to the size of a modal.

The advanced tab could handle series selection, and the elements we have already in the modal.

What about that?

 

[Cheesegeezer 3089]

18 minutes ago, chef said:

Nice!

What about creating tabs at the top of the page. I know how to do that. 

The new layout would be similar to auto organize, where the first tab would be the series selection/editing tab.  The second tab could be advanced settings.

Then we'd have an entire page to handle/add settings and we wouldn't be limited to the size of a modal.

The advanced tab could handle series selection, and the elements we have already in the modal.

What about that?

 

I like that!!! using the Auto Organise Layout with Tabs at the top would provide a very nice clean and full experience to the user.

 

[CBers 6806]

1 hour ago, chef said:

Currently the task will not take a sort order

So how does it start the processing on a completely new system then?

You must have programmed the order?

Just curious.
 

[Cheesegeezer 3089]

24 minutes ago, Micael456 said:

Happy to turn on debug and provide any logs, just don't think there's much in there. I'm running Ubuntu, is there anything I can enable at the OS level to monitor the embyserver process and provide more useful stats?

I think some screenshots prior to the detection process of cpu and memory usage for emby to show idle state demand, then running the process with some more screenshots at various points from emby usage.

I don't know Ubuntu at all so pretty useless for info on best ways.

[Cheesegeezer 3089]

6 minutes ago, CBers said:

So how does it start the processing on a completely new system then?

VOODOO MAGIC MAN!!

Edited September 15, 2021 by Cheesegeezer

[chef 3750]

1 hour ago, CBers said:

So how does it start the processing on a completely new system then?

You must have programmed the order?

Just curious.
 

Fingerprinting  first.

Title sequence Detection second.

It currently scans everything TV show releated.

Edited September 15, 2021 by chef

[chef 3750]

Okay bare with me, I have something for a new layout, and it'll be good. It's gonna take me 30 minutes to write, but it'll be better. I'm pretty sure.

[rbjtech 4346]

3 hours ago, Cheesegeezer said:

No, because all the required information is stored in the TitleSequence.db so this can be backed up using the backup configuration utility and transferred easily - we don't want any other apps just using our data that's not inside the Emby Environment.

wow - you guys have been busy - sorry been away on some other work.

On the above - I need to confirm what you replied Cheese as I'm not 100% sure that is correct ?

If you create a new instance of emby, then any emby media ID's will be unique to that instance - thus the titlesequence.db will not be transferrable ?

For the same instance (ie a backup/restore) using the original media ID's - then yes, the titlesequence.db can be restored and re-used - that is fine.

to note - this was actually mentioned way back, I had an idea to use the tvdb unique episode and show ID as the 'link' between emby and the titlesequence.db - that way - it becomes portable between multiple emby instances - as the tvdb id's will not change.  Maybe something to put on the concept list - as it's not a big issue for the majority of users.

For those that just want to test on an isolated Beta but using their existing library storage - I just setup a new library and pointed it to c:\test\, set everything to no metadata providers (read nfo only) - and then I used mklink to create symbolic links to a list of folders got from a text file (tv-shows.txt)

You can then pick the shows you want - and they appear in this new library and you don't have to copy the shows anywhere.

@echo off

FOR /F "delims=" %%x in (tv-shows.txt) DO (

	if not exist "c:\test\%%x" mklink /D "%%x" "\\media\TV Series\%%x"

	)

pause

Example tv-shows.txt file - these are just the folder names.

Breaking Bad (2008) [tvdbID=81189]
Westworld (2016) [tvdbID=296762]

Result is a 'virtual' library of shows that point to the real files .. 

 Directory of C:\Test

15/09/2021  14:20    <DIR>          .
15/09/2021  14:20    <DIR>          ..
15/09/2021  14:20    <SYMLINKD>     Breaking Bad (2008) [tvdbID=81189] [\\media\TV Series\Breaking Bad (2008) [tvdbID=81189]]
07/09/2021  16:03               164 makevlib.bat
15/09/2021  14:20                72 tv-shows.txt
07/09/2021  09:40            21,942 tvshows-full.txt
15/09/2021  14:20    <SYMLINKD>     Westworld (2016) [tvdbID=296762] [\\media\TV Series\Westworld (2016) [tvdbID=296762]]
               4 File(s)         23,127 bytes
               4 Dir(s)  133,536,813,056 bytes free

 

Edited September 15, 2021 by rbjtech

[Cheesegeezer 3089]

2 minutes ago, chef said:

Okay bare with me, I have something for a new layout, and it'll be good. It's gonna take me 30 minutes to write, but it'll be better. I'm pretty sure.

Chef, i'm going to upload a pull request for you, i'm gonna keep cheesychapters going an let you do the pull requests.  If there are any conflicts with my pull, let me know and i will fix them, before you pull them.

I also added you on discord as a friend.

I deleted my config UI updates so you can work on yours.

Cheers

[Cheesegeezer 3089]

4 minutes ago, rbjtech said:

On the above - I need to confirm what you replied Cheese as I'm not 100% sure that is correct ?

If you create a new instance of emby, then any emby media ID's will be unique to that instance - thus the titlesequence.db will not be transferrable ?

Yes you are correct, i didn't even think of the UID's that emby creates on a new library.

4 minutes ago, rbjtech said:

For the same instance (ie a backup/restore) using the original media ID's - then yes, the titlesequence.db can be restored and re-used - that is fine.

to note - this was actually mentioned way back, I had an idea to use the tvdb unique episode and show ID as the 'link' between emby and the titlesequence.db - that way - it becomes portable between multiple emby instances - as the tvdb id's will not change.  Maybe something to put on the concept list - as it's not a big issue for the majority of users.

This idea is awesome!!

[CBers 6806]

21 hours ago, chef said:

BOOM! Memory usage down by 1000 MB running five series at once!

CPU is high because I have 5 processing at once, but memory issue is squashed!

Nice catch @rbjtech! That one's all yours!

So what part of the process is memory intensive, as my EmbyServer.exe process is doing diddly squat with 4 "Episode Audio Fingerprinting" (ffmpeg) processes running?



This is running on a sever with a Ryzen 5 3600X 6-Core processor and 40Gb RAM.
 

[Cheesegeezer 3089]

1 minute ago, CBers said:

So what part of the process is memory intensive, as my EmbyServer.exe process is doing diddly squat with 4 "Episode Audio Fingerprinting" (ffmpeg) processes running?



This is running on a sever with a Ryzen 5 3600X 6-Core processor and 40Gb RAM.
 

the fingerprinting is fairly harmless, it's the detection task that is the black hole of resources (well it was, but hopefully shouldn't be anymore)

[chef 3750]

55 minutes ago, CBers said:

So what part of the process is memory intensive, as my EmbyServer.exe process is doing diddly squat with 4 "Episode Audio Fingerprinting" (ffmpeg) processes running?



This is running on a sever with a Ryzen 5 3600X 6-Core processor and 40Gb RAM.
 

 

53 minutes ago, Cheesegeezer said:

the fingerprinting is fairly harmless, it's the detection task that is the black hole of resources (well it was, but hopefully shouldn't be anymore)

 

Yup, the title sequence detection gets increasingly intensive. It can really drive the CPU. 

[rbjtech 4346]

The next stage @CBers - it was loading the entire titlesequence db into memory each time it did a detect query ..  .. so a 1.5Gb file meant ~1.5Gb of memory being used - it was also dog slow (as you can imagine) - @chef has now fixed this to only load the show involved, thus memory footprint is a fraction of what it was.

FP just uses ffmpeg as you say - it doesn't need much memory or cpu per process.

[chef 3750]

1 hour ago, Cheesegeezer said:

Yes you are correct, i didn't even think of the UID's that emby creates on a new library.

This idea is awesome!!

If that is what you guys want to do, then we might want to start thinking about implementing this sooner then later.

As more users start to use the plugin beta, we should be cautious about asking them to run the task, and then asking them to remove the DB.

But, there are other important things things to do before we get to that part of it, like a more intuitive UI (which I'm working on now). 

@CBers has brought up some good points that need addressing here first. 

 

@Cheesegeezer I know you probably have some fixes to push to the repo. Can you wait on syncing anything until I push the UI update?

Edited September 15, 2021 by chef