Chapter Images Extraction - Happening Twice?

[PenkethBoy 2068]

If i have just run a full library scan with Chapter Image extraction and Extract during library scan - both enabled for all libraries - then the scheduled Image extraction task (which ran immediately after the scan finished) should actually not need to do anything?

 

If that's so then the image extraction is being done twice for any new or changed media - possible flag not being set in db?

 

I can see ffmpeg etc running on the server as it works its way through the media it thinks needs chapters extracting.

 

Checking the new media the chapter images already exist as i can see them in the item details screen - created by the realtime monitor process trigger or the scan that just finished

 

Am i missing something?

 

Appears to happen on stable and beta servers

Edited June 23, 2017 by PenkethBoy

[Luke 42083]

Why do you feel that it's being done twice? Best thing to do is discuss a specific example, and attach the information requested in how to report a problem. thanks !

[PenkethBoy 2068]

I explained why i think its being done twice as i can see it doing it - i.e. ffmpeg is running while the chapter extraction scheduled task is active

 

and the new media already has chapters extracted - hence why i think it gets done twice

[Luke 42083]

We will need to look at a specific example then, thanks.

[PenkethBoy 2068]

 

 

Here's a log from my stable server - appears to happen on the beta server as well

 

Rebooted server so a clean start with debugging on

 

Manually ran a full Library Scan

Manually ran the Chapter extraction task - took 7 seconds as nothing new been added to server in last few days

Added 3 new episodes to "The Professionals" Season 5 

These were picked up by the real time monitor - metadata download etc and chapter images extracted - can see this in the log

Ran the full library scan again 

then ran chapter extraction task again - took 37 seconds and extracted new images (at slightly different times??)

 

So it appears to me that work is being done twice - hence the query

 

 

[Luke 42083]

Where were these files copied from? Or were they downloaded? In other words, is it a place where the transfer could have taken a little time? I think the file timestamp changed and that is why it ran twice.

 

My guess is that the first time the file hadn't actually completed copying in yet. There's no bullet proof way for us to determine that so we just have something like a 45 second delay. So I am thinking that the file was imported into the library and probed before it actually completed copying in or downloading.

[PenkethBoy 2068]

Hi @@Luke

 

Files were copied across the lan from a pc to the server - whole copy took less than 20 seconds for the three files - they are small ~700mb so quick to copy

 

time stamps are the same not changed as i compared them with the originals as in date modified - i assume modified date is what you check or do you use created date/accessed date?

 

Although the Stable Server library is on a linux nas - could that have an effect?

 

Actually my beta server is on 2012r2 and connected with a 10GB network and exhibits the same behavior so doubt its a linux "effect"

 

You have a 60 second delay on the realtime monitor so that would give 105 seconds (assuming they are cumulative) before doing any probing etc

 

Yes i can see why different time stamps (hence potential new file) would require a second run and thats fine but i dont think there is a change if i am understanding it correctly

 

 

However i did a full library scan before the Chapter extraction task - so that should  have picked up any time stamp changes first - and that scan extract any images - so when the Chapter extraction ran it would then not have needed to do anything.

 

Also if i change the name of say a movie on the server - so virtually instant change - run the full library scan after and then do a chapter extraction task - it does the extraction twice as discussed here.

 

I have seen this behaviour for quite a long time (but never got around to testing directly) and put it down to me constantly tinkering and changing/modifying files but i dont think thats the case anymore 

 

 

Happy to try different option here if you need more info

[Luke 42083]

No what I mean is timestamps must have changed from the time of the first extraction to the second. Actually they had to have changed because you can see in the log that the chapter image file names are different. the chapter image file names are driven based on the date timestamp of the file + the timetstamp of each chapter, so that means one of those two factors had to have changed between the first extraction and the second.

[PenkethBoy 2068]

I checked the original file and the copy have the same timestamp - if you mean modified time - or the separate process are reading different attributes

 

as i said just changing the name of a file or internal metadata will cause it to have chapters extracted twice

 

give it a try

[Luke 42083]

This is true:

as i said just changing the name of a file

this is not:

 or internal metadata will cause it to have chapters extracted twice

[CBers 7452]

If i have just run a full library scan with Chapter Image extraction and Extract during library scan - both enabled for all libraries - then the scheduled Image extraction task (which ran immediately after the scan finished) should actually not need to do anything?

 

 

Why do you have them both enabled?

 

I don't extract them via a library scan, as it would take too long.

 

I only have one library scan per day (overnight) on a schedule and the image extraction on a separate schedule.

 

If I add a movie during the day and want to see the chapter images, then I just run the separate schedule task.

 

Just an example of how I do it.

 

The way you do it, the separate chapter extraction task will still run as that is how you've scheduled it.

 

Either remove the scheduled task, or remove the extraction during the library scan. You don't really need both, unless you OCD kicks in

[PenkethBoy 2068]

No what I mean is timestamps must have changed from the time of the first extraction to the second. Actually they had to have changed because you can see in the log that the chapter image file names are different. the chapter image file names are driven based on the date timestamp of the file + the timetstamp of each chapter, so that means one of those two factors had to have changed between the first extraction and the second.

Hi Luke

 

If as you say its a combination of Timestamp (modified time) and chapter time then nothing changed as the scan, then chapter extraction-  took place within a minute of each other.

 

I have seen this and tested the double extraction on two completely different machines and two versions of the server - with libraries on local as well as remote disks and see the same result.

 

Looking at the log (see my post from 23rd June above) you see the original images being deleted as "dead chapter" images - although the image filenames listed are different and extracted at different chapter times (by their filenames) although the initial extraction is done at 5 min intervals but the "dead chapter" were "done" at 3 min intervals by name - although the log shows them being done at 5 min intervals?

 

Could that be the route of the problem - i.e extracted at 5 min intervals and saved with 3 min interval names?

 

 

This is true:

as i said just changing the name of a file

this is not:

 or internal metadata will cause it to have chapters extracted twice

Internal metadata (tag fields) within the MP4 file will cause this as it changes the timestamp of the file.

[Luke 42083]

Internal metadata (tag fields) within the MP4 file will cause this as it changes the timestamp of the file.

Yes you are right.

[PenkethBoy 2068]

Why do you have them both enabled?

 

I don't extract them via a library scan, as it would take too long.

 

I only have one library scan per day (overnight) on a schedule and the image extraction on a separate schedule.

 

If I add a movie during the day and want to see the chapter images, then I just run the separate schedule task.

 

Just an example of how I do it.

 

The way you do it, the separate chapter extraction task will still run as that is how you've scheduled it.

 

Either remove the scheduled task, or remove the extraction during the library scan. You don't really need both, unless you OCD kicks in

Thanks @CBers 

 

but although that would stop the double extraction a) it should not be necessary i am trying (failing - lol) to report a problem c) the average user may have the same settings d)Its a waste of energy etc

 

I could just ignore it/change my settings but where is the fun in that!   

[PenkethBoy 2068]

Internal metadata (tag fields) within the MP4 file will cause this as it changes the timestamp of the file.

Yes you are right.

 

Lol we agree on something