Jump to content

Show Intro Skip Option (Plugin)


Liquidfire88

Recommended Posts

Almost done a plugin layout. After the task is complete... Probably a long time first time run... There is a service entry point created, so clients can request the data from the server, it's by item.InternalId it will return a json object with intro times.

I suppose... It would then be up to client developers to take advantage of that data ūüė≥ūüė≥ūüė≥ {blink}{blink}ūüė≥ūüė≥ūüė≥

 

Edited by chef
  • Like 2
  • Thanks 1
Link to comment
Share on other sites

TeamB
1 hour ago, chef said:

Almost done a plugin. After the task is complete... Probably a long time first time run... There is a service entry point created, so clients can request the data from the server, it's by item.InternalId it will return a json object with intro times.

I suppose... It would then be up to client developers to take advantage of that data ūüė≥ūüė≥ūüė≥ {blink}{blink}ūüė≥ūüė≥ūüė≥

 

So how would a user use this?

Does the plugin scan seasons looking for the info and then store the info somewhere for each epp?
How does the plugin compare, does it use the first epp in a season and compare to all the other epps in the season?
How are you currently calculating the fingerprint, using pfpcalc? (I think the version of ffmpeg bundled with emby does not have chromaprint included for some reason)

Edited by TeamB
Link to comment
Share on other sites

{
InternalId:long,
IntroStart: TimeSpan,
IntroEnd:TimeSpan
}
26 minutes ago, TeamB said:

So how would a user use this?

Does the plugin scan seasons looking for the info and then store the info somewhere for each epp?
How does the plugin compare, does it use the first epp in a season and compare to all the other epps in the season?

It's a scheduled task and API endpoint

We can move through the library by series. An API request for series, managing each season one at a time.

It's going to be a long task, so it's best to move through series by series, season by season, episode by episode. We'll so have to keep tabs of completed data.

Each time we successfully match episodes in a season, it will be two at a time.

If there is no match, we can either do a follow up scan, or we try again immediately with another two episodes from the season.

Narrowing down all the episodes.

 

We could eventually create a db file with the data, but perhaps for now we save the data in the plugins configuration.

The service endpoint will take a baseItem InternalId, and return the start and end data for that id.

If we test the plugin first, because there is probably going to be issues until it's solid, I can limit series results for testing. Maybe 3 or 4 series, just to make sure it is somewhat smooth.

Is that a good idea?

 

Edit: we could try epp 1 and two, if it passes, we can try 1 and 3, untill the end of the season count.... To much trial and error?

Edited by chef
Link to comment
Share on other sites

samuelqwe

@chef in my testing, I've noticed that sometimes an intro end time stamp can be extended if there happens to be a few similar sounds after the intro has ended, so I suggest we stop looking for the contiguous region if the intro end time stamp hasn't changed in many samples. Something like this (if my knowledge of C# is good enough) where it stops after 100 samples (~10 secs) if the end time stamp has not changed at all:

// Stop the execution after we've been far enough past the found intro region
if (start != -1 && i - end >= 100) {
  break;
}

This would be placed at the bottom of the foreach loop, under the other if statement in the loop for the "findContiguousRegion" function. Not only would it prevent the intro from being misreported as longer than it actually is, but it would also stop the execution early if the intro is already found.

Feel free to do your own testing before you decide if you want to add this or not.

  • Like 1
Link to comment
Share on other sites

1 minute ago, samuelqwe said:

@chef in my testing, I've noticed that sometimes an intro end time stamp can be extended if there happens to be a few similar sounds after the intro has ended, so I suggest we stop looking for the contiguous region if the intro end time stamp hasn't changed in many samples. Something like this (if my knowledge of C# is good enough) where it stops after 100 samples (~10 secs) if the end time stamp has not changed at all:



// Stop the execution after we've been far enough past the found intro region
if (start != -1 && i - end >= 100) {
  break;
}

This would be placed at the bottom of the foreach loop, under the other if statement in the loop for the "findContiguousRegion" function. Not only would it prevent the intro from being misreported as longer than it actually is, but it would also stop the execution early if the intro is already found.

Feel free to do your own testing before you decide if you want to add this or not.

I can test it,

So now we look like this:

       // Find the intro region based on Hamming distances
        private static Tuple<int, int> findContiguousRegion(List<uint> arr, int upperLimit)
        {
            var start = -1;
            var end = -1;
            foreach (var i in Enumerable.Range(0, arr.Count()))
            {
                if (arr[i] < upperLimit && nextOnesAreAlsoSmall(arr, i, upperLimit))
                {
                    // Stop the execution after we've been far enough past the found intro region
                    if (start != -1 && i - end >= 100) {
                        break;
                    }

                    if (start == -1)
                    {
                        start = i;
                    }

                    end = i;
                }
            }

            return Tuple.Create(start, end);
        }

 

Edited by chef
Link to comment
Share on other sites

samuelqwe
4 minutes ago, chef said:

I can test it,

So now we look like this:



       // Find the intro region based on Hamming distances
        private static Tuple<int, int> findContiguousRegion(List<uint> arr, int upperLimit)
        {
            var start = -1;
            var end = -1;
            foreach (var i in Enumerable.Range(0, arr.Count()))
            {
                if (arr[i] < upperLimit && nextOnesAreAlsoSmall(arr, i, upperLimit))
                {
                    // Stop the execution after we've been far enough past the found intro region
                    if (start != -1 && i - end >= 100) {
                        break;
                    }

                    if (start == -1)
                    {
                        start = i;
                    }

                    end = i;
                }
            }

            return Tuple.Create(start, end);
        }

 

Actually I was thinking it could be below the outer if statement. That way we can stop the execution no matter what if there hasn’t been a change in 100 samples.

EDIT: like this

       // Find the intro region based on Hamming distances
        private static Tuple<int, int> findContiguousRegion(List<uint> arr, int upperLimit)
        {
            var start = -1;
            var end = -1;
            foreach (var i in Enumerable.Range(0, arr.Count()))
            {
                if (arr[i] < upperLimit && nextOnesAreAlsoSmall(arr, i, upperLimit))
                {

                    if (start == -1)
                    {
                        start = i;
                    }

                    end = i;
                }
              
              	// Stop the execution after we've been far enough past the found intro region
                if (start != -1 && i - end >= 100) {
                    break;
                }
            }

            return Tuple.Create(start, end);
        }

 

Edited by samuelqwe
Link to comment
Share on other sites

Just now, samuelqwe said:

Actually I was thinking it could be below the outer if statement. That way we can stop the execution no matter what if there hasn’t been a change in 100 samples.

right! 

// Find the intro region based on Hamming distances
        private static Tuple<int, int> findContiguousRegion(List<uint> arr, int upperLimit)
        {
            var start = -1;
            var end = -1;
            foreach (var i in Enumerable.Range(0, arr.Count()))
            {
                // Stop the execution after we've been far enough past the found intro region
                if (start != -1 && i - end >= 100) {
                    break;
                }
                if (arr[i] < upperLimit && nextOnesAreAlsoSmall(arr, i, upperLimit))
                {
                    

                    if (start == -1)
                    {
                        start = i;
                    }

                    end = i;
                }
            }

            return Tuple.Create(start, end);
        }

 

  • Like 1
Link to comment
Share on other sites

samuelqwe
Just now, chef said:

right! 


// Find the intro region based on Hamming distances
        private static Tuple<int, int> findContiguousRegion(List<uint> arr, int upperLimit)
        {
            var start = -1;
            var end = -1;
            foreach (var i in Enumerable.Range(0, arr.Count()))
            {
                // Stop the execution after we've been far enough past the found intro region
                if (start != -1 && i - end >= 100) {
                    break;
                }
                if (arr[i] < upperLimit && nextOnesAreAlsoSmall(arr, i, upperLimit))
                {
                    

                    if (start == -1)
                    {
                        start = i;
                    }

                    end = i;
                }
            }

            return Tuple.Create(start, end);
        }

 

That should work too, haha

  • Like 1
Link to comment
Share on other sites

PenkethBoy

@chef- will the plugin or could it write a file (chapter file or similar) to the folder of the episode - so alongside - named like the episode - with a suitable extension - say .chap or something emby currently would ignore

as i would like to take those files and remux the extra chapters into my files etc

  • Like 1
Link to comment
Share on other sites

4 hours ago, PenkethBoy said:

@chef- will the plugin or could it write a file (chapter file or similar) to the folder of the episode - so alongside - named like the episode - with a suitable extension - say .chap or something emby currently would ignore

as i would like to take those files and remux the extra chapters into my files etc

it is possible we could hold data like that.

Link to comment
Share on other sites

Here is the plugin code on Github so far. Most of the layout is there.

https://github.com/chefbennyj1/Emby.IntroSkip

 

I have a questions:

If there is not a good finger print match, and the FileRegions return -1,

at what point do we decide there is no intro for the episode, or try another pair of episodes to match?

I added an Exception in the code which gets thrown if the Regions are -1. This tells the scheduled task to try another episode pair.

You can see this here: https://github.com/chefbennyj1/Emby.IntroSkip/blob/master/IntroSkip/IntroDetection.cs#L344

Edited by chef
  • Like 1
Link to comment
Share on other sites

PenkethBoy
1 hour ago, chef said:

it is possible we could hold data like that.

thats almost a luke answer :)

is that a yes we can have it write out the intro info to files beside the videos

can be kept in a db file - but thats subject to corruption etc should something happen

  • Like 1
  • Haha 1
Link to comment
Share on other sites

Just now, PenkethBoy said:

thats almost a luke answer :)

is that a yes we can have it write out the intro info to files beside the videos

can be kept in a db file - but thats subject to corruption etc should something happen

LOL, 

Keep an episode intro json file in the season folder for all the episodes for each series?

Link to comment
Share on other sites

samuelqwe
2 hours ago, chef said:

Here is the plugin code on Github so far. Most of the layout is there.

https://github.com/chefbennyj1/Emby.IntroSkip

 

I have a questions:

If there is not a good finger print match, and the FileRegions return -1,

at what point do we decide there is no intro for the episode, or try another pair of episodes to match?

I added an Exception in the code which gets thrown if the Regions are -1. This tells the scheduled task to try another episode pair.

You can see this here: https://github.com/chefbennyj1/Emby.IntroSkip/blob/master/IntroSkip/IntroDetection.cs#L344

That’s some very good progress! Amazing work!

To answer your question, here is my idea:

The way I see it, we should return no intro if:
1. The region found is -1, -1
2. The region found is too short to seem like an intro (10 seconds should be good enough)

From there, I would likely retry two other times (if possible). If after three tries there’s still no intro found, I think it’s safe to assume the episode doesn’t have an intro. We could probably do the same thing for the season, if three tries on three different episodes have all failed to find an intro, the show probably doesn’t have an intro.

We could probably test and see how that does, and make some adjustments if necessary.

Does that seem like a good idea?

  • Like 1
Link to comment
Share on other sites

        private static bool IsEquitableIntro(double fileRegionStart, double fileRegionEnd)
        {
            if(TimeSpan.FromSeconds(Math.Round(fileRegionEnd)) - TimeSpan.FromSeconds(Math.Round(fileRegionStart)) <= TimeSpan.FromSeconds(10))
            {
                return false;
            }
            return !(fileRegionStart <= -1) && !(fileRegionEnd <= -1);
        }

 

LOL, I didn't know what else to call it.

This is one of the only times where "!notting" out a less-then condition seem to be okay to do.

Edited by chef
Link to comment
Share on other sites

samuelqwe
23 minutes ago, chef said:

        private static bool IsEquitableIntro(double fileRegionStart, double fileRegionEnd)
        {
            if(TimeSpan.FromSeconds(Math.Round(fileRegionEnd)) - TimeSpan.FromSeconds(Math.Round(fileRegionStart)) <= TimeSpan.FromSeconds(10))
            {
                return false;
            }
            return !(fileRegionStart <= -1) && !(fileRegionEnd <= -1);
        }

 

LOL, I didn't know what else to call it.

This is one of the only times where "!notting" out a less-then condition seem to be okay to do.

Oops, I forgot to mention the code already did that check here (since I was already doing it in my script):

else if (commonRegionEnd - commonRegionStart < 10) {
	// -1 means intro does not exists
	firstFileRegionStart  = -1.0;
	firstFileRegionEnd    = -1.0;
	secondFileRegionStart = -1.0;
	secondFileRegionEnd   = -1.0;
}

These are the line 343 to 350 in your latest commit (you'll see that comment a few lines above that mentions the length check). You could move that check to a function if you want though.

Link to comment
Share on other sites

2 minutes ago, samuelqwe said:

Oops, I forgot to mention the code already did that check here (since I was already doing it in my script):



else if (commonRegionEnd - commonRegionStart < 10) {
	// -1 means intro does not exists
	firstFileRegionStart  = -1.0;
	firstFileRegionEnd    = -1.0;
	secondFileRegionStart = -1.0;
	secondFileRegionEnd   = -1.0;
}

These are the line 343 to 350 in your latest commit (you'll see that comment a few lines above that mentions the length check). You could move that check to a function if you want though.

Cool,  only have to check for -1 then, and we can do that before the return.

We only really have to check firstFileRegionStart, and secondFileRegionStart for -1? 

Edited by chef
Link to comment
Share on other sites

samuelqwe
18 minutes ago, chef said:

We only really have to check firstFileRegionStart, and secondFileRegionStart for -1? 

Yeah, probably. Since otherwise if there was a region found, the start would be at least index 0.

If you look right above the length check I mentioned in my previous post, you can probably modify that there to check for -1, since I think that’s what it was already kinda doing.

Most of that logic was taken from the original Go version I translated the code from, so I haven’t really made any changes to that logic.

Edited by samuelqwe
  • Like 1
Link to comment
Share on other sites

Creating the episode index offset chooser for detection is a little bit complicated. 

Pretty much returning a Tuple of two integers (which are Episode indexes) and incrementing the index of  "EpisodeToCompare" with "EpisodeComparable", making sure to skip the index if it is the same, so we don't compare an episode with it's self.

This is also based on a successful match. 

 

I limited the series scan to 3 series results when it comes time to test it out.

After an initial scan of the library, it will be a really quick task, only having to scan a couple of new items as they are added, but the initial scan, it's going to be a big one.

Edited by chef
Link to comment
Share on other sites

samuelqwe
22 minutes ago, chef said:

Creating the episode index offset chooser for detection is a little bit complicated. 

Pretty much returning a Tuple of two integers (which are Episode indexes) and incrementing the index of  "EpisodeToCompare" with "EpisodeComparable", making sure to skip the index if it is the same, so we don't compare an episode with it's self.

This is also based on a successful match. 

 

I limited the series scan to 3 series results when it comes time to test it out.

After an initial scan of the library, it will be a really quick task, only having to scan a couple of new items as they are added, but the initial scan, it's going to be a big one.

Maybe something like, if the intro is found, increase episodeToCompareIndex by 1, and if no intro is found, increase episodeComparableIndex by 1.

But then you would have to make sure those indexes don’t become the same as to not compare the episode to itself, and you would probably want a fail counter that could reset when successful to see how many times the intro detection fails in row. Then you could use the fail counter to stop the intro detection for a season that keeps failing without having to go through all the episodes.

Edited by samuelqwe
Link to comment
Share on other sites

Once we encode two episodes for a series, and we are successful, we only have to encode the next episode of the series.

That'll cut down on time.

If the second attempt to compare episodes fails, then we have to encode two more, but, so far I haven't run into many issues with pin pointing intros, and getting the proper data.

The margin of error is very, very low.

 

  • Like 2
Link to comment
Share on other sites

10 minutes ago, crusher11 said:

Did you end up looking at Psych?

 

A friend of mine has it, but they had a technical issue and are currently unavailable.  The moment I see them again, they'll lend me a copy ;)

Link to comment
Share on other sites

@PenkethBoy  or

@TeamB

because eMby will show episodes in season which are Missing, or upcoming, I need to exclude these from my api query, so I don't have to create a separate list of episodes that are actually there.

Do you know what param I need to set in the InternalItemQuery to get only episodes which actually exist?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...