Jump to content

Show Intro Skip Option (Plugin)


Liquidfire88

Recommended Posts

Just now, PenkethBoy said:

how in all the noise are you going to determine thats the ONE?

I checked the actual stream, and found the closest one. LOL!

  • Haha 1
Link to comment
Share on other sites

Okay! I think I have something!

this command line will select the end of the intro!

 

$"-t 00:10:00 -i \"{input}\" -af silencedetect=noise=-50dB:d=0.2 -f null -";

 

this says: only register a decibel drop of -50 if it happens for only 0.2 of a second.  This happens right after an intro ends. 

 

To test: 

1. run the command line

2. take the read out from the second set of silence numbers (the first silence is always just before the stream starts)

3. take the silence ending number, and divide by 60 to get the position in the stream (because it's seconds)

4. open the stream in emby, and skip forward to that position... it's most likely the end of the intro.

 

 

Edited by chef
Link to comment
Share on other sites

16 minutes ago, chef said:

Okay! I think I have something!

this command line will select the end of the intro!

 


$"-t 00:10:00 -i \"{input}\" -af silencedetect=noise=-50dB:d=0.2 -f null -";

 

this says: only register a decibel drop of -60 if it happens for only 0.2 of a second.  This happens right after an intro ends. 

 

To test: 

1. run the command line

2. take the read out from the second set of silence numbers (the first silence is always just before the stream starts)

3. take the silence ending number, and divide by 60 to get the position in the stream (because it's seconds)

4. open the stream in emby, and skip forward to that position... it's most likely the end of the intro.

 

 

I can confirm, that on shows like "Mr. Robot" which doesn't have an intro sequence, only a recap, that it works.

Shows like "Rick and Morty" will pin point the end of the intro perfectly for each episode.

I'm about to test Game of Thrones because they have a recap and a really long into.

  • Thanks 1
Link to comment
Share on other sites

5 minutes ago, chef said:

I can confirm, that on shows like "Mr. Robot" which doesn't have an intro sequence, only a recap, that it works.

Shows like "Rick and Morty" will pin point the end of the intro perfectly for each episode.

I'm about to test Game of Thrones because they have a recap and a really long into.

This does work for Game of Thrones as well. It will find the end of the HBO TV static logo, and then again it will find the end of the intro.

 

  • Like 1
Link to comment
Share on other sites

42 minutes ago, Sammy said:

@chef will you be able to write a plugin or do we need @Luke to put it in the main code?

 

I am pretty sure this is the closest way of getting this feature to work with a scan. 

I am currently doing some more testing with ffmpeg to see how close I can get with random episodes from random series. 

I'll definitely put my results here and if it works enough to implement, then yeah. 

  • Like 2
  • Thanks 2
Link to comment
Share on other sites

Apparently we can check for a black screen too, using ffmpeg.

This is a huge help. Now, if the audio drops out and there is a black screen at the same time near the beginning of the media stream, we can assume with almost 99% success that that is the end of the episode intro.

I have a proof of concept test plugin almost completed. 

 

For now it will scan a couple episodes you want to try out, the simulate a 'skip' button  with a popup in the web app.

 

  • Like 3
  • Thanks 1
Link to comment
Share on other sites

19 minutes ago, Sammy said:

So how's the Plugin coming along?

 

HeHe!

I can't test from work, I wrote it here, but I need to check the ffmpeg instance on the actual server to see if it is actually doing what I want... So I'm going home. 😂

  • Like 2
Link to comment
Share on other sites

Woot woot!

Working!

(I put music to the video because otherwise, it's boring...)

 

Edited by chef
  • Like 6
Link to comment
Share on other sites

@chefnice :)

now how do you expect emby to deal with this so a user can skip to the end of the intro?

chapter markers in the file?

or some plugin "remote" control command

or ......

 

Link to comment
Share on other sites

being lazy (not re reading the whole thread)- but whats the ffmpeg command and how are you picking the beginning and end of the intro?

i'm currently thinking of a script to process files offline and add chapter markers to the files

Link to comment
Share on other sites

7 hours ago, PenkethBoy said:

@chefnice :)

now how do you expect emby to deal with this so a user can skip to the end of the intro?

chapter markers in the file?

or some plugin "remote" control command

or ......

 

I would say that each client will have to implement the skipping option.

What this code could do is flag three new xml tags inside the episodes metadata.

<HasIntro>  boolean

<IntroStart> TimeSpan

<IntroEnd> TimeSpan

The client would just watch the "PlaybackProgressEvent" for the "skip" button to show up and if the user presses it, remote control to the "IntroEnd" time stamp of the stream. That is, if the "HasIntro" is flagged as true.

 

To figure out where the intro starts, I read that PDF I posted a while back.

In that article, it was mentioned that the average time of an intro was 36.5 seconds.

It was also mentioned that there is a 15 second discrepancy in that average.

So...

After ffmpeg finds the timestamp of the end of the intro (using blackdetect, and silencedetect), I take that time stamp, and subtract 36.5s.  (Call this number "possibleStart").

Then use a condition that says that if the remaining time between "possibleStart" and the beginning of the stream is less then the discrepancy (15 seconds), we assume that the intro starts at the begining of the stream.

At this point we show the skip option at the streams 2 second mark. (This allows the stream to fully start, and the user to realize that an intro is starting...)

If the "possibleStart" is greater, then we know that the intro is happening later in the stream, we don't show the skip option until the calculated "possibleStart" time.

 

I think my maths are right... But I can get confused sometimes, so if that doesn't sound right, let me know... 🤤

I'm doing more tests today, and I have fit the entire flagging system in a scheduled task. 

 

 

Edited by chef
Link to comment
Share on other sites

Here is the entire Scheduled Task code to see how it works (currently only having to utilize "blackdetect" in ffmpeg) the code needs a bit of refactoring for certain.

Take note of the thresholds of ffmpeg's "blackdetect" command line

Also another interesting fact is that ffmpeg doesn't send standard output... Instead it sends data on the StandardError... WTF?? Took me a while to figure that out. 

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Library;
using MediaBrowser.Model.Logging;
using MediaBrowser.Model.Tasks;
using VideoIntroMaker.Configuration;

namespace VideoIntroMaker
{
    public class IntroDetectionScheduleTask : IScheduledTask, IConfigurableScheduledTask
    {
        private static ILogger Log             { get; set; }
        private ILibraryManager LibraryManager { get; }
        private static double Step = 0.0;
        public IntroDetectionScheduleTask(ILogManager logManager, ILibraryManager libMan)
        {
            Log = logManager.GetLogger(Plugin.Instance.Name);
            LibraryManager = libMan;
        }

        private IntroData DetectIntroData(BaseItem episode, IProgress<double> progress)
        {
            var blackFrames = DetectBlackFrame(episode.Path, progress);
            var introBlackFrame = blackFrames.FirstOrDefault(i => i.time != "");
            
            if (introBlackFrame != null)
            {
                Log.Info("INTRO DETECT HAS " + introBlackFrame.time);
                return new IntroData()
                {
                    Id = episode.InternalId,
                    HasIntro = true,
                    IntroEnd = CalculateIntroEndTime(introBlackFrame.time),
                    IntroStart = CalculateIntroStartTime(CalculateIntroEndTime(introBlackFrame.time))
                };
                
            }
            return new IntroData()
            {
                Id = episode.InternalId,
                HasIntro = false,
                IntroStart = TimeSpan.MinValue,
                IntroEnd = TimeSpan.MinValue
            };
        }

        private static List<Detection> DetectBlackFrame(string input, IProgress<double> progress)
        {
            var blackness = new List<Detection>();
            var ffmpegPath = "ffmpeg.exe";
            progress.Report(Step + 1.0);
            var procStartInfo = new ProcessStartInfo(ffmpegPath,$"-t 00:05:00 -i \"{input}\" -vf \"blackdetect=d=0.5:pix_th=0.00\" -f null -")
            {
                RedirectStandardOutput = true,
                RedirectStandardError = true,
                UseShellExecute = false,
                CreateNoWindow = true,
            };

            var process = new Process {StartInfo = procStartInfo};

            process.Start();

            string processOutput = null;
           
            while ((processOutput = process.StandardError.ReadLine()) != null)
            {
                try
                {
                    if (processOutput.Contains("blackdetect"))
                    {
                        Log.Info("FFMPEG INTRO SCAN: " + processOutput.Split(':')[1].Split(' ')[0]);
                        blackness.Add(new Detection()
                        {
                            time = processOutput.Split(':')[1].Split(' ')[0]
                        });
                    }
                }
                catch { }
            }
            
            return blackness;
        }

        public async Task Execute(CancellationToken cancellationToken, IProgress<double> progress)
        {
            //For now we will test by saving data to a plugin configuration XML
            //In the future this can be edited to save the three data points to the episodes .nfo/.xml metadata file
            var config = Plugin.Instance.Configuration;
            var tempIntroList = new List<IntroData>();
            if (config.Intros.Any())
            {
                foreach (var item in config.Intros)
                {
                    var episode = LibraryManager.GetItemById(item.Id); 
                    var introData = DetectIntroData(episode, progress); //<-- Here is out intro detection routine
                    progress.Report(Step + 1.0);
                    tempIntroList.Add(introData);
                }
                config.SavedIntros.AddRange(tempIntroList);
                config.Intros.Clear();
                Plugin.Instance.UpdateConfiguration(config);
            }
            progress.Report(100.0);
        }

        public IEnumerable<TaskTriggerInfo> GetDefaultTriggers()
        {
            return new[]
            {
                new TaskTriggerInfo
                {
                    Type = TaskTriggerInfo.TriggerInterval,
                    IntervalTicks = TimeSpan.FromHours(24).Ticks
                }
            };
        }


        private TimeSpan CalculateIntroEndTime(string blackFrameDetect)
        {
            //Format the seconds found into a TimeSpan for easy conditioning (converting to Ticks) later on
            return TimeSpan.Parse($"00:00:{Math.Round(Convert.ToDouble(blackFrameDetect))}");
        }

        private TimeSpan CalculateIntroStartTime(TimeSpan introEnd)
        {
            //Here is our calculation to figure out the intro start time
            return (introEnd - TimeSpan.FromSeconds(35.6)) < TimeSpan.FromSeconds(15.7)
                ? TimeSpan.FromSeconds(2)
                : introEnd - TimeSpan.FromSeconds(35.6);
        }


        public string Name        => "Detect Episode Intro Skip";
        public string Key         => "Intro Skip Options";
        public string Description => "Detect start and finish time of episode title sequences to allow for a 'skip' option";
        public string Category    =>  "Detect Episode Intro Skip";
        public bool IsHidden      => false;
        public bool IsEnabled     => true;
        public bool IsLogged      => true;
    }
}

 

Edited by chef
  • Like 1
Link to comment
Share on other sites

1 hour ago, chef said:

I would say that each client will have to implement the skipping option.

What this code could do is flag three new xml tags inside the episodes metadata.

<HasIntro>  boolean

<IntroStart> TimeSpan

<IntroEnd> TimeSpan

The client would just watch the "PlaybackProgressEvent" for the "skip" button to show up and if the user presses it, remote control to the "IntroEnd" time stamp of the stream. That is, if the "HasIntro" is flagged as true.

 

To figure out where the intro starts, I read that PDF I posted a while back.

In that article, it was mentioned that the average time of an intro was 36.5 seconds.

It was also mentioned that there is a 15 second discrepancy in that average.

So...

After ffmpeg finds the timestamp of the end of the intro (using blackdetect, and silencedetect), I take that time stamp, and subtract 36.5s.  (Call this number "possibleStart").

Then use a condition that says that if the remaining time between "possibleStart" and the beginning of the stream is less then the discrepancy (15 seconds), we assume that the intro starts at the begining of the stream.

At this point we show the skip option at the streams 2 second mark. (This allows the stream to fully start, and the user to realize that an intro is starting...)

If the "possibleStart" is greater, then we know that the intro is happening later in the stream, we don't show the skip option until the calculated "possibleStart" time.

 

I think my maths are right... But I can get confused sometimes, so if that doesn't sound right, let me know... 🤤

I'm doing more tests today, and I have fit the entire flagging system in a scheduled task. 

 

 

Since finding the start of the intro with this method is dicey at best, I would think we'd need to have a way for the user to initiate the skip without needing any sort of special button appearing on the screen.  Because, if they have to sit through potentially 15 seconds of the intro to be able to skip it, that really doesn't do much.  Therefore, if this technique were used, I  would think some sort of use of chapters would be the way to go.

Also, there may not be an intro at all.  You may just be finding the first commercial break.

  • Like 1
Link to comment
Share on other sites

40 minutes ago, ebr said:

Since finding the start of the intro with this method is dicey at best, I would think we'd need to have a way for the user to initiate the skip without needing any sort of special button appearing on the screen.  Because, if they have to sit through potentially 15 seconds of the intro to be able to skip it, that really doesn't do much.  Therefore, if this technique were used, I  would think some sort of use of chapters would be the way to go.

Also, there may not be an intro at all.  You may just be finding the first commercial break.

 

I agree there is a lot of assumptions in the code.

Can we assume intros happen before the first commercial break?

Perhaps, adding this sort of detection code to the chapter task? Using blackdetect to mark the first chapter as intro?

Edited by chef
Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...