Jump to content

Show Intro Skip Option (Plugin)

Recommended Posts

how in all the noise are you going to determine thats the ONE?

Share on other sites

Just now, PenkethBoy said:

how in all the noise are you going to determine thats the ONE?

I checked the actual stream, and found the closest one. LOL!

• 1

see my dm

• 1
Share on other sites

Okay! I think I have something!

this command line will select the end of the intro!

\$"-t 00:10:00 -i \"{input}\" -af silencedetect=noise=-50dB:d=0.2 -f null -";

this says: only register a decibel drop of -50 if it happens for only 0.2 of a second.  This happens right after an intro ends.

To test:

1. run the command line

2. take the read out from the second set of silence numbers (the first silence is always just before the stream starts)

3. take the silence ending number, and divide by 60 to get the position in the stream (because it's seconds)

4. open the stream in emby, and skip forward to that position... it's most likely the end of the intro.

Edited by chef
Share on other sites

I attached an example from the office with my cobbled together script, it isn't perfect and can be refined but does do the trick.

You run this like so:
`python3 finger.py bad.mp3 epsidodeTitle.mkv`

• 1
Share on other sites

16 minutes ago, chef said:

Okay! I think I have something!

this command line will select the end of the intro!

\$"-t 00:10:00 -i \"{input}\" -af silencedetect=noise=-50dB:d=0.2 -f null -";

this says: only register a decibel drop of -60 if it happens for only 0.2 of a second.  This happens right after an intro ends.

To test:

1. run the command line

2. take the read out from the second set of silence numbers (the first silence is always just before the stream starts)

3. take the silence ending number, and divide by 60 to get the position in the stream (because it's seconds)

4. open the stream in emby, and skip forward to that position... it's most likely the end of the intro.

I can confirm, that on shows like "Mr. Robot" which doesn't have an intro sequence, only a recap, that it works.

Shows like "Rick and Morty" will pin point the end of the intro perfectly for each episode.

I'm about to test Game of Thrones because they have a recap and a really long into.

• 1
Share on other sites

5 minutes ago, chef said:

I can confirm, that on shows like "Mr. Robot" which doesn't have an intro sequence, only a recap, that it works.

Shows like "Rick and Morty" will pin point the end of the intro perfectly for each episode.

I'm about to test Game of Thrones because they have a recap and a really long into.

This does work for Game of Thrones as well. It will find the end of the HBO TV static logo, and then again it will find the end of the intro.

• 1
Share on other sites

@chef will you be able to write a plugin or do we need @Luke to put it in the main code?

Share on other sites

42 minutes ago, Sammy said:

@chef will you be able to write a plugin or do we need @Luke to put it in the main code?

I am pretty sure this is the closest way of getting this feature to work with a scan.

I am currently doing some more testing with ffmpeg to see how close I can get with random episodes from random series.

I'll definitely put my results here and if it works enough to implement, then yeah.

• 2
• 2
Share on other sites

Apparently we can check for a black screen too, using ffmpeg.

This is a huge help. Now, if the audio drops out and there is a black screen at the same time near the beginning of the media stream, we can assume with almost 99% success that that is the end of the episode intro.

I have a proof of concept test plugin almost completed.

For now it will scan a couple episodes you want to try out, the simulate a 'skip' button  with a popup in the web app.

• 3
• 1
Share on other sites

@PenkethBoy do you know how to get the physical full path name for a video file from the API?

Share on other sites

emby API i assume

&fields=path

• 1
Share on other sites

12 minutes ago, PenkethBoy said:

emby API i assume

&fields=path

Excellent! That is correct

Share on other sites

tut tut - you doubted me!

• 1
• 1
Share on other sites

So how's the Plugin coming along?

HeHe!

Share on other sites

19 minutes ago, Sammy said:

So how's the Plugin coming along?

HeHe!

I can't test from work, I wrote it here, but I need to check the ffmpeg instance on the actual server to see if it is actually doing what I want... So I'm going home.

• 2
Share on other sites

Oh boy! I've done it... It is working. A couple more tests, and we'll see how it goes.

Share on other sites

Woot woot!

Working!

(I put music to the video because otherwise, it's boring...)

Edited by chef
• 6
Share on other sites

@chef Cool weekend project

• 1
• 1
Share on other sites

@chefnice

now how do you expect emby to deal with this so a user can skip to the end of the intro?

chapter markers in the file?

or some plugin "remote" control command

or ......

Share on other sites

being lazy (not re reading the whole thread)- but whats the ffmpeg command and how are you picking the beginning and end of the intro?

i'm currently thinking of a script to process files offline and add chapter markers to the files

Share on other sites

7 hours ago, PenkethBoy said:

@chefnice

now how do you expect emby to deal with this so a user can skip to the end of the intro?

chapter markers in the file?

or some plugin "remote" control command

or ......

I would say that each client will have to implement the skipping option.

What this code could do is flag three new xml tags inside the episodes metadata.

<HasIntro>  boolean

<IntroStart> TimeSpan

<IntroEnd> TimeSpan

The client would just watch the "PlaybackProgressEvent" for the "skip" button to show up and if the user presses it, remote control to the "IntroEnd" time stamp of the stream. That is, if the "HasIntro" is flagged as true.

To figure out where the intro starts, I read that PDF I posted a while back.

In that article, it was mentioned that the average time of an intro was 36.5 seconds.

It was also mentioned that there is a 15 second discrepancy in that average.

So...

After ffmpeg finds the timestamp of the end of the intro (using blackdetect, and silencedetect), I take that time stamp, and subtract 36.5s.  (Call this number "possibleStart").

Then use a condition that says that if the remaining time between "possibleStart" and the beginning of the stream is less then the discrepancy (15 seconds), we assume that the intro starts at the begining of the stream.

At this point we show the skip option at the streams 2 second mark. (This allows the stream to fully start, and the user to realize that an intro is starting...)

If the "possibleStart" is greater, then we know that the intro is happening later in the stream, we don't show the skip option until the calculated "possibleStart" time.

I think my maths are right... But I can get confused sometimes, so if that doesn't sound right, let me know...

I'm doing more tests today, and I have fit the entire flagging system in a scheduled task.

Edited by chef
Share on other sites

Here is the entire Scheduled Task code to see how it works (currently only having to utilize "blackdetect" in ffmpeg) the code needs a bit of refactoring for certain.

Take note of the thresholds of ffmpeg's "blackdetect" command line

Also another interesting fact is that ffmpeg doesn't send standard output... Instead it sends data on the StandardError... WTF?? Took me a while to figure that out.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using MediaBrowser.Controller.Entities;
using MediaBrowser.Controller.Library;
using MediaBrowser.Model.Logging;
using MediaBrowser.Model.Tasks;
using VideoIntroMaker.Configuration;

namespace VideoIntroMaker
{
public class IntroDetectionScheduleTask : IScheduledTask, IConfigurableScheduledTask
{
private static ILogger Log             { get; set; }
private ILibraryManager LibraryManager { get; }
private static double Step = 0.0;
public IntroDetectionScheduleTask(ILogManager logManager, ILibraryManager libMan)
{
Log = logManager.GetLogger(Plugin.Instance.Name);
LibraryManager = libMan;
}

private IntroData DetectIntroData(BaseItem episode, IProgress<double> progress)
{
var blackFrames = DetectBlackFrame(episode.Path, progress);
var introBlackFrame = blackFrames.FirstOrDefault(i => i.time != "");

if (introBlackFrame != null)
{
Log.Info("INTRO DETECT HAS " + introBlackFrame.time);
return new IntroData()
{
Id = episode.InternalId,
HasIntro = true,
IntroEnd = CalculateIntroEndTime(introBlackFrame.time),
IntroStart = CalculateIntroStartTime(CalculateIntroEndTime(introBlackFrame.time))
};

}
return new IntroData()
{
Id = episode.InternalId,
HasIntro = false,
IntroStart = TimeSpan.MinValue,
IntroEnd = TimeSpan.MinValue
};
}

private static List<Detection> DetectBlackFrame(string input, IProgress<double> progress)
{
var blackness = new List<Detection>();
var ffmpegPath = "ffmpeg.exe";
progress.Report(Step + 1.0);
var procStartInfo = new ProcessStartInfo(ffmpegPath,\$"-t 00:05:00 -i \"{input}\" -vf \"blackdetect=d=0.5:pix_th=0.00\" -f null -")
{
RedirectStandardOutput = true,
RedirectStandardError = true,
UseShellExecute = false,
CreateNoWindow = true,
};

var process = new Process {StartInfo = procStartInfo};

process.Start();

string processOutput = null;

while ((processOutput = process.StandardError.ReadLine()) != null)
{
try
{
if (processOutput.Contains("blackdetect"))
{
Log.Info("FFMPEG INTRO SCAN: " + processOutput.Split(':')[1].Split(' ')[0]);
blackness.Add(new Detection()
{
time = processOutput.Split(':')[1].Split(' ')[0]
});
}
}
catch { }
}

return blackness;
}

public async Task Execute(CancellationToken cancellationToken, IProgress<double> progress)
{
//For now we will test by saving data to a plugin configuration XML
//In the future this can be edited to save the three data points to the episodes .nfo/.xml metadata file
var config = Plugin.Instance.Configuration;
var tempIntroList = new List<IntroData>();
if (config.Intros.Any())
{
foreach (var item in config.Intros)
{
var episode = LibraryManager.GetItemById(item.Id);
var introData = DetectIntroData(episode, progress); //<-- Here is out intro detection routine
progress.Report(Step + 1.0);
tempIntroList.Add(introData);
}
config.SavedIntros.AddRange(tempIntroList);
config.Intros.Clear();
Plugin.Instance.UpdateConfiguration(config);
}
progress.Report(100.0);
}

public IEnumerable<TaskTriggerInfo> GetDefaultTriggers()
{
return new[]
{
new TaskTriggerInfo
{
Type = TaskTriggerInfo.TriggerInterval,
IntervalTicks = TimeSpan.FromHours(24).Ticks
}
};
}

private TimeSpan CalculateIntroEndTime(string blackFrameDetect)
{
//Format the seconds found into a TimeSpan for easy conditioning (converting to Ticks) later on
return TimeSpan.Parse(\$"00:00:{Math.Round(Convert.ToDouble(blackFrameDetect))}");
}

private TimeSpan CalculateIntroStartTime(TimeSpan introEnd)
{
//Here is our calculation to figure out the intro start time
return (introEnd - TimeSpan.FromSeconds(35.6)) < TimeSpan.FromSeconds(15.7)
? TimeSpan.FromSeconds(2)
: introEnd - TimeSpan.FromSeconds(35.6);
}

public string Name        => "Detect Episode Intro Skip";
public string Key         => "Intro Skip Options";
public string Description => "Detect start and finish time of episode title sequences to allow for a 'skip' option";
public string Category    =>  "Detect Episode Intro Skip";
public bool IsHidden      => false;
public bool IsEnabled     => true;
public bool IsLogged      => true;
}
}

Edited by chef
• 1
Share on other sites

1 hour ago, chef said:

I would say that each client will have to implement the skipping option.

What this code could do is flag three new xml tags inside the episodes metadata.

<HasIntro>  boolean

<IntroStart> TimeSpan

<IntroEnd> TimeSpan

The client would just watch the "PlaybackProgressEvent" for the "skip" button to show up and if the user presses it, remote control to the "IntroEnd" time stamp of the stream. That is, if the "HasIntro" is flagged as true.

To figure out where the intro starts, I read that PDF I posted a while back.

In that article, it was mentioned that the average time of an intro was 36.5 seconds.

It was also mentioned that there is a 15 second discrepancy in that average.

So...

After ffmpeg finds the timestamp of the end of the intro (using blackdetect, and silencedetect), I take that time stamp, and subtract 36.5s.  (Call this number "possibleStart").

Then use a condition that says that if the remaining time between "possibleStart" and the beginning of the stream is less then the discrepancy (15 seconds), we assume that the intro starts at the begining of the stream.

At this point we show the skip option at the streams 2 second mark. (This allows the stream to fully start, and the user to realize that an intro is starting...)

If the "possibleStart" is greater, then we know that the intro is happening later in the stream, we don't show the skip option until the calculated "possibleStart" time.

I think my maths are right... But I can get confused sometimes, so if that doesn't sound right, let me know...

I'm doing more tests today, and I have fit the entire flagging system in a scheduled task.

Since finding the start of the intro with this method is dicey at best, I would think we'd need to have a way for the user to initiate the skip without needing any sort of special button appearing on the screen.  Because, if they have to sit through potentially 15 seconds of the intro to be able to skip it, that really doesn't do much.  Therefore, if this technique were used, I  would think some sort of use of chapters would be the way to go.

Also, there may not be an intro at all.  You may just be finding the first commercial break.

• 1
Share on other sites

40 minutes ago, ebr said:

Since finding the start of the intro with this method is dicey at best, I would think we'd need to have a way for the user to initiate the skip without needing any sort of special button appearing on the screen.  Because, if they have to sit through potentially 15 seconds of the intro to be able to skip it, that really doesn't do much.  Therefore, if this technique were used, I  would think some sort of use of chapters would be the way to go.

Also, there may not be an intro at all.  You may just be finding the first commercial break.

I agree there is a lot of assumptions in the code.

Can we assume intros happen before the first commercial break?

Perhaps, adding this sort of detection code to the chapter task? Using blackdetect to mark the first chapter as intro?

Edited by chef
Share on other sites

This topic is now closed to further replies.
×
×
• Create New...