Completed Show Intro Skip Option

[rbjtech 4257]

1 hour ago, samuelqwe said:

The plugin is mostly in a proof of concept phase to show that it’s possible to detect intros automatically fairly well, so the data currently isn’t really used by anything.

Eventually, I think the goal would be to have a Netflix-like “skip intro” button, but that’s not something a plugin can add and would require work by the Emby team to add it into clients. Perhaps chapters is doable in the meantime, but there are only a few clients that let you skip chapters during playback.

Hopefully, as @chef improves the plugin, we can get to a point where it works well enough to get implemented natively by Emby.

ok that's cool - thanks for the info.

For chapters, simply writing to an XML file as an import to a MKV would be a great stepping stone .. Hint, Hint .. 

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Chapters SYSTEM "matroskachapters.dtd">

<Chapters>
  <EditionEntry>
    <ChapterAtom>
      <ChapterTimeStart>00:00:00.000</ChapterTimeStart>
      <ChapterDisplay>
        <ChapterString>Chef'sSkipIntroStart</ChapterString>
        <ChapterLanguage>eng</ChapterLanguage>
      </ChapterDisplay>
    </ChapterAtom>
    <ChapterAtom>
      <ChapterTimeStart>00:00:10.010</ChapterTimeStart>
      <ChapterDisplay>
        <ChapterString>Chef'sSkipIntroEnd</ChapterString>
        <ChapterLanguage>eng</ChapterLanguage>
      </ChapterDisplay>
    </ChapterAtom>
     </EditionEntry>
</Chapters>

Then a simple mkvpropedit --chapters abovefile.XML "C:\path...\your.mkv" will add these chapter points ...

Maybe write the chapter XML to the media location and then a simple script could pick them up and apply them, then remove the XML ..

I believe chapter navigation is reasonably close to being released on some of the clients ? - well if @Luke's recent comments are anything to go by .. 

Exciting stuff ! 

 

[chef 3745]

17 minutes ago, rbjtech said:

ok that's cool - thanks for the info.

For chapters, simply writing to an XML file as an import to a MKV would be a great stepping stone .. Hint, Hint .. 

<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Chapters SYSTEM "matroskachapters.dtd">

<Chapters>
  <EditionEntry>
    <ChapterAtom>
      <ChapterTimeStart>00:00:00.000</ChapterTimeStart>
      <ChapterDisplay>
        <ChapterString>Chef'sSkipIntroStart</ChapterString>
        <ChapterLanguage>eng</ChapterLanguage>
      </ChapterDisplay>
    </ChapterAtom>
    <ChapterAtom>
      <ChapterTimeStart>00:00:10.010</ChapterTimeStart>
      <ChapterDisplay>
        <ChapterString>Chef'sSkipIntroEnd</ChapterString>
        <ChapterLanguage>eng</ChapterLanguage>
      </ChapterDisplay>
    </ChapterAtom>
     </EditionEntry>
</Chapters>

Then a simple mkvpropedit --chapters abovefile.XML "C:\path...\your.mkv" will add these chapter points ...

Maybe write the chapter XML to the media location and then a simple script could pick them up and apply them, then remove the XML ..

I believe chapter navigation is reasonably close to being released on some of the clients ? - well if @Luke's recent comments are anything to go by .. 

Exciting stuff ! 

 

Serialize and edit the chapter xml, sounds like something to look into.

[Dmunozv04 1]

@cheffpcalc binaries are only provided for amd64, but I imagine it doesn’t matter because it will be implemented on emby’s own ffmpeg soon. 
If anyone needs help using it on arm right now, reply. 

[chef 3745]

16 hours ago, Dmunozv04 said:

@cheffpcalc binaries are only provided for amd64, but I imagine it doesn’t matter because it will be implemented on emby’s own ffmpeg soon. 
If anyone needs help using it on arm right now, reply. 

I've got the ffmpeg version and I'm trying to implement the chromaprint this weekend.

@TeamB what was that chromaprint flag on ffmpeg? Or... Is it -chromaprint

[samuelqwe 313]

7 hours ago, chef said:

@TeamB what was that chromaprint flag on ffmpeg? Or... Is it -chromaprint

@chef, something like this?
 

\Install\ffmpeg\ffmpeg-4.3.1-2021-01-01-full_build\bin\ffmpeg.exe ^
-i "C:\Data\media\movie.mkv" ^
-ss 00:00:30 ^
-t 00:01:00 ^
-ac 1 ^
-acodec pcm_s16le ^
-ar 16000 ^
-c:v nul ^
-f chromaprint ^
-fp_format raw ^
audio_chromaprint.bin

This is what TeamB sent a while back in our message chain.

Edited February 28, 2021 by samuelqwe

[chef 3745]

1 hour ago, samuelqwe said:

@chef, something like this?
 

\Install\ffmpeg\ffmpeg-4.3.1-2021-01-01-full_build\bin\ffmpeg.exe ^
-i "C:\Data\media\movie.mkv" ^
-ss 00:00:30 ^
-t 00:01:00 ^
-ac 1 ^
-acodec pcm_s16le ^
-ar 16000 ^
-c:v nul ^
-f chromaprint ^
-fp_format raw ^
audio_chromaprint.bin

 

Thank you sir! Let's see if this thing'll work

Edited February 28, 2021 by chef

[GiGo 103]

This would be a brilliant feature, also credit skip (easpically ripped Netflix shows  )

I'm not a programmer and have no clue on these types of things BUT... could there not be a database of theme songs, when the file is processed it detects the theme song, marks the posistion of theme song and then it can skip the intro? No idea if or how that could work, maybe a similar why to Picard? All theme songs can be referenced by the IMDB/TVDB tag?

[samuelqwe 313]

50 minutes ago, GiGo said:

I'm not a programmer and have no clue on these types of things BUT... could there not be a database of theme songs, when the file is processed it detects the theme song, marks the posistion of theme song and then it can skip the intro? No idea if or how that could work, maybe a similar why to Picard? All theme songs can be referenced by the IMDB/TVDB tag?

That’s a lot easier said than done, since making and maintaining a database like that is a huge undertaking. The method we are currently trying figures out the intro all by itself, by simply comparing common audio segments in the beginning of two episodes, which in most cases will be the intro. We can then use that to calculate when the intro starts and stops.

52 minutes ago, GiGo said:

also credit skip (easpically ripped Netflix shows  )

That’s a little harder to do, since the credits don’t always have recurring theme music. There may be a way besides analyzing the audio, but for the moment there doesn’t seem to be a good way to accurately detect credits.

[GiGo 103]

6 minutes ago, samuelqwe said:

That’s a lot easier said than done, since making and maintaining a database like that is a huge undertaking. The method we are currently trying figures out the intro all by itself, by simply comparing common audio segments in the beginning of two episodes, which in most cases will be the intro. We can then use that to calculate when the intro starts and stops.

That’s a little harder to do, since the credits don’t always have recurring theme music. There may be a way besides analyzing the audio, but for the moment there doesn’t seem to be a good way to accurately detect credits.

I've installed the plugin and will let you know how I get on with it. Am I right is assuming it doesn't work on a Roku?

[samuelqwe 313]

32 minutes ago, GiGo said:

I've installed the plugin and will let you know how I get on with it. Am I right is assuming it doesn't work on a Roku?

The plugin currently doesn’t actually add any skipping functionality, that would need to be added by the Emby team.

The plugin simply serves as a proof of concept to test out if detecting the intros automatically is viable (you can see the detected intros on the plugin page). If we can prove that it works well, then perhaps we can convince the Emby team to implement the functionality.

Edited February 28, 2021 by samuelqwe

[chef 3745]

@samuelqwe

Holy $#!7 Sam she workin'

The code needs refactoring, it's dirty because the finger prints was split up from the encoding. I need to put it all together.

Edited February 28, 2021 by chef

[GiGo 103]

So we should be looking forward to this nice new feature soon then? 

[samuelqwe 313]

51 minutes ago, chef said:

@samuelqwe

Holy $#!7 Sam she workin'

Awesome work, can’t wait to test it out and see how this all in one version performs!

56 minutes ago, chef said:

The code needs refactoring, it's dirty because the finger prints was split up from the encoding. I need to put it all together.

That’s definitely an advantage of having FFmpeg do all the work. Should simplify the whole process once it’s all refactored.

[rbjtech 4257]

On 28/02/2021 at 08:41, samuelqwe said:

That’s a little harder to do, since the credits don’t always have recurring theme music. There may be a way besides analyzing the audio, but for the moment there doesn’t seem to be a good way to accurately detect credits.

I definitely think the chapter points on the original media are key here - On a well authored title, a chapter point is always put on the start of the credits - thus, if emby had a 'skip last chapter' then that is a simple solution to this problem that doesn't need any further development.

Of course if the media doesn't have these chapter points THEN identifying the start of the credits is the challenge, but it's the same process once you do.

If you could get a hash match to the original media somehow, then I guess a database could work (the same as it does for subtitle timing today) - but a local reference held in the .nfo or tagged as 'Credits' in the chapters would be a better solution imo.

[samuelqwe 313]

2 minutes ago, rbjtech said:

I definitely think the chapter points on the original media are key here - On a well authored title, a chapter point is always put on the start of the credits - thus, if emby had a 'skip last chapter' then that is a simple solution to this problem that doesn't need any further development.

Of course if the media doesn't have these chapter points THEN identifying the start of the credits is the challenge, but it's the same process once you do.

If you could get a hash match to the original media somehow, then I guess a database could work (the same as it does for subtitle timing today) - but a local reference held in the .nfo or tagged as 'Credits' in the chapters would be a better solution imo.

Yes, having those chapter points would likely be the ideal way to detect credits.

A database should always be a last resort IMO, as that’s usually very hard to maintain, but is technically doable.

I think currently the focus is intros, but once that is further refined I think it would definitely be interesting to experiment with different ways of detecting credits.

[Mibok 118]

Maybe an skip chapter option, and then a plugin that identify and writes a chapter named intro into the file. That could be the best of two world. Anyway i'm happy that we are getting a little closer to have an intro skip option 

[Micael456 43]

I was looking into how to make an emby plugin to do this very thing, so glad to hear there's one on its way soon. That sounds amazing! Means I don't need to figure out Emby's plugin system now :D.

 

For the credits, I was personally thinking that OCR would probably be the way to go. Emby already knows the name of the cast and crew, tie that in with some OCR scanning towards the end of the episode, working backwards, until you find the start of the TEXT: Cast Member sequence, would probably be a pretty safe way to detect the credits.

 

Just an idea, thought I may as well provide something useful since I spent that time thinking about it ;).

Edited March 11, 2021 by Micael456

[rbjtech 4257]

I believe @chef is actively working on this and is testing a new method of identifying the intro using new features in the latest ffmpeg.

Having played with the previous method, I was extremely impressed with it's accuracy so I am excited to see this development.

One of the key issues however, is getting it incorporated into the Apps which may take a lot longer than developing the code to identify them in the first place ... 

Chapter points may be an interim solution - but even those are not available in the majority of apps.

Nice idea on using the credits - especially on shows that don't use the same close music (which could use the same 'process' for intro's but in reverse)

 

 

[chef 3745]

1 hour ago, Micael456 said:

I was looking into how to make an emby plugin to do this very thing, so glad to hear there's one on its way soon. That sounds amazing! Means I don't need to figure out Emby's plugin system now :D.

 

For the credits, I was personally thinking that OCR would probably be the way to go. Emby already knows the name of the cast and crew, tie that in with some OCR scanning towards the end of the episode, working backwards, until you find the start of the TEXT: Cast Member sequence, would probably be a pretty safe way to detect the credits.

 

Just an idea, thought I may as well provide something useful since I spent that time thinking about it ;).

Ocr, sounds interesting. I was also thinking that credits could actually use the silence param in ffmpeg ad long as we look at the proper duration from the end.

 

I got a job cooking of you can believe that... So this new kitchen has me running... But tomorrow is a day off, so I was hoping to get back at this.

[Sammy 735]

26 minutes ago, chef said:

Ocr, sounds interesting. I was also thinking that credits could actually use the silence param in ffmpeg ad long as we look at the proper duration from the end.

 

I got a job cooking of you can believe that... So this new kitchen has me running... But tomorrow is a day off, so I was hoping to get back at this.

FANTASTIC!

Now, how'd the new guy get a Friday off???

[Banquero 30]

He instalado este plugin en Synology, y me ha reconocido las entradas y finales de las introducciones en muchas series, pero no veo que funcione, viendolo en mi TV Samsung..Tengo que instalar algo o simplemente ahora mismo es que no está implementado?..Muchas gracias...

[samuelqwe 313]

7 minutes ago, Banquero said:

He instalado este plugin en Synology, y me ha reconocido las entradas y finales de las introducciones en muchas series, pero no veo que funcione, viendolo en mi TV Samsung..Tengo que instalar algo o simplemente ahora mismo es que no está implementado?..Muchas gracias...

The plugin doesn’t actually add any intro skipping functionality, it currently only serves as a proof of concept to show that it is possible to automatically find and detect intros. The ability to skip the intros would need to be added by the Emby team.

[Banquero 30]

1 minute ago, samuelqwe said:

El complemento en realidad no agrega ninguna funcionalidad de omisión de intro, actualmente solo sirve como una prueba de concepto para mostrar que es posible encontrar y detectar intros automáticamente. El equipo de Emby debería agregar la capacidad de omitir las introducciones.

Vaya, me he quedado helado..Pensaba que funcionaría...espero que lo implementen pronto..En este sentido Plex, nos lleva mucha ventaja...Gracias por la información...Saludos...

[samuelqwe 313]

1 minute ago, Banquero said:

Vaya, me he quedado helado..Pensaba que funcionaría...espero que lo implementen pronto..En este sentido Plex, nos lleva mucha ventaja...Gracias por la información...Saludos...

The Emby team is currently working on many other important features behind the scenes, so it may be a while until we see this implemented in Emby. However, the plugin has allowed us to show the Emby team how it could potentially be done.

I’m sure the team is working as fast as possible and that other features just have a higher priority, but I would expect this to implemented eventually.

[tchirou 23]

Personally, the lack of this feature made me switch to plex with a broken heart after 5 years of Emby and a lifetime license. But for tv shows, skipping intros is mandatory once you are used to it.