Completed Show Intro Skip Option

[chef 3749]

Overall I would like the clients to offer a skip button. Chapters are a fine thing to do now, but in the future, if the data is solid enough, that is what I would like to see. 

[rbjtech 4316]

9 hours ago, Micael456 said:

Not necessarily true. We can read in existing chapters, and re-write it out including the originals plus the skip ones (offsetting skip start and end if required). That way the user can have their chapters, as well as the IntroSkip ones.

ffmpeg should be able to map metadata and chapters for common formats if we wanted to actually touch the file, or we just save it internally in the Emby DB instead.

 

9 hours ago, crusher11 said:

I don't see why you couldn't simply add the required chapters to what's already there. If there's a chapter within, say, two seconds of the detected intro end time, use that chapter point instead of adding one.

But I'm not sure why you're surprised that shows have chapters. As you can see from my sample the vast majority do.

Alternatively, if this actually gets put into the UI the way Netflix and Amazon Prime do it - which one has to assume is the end goal here - do chapter markers need to be added at all? Surely that UI piece could simply read directly from the database of intro points?

Go back and read what is being proposed.  Nobody is suggesting we get rid of ANY chapter points.  The proposal was always to ADD just the Intro chapters as a temporary solution until the Core catches up with the development.  Given it has taken YEARS to even get chapter points included in the clients, do you honestly think we are going to get a proper 'Intro' skip any time soon ?

[rbjtech 4316]

Have a read of this post - this is the 'roadmap' -

https://emby.media/community/index.php?/topic/48304-show-intro-skip-option/&do=findComment&comment=1063686

To quote - these manual chapter points are just item #5 below.  Note we are at the 'Design' stage - hence the open discussion about it.  Once the Core can use them, they can probably even be removed/hidden.

#	Item	Description	Stage	Enhancements	Needs Core Updates ?	ETA
1	FingerPrint(Chroma)	Creates a template based on a percentage of the Audio runtime.	Ready	More efficient use of dB but good enough for release when used with scheduler	No	Complete
2	Detection	Detects the 'repeating Audio' vs other episodes and calculates the IntroStart and IntroEnd	Ready	Possibly more accuracy but certainly good enough at the moment with 90-95%	No	Complete
3	Schedule	Due to the Detection needing the FP database, a schedule needs to be run to ensure the data is available.	In Progress	Working now but needs manual config	No	~ 1 Week
4	Scaling	Depending on library size, these tasks can take days, even weeks - so it's important it's broken into manageable chunks.	In Progress		No	~ 1 Week
5	Manual Skip	The IntroStart/IntroEnd can be added into the existing chapter points without needing Core development work	Design		No	~ 2 Weeks
6	IntroSkip - Manual	Detection of an 'IntroStart/IntroEnd (in emby Chapters?) should bring a 'Skip Intro?' button to the front of the playback screen.	Concept		Yes	?
7	IntroSkip - Auto	Assuming accuracy is good enough, then #6 could also be automatic if given the option.	Concept		Yes

 

Edited September 8, 2021 by rbjtech

[rbjtech 4316]

So going back to the issue - here is an example of the challenge -

The Plugin has found Intro points in the following file - Breaking Bad s01e03

So IntroStart = 03:03, IntroEnd = 03:21

The original MKV and thus Emby has the chapter points listed as -

From the Emby 'chapter dB' - extracted from the MKV

Spoiler

{
  "Items": [
    {
      "Name": "...And the Bag's in the River",
      "ServerId": "72e5b94bca5342b0bbf22cd789944b49",
      "Id": "1153",
      "RunTimeTicks": 28919560000,
      "IndexNumber": 3,
      "ParentIndexNumber": 1,
      "IsFolder": false,
      "Type": "Episode",
      "ParentLogoItemId": "1114",
      "ParentBackdropItemId": "1114",
      "ParentBackdropImageTags": [
        "16356a5072438906a6756d8a99a18d55"
      ],
      "SeriesName": "Breaking Bad",
      "SeriesId": "1114",
      "SeasonId": "1119",
      "SeriesPrimaryImageTag": "64fd22a9e0809fff37b022a6abada931",
      "SeasonName": "Season 1",
      "ImageTags": {
        "Primary": "295996680d4bbce8fc065639a9d581c0"
      },
      "BackdropImageTags": [],
      "ParentLogoImageTag": "c4a6a14bcd7702c7b0e371dd081d0993",
      "ParentThumbItemId": "1114",
      "ParentThumbImageTag": "af94506bcd3cf98944ebd3b1ad8baa07",
      "Chapters": [
        {
          "StartPositionTicks": 0,
          "Name": "Chapter 01",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 2008260000,
          "Name": "Chapter 02",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 7754000000,
          "Name": "Chapter 03",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 14556630000,
          "Name": "Chapter 04",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 24120350000,
          "Name": "Chapter 05",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 28464690000,
          "Name": "Chapter 06",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        },
        {
          "StartPositionTicks": 28905130000,
          "Name": "Chapter 07",
          "ImageTag": "6be27cfeb0cf4564c86192e669990b52"
        }
      ],
      "MediaType": "Video"
    }
  ],
  "TotalRecordCount": 1
}

This displays as -

Chapter 1 - Is from 0:00 to 03:20.825 

So Chapter 2 is actually the END of the Intro - Great

But I don't want to skip directly to Chapter 2, because I'll be missing the first 3 minutes of the episode.

So I'm going to ADD/INSERT the 'Intro' chapter point.

It now looks like this -

(I'm not sure why the thumbnail is the same, need to look into that..)

So I play from the Start - as soon as I get to the 'Intro' chapter START (03:03) - I am in the Intro Chapter.  Ideally yes, it should then prompt me on the screen with a 'Skip Intro' button, but we don't have that, so the next best thing is just to 'skip to the next chapter'.  So we do that - and we are now @ 3:21 - which happens to also be Chapter 2

If the next chapter point was an auto generated point @ say 05:00 - then we need to re calculate the points FROM 03:21 - otherwise when you skip to next chapter - you will miss the time inbetween 03:31 and 05:00.  

These are the issues we need to understand and get right .. 

 

Edited September 8, 2021 by rbjtech

[Micael456 43]

30 minutes ago, rbjtech said:

If the next chapter point was an auto generated point @ say 05:00 - then we need to re calculate the points FROM 03:21 - otherwise when you skip to next chapter - you will miss the time inbetween 03:31 and 05:00.  

These are the issues we need to understand and get right .. 

 

Ah I get you now.

I was thinking that (in Emby world at least), since we're considering chapters as a short-term measure anyway, we'd have tiny chapter stubs (basically bookmark points), rather than a full contiguous "intro" chapter.

The trigger for the skip behaviour would be the "IntroStart" chapter, and we'd just tell the client to move to the "IntroEnd" 'chapter', rather than the next chapter specifically. But you're considering it from a perspective even before the client knows how to do that.

 

Then if we found that the Intro start/end points sit within a chapter, or even two chapters respectively, we can programatically split them/ insert new ones.

e.g. with your example

 

 

I mean, this is only while we're fudging around with chapters anyway. Once it's integrated directly into Emby I'd imagine it would be completely timestamp based (i.e. DB says at time x load skip button which seeks to time Y). But the general principle and the code would apply.

Edited September 8, 2021 by Micael456

[rbjtech 4316]

The Emby 'chapter' database does actually only record the chapter 'start' point - which tbh makes sense, as a new start point dictates the previous chapter has ended.

So from a pure 'database' perspective, yes, we need to write both the IntroStart AND IntroEnd.  But IntroEnd is actually the start of the next chapter - so we would display that as 'Chapter X' - not IntroEnd.

I think .. haha.. totally open to ideas here.

On the real 'Skip Info' functionality - really not sure, but considering there is a dedicated chapters table in the dB for navigation - it's seems odd if emby decided to use something else to store the Intro information.

On a personal note - I'd like it stored externally as well - as I want to write that info back into the Media file, to make it ultra portable on other systems/players - VLC/Kodi etc etc.

[Painkiller8818 203]

How does this work with tvshows not having the intro in the beginning? I have a lot of shows showing 1-3mins of the show, and after that the intro appears?

Is this also working or just for shows where the intro is the very first chapter?

 

thanks

[rbjtech 4316]

1 minute ago, Painkiller8818 said:

How does this work with tvshows not having the intro in the beginning? I have a lot of shows showing 1-3mins of the show, and after that the intro appears?

Is this also working or just for shows where the intro is the very first chapter?

 

thanks

See the example above - this is the reason for the discussion.  

Yes it works fine for Intro's 'anywhere' in the first 30% (?) of the show duration.

If the Intro was at the start - then we could simply write the IntroEnd - hit 'next chapter' and off we go... 

[Micael456 43]

18 minutes ago, rbjtech said:

The Emby 'chapter' database does actually only record the chapter 'start' point - which tbh makes sense, as a new start point dictates the previous chapter has ended.

So from a pure 'database' perspective, yes, we need to write both the IntroStart AND IntroEnd.  But IntroEnd is actually the start of the next chapter - so we would display that as 'Chapter X' - not IntroEnd.

I think .. haha.. totally open to ideas here.

On the real 'Skip Info' functionality - really not sure, but considering there is a dedicated chapters table in the dB for navigation - it's seems odd if emby decided to use something else to store the Intro information.

On a personal note - I'd like it stored externally as well - as I want to write that info back into the Media file, to make it ultra portable on other systems/players - VLC/Kodi etc etc.

Just had a thought. For MKVs at least- we could use subchapters.

e.g

.

Which VLC displays as

 

 

I wonder if MP4s and others have similar functionality?

 

EDIT: Upon further testing we'd still need an "introEnd" subchapter as well for the native skip to next to work.

 

Quote

it's seems odd if emby decided to use something else to store the Intro information.

I think this is where we need some steer from the emby devs.

On the one hand, it makes sense to keep everything in the one "chapterdb". Then it's all in one place, can treat like chapters.

Then again on the other hand, it could also make sense to have Emby chek for the presence of the "IntroDB", and if so enable the "Skip" button/ functionality. It's a clear and easy way to determine if there's an intro skip deployment.

Is the IntroSkip plugin going to be calling the shots and overlaying the button on the client, or is the client calling the shots and polling for info? Is IntroSkip something that end users will search for in the catalogue, or will it become a default plugin and just be there from the start? etc.

 

Edited September 8, 2021 by Micael456

[rbjtech 4316]

15 minutes ago, Micael456 said:

Is the IntroSkip plugin going to be calling the shots and overlaying the button on the client, or is the client calling the shots and polling for info? Is IntroSkip something that end users will search for in the catalogue, or will it become a default plugin and just be there from the start? etc.

The Core emby code/clients are calling the shots on any sort of 'Intro' button.  We know this is 'doable' simply because the clients today bring the 'Next Up' to the front of the UI (based on X minutes to the end of the show) - so it just needs something similar to 'watch' for the Intro markers/chapters (whatever you want to call them), show the button when it's hits the IntroStart and remove the button when it hits IntroEnd or you hit the 'action' button and it skips to the IntroEnd - or possibly even Auto Skips if you decided the markers were reliable and accurate. 

All tbc ..

But for me TODAY - hitting 'next chapter' when I see the Intro Start, is significantly better than FWD (30 sec), FWD (30 sec), FWD (30 sec), BACK (10 sec), BACK (10 sec) - to get to the end of an Intro !

Edited September 8, 2021 by rbjtech

[Micael456 43]

@rbjtech @chef

The more I think about chapters, the more I think this is actually a bad idea- at least for the general use case of the Intro Skip. I'm also beginning to understand why Emby generates it's own 5 minute chapter blocks to aid navigation.

Chapters should work well in MKV, MP4, and WebM, though with subtly different capabilities. But if we assume that Emby supports all of the "common" container formats though, the older we get the worse it is. Some, like AVI don't support chapters at all. Wikipedia has this lovely table on it. We could spend a long time trying to hash out a solution to this, when we don't actually need to.

I think it might be best to forget chapters as a temporary thing, limit it to those three containers above, or simply MKV (as I've confirmed that it works v. nicely there- see below).

 

41 minutes ago, rbjtech said:

We know this is 'doable' simply because the clients today bring the 'Next Up' to the front of the UI (based on X minutes to the end of the show)

For the actual "production" version, just have the button load up the timestamp details for that episode and set accordingly. As @rbjtech says, it works for the "next up", so the base logic that we'd need (understand time, pulling data from db) is present and functional.

The clients' button could simply check our DB for the intro and out start times, once intro start has passed, button appears. If pressed, seek to end time.

 

With regards to chapters in general, and pursuant to exporting them...

41 minutes ago, rbjtech said:

But for me TODAY - hitting 'next chapter' when I see the Intro Start, is significantly better than FWD (30 sec), FWD (30 sec), FWD (30 sec), BACK (10 sec), BACK (10 sec) - to get to the end of an Intro !

Agreed. I've tested with subchapters on MKV now, it works beautifully in VLC with a microsecond chapter for both intro/end. Not sure about MP4.

 

Thoughts?

Edited September 8, 2021 by Micael456

[Micael456 43]

Update: Seems that webm chapter support is a forked subset from MKV, so likely it will also support subchapters. Need to test.

mp4 does not support subchapters, however there's no issue with just inserting a new chapter in for the "end" of the intro. From my testing with Drax, the chapters are single timestamps, and just continue to the "next" chapter.

[rbjtech 4316]

Saving the 'External' chapters in whatever file format is not actually on the 'to do' list (ie out of scope) - as I don't believe that is something which emby should be responsible for.

We should (as we can) provide an 'enabler' for other tools to do it (a chapter.xml file has been discussed) but the chapter data and any modifications stays in the emby database only.

This info is originally populated FROM the external media file IF it exists, if not, then it's auto generated.  So currently, on a metadata refresh, any Intro chapter we write will be removed - thus we need to keep it in the 'Intro db' and re-populate on a schedule.

When (and if..) the real Intro button is added, then we can probably do away with keeping the Intro Data in the chapter dB, and simply lookup the 'Intro dB' instead - but that is some way off - so I would rather have the chapter 'compromise' in the meantime.

 @chef - what's your thoughts ?  I think it would be a REAL shame as we have come so far not to use the Intro data provided ..

[Micael456 43]

12 minutes ago, rbjtech said:

so I would rather have the chapter 'compromise' in the meantime.

Well in that case, why don't we just (whilst in alpha/beta) write a microsecond-chapter in the Emby DB for now where the "end" of the intro is. It shouldn't affect any existing chapters (and can sanity-check and offset by a second if needed), and would allow you as user to manually push to the "next chapter" when you see the intro start.

[rbjtech 4316]

59 minutes ago, Micael456 said:

Well in that case, why don't we just (whilst in alpha/beta) write a microsecond-chapter in the Emby DB for now where the "end" of the intro is. It shouldn't affect any existing chapters (and can sanity-check and offset by a second if needed), and would allow you as user to manually push to the "next chapter" when you see the intro start.

Maybe we have some wires crossed somewhere, but that's exactly what we are trying to do ..  

[Micael456 43]

9 minutes ago, rbjtech said:

but that's exactly what we are trying to do ..  

I blame the heat!

My bad, I must have missed something- I thought you wanted the whole range.

[chef 3749]

Hi @Cheesegeezer

I was wondering if you knew the answer to this.

When we request a BaseItem (our initial request with limited data), we can use that BaseItem to request the full BaseItemDto. 

Using the ILibraryManager we are able to save changes we have made to the BaseItem using the API.

However, the ILibraryManager  doesn't seem to have a specific method to save the BaseItemDto. 

It is unclear if, requesting the BaseItemDto using the BaseItem, will save any edited data in the BaseItemDto when the BaseItem is saved.

 

I had expected, perhaps, the IDtoService to have a save method, but it does not. It would seem that the LibraryManager is responsible for saving data. 

 

 

[Cheesegeezer 3089]

10 minutes ago, chef said:

Hi @Cheesegeezer

I was wondering if you knew the answer to this.

When we request a BaseItem (our initial request with limited data), we can use that BaseItem to request the full BaseItemDto. 

Using the ILibraryManager we are able to save changes we have made to the BaseItem using the API.

However, the ILibraryManager  doesn't seem to have a specific method to save the BaseItemDto. 

It is unclear if, requesting the BaseItemDto using the BaseItem, will save any edited data in the BaseItemDto when the BaseItem is saved.

 

I had expected, perhaps, the IDtoService to have a save method, but it does not. It would seem that the LibraryManager is responsible for saving data. 

 

 

Hi Chef,

I will have a look, but as far as I know the DTO's are just getters and setters that can be accessed by different languages, the Baseitem's are what we are using in C# to access the DTO info.

EDIT: the library manager will do the heavy lifting for us.  

But i will have a look.

Do you have a specific application i can look at. I have the source now from Git.  And have been familiarizing myself with it.

 

Edited September 8, 2021 by Cheesegeezer

[Cheesegeezer 3089]

EDIT ^^^^

[chef 3749]

Hi @Cheesegeezer

Yes, I created a class which handles chapters. Although editing chapters seems to have some divided. 

We can definitely have a toggle button in the config to opt in or out of the feature. As long as it doesn't take to long to implement. I think it is possible.

I'll update the git, or post my method here,  so you can see what I figured out so far... Just gotta drop the kids off a school first

Edited September 8, 2021 by chef

[Cheesegeezer 3089]

2 minutes ago, chef said:

Hi @Cheesegeezer

Yes, I created a class which handles chapters. Although editing chapters seems to have some divided. 

We can definitely have a toggle button in the config to opt in or out if the feature. As long as it doesn't take to long to implement, I think it is possible to edit.

I'll update the git, or post my method here,  so you can see what I figured out so far... Just gotta drop the kids off a school first

At least it's not at the pool hahaha!!!

just update git and i'll download,  I think it needs to be implemented as part of the plugin because the videoOSD would need to see the chapter marker in order to create a popup "Skip Intro" button that appears at the start and disappears at the end of the Intro.

[chef 3749]

@Cheesegeezer

Do you know the equivalent of the  ItemUpdateService in the c# API?

I think that is what we need to use to update the BaseItemDto.

I thought maybe the IMetadataSaver interface, but that isn't it.

Maybe we have to wrap our own?

 

[chef 3749]

6 hours ago, Painkiller8818 said:

How does this work with tvshows not having the intro in the beginning? I have a lot of shows showing 1-3mins of the show, and after that the intro appears?

Is this also working or just for shows where the intro is the very first chapter?

 

thanks

Yes, this has been taken into account. 

If the show has no intro (which is possible, ie. Mr. Robot) then the entry is marked as HasSequence = false.

If the intro is too short,  we ignore it.  The option to skip wouldn't show.

As long as the intro appears within the first 15 minutes of a sitcom, or 20 minutes of any show longer then the length of a sitcom,  we'll find it

The detection algorithm doesn't use chapters

 

It's pretty cool  how we were able to do it  

You can see here, that we were able to detect the intro for Archer season 1 episode 1, and there was obvious story line prior to the title sequence starting

Edited September 8, 2021 by chef

[Cheesegeezer 3089]

25 minutes ago, chef said:

@Cheesegeezer

Do you know the equivalent of the  ItemUpdateService in the c# API?

I think that is what we need to use to update the BaseItemDto.

 

I completely agree, seems to be the only way to do it.  However, if this is for Chapter injections then we should be using the IChapterManager to do this

 

[chef 3749]

53 minutes ago, Cheesegeezer said:

I completely agree, seems to be the only way to do it.  However, if this is for Chapter injections then we should be using the IChapterManager to do this

 

I can't find IChapterManager