End Credits Detection for resume/next episode

[rbjtech 4772]

4 minutes ago, kingom said:

Thanks for the insight

Do you know how plex does this?

Sorry i don't but probably a combo of the above.   To note, the Introskip Plugin has credit detection before Plex did ...  .. 

[neik 858]

1 hour ago, rbjtech said:

To note, the Introskip Plugin has credit detection before Plex did ...  .

And is working fairly well for TV shows.

A real masterpiece what you guys created and which continues to work just fine. In fact, it's one of my daily drivers plugin wise.

[samuelqwe 316]

3 hours ago, rbjtech said:

Sorry i don't but probably a combo of the above. 

You would be right, according to their blog when this was released: https://www.plex.tv/en-ca/blog/let-the-next-episode-roll/

Quote

We’ve worked hard to tune a machine learning algorithm to make sense of several inputs (text detection, the presence of black frames, and a few other secret ingredients) to come up with a reliable set of markers when credits begin and end.

Netflix also shared how they did this, or at least some details about it, in this article (which I've also shared previously): https://netflixtechblog.com/extracting-contextual-information-from-video-assets-ee9da25b6008

It would seem that text detection is often used, as well as some form of image fingerprinting (similar to how Intro Skip uses audio, but with frames of video), and black frame detection to fine-tune the result. I believe I also saw somewhere (though I can't seem to recall where exactly) that extracting motion vectors from video were also used to identify credits because end credits will typically have no motion or have constant motion in one direction.

So essentially, it would seem that in order get this right, a combination of inputs would have to be used to accurately detect credits, and doing so would likely be significantly more demanding on any given system than the method used for Intro Skip. This is why Plex chose to anonymously (and by default) upload the identified credit markers in a cloud database to reduce the system ressources required for this process, though this would go against Emby's stance on data privacy.

[crusher11 896]

On 2/12/2024 at 10:11 PM, ebr said:

Its the same process and basically the same thing - detecting and reacting to credits just like intros.  This specific request only outlines one use-case of the functionality.  There are multiple use cases but one basic function.

Trying to detect post-credit and in-credit sequences is a whole new challenge. Detecting credits is already doable in the IntroSkip plugin, or they can be manually added with ChapterAPI. All this request is for is acknowledging those markers for the "next episode" pop-up (already present in the Android apps, not sure about other clients) and for the purpose of marking something as played vs in-progress. Much, much simpler than entirely new detection features and skipping functionality.

There's no need to put a simple request on the back burner just because someone else attached a more complex request to it.

[rbjtech 4772]

7 hours ago, crusher11 said:

Trying to detect post-credit and in-credit sequences is a whole new challenge. Detecting credits is already doable in the IntroSkip plugin, or they can be manually added with ChapterAPI. All this request is for is acknowledging those markers for the "next episode" pop-up (already present in the Android apps, not sure about other clients) and for the purpose of marking something as played vs in-progress. Much, much simpler than entirely new detection features and skipping functionality.

There's no need to put a simple request on the back burner just because someone else attached a more complex request to it.

But without the detection, there is little reward for the majority of users and thus a low priority for emby as a whole.

Yes both the Introskip Plugin detected them and Chapter API can manually add them - but both 3rd party plugins and I'd imagine used by a fraction of the overall users.   The Introskip plugin is no longer distribured, nor supported.

So the ability to 'use' the CreditStart/CreditEnd hidden markers in the clients would go with the 'Add Credit Detection' feature request, not sit on it's own as a leftover from a previously explored venture.

[Nillaz 6]

I'm new to Emby and I know this is an older thread so please forgive me for the necropost but it seems like this is still the most appropriate place to ask: To be clear, with the intro plugin now deprecated there is currently no possible way to automate the detection of end credits? I have to manually add a marker for each title using the Chapter API plugin?

[seanbuff 995]

48 minutes ago, Nillaz said:

I'm new to Emby and I know this is an older thread so please forgive me for the necropost but it seems like this is still the most appropriate place to ask: To be clear, with the intro plugin now deprecated there is currently no possible way to automate the detection of end credits? I have to manually add a marker for each title using the Chapter API plugin?

Correct. There is currently no automated way of doing this for credits.

[Nillaz 6]

43 minutes ago, seanbuff said:

Correct. There is currently no automated way of doing this for credits.

Sounds like I have some soul searching to do. I appreciate the confirmation though.

[Nillaz 6]

So I've been using Emby for a few weeks and want to revisit this with my thoughts. We use our media server like our own personal Netflix complete with binge watching, so credit markers are important to us, and dealing with credit markers in Emby genuinely sucks. To call them frustrating to work with doesn't even begin to do it justice.

I get it that automatically detecting credits can be resource intensive for the host machine and doesn't have a 100% success rate for accurate detection regardless of method used, but this is one area where I would encourage the devs to consider that perfect is definitely the enemy of the good. When you have a large existing library it is absolutely painful to manually add these markers for each and every video. This is compounded by the fact that you have to do this for every video added to the server going forward as well, and it makes upkeep an enormous task. I would MUCH rather go back and manually edit videos that didn't automatically get it right the first time.

The whole reason I started experimenting with EMBY in the first place is my dissatisfaction with the direction of PLEX, but it is honestly hard to get away from their software because of this. To be fair PLEX's system isn't perfect either, but once again in this instance perfect is the enemy of the good. Even with being imperfect their system is so much more advanced as to be almost laughable.

Given that the intro/credit detection plugin is no longer available I implore the devs to reconsider their stance on this. I understand that the Emby team is relatively small and you have to be smart about where you allocate your resources but people have been asking for this for more than two years now across multiple threads.

[BenjaminAaron 1]

I am a paying Plex user, but I recently switched to Emby because I dislike Plex’s increasing focus on streaming services. After using it for several months, I can confidently say that, apart from the number of client apps (which isn’t a major issue), Plex’s most notable advantage is its relatively accurate detection of intros and outros. This feature has the most noticeable impact on the user experience. Other than that, I find Emby superior to Plex in almost every way. It seems Plex uses voice analysis to determine these segments because, even when there are additional scenes after the credits, it can still accurately identify the outro section. I hope the developers can pay more attention to this kind of feature request. Thank you very much!

[rbjtech 4772]

42 minutes ago, BenjaminAaron said:

I am a paying Plex user, but I recently switched to Emby because I dislike Plex’s increasing focus on streaming services. After using it for several months, I can confidently say that, apart from the number of client apps (which isn’t a major issue), Plex’s most notable advantage is its relatively accurate detection of intros and outros. This feature has the most noticeable impact on the user experience. Other than that, I find Emby superior to Plex in almost every way. It seems Plex uses voice analysis to determine these segments because, even when there are additional scenes after the credits, it can still accurately identify the outro section. I hope the developers can pay more attention to this kind of feature request. Thank you very much!

Agreed - another area where Emby have lost focus and are now way behind the competition - it doesn't even have the ability to edit the Intro's without the support of a 3rd party plugin...   Emby (with the 3rd party Plugin) used to lead in this area as it had both Intro and Outro detection and all the support tools (editing, stats etc).    Now even JF has long passed them and have recently introduced the architecture called 'media segments' which is tagging for for Commercial, Preview, Recap, Outro and Intro.   Their clients need to catch up, but the concept is solid.   A shame, as the same concept of 'MarkerTypes' is actually in Emby (and was discussed with them), but beyond Intro's, it has have never been developed further ...