Trying to make a machine-learning driven recommender system plugin

[isaaccodes 2]

Hey all, i'm a former full-stack javascript dev now doing data science with machine learning and im a big fan of emby. Been using it for years and i've always wanted to do a "passion-project" where I either clone or expand on the "More Like This" section under on a show's page using a variety of ML/data science techniques for generating a lot of different but relevant recommendations. Pic below of what i'm talking about with the "More Like This" section:



I've read through a lot of 

https://github.com/MediaBrowser/Emby/wiki/How-to-build-a-Server-Plugin

assuming that's still current a lurked the boards for a bit, and i'm able to make a general plugin, but I'm confused as to what interfaces i'd need to implement to accomplish what i'm looking to do with making a second "More Like This" row on a show populated by recommendations i'd grab from a separate backend i'd release, or if it's even possible with the interfaces that i've seen so far? 

Looking through the list of interfaces here:

https://github.com/MediaBrowser/Emby/wiki/Automatic-Type-Discovery
https://github.com/MediaBrowser/Emby/wiki/Dependency-Injection
https://github.com/MediaBrowser/Emby/wiki/Other-Interfaces

As far as i can tell i don't think any of these would let me add a row of shows as "Recommendations" under a show's "More Like This" or "Seasons" or "Cast & Crew" sections, unless i've missed it?

Are there other interfaces for that, or is it one of the above and I just missed that from the descriptions? Is this only possible with client side code modifications? If so, is there a client side plugin ecosystem i can contribute to?

As far as the rest of the design, i already have a rudimentary node.js REST api that i know i can use the http interface for in my emby plugin to fetch the recommendations from, so i'm not worried about the rest of the design or architecture, i just don't know how to insert these recommendations into a "shows" page.

I've seen in the frontend repo the actual "More Like This" sections code is dated, but assuming it hasn't been changed specifically it's

<div id="similarCollapsible" class="verticalSection itemVerticalSection verticalSection-extrabottompadding hide">
            <h2 class="sectionTitle sectionTitle-cards padded-left padded-right">${HeaderMoreLikeThis}</h2>

            <div is="emby-scroller" class="emby-scroller" data-mousewheel="false" data-centerfocus="true" data-horizontal="true">

                <div class="scrollerframe padded-top-focusscale padded-bottom-focusscale">
                    <div is="emby-itemscontainer" class="scrollSlider focuscontainer-x itemsContainer padded-left padded-right similarContent"></div>
                </div>
            </div>
        </div>

from MediaBrowser.WebDashboard/dashboard-ui/itemdetails.html which i can make sense of, but i just can't tell what backend interface is injecting the data there i guess?

[sydlexius 243]

Just a note that you might want to request that a moderator move this topic to the Developer API forum.  In the meanwhile, have a look at the official Dev site, where there is a link to a sample plugin project.

[isaaccodes 2]

Thanks for the respons Sydlexius! Is this more applicable for the Developer API forum? I thought since I was trying to develop a plugin it'd go in this forum, but i'll defer to your expertise.

Also Thank you for the link, there seems to be some new info in there for sure but a lot of the server plugin information that i'm referencing specifically seems to be identical to the github wiki i linked earlier.

I guess the core of my question to anyone familiar with the emby frontend/backend code is, is it even possible to modify a show's page to display a new row of shows or recommendations (basically cloning or editing the "More Like This" row) via serverside plugins or is this only doable through a combination of serverside plugins to change the show's datamodel or properties and then clientside changes to render those new changes? If it requires client side modification, is this basically unsupported?

[TeamB 2353]

13 hours ago, isaaccodes said:

is it even possible to modify a show's page to display a new row of shows or recommendations

the simple answer is no, you can not, emby server plugins can not ADD to the general client interface.

However, you might be able to use an existing mechanisum like tags to get you closer to you you want though.

Tags are an example of ways you create shared targeted groups, your similar system could find a bunch of similar movies and add tags to them all, the tags would show up and as long as you named them well you might be able to put together a good system.

This is just a suggestion and I have no idea if it would work just playing with ideas.

 

[isaaccodes 2]

thanks for the response @TeamBIt sounds like the idea is dead in the water then. I thought about piggy-backing off a general serverside feature that already flows into the clientside UI, which seems like the tag system as you described it, but i think the data model would end up being too much to manage if i add the recommendations as tags to each individual show, because some shows could be tagged in dozens of other shows recommendations and it'd get hairy fast.

 

[Luke 37095]

Hi, he's right that there's no way to inject the content into existing screens. The closest thing you could do right now is have the plugin add new screens, although I'm guessing that's going to have much lower value for you. 

[TeamB 2353]

3 hours ago, Luke said:

The closest thing you could do right now is have the plugin add new screens

are you refering to plugin pages?

but even with adding new plugin pages there is no current way of linking to them from the standard media pages correct?

Edited November 15, 2023 by TeamB

[Luke 37095]

33 minutes ago, TeamB said:

are you refering to plugin pages?

but even with adding new plugin pages there is no current way of linking to them from the standard media pages correct?

Yes that is correct.

[rbjtech 4283]

@isaaccodes

It's not elegant - but by adding entries into a new library as strm links - you can simply display those as 'Recommendations'.    Or you can also assign them unique Id's and display them that way.

This has been done before, have a search for 'Recommendations' - also been done as unique id's which is what 'Top Picks' uses.

The downside is you will get duplicate items listed in searches etc - so until we have an elegant way of adding 'items' into the main screens this is all emby can offer at this time.

We are all waiting for this functionality to exist - then exciting Plugins such as yours and Smart Playlist type plugins will start to become a true reality

[TeamB 2353]

1 hour ago, rbjtech said:

@isaaccodes

It's not elegant - but by adding entries into a new library as strm links - you can simply display those as 'Recommendations'.    Or you can also assign them unique Id's and display them that way.

This has been done before, have a search for 'Recommendations' - also been done as unique id's which is what 'Top Picks' uses.

The downside is you will get duplicate items listed in searches etc - so until we have an elegant way of adding 'items' into the main screens this is all emby can offer at this time.

We are all waiting for this functionality to exist - then exciting Plugins such as yours and Smart Playlist type plugins will start to become a true reality

not sure that would help in this ask, they want to be able to add recommendations to an individual movie page, like the 'more like this' but based on a new ai approach.

[bakes82 90]

Add them to a collection and cross your fingers and wait for the ability to show collections/smart views on the home screen "soon (tm)".  Ask @rbjtechhe knows when soon is.  You can do the parsing/collection work all using the REST API and not need to use the server plugin, that way you could use other tools to do your analysis since well you cant really inject other DLLs to emby easily per say, so if you wanted to use ML.NET its not part of the emby base dlls.

[TeamB 2353]

6 hours ago, bakes82 said:

Add them to a collection and cross your fingers and wait for the ability to show collections/smart views on the home screen "soon (tm)".

the original ask is to add to an item page not the home page, I get where you are coming from but that is not what is being asked for here.

also @bakes82you seam to be a bit upset about something, this is like the 4th thread I have seen you pop up in with some sort of axe to grind. I am not saying it is not frustrating seeing the "soon"™ response and waiting for new stuff is frustrating but perhaps take a step back, head back to plex and relax for a while, come back in a few months and see how things are then.

Edited November 16, 2023 by TeamB

[isaaccodes 2]

So it looks like from everyone's responses, even if i were to use a different page to or "collection" based approach, there's no way to embed any of these approaches in the standard media pages.

Oh well. I'll be working on the AI recommendations side of the app in the meantime as a passion project, so i really just need the interface to embed links into the a specific emby show's page, so per @rbjtechif this is something that a lot of the community has wanted and it does eventually end up getting developed, then i can simple wire it up when/if this functionality gets added!

Thanks all!

[Luke 37095]

The plan is to have a smart view feature where users can create views based on filter sets. I think once we have that, then a plugin could be used to create custom ones, so that would be your best bet.

[isaaccodes 2]

that'd be fantastic! assuming this feature would come months down the line, if at all, where would be the best place to check back to find news of when it ships? do you guys have an email list or rss feed or what do you recommend?

[Luke 37095]

15 minutes ago, isaaccodes said:

that'd be fantastic! assuming this feature would come months down the line, if at all, where would be the best place to check back to find news of when it ships? do you guys have an email list or rss feed or what do you recommend?

I would follow the blog for release announcements, but it will be in the beta channel much sooner so for that you can follow the testing area of the community.

[TeamB 2353]

2 hours ago, Luke said:

The plan is to have a smart view feature where users can create views based on filter sets. I think once we have that, then a plugin could be used to create custom ones, so that would be your best bet.

i still dont think this is what @isaaccodeswants, they want to be able to add a list of movies or a custom link to list of movies to the movie details page, just like the more like this area in a movie details page.

[Luke 37095]

3 minutes ago, TeamB said:

i still dont think this is what @isaaccodeswants, they want to be able to add a list of movies or a custom link to list of movies to the movie details page, just like the more like this area in a movie details page.

True, but a generalized set of recommendations on the home screen would be a good starting point and then if it takes off you never know.

[TeamB 2353]

@Luke

thinking about this a little more, the only way i see this working would for the more like this to be moved to a plugin as a provider, a bit like a type of metadata provider, then people could select a different more like this provider when setting up their libs. plugin devs could then create new variations that used different approaches.

Edited November 20, 2023 by TeamB