rbjtech 4265 Posted February 16, 2023 Share Posted February 16, 2023 On 06/02/2023 at 16:40, Riddler84 said: Their code is public and if you look at it, you can see that they pretty much scrape the site. They're requesting the HTML and then using xpath to extract the info they need. I don't know if they have IMDB's consent to do this (you can get it according to IMDb's TOS), but even if not. I'm not a legal expert by any means, but I doubt that IMDb can forbid anyone to access their public available HTML and extracts every data they want from it. Otherwise, Google wouldn't work at all. They're scraping millions of sites every day without asking. All that IMDb can do against it, is trying to block the access for everyone who is trying to scrape the site, but this is extremely hard to do or to detect and probably not worth the effort. And in the case of PMM, there's not a central IP which is requesting all the data, because every PMM user is making requests from his own client. A plugin wouldn't be necessary, I guess. It's a service that has to be hosted separately by every single user. It has its own configurable schedule, so it's not needed to make a plugin for Emby to use Emby's schedule feature. I guess it would just make use of Emby's REST API to make direct changes to the database. That's how they do it for Plex, at least. But sadly, I doubt that they will create an Emby version anytime soon. @Cheesegeezer so having a very quick look at it - they appear to be using Top Lists - mdblist.com as their main source - which in itself if available via an API (free + various tiers) - this is the source of the list information - and therefore, they are not scaping it from the key providers - output is available via JSON. Where 'they' get it from, is up to them... but I don't think there is a need to 'scrape' the provider sites .. So yes, getting them to support emby would be ideal (the Emby API has Collection functionality) - so technically, they just need to query the emby API (by provider ID) based on the results from the mdblist API - and you have a 'List' based collection. You can create a 'missing' virtual item or episode via the API as well if you want the complete list, not just those in the list that you have. Link to comment Share on other sites More sharing options...
jaketame 86 Posted February 16, 2023 Author Share Posted February 16, 2023 12 minutes ago, rbjtech said: @Cheesegeezer so having a very quick look at it - they appear to be using Top Lists - mdblist.com as their main source - which in itself if available via an API (free + various tiers) - this is the source of the list information - and therefore, they are not scaping it from the key providers - output is available via JSON. Where 'they' get it from, is up to them... but I don't think there is a need to 'scrape' the provider sites .. So yes, getting them to support emby would be ideal (the Emby API has Collection functionality) - so technically, they just need to query the emby API (by provider ID) based on the results from the mdblist API - and you have a 'List' based collection. You can create a 'missing' virtual item or episode via the API as well if you want the complete list, not just those in the list that you have. There are multiple providers they use and they call these builders so you can build from lots of different sources - https://metamanager.wiki/en/latest/metadata/builders.html Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted February 16, 2023 Share Posted February 16, 2023 (edited) Sure - I guess the point I'm making is it's very easy to create a collection in emby using the API - the 2 main steps are below - 1. Create the 'list' using provider id numbers - in this case IMDB Top 5 films - this gets you the associated emby ID's .. curl -X GET "http://***:8096/emby/Items?Recursive=true&AnyProviderIdEquals=imdb.tt0111161%2Cimdb.tt0068646%2Cimdb.tt0468569%2Cimdb.tt0071562%2Cimdb.tt0050083&api_key=emby_api_key" -H "accept: application/json" 2. JSON output from that (emby ID's) is used to create the Collection curl -X POST "http://***:8096/emby/Collections?Name=IMDB%20Top%205%20Movies&Ids=3024363%2C3024320%2C3024319%2C3024311%2C3023958&api_key=emby_api_key" -H "accept: application/json" That's it - collection created. Edited February 16, 2023 by rbjtech Link to comment Share on other sites More sharing options...
Riddler84 38 Posted March 8, 2023 Share Posted March 8, 2023 On 2/16/2023 at 5:34 PM, rbjtech said: so having a very quick look at it - they appear to be using Top Lists - mdblist.com as their main source - which in itself if available via an API (free + various tiers) - this is the source of the list information - and therefore, they are not scaping it from the key providers - output is available via JSON. Where 'they' get it from, is up to them... but I don't think there is a need to 'scrape' the provider sites .. No, I don't think that is correct. Yes, you can use mdblist as well, but that's up to the single user. They have many builders you can use, mdblist is just one of them. But you need to provide your own API key. And IMDb is just another builder. In the IMDb builder config, you can directly define IMDb lists as data sources. And as I said, the source code clearly shows that they are simply scraping the website. Link to comment Share on other sites More sharing options...
joggs 24 Posted August 16, 2023 Share Posted August 16, 2023 (edited) On 2/16/2023 at 4:42 PM, GrimReaper said: That's a lot of features - and I'm a sucker for overlays. Nice thing they've got goin' there - and nicely implemented. Yes, I also love the customizable overlays. In addition to video and audio formats, it is nice with ratings, and icons for movies that that have post credits scenes etc, such as this small example. I like that the images are modified, so it works in all apps and not only the web app. Edited August 16, 2023 by joggs Link to comment Share on other sites More sharing options...
Warzonefury 11 Posted August 16, 2023 Share Posted August 16, 2023 Big +1 !!! Link to comment Share on other sites More sharing options...
Luke 37066 Posted August 16, 2023 Share Posted August 16, 2023 4 hours ago, Warzonefury said: Big +1 !!! Hi, +1 for what exactly? 1 Link to comment Share on other sites More sharing options...
Warzonefury 11 Posted August 17, 2023 Share Posted August 17, 2023 On 8/16/2023 at 11:18 PM, Luke said: Hi, +1 for what exactly? Plex Meta Manager (Emby Support) This soft is just wonderful. Link to comment Share on other sites More sharing options...
Luke 37066 Posted August 17, 2023 Share Posted August 17, 2023 What does supporting it mean? Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted August 18, 2023 Share Posted August 18, 2023 (edited) All - this is not for emby to 'support' - it it for PMM to support emby Emby has great REST API support for creating and managing collections - and a few examples have been given in this thread. PMM therefore doesn't need tight integration with Emby - they should be able to simply 'import' the emby libraries via the provider ID's to get a list of what's 'available', then pipe that into the PMM 'engine' to create the collections and finally create those new collections via an emby API request. I don't believe there is anything emby need to do here .. IF those collections needed to be made into 'Channels' (as opposed to collections), THEN it would need an Emby PMM Plugin - but getting the collections into it looks to be an easy first step.. Edited August 18, 2023 by rbjtech 1 1 Link to comment Share on other sites More sharing options...
seanbuff 840 Posted August 18, 2023 Share Posted August 18, 2023 24 minutes ago, rbjtech said: PMM therefore doesn't need tight integration with PMM I'm assuming you meant 'Emby' in there somewhere 1 1 Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted August 18, 2023 Share Posted August 18, 2023 (edited) -delete- Edited August 18, 2023 by rbjtech Link to comment Share on other sites More sharing options...
joggs 24 Posted August 18, 2023 Share Posted August 18, 2023 13 hours ago, rbjtech said: All - this is not for emby to 'support' - it it for PMM to support emby Emby has great REST API support for creating and managing collections - and a few examples have been given in this thread. PMM therefore doesn't need tight integration with Emby - they should be able to simply 'import' the emby libraries via the provider ID's to get a list of what's 'available', then pipe that into the PMM 'engine' to create the collections and finally create those new collections via an emby API request. I don't believe there is anything emby need to do here .. IF those collections needed to be made into 'Channels' (as opposed to collections), THEN it would need an Emby PMM Plugin - but getting the collections into it looks to be an easy first step.. Correct. We need to let them know that we are really interested in this, so they prioritize it. Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted August 19, 2023 Share Posted August 19, 2023 10 hours ago, joggs said: Correct. We need to let them know that we are really interested in this, so they prioritize it. Looking at the code - while the providers are modular, the core looks very Plex centric (as you might expect, as it's written for Plex..) - so mapping it to emby doesn't look 'easy' and it's probably why this hasn't gained traction yet. As an open source project of this size then somebody interested from an emby perspective needs to work with the PMM developers to really get this integrated... Link to comment Share on other sites More sharing options...
Riddler84 38 Posted August 21, 2023 Share Posted August 21, 2023 On 8/16/2023 at 3:26 PM, joggs said: Yes, I also love the customizable overlays. In addition to video and audio formats, it is nice with ratings, and icons for movies that that have post credits scenes etc, such as this small example. I like that the images are modified, so it works in all apps and not only the web app. Yep, I used this when I still was using Plex. I really miss these overlays. I don't have hope that PMM will make an Emby version in the near future, so I'm currently in the process of copying some of my most wanted features from PMM and turn them into some Python scripts. I already made a script to scrape the tags for movies, series and episodes from IMDb. It's pretty fast and has its own configurable caching system to make sure it only processes new items or items that are past a time limit. Newly released items can have a different time limit to update them more often, as long as they are released within the last 14 days for example. I can use these tags now to make collections based on them. Which is the next step, making a script that will automatically maintain collections in Emby and is being able to create collections based on IMDb lists, like for trending stuff, the top 250 or Oscar winners etc. And lastly, I want to recreate the overlays system and make it as closely to PMM's design as possible. This will probably be the hardest task^^ And then I can just wait, until we can customize the Emby frontpage enough to place content from collections on it Link to comment Share on other sites More sharing options...
hthgihwaymonk 21 Posted August 21, 2023 Share Posted August 21, 2023 (edited) On 2/16/2023 at 9:14 AM, rbjtech said: Sure - I guess the point I'm making is it's very easy to create a collection in emby using the API - the 2 main steps are below - 1. Create the 'list' using provider id numbers - in this case IMDB Top 5 films - this gets you the associated emby ID's .. curl -X GET "http://***:8096/emby/Items?Recursive=true&AnyProviderIdEquals=imdb.tt0111161%2Cimdb.tt0068646%2Cimdb.tt0468569%2Cimdb.tt0071562%2Cimdb.tt0050083&api_key=emby_api_key" -H "accept: application/json" 2. JSON output from that (emby ID's) is used to create the Collection curl -X POST "http://***:8096/emby/Collections?Name=IMDB%20Top%205%20Movies&Ids=3024363%2C3024320%2C3024319%2C3024311%2C3023958&api_key=emby_api_key" -H "accept: application/json" That's it - collection created. This is how I went about doing it as well, using "ProviderId's", and then creating the collections I wanted based on mdblist lists as it has access to all of the different providerIds Edited August 21, 2023 by hthgihwaymonk Link to comment Share on other sites More sharing options...
hthgihwaymonk 21 Posted August 30, 2023 Share Posted August 30, 2023 making progress... need to speed up collection creating/updating though. 3 Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted August 30, 2023 Share Posted August 30, 2023 6 hours ago, hthgihwaymonk said: making progress... need to speed up collection creating/updating though. Nice - what are you using to do the API collection requests - a script ? Link to comment Share on other sites More sharing options...
hthgihwaymonk 21 Posted August 30, 2023 Share Posted August 30, 2023 7 hours ago, rbjtech said: Nice - what are you using to do the API collection requests - a script ? wrote it using python 2 Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted August 30, 2023 Share Posted August 30, 2023 3 minutes ago, hthgihwaymonk said: wrote it using python If it's portable - then maybe consider sharing it ... Link to comment Share on other sites More sharing options...
Grumpy1 44 Posted August 30, 2023 Share Posted August 30, 2023 @hthgihwaymonk Did you make those icons ? 4k dv and so forth they are very nice Link to comment Share on other sites More sharing options...
hthgihwaymonk 21 Posted August 30, 2023 Share Posted August 30, 2023 (edited) 9 minutes ago, Grumpy1 said: @hthgihwaymonk Did you make those icons ? 4k dv and so forth they are very nice I cheated and pulled them from https://github.com/meisnate12/Plex-Meta-Manager/tree/master/defaults/overlays/images I did have to make some for the 3D content overlays though Edited August 30, 2023 by hthgihwaymonk 2 Link to comment Share on other sites More sharing options...
mxmh 7 Posted September 20, 2023 Share Posted September 20, 2023 is there an update? Link to comment Share on other sites More sharing options...
rbjtech 4265 Posted September 20, 2023 Share Posted September 20, 2023 21 minutes ago, mxmh said: is there an update? Who are you asking ? Unless the Dev of this utility is working on Emby Integration, then I don't believe anybody else is ... Link to comment Share on other sites More sharing options...
mxmh 7 Posted September 20, 2023 Share Posted September 20, 2023 27 minutes ago, rbjtech said: Who are you asking ? Unless the Dev of this utility is working on Emby Integration, then I don't believe anybody else is ... could you share your python script to set collections using imdb ids Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now