softworkz 3683 Posted November 27, 2022 Share Posted November 27, 2022 (edited) This is another brand-new plugin which is available in the catalog now: Emby Data Explorer Important Note Make sure to clear your browser cache after installation and restarting the server! History In 2016 I was working on Metadata Providers and wanted to see and better understand which metadata is coming from which provider and which information was actually taken by Emby Server. In turn, I had created the MetadataViewer Plugin. Later, it became outdated and I was no longer around to update, so eventually it had to be pulled due to incompatibilities after the server had evolved. Meanwhile, there had also been an idea for a "Backstage View" plugin with the purpose to provide some low-level insights into Emby item data, but it never took off. This month, I came back to some work on metadata providers and I had the same problems again, which ended up in creating this plugin. It's a fusion of the MetadataViewer plugin and the Backstage View concept - with a name that leaves room for more... Beta For now, it's available for the beta server only It's been done in two and a half days, so please report bugs in case you find any How it Works The plugin adds a new entry to the Item Context Menu: This will open a dialog for data display: This dialog is the one and only UI that the plugin provides. Everything is happening there. I will describe individual features in subsequent topics. ENJOY! Edited November 27, 2022 by softworkz 10 Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 (edited) Exploring Metadata Sources Exploring the flow of metadata in Emby is one of the core features of the plugin. It provides answers to questions like: Where does my metadata come from? Which data does each metadata provider supply exactly? The data of which providers is actually chosen and used by Emby and why? How does Emby merge the data together from multiple providers? What influence does the selected metadata language have on the results? Which are the best providers for my content (in a specific library), which ones do I really need and which can I remove? How should in order the providers in the library settings to get best results? Basics General A metadata provider is always specific to a certain kind of item in Emby (e.g. movie, series, season, episode, person, music artist, song, etc) Metadata plugins typically include a smaller or larger set of providers, each for a specific kind of item Provider types There are three types of metadata providers in Emby: (we're ignoring image providers here) Remote Metadata Providers Those are querying a remote service to identify and retrieve metadata information about items in Emby Input: ItemLookupInfo A set of data which contains everything that Emby knows about and item and what might be helpful to acquire matching data. This includes things like item title, year, index numbers like season/episode numbers, etc. Output: A metadata result information which includes an artificial "dummy item" to which the metadata results are applied Local Metadata Providers These are typically for reading data that is stored in the file system alongside the actual media files (e.g. Nfo files) Input: ItemInfo and local path ItemInfo contains some basic information about an item. Also the path is important for those providers to know where to look for relevant files Output: A metadata result information which includes an artificial "dummy item" to which the metadata results are applied Custom Metadata Providers These are general--purpose providers and can be used in many different ways. They are different from other providers as they get the actual item as input which they can modify directly. The Ffprobe is a provider of this kind Input: MetadataRsult with the actual item, possibly containing the resulting and merged data from other providers which have run before Output: The same MetadataResult, possibly with changes made to the item directly Variations Two variation exist for those kinds of providers: Pre-Refresh: These are run before all others on metadata retrieval Normal: Are run at the end after all other providers, which allows them to make changes to the metadata that came from other providers Provider Configuration Remote and Local providers can be configured in the options for each library Providers can be enabled or disabled The order of provider invocation can be configured Custom providers cannot be configured as these are meant for specific tasks and they are expected to determine on their own when they should become active or not Execution Order It is important to understand the order of provider execution, because it depends on this order, which data Emby is finally using from those providers. There is a fixed order by provider type which cannot be configured. It is fixed and stands above everything else. This is the order when doing a full metadata refresh or on initial metadata retrieval for an item. Custom Pre-Refresh providers Remote Metadata providers In the order which is configured in library options Local Metadata providers In the order which is configured in library options Custom Providers The non-pre-refresh ones Metadata Retrieval Language Remote metadata providers often can provide data in multiple languages. Emby needs to decide in which language it should request metadata for an item. This is determined over multiple levels: Check whether "PreferredMedataLanguage" is set on the item directly (by the user) if not, then Go up in the hierarchy, parent-by-parent and check whether PreferredMedataLanguage is set on any of them if not, then Look at the library options and see whether PreferredMedataLanguage is configured for the library if not, then Use the PreferredMedataLanguage from the global Emby Server configuration which users are choosing when setting up Emby Server Sections With that knowledge in mind, we can look at the individual section that are shown Options Section This section allows to control what is being shown and how it's presented: Request Language The language for which to retrieve metadata from remote providers Metadata Providers Allows to choose the set of metadata providers to be shown as columns Show all Fields Normally, only a common set of properties is shown to allow for a better overview Also, rows without any values are hidden by default Select this to show everything Show Full Screen Does what it says Lookup Info Section This includes (roughly) all the data that is used for querying remote providers. The "Requested Language" corresponds to the selection made in the Options Section. The "Configured Language" row is indicating which language would normally be queried for the item when you do a metadata refresh. It also tells why, or rather where the configured language is taken from. What's stull planned is providing more control over this, to allow querying the providers in the same way as when the item would have been newly added and no provider ids are known yet. Item/Result Properties Section This section displays all the data in a single tabular view: First Column Shows the current values of the item in the Emby database. Subsequent Columns Each of those columns shows the results of a single metadata provider. The columns are in the configured execution order from left to right! This means that Emby would run the providers in the same order in case of a full metadata refresh. But take also note of the Is Enabled row. Emby will only run the providers which are enabled for the library. Understanding the Data Merging Basically (= besides some exceptions) for remote providers, Emby is taking the first data that is available. So you can go row by row from left to right to see why a certain value is taken by Emby and not one of the values in a different column. Controlling Column Display To simplify the view, you can control the columns to be shown with the dropdown selection in the Options Section: All Shows all providers which are applicable to the item (excluded are custom providers which do not provide a result) Only Remote Providers Often you'll be interested in the remote providers' results only. This setting will hide all other providers Only Enabled Providers Shows only those providers as columns that Emby would use for retrieving metadata with the current configuration Only Enabled Remote Providers Shows only remote providers which are enabled None Use this, when you are not interested in metadata at all and you want to view the item's data only Edited November 27, 2022 by softworkz Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 (edited) Data Display General Empty Cells When cells are empty, it means that there's no value - or that the value is identical to the default value for an item of that type Numbers equal to zero are hidden as well Word Wrapping When many columns are shown, there's often not much space to display longer values and text. To make this more efficient and avoid rows getting overly tall, there's a word-wrapping by-letter. It's not nice for reading, but it saves space and allows to compare texts more quickly and easily Data Types Booleans are shown as checkmark/cross symbols. This makes visual comparison much easier than having TRUE/FALSE text everywhere Dates Are shown in ISO format: yyyy-mm-dd Item Links Every value that represents an Emby item is represented as link which allows to open that item in the data explorer. Currently there is no way back. The only option to return to a previous item is to go to the Relations Section and find a link that leads to the previous item. Local File and Folder Links Values that are indicating a patch in the local file system are also shown as links: Unfortunately, browsers do not allow anymore to open file:/// links to open from web pages. But there's a solution. It works on Windows only with Chrome (probably Edge as well) after installing the following Chrome extension: https://chrome.google.com/webstore/detail/local-explorer-file-manag/eokekhgpaakbkfkmjjcbffibkencdfkl Once everything is set up and working, you can even DISABLE the extension, so it won't be able to read the content of the sites you are visiting. The Data Explorer is already taking care of preparing the links as needed. You need to keep the native installation part of the extension, though. File Links For file links, there's also a small "folder symbol" shown. This allows you to open the containing folder in Windows Explorer rather than opening a file directly. Edited November 27, 2022 by softworkz 1 Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 features Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 (edited) Planned Features (as time will allow) Allow querying the providers in the same way as if the item would have been newly added and no provider ids are known yet Show and compare media sources Provide back/forward navigation Provide link buttons to show items in the Emby UI in a new window Keep row headings sticky on top Edited November 27, 2022 by softworkz Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 Very nice - thanks for sharing @softworkz - I'll take a look later on today. Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 (edited) Hmm - Plugin installed and emby rebooted, plugin is loading (in the logs) but no extra item in the context menu - tried multiple libraries. Latest beta - 4.8.0.19 Would a log help ? Edited November 27, 2022 by rbjtech Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 Press F12, go to the network tab, disable caching and press F5 1 Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 Rookie mistake - yep, clearing the browser cache did it - thanks ! Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 Just now, rbjtech said: Rookie mistake Nah, I'll need to find a better way to inject the menu. Of course it should work without needing to do that. PS: you can enable the cache again, it's not changing over time. Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 (edited) 6 minutes ago, softworkz said: Nah, I'll need to find a better way to inject the menu. Of course it should work without needing to do that. PS: you can enable the cache again, it's not changing over time. Browser Cache was just cleared; I didn't disable it. Of interest and for another thread, how to get the function on the context menu via a plugin is of great interest ! @chef @Cheesegeezer Edited November 27, 2022 by rbjtech 1 1 Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 1 minute ago, rbjtech said: Browser Cache was just cleared; I didn't disable it. Of interest and for another thread, how to get the function on the context menu via a plugin is of great interest ! @chef @Cheesegeezer I'm not sure whether we want to make this generally available. I started an internal discussion about it, but I can't make any promises.. 1 Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 Some immediate feedback - looks fantastic - all the info you can ever want.. but - multi-versions of the same ItemId do not show the details of the various versions. ie it needs to show all MediaSourceID's from the ItemId ? I think that would be useful. 1 Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 4 minutes ago, rbjtech said: but - multi-versions of the same ItemId do not show the details of the various versions. ie it needs to show all MediaSourceID's from the ItemId ? LOL, I had written the first lines for this and then I decided to make a cut and get an initial version out. Of course this can be added! BTW, when you install the "Local Explorer" extension for Chrome or Edge, you can click on the path links and open the items locally. Folders open in Windows explorer and files have that folder symbol at the right, which opens the folder where the file is located. 1 Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted November 27, 2022 Share Posted November 27, 2022 haha sorry, I didn't mean to sound ungrateful - but I use multi-versions a lot, so it was the very first thing I looked for .. 1 Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 As I already have what I need, I wanted to check out first, what is being asked for. But this is surely a hot candidate. Multiple sources could be nicely shown in a multi-column table for comparison. 1 Link to comment Share on other sites More sharing options...
Cheesegeezer 3093 Posted November 27, 2022 Share Posted November 27, 2022 Nice one @softworkz!!! this will be a very useful and powerful tool i can see, especially for a lot of the power users, thanks for sharing bud 1 Link to comment Share on other sites More sharing options...
Cheesegeezer 3093 Posted November 27, 2022 Share Posted November 27, 2022 just giving it a run now. took me a while to find it, i was looking in the Metadata section but it's up in the General lol Link to comment Share on other sites More sharing options...
ebr 15186 Posted November 27, 2022 Share Posted November 27, 2022 Apologize for not installing to look myself, but this is restricted to Admin users, right? Link to comment Share on other sites More sharing options...
softworkz 3683 Posted November 27, 2022 Author Share Posted November 27, 2022 19 minutes ago, ebr said: Apologize for not installing to look myself, but this is restricted to Admin users, right? Good point. Update is out. It's not yet available for the release channel anyway. 1 Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted March 25, 2023 Share Posted March 25, 2023 @softworkz Appears to be broken in the latest .26 beta FYI Thanks. Link to comment Share on other sites More sharing options...
softworkz 3683 Posted March 25, 2023 Author Share Posted March 25, 2023 (edited) Yea I knew it will break. A lot of my plugins are broken now. I'm still waiting for @Luketo apply the GenericUI fixes I have submitted, so I can update all the plugins. Edited March 25, 2023 by softworkz 1 Link to comment Share on other sites More sharing options...
rbjtech 4588 Posted March 26, 2023 Share Posted March 26, 2023 This now appears to be fixed - After a Plugin update to v4.8.0.26 Thanks ! @Luke@softworkz@Happy2Play 1 Link to comment Share on other sites More sharing options...
Multiverse 5 Posted October 9, 2023 Share Posted October 9, 2023 (edited) I've tried to install it on 4.8.0.51, but after the restart threre is no plugin inside Emby. But it is inside my plugin folder! BTW: I'm running Emby Server on Debian Bookworm (Stable). Edited October 9, 2023 by Multiverse Link to comment Share on other sites More sharing options...
Luke 37994 Posted October 9, 2023 Share Posted October 9, 2023 11 minutes ago, Multiverse said: I've tried to install it on 4.8.0.51, but after the restart threre is no plugin inside Emby. But it is inside my plugin folder! BTW: I'm running Emby Server on Debian Bookworm (Stable). HI, what folder did you put it in? Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now