Anything todo with emby scanning updating metadata library

[Ugnaughts 53]

i wanted to start a topic to consolidate any knowledge on the topic of scanning library's for emby. so anyone who has info related to how the scanners work. how they are organized in the database. people that are comfortable with developing plugins. people comfortable with database query's cmds. this topic is not designed to have one answer, but on going so anyone that has any new helpful details can put in this topic. so we can work together in fine tuning the emby scanning/updating process. since we cant update the original post with the updates that we have learned so far. we can occasionally just post within the thread what we have learned from this thread to that point. this will help future developers of plugins and others learn anything about this topic on an advanced level. 

[Luke 40154]

HI, can you please be more specific about your question? Thanks.

[Ugnaughts 53]

so far ive learned that tvmaze is very fast at scanning and updating tv shows, nearly perfect at finding all non-adult shows. so i use that as first choice in metadata for tvshows. also i have a very large library because i parse strm files from iptv provider. which is about 2700 series and 15000 movies. i  use tvdb as second choice, which seems to be more perfect, but much slower than tvmaze. 

i use moviedb as first choice for metadata for movies folder. and tvdb as the second choice. 

if i have to add /remove those library's for what ever reason. it takes 3-4 days of constant scanning to get all the metadata\images. if it gets interrupted or i choose the wrong metadata providers then there will be some random amount of titles that never got scanned for metadata. and others that got scanned but never got images like posters. ive tried the search for missing metadata but it seems to take just as long as a full scan. 

i have found by using a free portable program called dbbrowser for sqlite. that in the metaitems table. if i filter type to 6, which means tv show title. it will list all tv shows in database that the basic scanner has added. if i then filter the images field to a-z. i will get basically 3 groups.

1.will be blank or null which means these are the ones that it scanned but didnt find metadata for.

2. some will start with just %MetadataPath%\library which means it found metadata but didnt find poster image and emby might have created an image instead of leaving blank. if it creates an image for the poster.

3. others that have metadata and a real poster, aka good for the most part. will start with "http" something then "|%MetadataPath%\library"

so if someone doing a database query could filter the few that need updating and then update based off there unique id on that row. which would make updating missing meta or posters lightning fast and effective. in theory.....

then the scan for refresh metadata or search for missing metadata will be a waste of time. if you click replace all images then it will scan the whole library and replace all images. which will be a waste of time for finding these missing posters. you need to click refresh and replace all images, which will do a scan and replace of all files in the database, and replace all metadata and images, which is not ideal. 

i suggest to have some more clarity into current scans and add some more. 

like have check boxes of what you what todo in your scan of that library. scan for new media// update all metadata for what has already been scanned // search for metadata that is missing from what has already been scanned into database // update all images for what has already been scanned in database // search for images that are already scanned into database, wither they are blank or default image saved and update them only.

 

Edited January 21 by joekingcool
update

[Ugnaughts 53]

also when i select select more than one metadata provider in library settings. does it use both for all items it finds. or does it use the first choice for all items until it dont find meta and then it will try the second meta provider?

 

[Happy2Play 9460]

6 minutes ago, joekingcool said:

also when i select select more than one metadata provider in library settings. does it use both for all items it finds. or does it use the first choice for all items until it dont find meta and then it will try the second meta provider?

 

When multiple providers are enabled, all will be used to a point.  Emby attempts to get everything it need/wants from first provider but still queries all other providers just in case there is something the first provider does not have and so one.  But secondary info is only used for NULL info from first provider it will not supplement existing data.

 

[Ugnaughts 53]

14 minutes ago, Happy2Play said:

But secondary info is only used for NULL info from first provider it will not supplement existing data.

so it will contact both meta providers for each item. use the meta from first provider. then only the blank fields are filled from second meta provider? 

[Happy2Play 9460]

3 minutes ago, joekingcool said:

so it will contact both meta providers for each item. use the meta from first provider. then only the blank fields are filled from second meta provider? 

Correct.

[bakes82 160]

Doesnt the library manager have options to look for missing things like images and what not. I built my own plugin to match items that emby doesnt find or that I reset using trakt, so it pulls all the unmatched items in emby, then uses the folder/file name to search trakt and pulls back what it thinks is the best option, and then if the score is greater than the confidence level I select it will auto select the "associate media" button, and then I can just press "Match Items" and it will issue the match for emby refresh meta, and pull images. So ultimately yes I believe using the library rest/sdk you could probably do what you want. Find items with missing data, loop them issue the metarefresh for those specefic items.

[Ugnaughts 53]

@bakes82

sounds interesting and on path of what i wanted to accomplish. i wanted to fully understand behind the scenes of the scans as to what they actually do. and save time on big library's  and be more effective to what people expect when they run a scan. 

i expect that a scan media (scheduled task) would check for new items and update images/meta for only those. 

i expect that scan metadata folder (scheduled task), would do. well not sure what it does. they say it looks for changes in that folder. but who is really editing that folder which has a cryptic file system names. that can only be used with that database? well i guess at one point i did edit the custom logos when they actually made it that far. in attempts to get custom logos. but once again thats only for that database. so if you need to nuke things and start fresh. not good day.

then there is refresh metadata. ether refresh metadata all or refresh metadata missing.  both take same amount of time. you can only replace all images not the missing ones. 

then there is scan library files. which i assume looks for anything new, then gets metadata /images for just those.  

so maybe @Lukecan clarify 

1.i would have library scan

2. then a missing metadata , with options you care about

3. then missing images, with options you care about

and dont put those generated images when it cant find one. 

have a filter from the scan that shows whats missing

another question i have is. whats the benefit of having the nfo and images in the media folder vs metadata folder. i understand its nice and tidy in the medtadata folder. but if you have to delete the db and start over. or the more common thing. your db don't work with that version of emby because they changed where things are in db. i understand why they do it, in the name of progress. so ive learned i now need to backup the system folder with every full backup. just so i know it will work together. so i recently switched to having the meta files in the media folder. but now when it runs those daily scans. it went from 6 minutes to 46 minutes. i deleted the meta files and images for one strm file. and after that nothing was replaced when i the scheduled metadata folder scan. still waiting on results for the second scheduled library scan to finish. but if it dont replace those nfo and images files, then why is it taking so much longer and not doing anything of use that i know of?

sql query's for filtering are within seconds to find the files that are missing things that are needed. so if you have a library of 100,000 it should narrow that down to what is actually needed in few seconds. then update those medias with missing posters, etc. i know the pc can only send cmd so fast. so maybe it can only send a few quests a second. but at least focus on whats missing. and use the regular scan for new stuff.

it will eliminate all the confusion on so many posts over the years.  this is not only an emby issue. its will all providers. but i believe emby is so much better in everyway. or i wouldn't waste my time trying to make it better. embys support team is by far the best there is in this category of software. yes im ocd and perfectionist and proably a few other choice words lol 

 

[Ugnaughts 53]

 

[Luke 40154]

Quote

then there is refresh metadata. ether refresh metadata all or refresh metadata missing.  both take same amount of time. you can only replace all images not the missing ones. 

Hi ,this is not true. Just don't check the box to replace existing images.

[Ugnaughts 53]

On 2/2/2025 at 9:06 PM, Luke said:

Hi ,this is not true. Just don't check the box to replace existing images.

they did take the same time.  but now i realize that doing the refresh from folder and unchecking replace images. will replace missing meta files like png jpg nfo. which is the main thing i wanted. i would occasionally have the initial scan be interrupted because of the time to scan all the strms. i dont want anything fancy, just the default images and basic info on movies and shows. so i tested each provider. and found that tvdb is the most if not fully complete library for both movies and shows. the quality compared to the fanart. the tvmaze did seem allot faster, mostly because it downloads all the info for that title at once, from want i understand. but for simplify am just going to use tvdb.

so if run the scan from ether scheduled or folder view. and its complete. then go to folder view and do a refresh with images unchecked. then it will find missing meta files.  

thank you for your help @Luke