Jump to content

Large Library.db file?


Go to solution Solved by Lessaj,

Recommended Posts

Posted

Do you guys only rely on Emby's nfo saver or are you using other programs to precure them?

I don't see this.

But big but.

I only scrape data to nfo on initial import.

After that its set to never.

This is just out of curiosity 😁

 

Posted

Emby completely manages my NFOs.

  • Like 1
Posted

Hmmm so its set to never or 

image.png.1401a34fffebed67bae8bd6b544b5f15.png

  • Solution
Posted

I refresh every 30 days. Anyway I fixed the example I provided, so you can ignore that. It does seem like it was due to changes from TVDB for that series, but my NFOs were still in the KB range just larger than expected.

If this does stem from an issue with the plugin earlier this year that I'm remembering, and my memory could be incorrect here since none of the plugin notes mention that specifically, but I highly suspect if you clean up any of these excessively large NFOs and then delete/rename library.db so it gets recreated, then start to recreate your libraries it may be fully resolved.

  • Like 1
  • Thanks 1
Posted
7 minutes ago, Lessaj said:

I refresh every 30 days. Anyway I fixed the example I provided, so you can ignore that. It does seem like it was due to changes from TVDB for that series, but my NFOs were still in the KB range just larger than expected.

If this does stem from an issue with the plugin earlier this year that I'm remembering, and my memory could be incorrect here since none of the plugin notes mention that specifically, but I highly suspect if you clean up any of these excessively large NFOs and then delete/rename library.db so it gets recreated, then start to recreate your libraries it may be fully resolved.

I think so. I'm going to try and vacuum the db and what-not and see if the db cleans itself up first so i can maybe keep my watch history lol. But if that doesn't clear it up by tomorrow I'll just do a rebuild. Thanks for your help! 

Posted
4 minutes ago, Coolbule said:

I think so. I'm going to try and vacuum the db and what-not and see if the db cleans itself up first so i can maybe keep my watch history lol. But if that doesn't clear it up by tomorrow I'll just do a rebuild. Thanks for your help! 

The watch history is maintained in the users DB to my knowledge, so it -should- be possible to just rebuild library.db, otherwise you can use the backup and restore plugin to restore the watch history/user data after a full rebuild.

Posted
Just now, Lessaj said:

The watch history is maintained in the users DB to my knowledge, so it -should- be possible to just rebuild library.db, otherwise you can use the backup and restore plugin to restore the watch history/user data after a full rebuild.

Does that include the continue watching? When i did the library.db rebuild earlier all of that seemed to go away....maybe it was just not loading yet because that .nfo file was breaking everything.

Happy2Play
Posted
1 minute ago, Lessaj said:

The watch history is maintained in the users DB to my knowledge, so it -should- be possible to just rebuild library.db, otherwise you can use the backup and restore plugin to restore the watch history/user data after a full rebuild.

No it is in the usersdata table of the library.db.  But should be able to extract userdata from a backup.

 

  • Thanks 1
Posted (edited)
2 minutes ago, Coolbule said:

Does that include the continue watching? When i did the library.db rebuild earlier all of that seemed to go away....maybe it was just not loading yet because that .nfo file was breaking everything.

To my knowledge yes, if restoring the user data via the plugin. Admittedly never actually had to restore data before.

EDIT: Per above would basically have to use the plugin to restore the user data from a backup. My bad on mixing that up.

Edited by Lessaj
Posted

Out curiosity what kind of useful data would be in an NFO file that would make it 1MB, 10MB or more in size? My largest NFO is around 20K but I let Emby create and manage and don't have much use for them myself. I can't think if what could push an NFO to almost 1GB in size and for that data to have any real purpose.

Wouldn't the cleanest approach be deleting all of them and letting Emby recreate. I get how some people customize their metadata using NFO so a clean slate would also mean more work to recreate the customizations.

 

Posted
Just now, Q-Droid said:

Out curiosity what kind of useful data would be in an NFO file that would make it 1MB, 10MB or more in size? My largest NFO is around 20K but I let Emby create and manage and don't have much use for them myself. I can't think if what could push an NFO to almost 1GB in size and for that data to have any real purpose.

Wouldn't the cleanest approach be deleting all of them and letting Emby recreate. I get how some people customize their metadata using NFO so a clean slate would also mean more work to recreate the customizations.

 

From what I saw, my largest valid NFO was ~.06MB. Its just a text file listing details about the movie/show.

The invalid had a lot of repetitive text on the title and original title xml tags, which is why it was >1MB.

Posted (edited)

@Coolbule were you using Jellyfin before and might that application been the one that created the NFO?

This is from your fat NFO:

Quote

    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\A\Alexis Denisof\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\A\Alyson Hannigan\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\C\Chelan Simmons\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\C\Cobie Smulders\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\E\Ellen D. Williams\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\J\Jason Segel\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\J\Josh Radnor\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\N\Neil Patrick Harris\folder.jpg</thumb>
    <thumb>C:\ProgramData\Jellyfin\Server\metadata\People\S\Seth Green\folder.jpg</thumb>
 

 

 

Edited by Q-Droid
Posted (edited)
31 minutes ago, Q-Droid said:

@Coolbule were you using Jellyfin before and might that application been the one that created the NFO?

This is from your fat NFO:

 

 

There is a good chance. I did turn off jellyfin's scrapers so it will only read the files Emby creates now. I am surprised you got that file open

Edited by Coolbule
Posted
48 minutes ago, Q-Droid said:

@Coolbule were you using Jellyfin before and might that application been the one that created the NFO?

This is from your fat NFO:

 

 

I do see that <thumb> added. That one specifically could have been added by jellyfin... but the other one(s) I have don't have the <thumb> for jellyfin and are also large. I do have others without the <thumb> jellyfin location that are >1GB also, so they might both be having the issue?

Posted
2 hours ago, Lessaj said:

I refresh every 30 days. Anyway I fixed the example I provided, so you can ignore that. It does seem like it was due to changes from TVDB for that series, but my NFOs were still in the KB range just larger than expected.

If this does stem from an issue with the plugin earlier this year that I'm remembering, and my memory could be incorrect here since none of the plugin notes mention that specifically, but I highly suspect if you clean up any of these excessively large NFOs and then delete/rename library.db so it gets recreated, then start to recreate your libraries it may be fully resolved.

Well i can confirm emby is the one creating the 800MB .nfo file. I just had emby refresh the metadata on that one How I met your mother episode and it is back to 884MB.

  • Sad 1
Posted
28 minutes ago, Coolbule said:

Well i can confirm emby is the one creating the 800MB .nfo file. I just had emby refresh the metadata on that one How I met your mother episode and it is back to 884MB.

Was the new NFO created from a clean DB or had the data been stored before you deleted the NFO file?

 

Posted
35 minutes ago, Q-Droid said:

Was the new NFO created from a clean DB or had the data been stored before you deleted the NFO file?

 

If i selected "Replace all metadata" that should delete then re-insert right? (for that specific episode). I do know the jellyfin tags were no longer in the file. when I had emby replace all metadata for it.

Posted
1 hour ago, Coolbule said:

If i selected "Replace all metadata" that should delete then re-insert right? (for that specific episode). I do know the jellyfin tags were no longer in the file. when I had emby replace all metadata for it.

I agree but don't know what Emby actually does.

 

Posted (edited)
On 06/12/2025 at 21:07, Coolbule said:

Well i can confirm emby is the one creating the 800MB .nfo file. I just had emby refresh the metadata on that one How I met your mother episode and it is back to 884MB

I am trying to see if I can reproduce the issue seen in the nfo file for Paw Patrol S01 E39-E40

The problem in your nfo is with the "title" and "originaltitle" fields for each of the two episodes where they are repeating two titles on and on reaching 32Mb + characters for the title line and 24.5Mb characters for the original title line !

At the moment my tests are picking different episodes from yours for E39 and E40. Your appears to be for the "US Airing Date" whereas my tests are picking the default "Airing date" ordering.

How did you get E39 and E40 pick US Airing Dates for episodes ? 

E39 Pups Save the Easter Egg Hunt
E40 Pups and the Lighthouse Boogie

instead of

E39 Pups and the Beanstalk
E40 Pups Save the Turbots

I wonder if the cause is related to having refreshes with different airing dates order

Edited by sa2000
Posted
4 hours ago, sa2000 said:

I am trying to see if I can reproduce the issue seen in the nfo file for Paw Patrol S01 E39-E40

The problem in your nfo is with the "title" and "originaltitle" fields for each of the two episodes where they are repeating two titles on and on reaching 32Mb + characters for the title line and 24.5Mb characters for the original title line !

At the moment my tests are picking different episodes from yours for E39 and E40. Your appears to be for the "US Airing Date" whereas my tests are picking the default "Airing date" ordering.

How did you get E39 and E40 pick US Airing Dates for episodes ? 

E39 Pups Save the Easter Egg Hunt
E40 Pups and the Lighthouse Boogie

instead of

E39 Pups and the Beanstalk
E40 Pups Save the Turbots

I wonder if the cause is related to having refreshes with different airing dates order

So, I'm actually getting the episode info from themoviedb first.... and I did find something interesting opening paw patrol in emby...

I guess this could be the TheMovieDB's API issue and not TVDB? I remember having issues getting the order for one show so I think that's how I got the US order.

Also I'll list my meta data options for TV Shows if that helps

Series Metadata  Downloaders 

TVDB - Movie DB - Open Movie DB

Season Metadata Downloaders

only TheMovieDb

Episode Metadata Downloaders

TheMovieDB - The TVDB - Open Movie DB

 

image.thumb.png.d83fdd42a7ff4fd0d88973f2df0e1994.png

  • Thanks 1
Posted
2 hours ago, Coolbule said:

Also I'll list my meta data options for TV Shows if that helps

Series Metadata  Downloaders 

TVDB - Movie DB - Open Movie DB

Season Metadata Downloaders

only TheMovieDb

Episode Metadata Downloaders

TheMovieDB - The TVDB - Open Movie DB

Creating a library with your metadata settings for Paw Patrol  Season 1 with media file "Paw Patrol S01E39-E40.mkv" now has E39 E40 being matched to what you have in the nfo file.

Ignoring the massive title and originaltitle fields that you have when comparing your nfo to mine, I have some extra info like: "<writer>" and "<credits>" and "<rating>". Also the "<plot> text in my case is for each of E39 and E40 separately whereas in yours the E39 plot has both concatenated and the E40 plot is empty string. And I do not have an "<originaltitle>" field.

So I am almost there in replicating your setup for the library. 

What version of Emby Server are you running. The S01E39-E40 file was added to your library on 30 October 2025 - what versions of Emby Server have been running at that time and after?

Also any changes to library metadata providers/options since 30 october?

Also - if you get me a log file I can see what plugins you have - in case there is some plugin involvement here

FYI these are the settings I have for my test library:

Preferred Metadata Language: English
Certification: United States
Preferred Image language: English
Prefer Embedded Titles over filenames: unticked
Metadata Readers: Nfo enabled
Series Metadata Downloaders: 3 enabled in following order: TheTVDB, TheMovieDB, The Open Movie Database
Season Metadata Downloaders: 1 enabled: TheMovieDB
Episode Metadata Downloaders: 3 enabled in following order: TheMovieDB, TheTVDB, The Open Movie Database
After initial import, automatically refresh metadata from the internet: Never
Automatically refresh metadata for episodes with placeholder titles such as TBA: Never
Metadata Savers: Nfo - Enabled
Series Image Fetchers: 2 selected in this order: TheTVDB, FanArt
Season Image Fetchers: 1 selected: TheTVDB
Episode Image Fetchers: 2 selected in this order: TheTVDB, Image Capture
Save artwork into media folders: enabled
Save metadata and images as hidden files: unticked

 

Posted (edited)

Current Version is 4.9.1.90, I just let it update automatically.

Here's my latest log file...if there's a different one you need let me know or if you need me to hit "refresh meta data" on that file and get the logs from that I can do that too.

The test library does look the same as mine.

I haven't touched the metadata stuff....except when i re-made the library.db and re-created the libraries they were probably set back to default, then i brought back my "66GB" library and removed the nfo and it went down to 49GB (just from the 800MB one).

Oh also, does the "Replace all metadata" re-grab everything from the sources and update the DB (when you run it on a single episode/file)?

 

Edit: I also just ran the refresh meta data on S1 paw patrol... they're all like 23MB now for the multi-episode files...i'll attach the new one and I think it matches your setup now too

Edited by sa2000 to remove raw unsanitized embyserver log file

 

 

Paw Patrol S01E39-E40.nfo

Edited by sa2000
removed unsanitized log
Posted
9 hours ago, Coolbule said:

ou need me to hit "refresh meta data" on that file and get the logs from that I can do that too.

The test library does look the same as mine.

Still having different content in the nfo file from yours - running now on 4.9.1.90 and started with a new library with just this file "Paw Patrol S01E39-E40.mkv" in "Paw Patrol\Season 1" folder. Even tried starting with 4.9.1.80 since the date added for your E39-E40 was before 4.9.1.90 was released.

Could you enable debug logging and refresh all metadata for this episode S01E39-E40 and let me have log and zipped nfo file again and also include season.nfo and tvshow.nfo

Also what settings do you have for the "Nfo Metadata" plugin?

Could you double check the order of metadata sources 

This shows the diffs between your nfo and mine. I am getting writer and credits entries

Paw Patrol S01E39-E40.nfo diffs

My nfo

E39
  <plot><![CDATA[It’s Adventure Bay’s annual Easter Egg Hunt and the pups are helping Mayor Goodway with the egg decorating. The hunt for a very special egg leads Ryder and the PAW Patrol on an exciting adventure.]]></plot>
  <title>Pups Save the Easter Egg Hunt</title>
  <writer>Ursula Ziegler-Sullivan</writer>
  <credits>Ursula Ziegler-Sullivan</credits>
  <rating>5</rating>

E40
  <plot><![CDATA[Cap'n Turbot has accidentally locked himself inside the lighthouse, so it is up to the pups to rescue a rowboat drifting out to sea.]]></plot>
  <title>Pups and the Lighthouse Boogie</title>
  <writer>Ursula Ziegler-Sullivan</writer>
  <credits>Ursula Ziegler-Sullivan</credits>
  <rating>5</rating>

Your nfo

E39  
  <plot><![CDATA[It’s Adventure Bay’s annual Easter Egg Hunt and the pups are helping Mayor Goodway with the egg decorating. The hunt for a very special egg leads Ryder and the PAW Patrol on an exciting adventure. / Cap'n Turbot has accidentally locked himself inside the lighthouse, so it is up to the pups to rescue a rowboat drifting out to sea.]]></plot>
  <title>Pups Save the Easter Egg Hunt / Pups and the Lighthouse Boogie / Pups Save the Easter Egg Hunt ...... etc repeated ...</title>
  <originaltitle>Pups and the Beanstalk / Pups Save the Turbots / Pups and the Beanstalk / Pups Save the Turbots ...... etc repeated ...</originaltitle>
  <rating>0</rating>

E40
  <plot />
  <title>Pups Save the Easter Egg Hunt / Pups and the Lighthouse Boogie / Pups Save the Easter Egg Hunt / Pups and the Lighthouse Boogie ...... etc repeated ...</title>
  <originaltitle>Pups and the Beanstalk / Pups Save the Turbots / Pups and the Beanstalk / Pups Save the Turbots</originaltitle ...... etc repeated ...>
  <rating>0</rating>

I could have a look at your library.db - you can use the link I provided you to upload a zip of the library.db - if you do daily backups, the zip of last night;s backup library.db would be ok

 

Happy2Play
Posted (edited)
12 hours ago, Coolbule said:

Oh also, does the "Replace all metadata" re-grab everything from the sources and update the DB (when you run it on a single episode/file)?

Yes/No as it will not clear out any fields that Emby is not providing (Once data is in db a Refresh will not remove all existing data in a Refresh).  Like I don't get your rogue <title>/<originaltitle> field at all in my test either.  So from your metadata standpoint I think it is just bloating your db per the rouge <originaltitle> field.  Importing your nfo change your title field to what online provider returned.  I would have to disable providers to get your exact nfo file.

Importing your nfo file really bloats the db table orginaltitle column.

image.thumb.png.1ef83c17a1d35a03a0d1983ad0e7b187.png

Can only guess third party metadata within nfo creation unless we can find the specific steps to recreate this nfo as the setting listed above will not create it.

Edited by Happy2Play
Happy2Play
Posted

Also note just deleting already import nfo files and refreshing will not clear the db either as Emby will just db info into new nfo file.  So if one where to just delete nfo files they would need to Manually Identify the show to sort of purge the db or remove the show/library, do a complete scan for Emby to purge the db and then readd show/library.

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...