Jump to content

Large Library.db file?


Go to solution Solved by Lessaj,

Recommended Posts

Posted (edited)
1 hour ago, sa2000 said:

Still having different content in the nfo file from yours - running now on 4.9.1.90 and started with a new library with just this file "Paw Patrol S01E39-E40.mkv" in "Paw Patrol\Season 1" folder. Even tried starting with 4.9.1.80 since the date added for your E39-E40 was before 4.9.1.90 was released.

Could you enable debug logging and refresh all metadata for this episode S01E39-E40 and let me have log and zipped nfo file again and also include season.nfo and tvshow.nfo

 

 

Where is the setting to enable debug logging?

 

Nevermind, found it.

Edited by Coolbule
Posted (edited)

So something weird happened.... I manually removed the duplicated stuff from the .nfo last night and today the refresh scan keeps that file to 7kb.

I did the same to the how i met your mother episode, but it reverted to the 800MB. Does this .nfo match yours?

I'm zipping my library file now and will send when its done.

Edited by sa2000 to remove raw unsanitized embyserver log

 

Paw Patrol S01E39-E40.nfo season.nfo tvshow.nfo

Edited by sa2000
Posted

Thanks for the database and the updated nfo for Paw Patrol S01E39-E40 

There are inconsistencies in the data in the nfo which appear to relate to thetvdb changes to introduce "US Airing Order". So at some point the metadata had the episode titles based on the "US Airing Order" rather than the default  "Aired Order". Whilst the title and original title are now based on the Aired Order, the sortname field still has the US Airing Order title. Not sure what we update on refreshes and it is possible some fields are not.

thetvdb has these titles for the different airing date orders

AirDate Order:
E39 - Pups and the Beanstalk
E40 - Pups save the turbots

US Airing Date order:
E39 - Pups Save the Easter Egg Hunt
E40 - Pups and The Lighthouse Boogie
 

Your nfo has the following now

E39
  <title>Pups Save the Easter Egg Hunt</title>
  <originaltitle>Pups and the Beanstalk / Pups Save the Turbots </originaltitle>
  <sorttitle>Pups Save the Easter Egg Hunt,  Pups and the Lighthouse Boogie</sorttitle>
  <imdbid>tt3461700</imdbid>
  <tvdbid>4856019</tvdbid>
  <uniqueid type="tvdb">4856019</uniqueid>
  <uniqueid type="imdb">tt3461700</uniqueid>
  <episode>39</episode>
  <season>1</season>
  <aired>2014-04-13</aired>

E40
  <title>Pups and the Lighthouse Boogie</title>
  <originaltitle>Pups and the Beanstalk / Pups Save the Turbots </originaltitle>
  <sorttitle>Pups Save the Easter Egg Hunt,  Pups and the Lighthouse Boogie</sorttitle>
  <episode>40</episode>
  <season>1</season>
  <aired>2014-04-13</aired>

Maybe the safest option is to move the files out of the current library paths, scan the library and then bring the files back 

I thought maybe the issue / cause is to do with switching back and forth between "US Airing Order" and "Aired Order" but looking at the other media files with issues - the "How I Met Your Mother" series does not have a different aired order - so that blows that idea. 

Looking at your database, the following are shows and seasons that need to be corrected

image.png

I have not been able to see a pattern to allow me to try and reproduce. The issue appears to be mostly with the originaltitle field. All my attempts so far have not reproduced the issue. 

This gives you the multi-episode files that have the issue in addition to Paw Patrol E39-E40 

image.png

Looking at this list, do you know what might have been common to cause the problem?

 

 

 

  • Like 1
Posted
On 08/12/2025 at 22:41, Coolbule said:

Also I'll list my meta data options for TV Shows if that helps

Series Metadata  Downloaders 

TVDB - Movie DB - Open Movie DB

Season Metadata Downloaders

only TheMovieDb

Episode Metadata Downloaders

TheMovieDB - The TVDB - Open Movie DB

I think it is a bad idea to have all 3 providers listed here for Episode Metadata 

You have opted for the TheMovieDB only for the season metadata, so I would suggest you just have that for Episode metadata as mixing metadata from different sources could be potential for issues

You can see here that the Movie Database has US Airing Dates by default for Paw Patrol

image.png

But the default for thetvdb which is what is used by Emby is not the US Airing Dates

image.png

thetvdb has this for US Airing Date which we would not be using - 

image.png

I will add a note in the Naming Standards article about this

 

  • Agree 1
  • Thanks 1
Posted (edited)

@Lessaj and any others with this issue could use this sqlite query to list media files with this issue

select id, LENGTH(name), LENGTH(OriginalTitle), SeriesName, SortName, ProviderIds, Path, DateCreated, DateModified, DateLastRefreshed, DateLastSaved, IsLocked, LockedFields  from mediaitems where ((type = 8) and ((name like '% / % / % / %') OR (originaltitle like '% / % / % / %')))

I like Paweł Salawa's SQLite Studio product which has a very easy to use "Export query results" feature in the tools menu for exporting to a csv which can be imported to excel

Note: As it is looking for media items with title or originaltitle including 3 slash characters - you may pick some that do have that many slashes and not actually the repeated two episode titles over and over again. So there may be false positives.

Update: changed the query to just pick episodes since we know the issue has only been seen for episodes. The query results gives the series name, episode path and length of each of the fields that have been seen to get corrupted. Also some date/time information lis returned, like date added, last refreshed etc .. These are epoch times and can be converted to the UTC date and time using tools such as EpochConverter

 

Edited by sa2000
Posted
1 minute ago, sa2000 said:

@Lessaj and any others with this issue could use this sqlite query to list media files with this issue

select id, LENGTH(name), LENGTH(OriginalTitle), SeriesName, type, SortName, ProviderIds, Path, DateCreated, DateModified, DateLastRefreshed, DateLastSaved, IsLocked, LockedFields  from mediaitems where ((name like '% / % / % / %') OR (originaltitle like '% / % / % / %'))

I like Paweł Salawa's SQLite Studio product which has a very easy to use "Export query results" feature in the tools menu for exporting to a csv which can be imported to excel

 

Thanks for providing this. It returns 30 rows for me and interestingly 18 of the 30 are songs, the rest are from Hey Arnold! and one episode of Ramen Akaneko. Doesn't seem like I have much of an issue overall, but I will look into the results. I just fixed Hey Arnold! the other day when I mentioned it...

Posted

There may be false positives - if the titles do genuinnely have 3 slashes

Posted

Ah, in the case of the music files, yes the titles do have more than one, thank you. For the Hey Arnold! entries I realized that this was a copy of my library before I fixed it, so nevermind all good now. And the Ramen Akaneko episode does have 4 titles with 3 slashes. So in my case, no issues found. :)

  • Thanks 1
Posted

Thanks. I have amended the query above to just pick episodes

Posted

I did realize you guys set up FK's so I went in and manually changed the fields instead of deleting them. Dropped 3GB from the file so now everything is running well. Its still pretty large at 12GB, so I may have more to look at later on, but for now its working again.

Posted (edited)
5 hours ago, Lessaj said:

Thanks for providing this. It returns 30 rows for me and interestingly 18 of the 30 are songs, the rest are from Hey Arnold! and one episode of Ramen Akaneko. Doesn't seem like I have much of an issue overall, but I will look into the results. I just fixed Hey Arnold! the other day when I mentioned it...

Also check fts_search9 the titles are duplicated on that table too

That saved me another 3GB on the .db file

Edited by Coolbule
  • Thanks 1
Posted

Okay final thing i found out.... @sa2000if you ever find out how to rebuild the fts_search9, I believe this is the only table left with the duplicated titles stored but since its stored as a BLOB its a lot harder to read through to see, without just rebuilding the entire db let me know...but that table is currently at 9.2GB and when i delete it the search no longer works (makes sense), but it doesn't rebuild the search either ( i was just testing and being hopeful). Everything is running fine though even with that table as large as it is and none of the duplicated titles have come back since. 

I looked at what I could and didn't really find out how that table related to the others and the blobs for some were massive and some were tiny. Thank you guys for all your help, server is running great now.

  • Thanks 1
Posted
4 hours ago, Coolbule said:

if you ever find out how to rebuild the fts_search9,

Have not been able to find out how to do this easily - will discuss with the development team

Posted
12 hours ago, Coolbule said:

if you ever find out how to rebuild the fts_search9, I believe this is the only table left with the duplicated titles stored but since its stored as a BLOB its a lot harder to read through to see, without just rebuilding the entire db let me know...but that table is currently at 9.2GB and when i delete it the search no longer works (makes sense), but it doesn't rebuild the search either ( i was just testing and being hopeful). Everything is running fine though even with that table as large as it is and none of the duplicated titles have come back since. 

I have discussed this with the development team. I understand that the search data gets recreated for an item when clicking Save for an item in Metadata Manager.

So for each of the affected 2-episodes item identified in my list that I pasted above here, select it in Metadata Manager and then click Save. Also do Paw Patrol S01E39-E40 which you fixed beforehand

You can try Metadata Manager at Season level and see if that propagates down to the episodes. If not, then do each multi-episode file in the list individually.

Let me know how it goes

Posted

I had a similar problem with a huge library.db. It drove me insane, and on top of that, I had the ram slowly filling up till it eventually crashed the server.

Over a period of 4 months time, I reinstalled my emby server 3 times - the last time on totally new hardware which is now running my emby while my old server is effectively just a network disk share.

What was ultimately my problem was actually some old videofiles that I guess got corrupted somewhere along the way.

My db would be looking fine, then when I added the folder with these files, it just went crazy - 50-100GB db file and ram usage through the roof.

I only noticed it when I accidentally forgot to add a share when setting it up again for the 16th time, trying to fix it.

 

Just leaving it here incase someone comes here looking for a solution and doesnt find one in the original posts.

Posted
11 hours ago, TimFlix said:

My db would be looking fine, then when I added the folder with these files, it just went crazy - 50-100GB db file and ram usage through the roof.

Did you have multi-episode media files ? Like in example seen here - "Paw Patrol S01E39-E40.mkv" for season 1 episodes 39 and 40.

Posted
On 15/12/2025 at 08:55, sa2000 said:

You can try Metadata Manager at Season level and see if that propagates down to the episodes. If not, then do each multi-episode file in the list individually.

@CoolbuleIt does not - so would need to be individually for the 61 media files known to have the issue.

Posted (edited)
On 12/16/2025 at 4:30 AM, sa2000 said:

@CoolbuleIt does not - so would need to be individually for the 61 media files known to have the issue.

Sorry for the late reply, thank you so much! I'll go through and do that. You guys have been super helpful thanks for reaching out to them and getting back with me!

Edit: Will i need to do a "fake change" to force the save, or will it save even if 'no change' it detected?

Edited by Coolbule
  • Thanks 1
Posted

The NFO will be updated when you hit save even if you didn't change anything.

Posted

Yes just clicking save and it is expected that the fts_search9 field would get updated for that item

  • Thanks 1

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...