Large Library.db file?

[Lessaj 467]

I am not sure what fts_search9_data is used for, my DB only has about 15k rows for that table. It makes sense that this would be large because one of the things it seems to store is a binary blob, it looks like each blob is about 4k - if you multiply by 3,791,378, divide by 1024 for MB and again for GB that's 14.46GB, pretty much exactly as you're seeing. Hopefully a dev can chime in on what this table is used for and why it might be so huge.

[Coolbule 12]

Yeah i pulled up my original 55GB db file and it showed the same...that one table exploded. Thank you for your help today. Hopefully a dev can give more insight now.

[Lessaj 467]

1 hour ago, Coolbule said:

When you were talking about my file naming scheme, could you explain more on that? Is it just the extra letter folders mostly?

I noticed some nesting, I guess because you have it organized into collections already on the file system? Like 101 Dalmations. MKVs within a folder with an indecipherable name. Episodes that haven't been renamed.  Stuff that shouldn't cause a huge library file but it helps for identification or multi version.

Z:\Movies\0\101 Dalmations\101 Dalmations\101 Dalmations.mpg
Z:\Movies\L\Land Before Time\The Land Before Time VI The Secret of Saurus Rock\B4_t00.mkv
Z:\TV Shows\H\Homeland\Season 2\Homeland.S02E01.1080p.BluRay.x265-RARBG.mp4

 

[Coolbule 12]

1 minute ago, Lessaj said:

I noticed some nesting, I guess because you have it organized into collections already on the file system? Like 101 Dalmations. MKVs within a folder with an indecipherable name. Episodes that haven't been renamed.  Stuff that shouldn't cause a huge library file but it helps for identification or multi version.

Z:\Movies\0\101 Dalmations\101 Dalmations\101 Dalmations.mpg
Z:\Movies\L\Land Before Time\The Land Before Time VI The Secret of Saurus Rock\B4_t00.mkv
Z:\TV Shows\H\Homeland\Season 2\Homeland.S02E01.1080p.BluRay.x265-RARBG.mp4

 

Since Emby read those in fine I just used that as an easier way for me to see movies. So like 101 Dalmations and 101 Dalmations 2 are in the same folder "0/101 Dalmations" 

Somewhere i put all my marvel movies together under /m/Marvel/Captn A/ (then the 2-3 movies here). 

and then the B4_t00.mkv, I am just lazy and don't re-name them unless I have multiple versions (or it has trouble finding the metadata)

Homeland i just forgot to rename lol

[Lessaj 467]

Yea not related to the issue here I don't think.

[Coolbule 12]

23 hours ago, Luke said:

You basically would just do as much as you're willing to allocate.

Hey so we found out which table in the db is exploding in size causing the issue of the server lagging/taking up 40+GB of RAM. ftw_search9_data, do you have any insight in why that table keeps growing uncontrollably? The logs at the start of this thread show the whole library being built which turned into ~33GB of data.

[Coolbule 12]

I honestly am at a loss on what to do now. I re-added my libraries so I could atleast try to watch my shows but I'm getting errors or extremely slow loading times for each screen. Play next episode fails to work, playing any show can get an error processing request. The skip ahead/jump back takes 30 seconds or longer to appear on screen. Attached is another recent log file....should show all the errors.

embyserver.txt

[Happy2Play 9780]

Devs will have to comment further but looks like a database issue.

Error BaseItem: Error refreshing item

Busy: cannot commit transaction - SQL statements in progress

 

Edited December 6, 2025 by Happy2Play

[Lessaj 467]

Do you store NFO next to your media? If yes, can you check if the NFO for this episode is massive? There may be others but this is the first one that comes up, after that the errors are SQL statements in progress.

Quote

System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Too many characters. The resulting number of bytes is larger than what can be returned as an int. (Parameter 'charCount')

2025-12-06 01:01:41.382 Error BaseItem: Error refreshing item The Final Page (1) / The Final Page (2) 25279 Z:\TV Shows\H\How I Met Your Mother\Season 8\S08E11-E12.mkv

Quote

    SQLitePCL.pretty.SQLiteException: Busy: cannot commit transaction - SQL statements in progress
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.

Saw the same error here and it was a big NFO file:

 

Edited December 6, 2025 by Lessaj

[Coolbule 12]

14 minutes ago, Lessaj said:

Do you store NFO next to your media? If yes, can you check if the NFO for this episode is massive? There may be others but this is the first one that comes up, after that the errors are SQL statements in progress.

Saw the same error here and it was a big NFO file:

Holy crap you are a genius. its 884MB. I guess I can make a script to look for .nfo files that are >1mb or something. I can't even open it in notepad.

Edited December 6, 2025 by Coolbule
moved text out of quote

[Lessaj 467]

I don't think that will contribute to the massive table we've identified but it's definitely something that is an issue! Notepad will not be able to open something that large, Notepad++ probably could but more than likely it's replicating the same information over and over again.

[Lessaj 467]

@sa2000This is the second report of an NFO being huge causing issues, and both instances seems to be related to a double episode. I don't have any double HIMYM episodes to directly compare against but I did find a few of my own double episode NFOs that have the same information repeated in them, luckily they are still very small. I'm not sure if there's an issue with the metadata plugins that may be causing this, I do recall an issue earlier this year where that might have been happening. Might be something to keep an eye out for, I know that I will be looking out for this for the episodes I've identified in my own library. Attaching an example NFO from my library.

EDIT: In my case it looks like there's something odd that happened with the double episodes, I noticed this happen with another show. May be what ended up causing that NFO to balloon but I think I can fix it on my end in this case.

Hey Arnold! - S05E41-E42 - The Journal.nfo

Edited December 6, 2025 by Lessaj

[Coolbule 12]

16 minutes ago, Lessaj said:

I don't think that will contribute to the massive table we've identified but it's definitely something that is an issue! Notepad will not be able to open something that large, Notepad++ probably could but more than likely it's replicating the same information over and over again.

Let's be honest....people only actually open notepad because they misclicked and didn't get the ++ one lol.

Is there a chance the .db file is being built off .nfo data? 

[Lessaj 467]

1 minute ago, Coolbule said:

Let's be honest....people only actually open notepad because they misclicked and didn't get the ++ one lol.

Is there a chance the .db file is being built off .nfo data? 

It does import the NFO if you have that enabled, I'm just not sure it's related to the massive table you're seeing. I don't know what it's used for.

[sa2000 674]

13 minutes ago, Lessaj said:

This is the second report of an NFO being huge causing issues, and both instances seems to be related to a double episode. I don't have any double HIMYM episodes to directly compare against but I did find a few of my own double episode NFOs that have the same information repeated in them, luckily they are still very small. I'm not sure if there's an issue with the metadata plugins that may be causing this, I do recall an issue earlier this year where that might have been happening. Might be something to keep an eye out for, I know that I will be looking out for this for the episodes I've identified in my own library. Attaching an example NFO from my library.

Is this related / similar to the crash when saving to an nfo file that was found to have existing one size 840Mb nfo file: see https://emby.media/community/index.php?/topic/144523-emby-server-crashing-emby-49190-debian-132/#findComment-1490574

[sa2000 674]

15 minutes ago, Lessaj said:

Might be something to keep an eye out for, I know that I will be looking out for this for the episodes I've identified in my own library. Attaching an example NFO from my library.

I will try and see if i can reproduce the problem

[Lessaj 467]

Just now, sa2000 said:

Is this related / similar to the crash when saving to an nfo file that was found to have existing one size 840Mb nfo file: see https://emby.media/community/index.php?/topic/144523-emby-server-crashing-emby-49190-debian-132/#findComment-1490574

In the case of what was identified from the latest log file provided, yes. I don't know if it's the root of the problem here.

[sa2000 674]

what i would be interested is an nfo file (or way to end up with it) that would lead to this error in the log

System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Too many characters. The resulting number of bytes is larger than what can be returned as an int. (Parameter 'charCount')
	   at System.Text.UTF8Encoding.UTF8EncodingSealed.<GetMaxByteCount>g__ThrowArgumentException|7_0(Int32 charCount)

 

[Coolbule 12]

1 minute ago, sa2000 said:

what i would be interested is an nfo file (or way to end up with it) that would lead to this error in the log

System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Too many characters. The resulting number of bytes is larger than what can be returned as an int. (Parameter 'charCount')
	   at System.Text.UTF8Encoding.UTF8EncodingSealed.<GetMaxByteCount>g__ThrowArgumentException|7_0(Int32 charCount)

 

What is the best way to upload an 800MB .nfo file to you

[sa2000 674]

2 minutes ago, Coolbule said:

What is the best way to upload an 800MB .nfo file to you

sending you an upload link by PM

[Coolbule 12]

I guess zipping it probably would have made it small enough to go through the forums chat.... but its also uploaded to your pm too
 

S08E11-E12.zip

[Coolbule 12]

Fun fact.... I have Paw Patrol on my server and almost all of those are 2 episode files... every single one is over 100MB now too. So that "2 episode" theory seems to be getting stronger

30 minutes ago, Lessaj said:

@sa2000This is the second report of an NFO being huge causing issues, and both instances seems to be related to a double episode. I don't have any double HIMYM episodes to directly compare against but I did find a few of my own double episode NFOs that have the same information repeated in them, luckily they are still very small. I'm not sure if there's an issue with the metadata plugins that may be causing this, I do recall an issue earlier this year where that might have been happening. Might be something to keep an eye out for, I know that I will be looking out for this for the episodes I've identified in my own library. Attaching an example NFO from my library.

EDIT: In my case it looks like there's something odd that happened with the double episodes, I noticed this happen with another show. May be what ended up causing that NFO to balloon but I think I can fix it on my end in this case.

Hey Arnold! - S05E41-E42 - The Journal.nfo 87.6 kB · 3 downloads

 

[Lessaj 467]

Digging more into my own example I mentioned earlier I'm finding a big crossover of episodes suddenly happened, and may be due to changes on TVDB since I see Hey Arnold was updated recently so now I'm going through and trying to fix it, maybe be unrelated but a similar symptom...

[Coolbule 12]

@sa2000Hey, so I found a 100MB file that is seemingly having the same issue, but it can actually open in notepad++ easily.... its the Title and the originalTitle lines. They're duplicated over and over and over...and over.. I'll attach that one since it should be easier to debug a smaller/more usable file.

 

This would make sense on the large db file now....since it is reading the title of the show/movie and putting it into the db table for search records.

Paw Patrol S01E39-E40.zip

Edited December 6, 2025 by Coolbule
2nd thought added

[Coolbule 12]

So I don't know why How I met your Mother was the one that caused the error.....i have some >1GB right now. I'll keep em for the rest of today or so if you need any more examples, then I'll have my python script delete them to keep my server from dying lol.