Jump to content

metadata cache purge feature / force update metadata


Riezel

Recommended Posts

No, it will get backported at some point but I think the change comes with some degree of risk so I think it's best that it sit in the beta channel for a little while. Thanks.

  • Like 1
Link to comment
Share on other sites

  • 3 weeks later...

@RiezelI have issues with memory on the Emby Server for Synology as well during Library scans.  I too have a very large library. What I can share with you is to make sure your log files are free from errors during library scans.  No errors, pretty much no memory blow up. Lots of errors in the log file during scanning and you'll see memory slowly climb until it runs out.

Your log shows lots of
SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
errors when scanning gets to /volume1/media/Anime/Series/!Collection/Cyberpunk Edgerunners/Cyberpunk.Edgerunners.

You'll notice the same FOREIGN KEY and 'SQLitePCL.pretty.SQLiteException' errors keep repeating over and over when trying to save info to your database. Your logging info with full debug logging turned on which actually will make this issue worse for you as memory is used at a faster rate dong that.

Try our recommended instructions in the following Knowledge Base Article: https://emby.media/support/articles/Corrupt-Database.html?q=SQLiteException

 

Link to comment
Share on other sites

Riezel
Posted (edited)
3 hours ago, Carlo said:

@RiezelI have issues with memory on the Emby Server for Synology as well during Library scans.  I too have a very large library. What I can share with you is to make sure your log files are free from errors during library scans.  No errors, pretty much no memory blow up. Lots of errors in the log file during scanning and you'll see memory slowly climb until it runs out.

Your log shows lots of
SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
errors when scanning gets to /volume1/media/Anime/Series/!Collection/Cyberpunk Edgerunners/Cyberpunk.Edgerunners.

You'll notice the same FOREIGN KEY and 'SQLitePCL.pretty.SQLiteException' errors keep repeating over and over when trying to save info to your database. Your logging info with full debug logging turned on which actually will make this issue worse for you as memory is used at a faster rate dong that.

Try our recommended instructions in the following Knowledge Base Article: https://emby.media/support/articles/Corrupt-Database.html?q=SQLiteException

 

sadly, kinda late already, i'm currently trying different approach atm

probably kinda same approach, but not via database level, i've doing full refresh meta data atm, sofar no memory spike up, i'll probably can start experiment again in 4 ~ 5 days (after full refresh my library), and yes it's that long for my huge library, what currently i did is, renaming all files (16k+ files) that potentially have this issue (detected as multiple episode) and go full refresh meta data in new library as well (not full scans)

Sofar havnt seen those sqlite error with this method (just checked the log from 19th), thanks for the reply btw, i'll keep it updated in the other thread later

Edited by Riezel
Link to comment
Share on other sites

On 3/20/2024 at 9:39 PM, Riezel said:

sadly, kinda late already, i'm currently trying different approach atm

probably kinda same approach, but not via database level, i've doing full refresh meta data atm, sofar no memory spike up, i'll probably can start experiment again in 4 ~ 5 days (after full refresh my library), and yes it's that long for my huge library, what currently i did is, renaming all files (16k+ files) that potentially have this issue (detected as multiple episode) and go full refresh meta data in new library as well (not full scans)

Sofar havnt seen those sqlite error with this method (just checked the log from 19th), thanks for the reply btw, i'll keep it updated in the other thread later

OK I'm trying to chase down why this is happening to you but worst case scenario you may want to consider starting with a new database.

Link to comment
Share on other sites

Riezel
4 hours ago, Luke said:

OK I'm trying to chase down why this is happening to you but worst case scenario you may want to consider starting with a new database.

i already fixed it, details in here, and how i achieve it, so far no problem after i did it (it just took time for my lib to fix it, due to huge size of it)

 

  • Thanks 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...