Music Library in Bad State

[lharris 3]

My music library is in a bad state and logs are not really helping even with debug on.    I ran a vacuum database and it cut it down by half, but it didn't help.

So what happened:

Backend storage moved and almost all of by libraries had a folder injected into their path.  A long time ago I went from using \\servername\TV Shows \\Servername\Movies to \\Severname\Media and then everything sat inside there.   I kept the legacy shares around for a while and finally moved them.   All went well until I got to the music library. 

Changed it from \\servername\music to \\servername\media\music and it just never seemed responsive... i tried it over and over... and now for some reason I have the folder in there twice.  

No matter how many times I click the minus, it wont remove the second entry.   Also, nothing interesting that I can find shows up in the log even with debug on except for the http call..... I used SysInternals processmonitor to confirm that Emby Server is not encountering a file system issue.     

So.... screw it.  I right click the library and click delete.... I don't care anymore right?   Does nothing... nothing of value in the logs, but I let it go with embyserver.exe using a high percentage of CPU, like 30 to 70% and went bed, then to work, then come home and still nothing.

OK... so let's just move all the sub folders out of that folder and scan library with nothing in there.... lets FORCE emby to do something.    Emby pegged 8 core's at 100% so hard that the web interface was barely responsive and Windows Server 2016 was stuggling... this was on a Dell R720xd with E5-2680 v2's...   I let it run for a quite a while...... nothing.  

So I killed emby... restarted it and well... the library is now empty...  Still can't clear the paths or delete the empty library. 

 ok... maybe I have to add a valid folder to delete the other "two" folders of that are the same  sooo \\STORAGE1.<domain>.home\Media\MusicTest  it is... now the library is 3 folders that I can't remove. 

Try to delete a library with three folders (actually 2) with nothing in them...  CPU pegged again

About ready to just nuke the database from orbit... is there any direction you can point me in?   Right now when deleteing the DB, it just sits there with a spinning circle, perptually using 40% CPU across 8 cores of a Xeon...   I don't get it.

logs.zip

[lharris 3]

So it looks like after several hours, I can get an error message to pop up when trying to delete the library, but I am still unable to pick out of the log what is actually erroring.   I'll attach another log.  There are a lot of ffmpeg errors for individual media files in other libraries, but I think that's unrelated.

logs.zip

Edited August 20, 2024 by lharris

[Happy2Play 9781]

Do you have a database backup for say the 18th you can restore?  Actually I see you retain the last 5 so you have previous dbs to restore and work with. @lharris

Edited August 20, 2024 by Happy2Play

[lharris 3]

I gave that a try and even though I am selecting the backup from the 16th, it restores the files from embyserver-backup-full which are dated yesterday and the server doesn't go back to before I started messing with it.   I don't get it.   I'm starting to think Emby's database game is really bad...

I select this:  

 

 

This is what happens in the log... 

2024-08-20 22:06:44.045 Info Backup & Restore: OnAppHostDisposed
2024-08-20 22:06:44.045 Info Backup & Restore: SyncDirectories c:\EmbyBackup\embyserver-backup-full\data to C:\Users\emby\AppData\Roaming\Emby-Server\programdata\data. recursive: False
2024-08-20 22:06:44.046 Info Backup & Restore: TryCopyFile c:\EmbyBackup\embyserver-backup-full\data\authentication.db to C:\Users\emby\AppData\Roaming\Emby-Server\programdata\data\authentication.db
2024-08-20 22:06:44.049 Info Backup & Restore: TryCopyFile c:\EmbyBackup\embyserver-backup-full\data\library.db to C:\Users\emby\AppData\Roaming\Emby-Server\programdata\data\library.db
2024-08-20 22:06:44.589 Info Backup & Restore: TryCopyFile c:\EmbyBackup\embyserver-backup-full\data\users.db to C:\Users\emby\AppData\Roaming\Emby-Server\programdata\data\users.db
2024-08-20 22:06:44.590 Info Main: Shutdown complete
 

And the files are dated.... early this morning.

 

Instead of grabbing these:

 

I don't really understand how this could be.... 

I'm about to copy these files into the embyserver-backup-full path and see what happens.  

Edited August 21, 2024 by lharris

[lharris 3]

OK manually copying the files from C:\EmbyBackup\Emby Server Databases - 2024-08-16 00.10.24   over top of the files in C:\EmbyBackup\embyserver-backup-full\data  actually got the database to go back to before I messed with the music library.   I don't know if that is a bug or database corruption or what. Super disapointed.

I then added \\STORAGE1.domain.home\Media\Music and removed \\STORAGE1.domain.home\Music and after several minutes it seemed to work!!!!  Metadata is a wreck right now, but I assume that will fix itself.  I have no artists at the moment... for example. 

I have to say my confidence in the database is really shaken.  I want to vacuum or somehow rebuild the database, but honestly at this point now that Emby is not having a repeated stroke, I want to back up and then see if I can remove the library completely and start over with the music libary.   

 

Edited August 21, 2024 by lharris

[Happy2Play 9781]

Yes this appears to be something @Lukeneeds to look at as I just tried selective db restore and it restored the full-backup not the date selected just as you showed above.

 

 

2 minutes ago, lharris said:

OK manually copying the files from C:\EmbyBackup\Emby Server Databases - 2024-08-16 00.10.24   over top of the files in C:\EmbyBackup\embyserver-backup-full\data  actually got the database to go back to before I messed with the music library.   I don't know if that is a bug or database corruption or what. Super disapointed.

Are you good now?  But have no idea how you got duplicated folder paths added to your library.

[lharris 3]

Just updated post above as you commented.

[Happy2Play 9781]

3 minutes ago, lharris said:

Just updated post above as you commented.

Dev will have to comment if it will cause issues, but you should not be putting the same info in both Folder and Optional path files.  Should only be Folder in your case.

[Happy2Play 9781]

Depending on how your music is organized you may need to set Music Folder Structure.

[lharris 3]

Oh it's a dumpster fire at the moment so I have it set to unstructured and just letting Emby identify it as best as it can.  I recently installed a head unit in my car with android auto and have been really playing around with Emby on the head unit.   Wrangling/building a nicer music library is something I plan to tackle as fall/winter marches on...

7 minutes ago, Happy2Play said:

Dev will have to comment if it will cause issues, but you should not be putting the same info in both Folder and Optional path files.  Should only be Folder in your case.

Good to know, wasn't sure about that so I added it to see what happens.  Only the user Emby runs as has write access to that path (not sure about read only for everyone else, but I'll look)... so I guess a client that's not a windows client/user that has authenticated may fail to connect to that path.   I can see a client stumbling on it trying to hit it direct vs pulling it through emby.   I'll check that out, thanks!

Edited August 21, 2024 by lharris

[Luke 42079]

1 hour ago, Happy2Play said:

Yes this appears to be something @Lukeneeds to look at as I just tried selective db restore and it restored the full-backup not the date selected just as you showed above.

 

 

Are you good now?  But have no idea how you got duplicated folder paths added to your library.

There is no selective restore. The dropdown selection lets you pick which db files to use but all of the data files will still be restored as well.

[Luke 42079]

Regarding the wrong backup files being restored, we'll look into this. Thanks.