Libraries missing - database is locked

[Oldjuju 4]

Hey guys,

hoping for some assistance.
I'm running Emby in a Docker container on a QNAP. Emby is the latest version.

A week ago I found half of my library missing. The media was still present on the system, just not showing within Emby.
I upgraded to the latest Emby docker image and the issue persisted.
Performing a full library scan looked like it picked up all of the media.

I later found that half of the movies don't show any media - ie, the movie shows up in Emby with background images, but it shows as an empty folder.
Various methods have been tried.
 - touching the files
 - moving the folder to a different location on the disk, scanning library and moving folder back
The only thing that has worked is having Radarr regrab the movie.

Further investigation shows the following error in the server logs:
SQLitePCL.pretty.SQLiteException: Busy: database is locked

I came across a couple of articles in the forums about moving the library.db and letting Emby recreate it. But neither of these has helped.
https://emby.media/community/index.php?/topic/67962-clean-up-and-rebuild-without-completely-starting-over/
https://emby.media/community/index.php?/topic/44238-database-help/

My library.db is 166M.
Since moving the db file and then moving it back, there is now nothing listed in the "LIbrary" section within Emby admin.

I've installed sqlitebrowser and can see the data.

I could just readd the libraries and rescan everything.
But, I have put a lot of time into the music organisation and am worried that I will need to restart that.
I also want to retain my tv show watch history. I don't really care about the movies watch history.

I'm on Emby Premiere, but couldn't find the backup plugin mentioned in one of the posts.

any assistance would be greatly appreciated.

Cam

[Happy2Play 9780]

11 minutes ago, Oldjuju said:

Further investigation shows the following error in the server logs:
SQLitePCL.pretty.SQLiteException: Busy: database is locked

See the Locked database section.

Corrupt Database : Emby

11 minutes ago, Oldjuju said:

came across a couple of articles in the forums about moving the library.db and letting Emby recreate it. But neither of these has helped.

But can not see how replacing library.db did not resolve your issue.

Edited May 27, 2021 by Happy2Play

[Oldjuju 4]

Thanks for the super quick reply and the link.
Deleting the library.db.shm and library.db-wal has returned the libraries now. That's a big recovery - huge thanks there!

I still have the initial problem though.
Half of my movies show as an empty folder within Emby even though the media is on the disk and the paths are correct.
Simply re-downloading them fixes the issue, but that's a bit extreme.

Any ideas about this issue? Does this sound like a corrupted database?
I will run the integrity check across the database. The integrity of the database is "ok".

Edited May 28, 2021 by Oldjuju

[Happy2Play 9780]

39 minutes ago, Oldjuju said:

I still have the initial problem though.
Half of my movies show as an empty folder within Emby even though the media is on the disk and the paths are correct.
Simply re-downloading them fixes the issue, but that's a bit extreme.

Any ideas about this issue? 

Has a Library scan finished?  Can you post a log after a library scan.

[Oldjuju 4]

Hmm, looks like I've now got a read only database.

2021-05-28 00:55:41.028 Error Server: Error processing request
    *** Error Report ***
    Version: 4.6.1.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Linux version 4.14.24-qnap (root@U16BuildServer44) (gcc version 4.9.2 (toolchain config: [gcc-4.9.2 binutils-2.25 glibc-2.21])) #1 SMP Sat May 15 17:3
    Framework: .NET Core 3.1.13
    OS/Process: x64/x64
    Runtime: system/System.Private.CoreLib.dll
    Processor count: 4
    Data path: /config
    Application path: /system
    SQLitePCL.pretty.SQLiteException: ReadOnly: attempt to write a readonly database
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at Emby.Server.Implementations.Data.SqliteUserDataRepository.SaveUserData(Int64 internalUserId, Int64 itemId, UserItemData userData, CancellationToken cancellationToken)
       at Emby.Server.Implementations.Library.UserDataManager.SaveUserData(User user, BaseItem item, UserItemData userData, UserDataSaveReason reason, CancellationToken cancellationToken)
       at Emby.Server.Implementations.Session.SessionManager.OnPlaybackProgress(User user, BaseItem item, PlaybackProgressInfo info, SessionInfo sessionInfo)
       at Emby.Server.Implementations.Session.SessionManager.OnPlaybackProgress(PlaybackProgressInfo info, Boolean isAutomated)
       at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCLEx.sqlite3, Int32)

[Happy2Play 9780]

3 minutes ago, Oldjuju said:

SQLitePCL.pretty.SQLiteException: ReadOnly: attempt to write a readonly database

This would be a new one as I have not seen this one before. @Luke@cayars

 

[Oldjuju 4]

Ok, permissions on the library.db file must have mucked up with my moving it around.
The read only database issue is now fixed.

Library scan on my TV library and Music library are working fine and shows the percentage circle while it's working.

The movies library flashes when I start a scan, but barely shows the percentage circle. It appears to finish immediately.
If I move a folder from the library path, it is removed from the library, and appears as an empty folder when moved back in.

Attached is a embyserver.txt

embyserver (2).txt

[Luke 42077]

Hi, please make sure to run a library scan after moving it back in.

[Oldjuju 4]

Hi Luke, thanks for the reply.

I ran a library scan when I moved the folder out and then again when I moved the folder back in.
Same result. The movie just shows as an empty folder with the background image of the movie.

Are the logs suggesting that I didn't run another scan after moving the folder back into the library path?

Edited May 28, 2021 by Oldjuju

[Oldjuju 4]

Just an update. 

I found while interrogating the DB that the missing movies had a record but were missing data, including the filename. 
I tried deleting a row but couldn't due to Foreign Key contraints.

Laziness took hold and I thought I'd just remove the entire "Movies" library and re-add it.
This worked a treat and kept all of my watch history.

Happy days. Thanks for the suggestions.

[Happy2Play 9780]

5 minutes ago, Oldjuju said:

Just an update. 

I found while interrogating the DB that the missing movies had a record but were missing data, including the filename. 
I tried deleting a row but couldn't due to Foreign Key contraints.

Laziness took hold and I thought I'd just remove the entire "Movies" library and re-add it.
This worked a treat and kept all of my watch history.

Happy days. Thanks for the suggestions.

Yes userdata is in a separate table and is tracked by providerid when available.  Only items without providerids will lose watched status.

[Luke 42077]

Thanks for the feedback. Apologies for the disruption.

[Oldjuju 4]

No stress....well, a bit of stress. But all good.

[Luke 42077]

Thanks for the feedback.

[tonyvilla 0]

estoy en la mism situacion no tengo acceso a escanear las librerias

[Happy2Play 9780]

4 minutes ago, tonyvilla said:

I'm in the same situation I don't have access to scan the libraries

Replied to your topic