Jump to content


Photo

The "database disk image is malformed" error has me stymied

docker

  • Please log in to reply
9 replies to this topic

#1 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 20 September 2019 - 11:22 PM

I'm running Emby via Docker on a Ubuntu 18.04 host. After I had issues with getting some new episodes detected, I started troubleshooting and eventually found that error.
 

2019-09-20 04:38:42.578 Error HttpServer: Error processing request
    *** Error Report ***
    Version: 4.2.1.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Unix 5.0.0.27
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Runtime: file:///system/System.Private.CoreLib.dll
    Processor count: 32
    Program data path: /config
    Application directory: /system
    SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at Emby.Sqlite.SqliteExtensions.ExecuteQuery(IStatement This)+MoveNext()
       at Emby.Server.Implementations.Data.SqliteItemRepository.QueryItemsInTransaction[T](InternalItemsQuery query, IDatabaseConnection db, String[] columnsToSelect, Func`5 rowReaderFn, Boolean singleResult, String methodName)
       at Emby.Server.Implementations.Data.SqliteItemRepository.GetItemList(InternalItemsQuery query)
       at Emby.Server.Implementations.Library.LibraryManager.GetItemsResult(InternalItemsQuery query)
       at MediaBrowser.Controller.Entities.Folder.GetItems(InternalItemsQuery query)
       at MediaBrowser.Api.UserLibrary.ItemsService.GetQueryResult(GetItems request, DtoOptions dtoOptions, User user)
       at MediaBrowser.Api.UserLibrary.ItemsService.GetItems(GetItems request)
       at MediaBrowser.Api.UserLibrary.ItemsService.Get(GetItems request)
       at Emby.Server.Implementations.Services.ServiceController.Execute(HttpListenerHost appHost, Object requestDto, IRequest req)
       at Emby.Server.Implementations.Services.ServiceHandler.ProcessRequestAsync(HttpListenerHost appHost, IRequest httpReq, IResponse httpRes, RestPath restPath, String responseContentType, CancellationToken cancellationToken)
       at Emby.Server.Implementations.HttpServer.HttpListenerHost.RequestHandler(IRequest httpReq, ReadOnlyMemory`1 urlString, ReadOnlyMemory`1 host, ReadOnlyMemory`1 localPath, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCL.sqlite3, Int32)

All the topics I found say to move/delete the *.db files. Usually library.db.

That has not fixed it.

I have tried moving the following files in config/data to new names:

  •     activitylog.db
  •     authentication.db
  •     displaypreferences.db
  •     library.db
  •     users.db

I first tried just the library.db file, but the library scan failed. After that I just moved all of them. The library scan failed again. The dashboard had this error:

Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
at SQLitePCL.pretty.StatementImpl.MoveNext()
at Emby.Sqlite.SqliteExtensions.ExecuteQuery(IStatement This)+MoveNext()
at Emby.Server.Implementations.Data.SqliteItemRepository.QueryItemsInTransaction[T](InternalItemsQuery query, IDatabaseConnection db, String[] columnsToSelect, Func`5 rowReaderFn, Boolean singleResult, String methodName)
at Emby.Server.Implementations.Data.SqliteItemRepository.GetItemList(InternalItemsQuery query)
at Emby.Server.Implementations.Library.LibraryManager.GetItemsResult(InternalItemsQuery query)
at MediaBrowser.Controller.Entities.Folder.GetItems(InternalItemsQuery query)
at MediaBrowser.Controller.Entities.Folder.GetChildren(InternalItemsQuery query)
at Emby.Server.Implementations.IO.LibraryMonitor.Start()
at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

Right now the thumbnail extraction job is running. I'm tailing the log file and I'm seeing a lot of logs like this:

2019-09-21 02:30:00.244 Error ChapterImagesTask: Error refreshing thumbnails for /mnt/tv/The Avengers/Season 4/The Avengers S04E02 The Gravediggers.mp4
    *** Error Report ***
    Version: 4.2.1.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Unix 5.0.0.27
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Runtime: file:///system/System.Private.CoreLib.dll
    Processor count: 32
    Program data path: /config
    Application directory: /system
    SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at Emby.Sqlite.SqliteExtensions.ExecuteQuery(IStatement This)+MoveNext()
       at Emby.Server.Implementations.Data.SqliteItemRepository.GetChapters(BaseItem item)
       at MediaBrowser.Providers.MediaInfo.ChapterImagesTask.Execute(CancellationToken cancellationToken, IProgress`1 progress)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCL.sqlite3, Int32)

I am running via docker-compose and systemd. The compose file is using the straight emby/embyserver:latest image, and I have my data mounted via Docker named volumes and nfs.

I think the db corruption might be due to my .service file not shutting down the Emby server before it kills the container. What is the command line command to run for that?

Is there a way to tell which .db file is corrupted? The error messages do not leave any clues.

Any ideas how I can get my Emby server back up and running? Right now when I try to browse, all I get is the loading circle animation. The log file pops up this kind of message:

2019-09-21 03:09:20.625 Info HttpServer: HTTP Response 500 to 192.168.10.3. Time: 6ms. http://hostname:8096/Users/784fca4761674832b51895c835a93714/Items?SortBy=SortName&SortOrder=Ascending&IncludeItemTypes=Movie&Recursive=true&Fields=BasicSyncInfo%2CMediaSourceCount%2CSortName%2CPrimaryImageAspectRatio%2CProductionYear&ImageTypeLimit=1&EnableImageTypes=Primary%2CBackdrop%2CThumb&StartIndex=0&Limit=100&ParentId=abebc196cc1b8bbf6f8bb5ca7b5ad6f1

Well, I went ahead and restored the backed up .db files. So all my old content is accessible now. But scanning for new results in a malformed database image message.


Since it is different than what I've already shared, here it is.
 

2019-09-21 03:15:14.041 Error ProviderManager: Error refreshing item
    *** Error Report ***
    Version: 4.2.1.0
    Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Unix 5.0.0.27
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Runtime: file:///system/System.Private.CoreLib.dll
    Processor count: 32
    Program data path: /config
    Application directory: /system
    SQLitePCL.pretty.SQLiteException: Corrupt: database disk image is malformed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItem(IDatabaseConnection db, BaseItem item, Int64 topParentId, Int64 userDataKeyId, IStatement insertItemStatement, IStatement updateItemStatement)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItemsInTranscation(IDatabaseConnection db, List`1 tuples)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItems(List`1 items, CancellationToken cancellationToken)
       at Emby.Server.Implementations.Library.LibraryManager.UpdateItems(List`1 items, BaseItem parent, ItemUpdateType updateReason, CancellationToken cancellationToken)
       at Emby.Server.Implementations.Library.LibraryManager.UpdateItem(BaseItem item, BaseItem parent, ItemUpdateType updateReason)
       at MediaBrowser.Providers.Manager.MetadataService`2.SaveItem(MetadataResult`1 result, LibraryOptions libraryOptions, ItemUpdateType reason, CancellationToken cancellationToken)
       at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
       at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
       at MediaBrowser.Providers.Manager.ProviderManager.RefreshCollectionFolderChildren(MetadataRefreshOptions options, CollectionFolder collectionFolder, CancellationToken cancellationToken)
       at MediaBrowser.Providers.Manager.ProviderManager.RefreshItem(BaseItem item, MetadataRefreshOptions options, CancellationToken cancellationToken)
       at MediaBrowser.Providers.Manager.ProviderManager.StartProcessingRefreshQueue()
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCL.sqlite3, Int32)

So, how do I get scanning working again?

Thanks!
 



#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138093 posts
  • Local time: 03:05 PM

Posted 21 September 2019 - 12:09 AM

Hi, that's strange. what kind of storage is this on? You may want to look into possible problems with that.



#3 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 21 September 2019 - 12:53 AM

Storage is a FreeNAS vm with HDD's passed directly through so it can do everything it needs to make ZFS work. Emby accesses the data via nfs. The docker volumes are defined something like:

volumes:
  embyconfig:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.122.214,nolock,soft,rw"
      device: ":/mnt/cedarchest/emby/config"
  embymovies:
    driver_opts:
      type: "nfs"
      o: "addr=192.168.122.214,nolock,soft,rw"
      device: ":/mnt/cedarchest/emby/movies"

The HDDs are less than a year old. Haven't checked their SMART output recently, but I believe FreeNAS has alerts for that...

 

Since FreeNAS and Emby are on the same host, there should be little network overhead. Um, virtualization is done via virt-manager/qemu.



#4 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138093 posts
  • Local time: 03:05 PM

Posted 21 September 2019 - 12:18 PM

Ok, deleting the library.db file is all you need to do. If it keeps happening then there is likely something causing the file to become corrupted.

#5 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 21 September 2019 - 02:36 PM

Ok, deleting the library.db file is all you need to do. If it keeps happening then there is likely something causing the file to become corrupted.

So, that was the first thing I tried. I'll go ahead and try again.

 

Any idea what is causing the corruption?

 

It worked fine when I set up the docker install from scratch.

 

Also, is there a command line command I can add to my .service file to make sure systemd cleanly stops emby before shutting down the container?



#6 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 21 September 2019 - 08:15 PM

The second try deleting library.db worked. Go figure.

 

Thanks for the help!

 

Now to figure out how to make sure my service stops Emby properly....



#7 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138093 posts
  • Local time: 03:05 PM

Posted 21 September 2019 - 10:30 PM

Thanks for the feedback.

 

 

 

Now to figure out how to make sure my service stops Emby properly....

How is it doing it?



#8 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 22 September 2019 - 12:43 AM

Here is my .service file.

[Unit]
Description=emby.tipperthecat.life
After=docker.service
Requires=docker.service

[Service]
ExecStartPre=/bin/bash -c 'while [ ! -d /home/username/shares/emby/config ]; do /usr/bin/sleep 5; done'
TimeoutStartSec=300
ExecStart=/usr/local/bin/docker-compose -f /srv/emby/docker-compose.yml up
ExecStop=/usr/local/bin/docker-compose -f /srv/emby/docker-compose.yml down -v
Restart=on-failure

[Install]
WantedBy=default.target

The ExecStartPre line makes sure the nfs server is up and running before starting Emby.

 

Earlier I ran a temp container to see if EmbyServer --help had anything to say. It didn't but I did notice that when it does this:

/ # exit
[cmd] sh exited 0
s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening
[cont-finish.d] executing container finish scripts...
[cont-finish.d] done.
[s6-finish] waiting for services.
[s6-finish] sending all processes the TERM signal.
[s6-finish] sending all processes the KILL signal and exiting.

So, does the container know how to shut down cleanly without my help?

 

I have had my desktop powered off by a circuit breaker tripping, so that might have been when the initial corruption occurred.


 



#9 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 138093 posts
  • Local time: 03:05 PM

Posted 22 September 2019 - 10:41 PM

 

 

So, does the container know how to shut down cleanly without my help?

Yes it does.

 

 

 

I have had my desktop powered off by a circuit breaker tripping, so that might have been when the initial corruption occurred.

Yup, that's a likely culprit.



#10 jerrac OFFLINE  

jerrac

    Newbie

  • Members
  • 7 posts

Posted 23 September 2019 - 04:18 PM

Thanks. :)

 

Guess I'll move finding a good UPS up my todo list....







Also tagged with one or more of these keywords: docker

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users