emby Scan media library Stuck 40.8%

[zysljd 3]

I had a very old media library that was filled with a lot of other content that caused the media library scanning to be very slow and would get stuck at 36%, I removed the entire media library today. I removed the whole library today, but the library progress is still stuck at 40.5%.
I see in the logs that it is still scanning the files that were removed from the library.
I'm pretty sure my other libraries don't contain this path.
Example.
A media library contains E:\video
B media library contains F:\video.
C Media Library contains G:\video
I removed the C media library, but the scanning of G:\video still appears in the log.
I tried restarting the emby server, but G:\video is still in the log.
I don't understand why, and how can I check where else the path to the deleted media library is contained?
Or is there any other way? What can I do?

[zysljd 3]

emby server version:4.8.8

[zysljd 3]

I found some tracks through the api.
I found the removed paths in the Library/PhysicalPaths return
Through the api's note "Gets a list of physical paths from virtual folders"
I found that the api "VirtualFolders/Query" returned no paths to the removed libraries.

[zysljd 3]

I found library.db, and in the MediaItems table and  fts_search9_content table there,

MediaItems table found the corresponding path, and the record shows type=3.path= folder path of the removed media library.

fts_search9_content found path name ,but not have disk drive number (A:\\ or B:\\)
I'm not entirely sure if this is the record that is causing the problem, but I've searched all the tables and this is the only one that matches.
Even if I am sure, I'm not sure how to proceed, so how do I identify the problem and solve it?

Edited August 13, 2024 by zysljd

[Happy2Play 9780]

How many libraries have setup and what paths did you uses?

 

[zysljd 3]

7 minutes ago, Happy2Play said:

I had 5 libraries and deleted 1. What is the path? The previously configured path is M:\movie.
The MediaItems table is M:\\movie.
I analyzed the records in the MediaItems table.
The parentid of the records that should be deleted points to root.
and name is also the name of the library I deleted. I would like to know how I can safely delete the remaining records.

 

 

[zysljd 3]

2 minutes ago, zysljd said:

 

The path field in the MediaItems table is M:\movie.

[Happy2Play 9780]

4 minutes ago, zysljd said:

I had 5 libraries and deleted 1. What is the path? The previously configured path is M:\movie.
The MediaItems table is M:\\movie.
I analyzed the records in the MediaItems table.
The parentid of the records that should be deleted points to root.
and name is also the name of the library I deleted. I would like to know how I can safely delete the remaining records.

Possibly but if they are not cleared with library removal I can only guess they are attached/link to another library.  Were there errors in the log when removing the library?  This is why specific information is usually required. 

[zysljd 3]

3 minutes ago, Happy2Play said:

Possibly but if they are not cleared with library removal I can only guess they are attached/link to another library.  Were there errors in the log when removing the library?  This is why specific information is usually required. 

And I'm pretty sure it's not in any of the other libraries.
How do I find the records? They're huge. They could be hundreds M.

[Happy2Play 9780]

3 minutes ago, zysljd said:

And I'm pretty sure it's not in any of the other libraries.
How do I find the records? They're huge. They could be hundreds M.

The How to Report a Problem tells you the log location.  Or go to App Settings-Logs, click on log and download.

[zysljd 3]

9 minutes ago, Happy2Play said:

Possibly but if they are not cleared with library removal I can only guess they are attached/link to another library.  Were there errors in the log when removing the library?  This is why specific information is usually required. 

I queried /Library/VirtualFolders/Query via api
The path I removed was not used in the results.
If it is a system deletion there is a bug, what should I do now?

[Neminem 1519]

server Logs, logs logs.

[zysljd 3]

7 minutes ago, Happy2Play said:

The How to Report a Problem tells you the log location.  Or go to App Settings-Logs, click on log and download.

The deletion happened a few days ago, how can I find the log at that time, the overall log size exceeds the forum limit.

[Neminem 1519]

That might be, but without logs its all guess work.

[zysljd 3]

3 minutes ago, JayceDK said:

That might be, but without logs its all guess work.

My logs for the last few days add up to 800MB.

When deleting libraries, is there any special text that is recorded?

So that I can find the logs at that time.

Edited August 13, 2024 by zysljd

[Happy2Play 9780]

Look for the same api endpoint calls but shouldn't be that hard to miss removal/deletion of library.

But yes a log that size will be an issue but all we can do is guess.

[zysljd 3]

2 minutes ago, Happy2Play said:

Look for the same api endpoint calls but shouldn't be that hard to miss removal/deletion of library.

But yes a log that size will be an issue but all we can do is guess.

I found the log that reported the error.

It happened 4 times.

This is the same as the number that remain in the data table

2024-08-13 01:48:54.425 Info App: Deleting path E:\embyserver\programdata\metadata\library\7e\7e0659c6f0d5b797118d897bd977be3f
2024-08-13 01:48:54.426 Error SqliteItemRepository: Error saving items
    *** Error Report ***
    Version: 4.8.8.0
    Command line: E:\embyserver\system\EmbyServer.dll
    Operating system: Microsoft Windows 10.0.17763
    Framework: .NET 6.0.30
    OS/Process: x64/x64
    Runtime: E:/embyserver/system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: E:\embyserver\programdata
    Application path: E:\embyserver\system
    SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at SQLitePCL.pretty.DatabaseConnection.Execute(IDatabaseConnection This, String sql)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItemsInTranscation(IDatabaseConnection db, List`1 tuples, MetadataRefreshOptions metadataRefreshOptions)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItems(List`1 items, MetadataRefreshOptions metadataRefreshOptions, Action`1 afterSave, Boolean disableForeignKeys, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCLEx.sqlite3, Int32)

2024-08-13 02:57:56.762 Error SqliteItemRepository: Error saving items
    *** Error Report ***
    Version: 4.8.8.0
    Command line: E:\embyserver\system\EmbyServer.dll
    Operating system: Microsoft Windows 10.0.17763
    Framework: .NET 6.0.30
    OS/Process: x64/x64
    Runtime: E:/embyserver/system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: E:\embyserver\programdata
    Application path: E:\embyserver\system
    SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at SQLitePCL.pretty.DatabaseConnection.Execute(IDatabaseConnection This, String sql)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItemsInTranscation(IDatabaseConnection db, List`1 tuples, MetadataRefreshOptions metadataRefreshOptions)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItems(List`1 items, MetadataRefreshOptions metadataRefreshOptions, Action`1 afterSave, Boolean disableForeignKeys, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCLEx.sqlite3, Int32)

2024-08-13 02:57:56.764 Error SqliteItemRepository: Error saving items
    *** Error Report ***
    Version: 4.8.8.0
    Command line: E:\embyserver\system\EmbyServer.dll
    Operating system: Microsoft Windows 10.0.17763
    Framework: .NET 6.0.30
    OS/Process: x64/x64
    Runtime: E:/embyserver/system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: E:\embyserver\programdata
    Application path: E:\embyserver\system
    SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at SQLitePCL.pretty.DatabaseConnection.Execute(IDatabaseConnection This, String sql)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItemsInTranscation(IDatabaseConnection db, List`1 tuples, MetadataRefreshOptions metadataRefreshOptions)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItems(List`1 items, MetadataRefreshOptions metadataRefreshOptions, Action`1 afterSave, Boolean disableForeignKeys, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCLEx.sqlite3, Int32)

2024-08-13 09:47:58.125 Error SqliteItemRepository: Error saving items
    *** Error Report ***
    Version: 4.8.8.0
    Command line: E:\embyserver\system\EmbyServer.dll
    Operating system: Microsoft Windows 10.0.17763
    Framework: .NET 6.0.30
    OS/Process: x64/x64
    Runtime: E:/embyserver/system/System.Private.CoreLib.dll
    Processor count: 8
    Data path: E:\embyserver\programdata
    Application path: E:\embyserver\system
    SQLitePCL.pretty.SQLiteException: Constraint: FOREIGN KEY constraint failed
    SQLitePCL.pretty.SQLiteException: Exception of type 'SQLitePCL.pretty.SQLiteException' was thrown.
       at SQLitePCL.pretty.SQLiteException.CheckOk(sqlite3 db, Int32 rc)
       at SQLitePCL.pretty.StatementImpl.MoveNext()
       at SQLitePCL.pretty.DatabaseConnection.Execute(IDatabaseConnection This, String sql)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItemsInTranscation(IDatabaseConnection db, List`1 tuples, MetadataRefreshOptions metadataRefreshOptions)
       at Emby.Server.Implementations.Data.SqliteItemRepository.SaveItems(List`1 items, MetadataRefreshOptions metadataRefreshOptions, Action`1 afterSave, Boolean disableForeignKeys, CancellationToken cancellationToken)
    Source: SQLitePCL.pretty
    TargetSite: Void CheckOk(SQLitePCLEx.sqlite3, Int32)

[Happy2Play 9780]

That suggests you possibly have overlapping libraries.  Do you have libraries that have a child path or parent library?  If so may require the removal of them and hope the db corrects the issue otherwise you will have to create a new db or possibly manually correct the issue in the database.

 

 

[ebr 16184]

Hi.  Have you modified the database using a tool outside of Emby?

[Happy2Play 9780]

But could be just the removal of large amounts of data if it only happened 4 times but all we can do is guess.

[zysljd 3]

4 minutes ago, ebr said:

Hi.  Have you modified the database using a tool outside of Emby?

I haven't modified the database, and I've only just queried the database for records.

[zysljd 3]

5 minutes ago, Happy2Play said:

But could be just the removal of large amounts of data if it only happened 4 times but all we can do is guess.

Yes, it is. Tens of thousands of records were deleted.

 

"If so may require the removal of them and hope the db corrects the issue otherwise you will have to create a new db or possibly manually correct the issue in the database."

 

How to fix it manually,Is it possible to remove the corresponding id via api?

 

 

[Happy2Play 9780]

2 minutes ago, zysljd said:

How to fix it manually,Is it possible to remove the corresponding id via api?

Really depends but am not a db person just tinker around.  But you end of removing the item from each table then from mediaitems.

I now there must be a correct/better way but I do something like this as you have to ensure the all the linking of an item is removed.

DELETE FROM AncestorIds2 where ItemId = 
DELETE FROM Chapters3 where ItemId = 
DELETE FROM ItemPeople2 where ItemId = 
DELETE FROM ItemLinks2 where ItemId = 
DELETE FROM MediaStreams2 where ItemId = 
DELETE FROM UserDataKeys2 where Id = 
DELETE FROM mediaitems where Id = 

 

[zysljd 3]

Just now, Happy2Play said:

Really depends but am not a db person just tinker around.  But you end of removing the item from each table then from mediaitems.

I now there must be a correct/better way but I do something like this as you have to ensure the all the linking of an item is removed.

DELETE FROM AncestorIds2 where ItemId = 
DELETE FROM Chapters3 where ItemId = 
DELETE FROM ItemPeople2 where ItemId = 
DELETE FROM ItemLinks2 where ItemId = 
DELETE FROM MediaStreams2 where ItemId = 
DELETE FROM UserDataKeys2 where Id = 
DELETE FROM mediaitems where Id = 

 

Okay, I'll try it.
If I recreate the database, does that mean I need to re-scrape all the media?

[Happy2Play 9780]

11 minutes ago, zysljd said:

If I recreate the database, does that mean I need to re-scrape all the media?

Depends on if you save with media or not.

Or if you are Premiere users have you configured Backup and Restore as you would have a db backup.  Otherwise you will want to ensure you keep backups of a minimum of the databases manually.