Reboot loses metadata and library entries

[JuJuJurassic 49]

As the UK's power costs have quadrupled over the last year, I'm turning my server off from 4am to 4 pm.

I've added "00 04 * * * /usr/sbin/shutdown -P now" to the crontab -e on the Emby server and it works fine. I have internal storage and 2 SMB servers offering shares. One is old, and stays on all the time. The other, films, is the new, power hungry, server. My plan is to move the old server to the new one in the longer term, eliminating a server.

Yesterday it worked fine.

Today I tested it and all the media that was internal or permantly on, had it's metadata, but the films, the server that turned off, seems to have lost most of it's metadata, even though the metadata is stored with the media folders, as in metadata.png attached. If I look at a folder from a film with it's images and metadata, it looks ok, as in metadata2.png, but if I look at a film with it's metadata missing from Emby, no images etc, it's folder looks like metadata3.png. But it looks like the metadata's there.

Initially I thought it was the problem I had previously, but the metadata is now stored with the media, here's the old problem

I've changed the timings so the emby turns off first, then 5 minutes later the films smb server, and on poweron the new SMB server powers up, but I put a 5 monite wait in grub before the Emby server completes it's boot.

Unfortunatley with the electricity prices as high as they are, I need to implement this, but has nay one any ideas why my metadata went, and how I can stop it happening again?

Also is there a way of making Emby realise the metadata's already there, it's now pulling it down again.

I'm running Emby on unbuntu 20.04.

Thanks

embyserver (11).txt

Edited January 18, 2023 by JuJuJurassic
reference old post

[Luke 42083]

Hi, why do you think it's lost the metadata?

[JuJuJurassic 49]

Hi Luke, I don't know, I was hoping you could help me.

I did wonder is the emby server came up before the films server and decided there was no metadata, but there would also haave been o movies either, I'm stuck

I did notice these errors in the log file, just before it decided to re read the metadata

------------

2023-01-18 08:32:44.966 Error FileSystem: Error determining CreationTimeUtc for /mnt/films/Suddenly, Last Summer (1959)/Suddenly, Last Summer (1959) BR-DISK.iso
    *** Error Report ***
    Version: 4.7.11.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
    Operating system: Linux version 5.4.0-137-generic (buildd@lcy02-amd64-009) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 20
    Framework: .NET 6.0.8
    OS/Process: x64/x64
    Runtime: opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 16
    Data path: /var/lib/emby
    Application path: /opt/emby-server/system
    System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Valid values are between -62135596800 and 253402300799, inclusive. (Parameter 'seconds')
       at System.DateTimeOffset.FromUnixTimeSeconds(Int64 seconds)
       at System.IO.FileStatus.GetCreationTime(ReadOnlySpan`1 path, Boolean continueOnError)
       at Emby.Server.Implementations.IO.ManagedFileSystem.GetCreationTimeUtc(FileSystemInfo info)
    Source: System.Private.CoreLib
    TargetSite: System.DateTimeOffset FromUnixTimeSeconds(Int64)
    
2023-01-18 08:32:44.967 Error FileSystem: Error determining LastAccessTimeUtc for /mnt/films/Suddenly, Last Summer (1959)/Suddenly, Last Summer (1959) BR-DISK.iso
    *** Error Report ***
    Version: 4.7.11.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
    Operating system: Linux version 5.4.0-137-generic (buildd@lcy02-amd64-009) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 20
    Framework: .NET 6.0.8
    OS/Process: x64/x64
    Runtime: opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 16
    Data path: /var/lib/emby
    Application path: /opt/emby-server/system
    System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Valid values are between -62135596800 and 253402300799, inclusive. (Parameter 'seconds')
       at System.DateTimeOffset.FromUnixTimeSeconds(Int64 seconds)
       at Emby.Server.Implementations.IO.ManagedFileSystem.GetLastWriteTimeUtc(FileSystemInfo info)
    Source: System.Private.CoreLib
    TargetSite: System.DateTimeOffset FromUnixTimeSeconds(Int64)
    
2023-01-18 08:32:47.322 Error FileSystem: Error determining CreationTimeUtc for /mnt/films/We Summon the Darkness (2019) []/We Summon the Darkness (2019) [Bluray-480p].mkv
    *** Error Report ***
    Version: 4.7.11.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
    Operating system: Linux version 5.4.0-137-generic (buildd@lcy02-amd64-009) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 20
    Framework: .NET 6.0.8
    OS/Process: x64/x64
    Runtime: opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 16
    Data path: /var/lib/emby
    Application path: /opt/emby-server/system
    System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Valid values are between -62135596800 and 253402300799, inclusive. (Parameter 'seconds')
       at System.DateTimeOffset.FromUnixTimeSeconds(Int64 seconds)
       at System.IO.FileStatus.GetCreationTime(ReadOnlySpan`1 path, Boolean continueOnError)
       at Emby.Server.Implementations.IO.ManagedFileSystem.GetCreationTimeUtc(FileSystemInfo info)
    Source: System.Private.CoreLib
    TargetSite: System.DateTimeOffset FromUnixTimeSeconds(Int64)
    
2023-01-18 08:32:47.323 Error FileSystem: Error determining LastAccessTimeUtc for /mnt/films/We Summon the Darkness (2019) []/We Summon the Darkness (2019) [Bluray-480p].mkv
    *** Error Report ***
    Version: 4.7.11.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
    Operating system: Linux version 5.4.0-137-generic (buildd@lcy02-amd64-009) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 20
    Framework: .NET 6.0.8
    OS/Process: x64/x64
    Runtime: opt/emby-server/system/System.Private.CoreLib.dll
    Processor count: 16
    Data path: /var/lib/emby
    Application path: /opt/emby-server/system
    System.ArgumentOutOfRangeException: System.ArgumentOutOfRangeException: Valid values are between -62135596800 and 253402300799, inclusive. (Parameter 'seconds')
       at System.DateTimeOffset.FromUnixTimeSeconds(Int64 seconds)
       at Emby.Server.Implementations.IO.ManagedFileSystem.GetLastWriteTimeUtc(FileSystemInfo info)
    Source: System.Private.CoreLib
    TargetSite: System.DateTimeOffset FromUnixTimeSeconds(Int64)

---------

but I'm not sure if they have anything to do with it. Could it be the size / number of files, 5500 videos. plus metadata. 

What makes it decide there's no metadata on an entry, or is it thinking the metadata's old / corrupt?

Any suggestions?

thanks

 

[JuJuJurassic 49]

I just checked the emby log files, it's deleting the metadata in /var/ folder

2023-01-18 20:31:12.413 Info App: Removing item from database, Type: Folder, Name: Georgy Jamieson, Path: , Id: 5649373

2023-01-18 20:31:12.427 Info App: Deleting path /var/lib/emby/metadata/library/f8/f8a06d19e37747da8fbd539516b13d99

2023-01-18 20:31:12.427 Info App: Deleting path /var/lib/emby/metadata/library/02/02b3c4aafe2d4b22adf69f5e6d05e380

 

 

What's going on?

embyserver (12).txt

[JuJuJurassic 49]

I've looked at the emby server today, it was left on last night, It's finished the metadata update on the films, but seems to still be deleting the metadata from somewhere else. Any suggestions why it deleted it in the first place?

It's also producing errors in the log, any idea what the errors are below? Are they relevant?

Thanks

--

*** Error Report ***
Version: 4.7.11.0
Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffdetect /opt/emby-server/bin/ffdetect -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-deb_{version}_amd64.deb
Operating system: Linux version 5.4.0-137-generic (buildd@lcy02-amd64-009) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #154-Ubuntu SMP Thu Jan 5 17:03:22 UTC 20
Framework: .NET 6.0.8
OS/Process: x64/x64
Runtime: opt/emby-server/system/System.Private.CoreLib.dll
Processor count: 16
Data path: /var/lib/emby
Application path: /opt/emby-server/system
System.NullReferenceException: System.NullReferenceException: Object reference not set to an instance of an object.
at Emby.Server.Implementations.Dto.DtoService.AttachPeople(BaseItemDto dto, BaseItem item, DtoOptions options)
at Emby.Server.Implementations.Dto.DtoService.GetBaseItemDtoInternal(BaseItem item, DtoOptions options, User user)
at Emby.Server.Implementations.Dto.DtoService.GetBaseItemDto(BaseItem item, DtoOptions options, User user)
at Emby.Api.UserLibrary.UserLibraryService.Get(GetItem request)
at Emby.Server.Implementations.Services.ServiceController.GetTaskResult(Task task)
 

--

[JuJuJurassic 49]

I changed the timings of the servers. The dell was taking much longer to boot than I thougth, so the emby server was up, when it's filestore wasn't. For som eason it then decided to delete all the metadata. I've tried it a few times making sure the library's there and all seems ok, but plex would have jusrt said the file wasn't there. Ah the joys, well at least I think it's fixed

 

[fabzo 1]

I'm wondering the same as well. Is there any situation in which Emby is actively deleting media entries? Two days ago I experienced this for the second time (a few month after the first). At some point entries in my media library were just gone. The first time this happened Emby was basically empty. The second time now only the TV shows were gone while the movies were still present. Each time I had to "Scan library files" again on the affected libraries. The only thing that seems slightly relevant in the logs, which I could find, is over 9k of "Creating virtual missing episode X" entries over the course of a few hours. 

The annoying part here is that even after rescanning the library I need to manually remove series and add them again in the correct order because Emby is associating some episodes with the wrong series and there is no built in feature to move episodes / files from one series to another.

If this is indeed the file server shortly not being available, is there a way of telling Emby to never remove entries from a library without starting a manual cleanup operation of sorts? As JuJuJurassic said ""plex would have jusrt said the file wasn't there.". That would be my expectation as well. Having to rescan the library is one thing, but having to manually reassociate the media again is really unfortunate

Edited March 20, 2023 by fabzo

[Happy2Play 9783]

There are some changes coming in 4.8 for this offline/unavailable libraries mentioned here.

 

[fabzo 1]

Thanks @Happy2Play. That does seem to be the exact same issue I'm seeing on my side and described in this thread here. Linux mounts indeed present an empty directory if the mount/remote storage isn't available. Thanks for pointing me to that. Looking forward to 4.8 then  

Edited March 21, 2023 by fabzo