Jump to content

Libraries vanish / empty themselves when network path is not directly accessible at startup


NicoLaz

Recommended Posts

Hi all

Half a year ago, I moved my emby instance from my NAS to a seperate machine (Win10) to get better performance. However, as I'm using emby as a DVR and my library got quite ... ähm ... "comprehensive" by now, I also upgraded my NAS and seperated media server and data storage. I'm using SMP paths in my libraries to add the shared folders to emby (location = \\SERVER\FOLDER1). I had a power outage today and (re-)noticed an odd behaviour of emby: as my media server is a lot faster in booting up as my NAS, accessing the library failed when emby service started. As a result, some of my libraries where just gone from the admin interface and in the remaining ones, nearly all the entries were missing. Restoring my nightly backup did fail as well (I experienced the already known bug with multiple entries after restore) ending up with me deleting all libraries and setting them up again.

Yes, one could think that the settings were damaged by the shutdown, but I already experienced the same behaviour when I shut down the NAS and media server in parallel a few month ago.

Did you experience something similar in the past? If so, it seems that emby doesn't destinguish between "non accessible" and "non existent" when accessing libraries. It might then be a good idea to catch that error and allow some access retries or even "halt" / "pause" a library that can't be accessed instead of just deleting the entries or even the whole library. If not, it might just be my setup and I would appreciate any tips on how to change it to avoid a repetition of that...

Cheers!

Link to comment
Share on other sites

PenkethBoy

too late now but i would invest in a UPS or two to run the pc and nas should you get an outage - at least you can shut down the machines gracefully (automatically) - and let them restart or not

then its a choice if you run as a service or not and let emby auto start or not - i dont - so i can control when things are back online etc

advice after the event is alway great but does not help apart from going forward

I have not lost data due to a network drive going off line as my data is local - although have had the odd HDD disappear on a reboot - but comes back after a power cycle - windows quirks - but as i dont have emby auto start (you could set a delay of say 10-20-30 mins as necessary to cover the NAS taking longer to appear etc)

Lots of things could be done differently depending on need and will etc

Link to comment
Share on other sites

Hi, if you could attach the emby server log from when this happened we can look at what happened. The library should not disappear. Thanks.

Link to comment
Share on other sites

Hi @Luke

Thank you for your fast reply! I wanted to prepare the logs this morning but noticed that they already went into rotation and got deleted :( I didn't realize the timespan was so short, sorry... The only thing I still habe is those two warning messages in the message history - they belong to the two libraries I was missing:

Could not find a part of the path 'C:\Users\cube\AppData\Roaming\Emby-Server\programdata\root\default\Recorded Movies11'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
at System.IO.Enumeration.FileSystemEnumerableFactory.FileSystemInfos(String directory, String expression, EnumerationOptions options, Boolean isNormalized)
at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileSystemEntries(String path, Boolean recursive)
at MediaBrowser.Controller.Providers.DirectoryService.GetFileSystemEntries(String path)
at MediaBrowser.Controller.IO.FileData.GetFilteredFileSystemEntries(IDirectoryService directoryService, String path, IFileSystem fileSystem, IServerApplicationHost appHost, ILogger logger, ItemResolveArgs args, Int32 flattenFolderDepth, Boolean resolveShortcuts)
at MediaBrowser.Controller.Entities.CollectionFolder.CreateResolveArgs(IDirectoryService directoryService)
at MediaBrowser.Controller.Entities.CollectionFolder.FolderIdsRequireUpdating(Boolean updateValues)
at MediaBrowser.Controller.Entities.CollectionFolder.BeforeMetadataRefresh(Boolean replaceAllMetdata)
at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateTopLibraryFolders(Boolean refreshMetadatas, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.PerformLibraryValidation(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)
Could not find a part of the path 'C:\Users\cube\AppData\Roaming\Emby-Server\programdata\root\default\Recordings1'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
at System.IO.Enumeration.FileSystemEnumerableFactory.FileSystemInfos(String directory, String expression, EnumerationOptions options, Boolean isNormalized)
at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileSystemEntries(String path, Boolean recursive)
at MediaBrowser.Controller.Providers.DirectoryService.GetFileSystemEntries(String path)
at MediaBrowser.Controller.IO.FileData.GetFilteredFileSystemEntries(IDirectoryService directoryService, String path, IFileSystem fileSystem, IServerApplicationHost appHost, ILogger logger, ItemResolveArgs args, Int32 flattenFolderDepth, Boolean resolveShortcuts)
at MediaBrowser.Controller.Entities.CollectionFolder.CreateResolveArgs(IDirectoryService directoryService)
at MediaBrowser.Controller.Entities.CollectionFolder.FolderIdsRequireUpdating(Boolean updateValues)
at MediaBrowser.Controller.Entities.CollectionFolder.RequiresRefresh()
at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateTopLibraryFolders(Boolean refreshMetadatas, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.PerformLibraryValidation(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

@PenkethBoy: Thank you, I appreciate your hint. A USP is already on my "should do" list but the last power outage here was 4 years ago and for an 8 bay NAS + media server with an i5 I will probably need a rather large one, so I never really put that plan into reality. But perhaps I should re-prioritize.

Cheers

NicoLaz

Link to comment
Share on other sites

Gilgamesh_48
28 minutes ago, NicoLaz said:

A USP is already on my "should do" list but the last power outage here was 4 years ago and for an 8 bay NAS + media server with an i5 I will probably need a rather large one, so I never really put that plan into reality. But perhaps I should re-prioritize.

I assume that was a typo and you meant UPS.

I have a few of these:
https://www.amazon.com/gp/product/B000FBK3QK/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1
and just one holds up two servers, a router, 15 or so external hard drives and a few other small devices plenty long enough to allow and orderly shutdown and it easily prevents any reboots or glitches resulting from momentary power losses. I believe there may be better or cheaper ones currently available BUT be careful when trying to save money. Power protection is not the place to try and save money at the expense of quality.

A good UPS, or several if needed, are a small investment toward preventing the problems that losses of power or power glitches can cause. Surge protectors are almost useless in the real world for meaningful protection. They are better than nothing but by only a small margin.

Link to comment
Share on other sites

On 9/27/2020 at 9:02 AM, NicoLaz said:

Hi @Luke

Thank you for your fast reply! I wanted to prepare the logs this morning but noticed that they already went into rotation and got deleted :( I didn't realize the timespan was so short, sorry... The only thing I still habe is those two warning messages in the message history - they belong to the two libraries I was missing:


Could not find a part of the path 'C:\Users\cube\AppData\Roaming\Emby-Server\programdata\root\default\Recorded Movies11'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
at System.IO.Enumeration.FileSystemEnumerableFactory.FileSystemInfos(String directory, String expression, EnumerationOptions options, Boolean isNormalized)
at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileSystemEntries(String path, Boolean recursive)
at MediaBrowser.Controller.Providers.DirectoryService.GetFileSystemEntries(String path)
at MediaBrowser.Controller.IO.FileData.GetFilteredFileSystemEntries(IDirectoryService directoryService, String path, IFileSystem fileSystem, IServerApplicationHost appHost, ILogger logger, ItemResolveArgs args, Int32 flattenFolderDepth, Boolean resolveShortcuts)
at MediaBrowser.Controller.Entities.CollectionFolder.CreateResolveArgs(IDirectoryService directoryService)
at MediaBrowser.Controller.Entities.CollectionFolder.FolderIdsRequireUpdating(Boolean updateValues)
at MediaBrowser.Controller.Entities.CollectionFolder.BeforeMetadataRefresh(Boolean replaceAllMetdata)
at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateTopLibraryFolders(Boolean refreshMetadatas, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.PerformLibraryValidation(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

Could not find a part of the path 'C:\Users\cube\AppData\Roaming\Emby-Server\programdata\root\default\Recordings1'.
at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
at System.IO.Enumeration.FileSystemEnumerator`1.Init()
at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options, Boolean isNormalized)
at System.IO.Enumeration.FileSystemEnumerableFactory.FileSystemInfos(String directory, String expression, EnumerationOptions options, Boolean isNormalized)
at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
at Emby.Server.Implementations.IO.ManagedFileSystem.GetFileSystemEntries(String path, Boolean recursive)
at MediaBrowser.Controller.Providers.DirectoryService.GetFileSystemEntries(String path)
at MediaBrowser.Controller.IO.FileData.GetFilteredFileSystemEntries(IDirectoryService directoryService, String path, IFileSystem fileSystem, IServerApplicationHost appHost, ILogger logger, ItemResolveArgs args, Int32 flattenFolderDepth, Boolean resolveShortcuts)
at MediaBrowser.Controller.Entities.CollectionFolder.CreateResolveArgs(IDirectoryService directoryService)
at MediaBrowser.Controller.Entities.CollectionFolder.FolderIdsRequireUpdating(Boolean updateValues)
at MediaBrowser.Controller.Entities.CollectionFolder.RequiresRefresh()
at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
at MediaBrowser.Controller.Entities.BaseItem.RefreshMetadata(MetadataRefreshOptions options, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateTopLibraryFolders(Boolean refreshMetadatas, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.PerformLibraryValidation(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.Library.LibraryManager.ValidateMediaLibraryInternal(IProgress`1 progress, CancellationToken cancellationToken)
at Emby.Server.Implementations.ScheduledTasks.ScheduledTaskWorker.ExecuteInternal(TaskOptions options)

@PenkethBoy: Thank you, I appreciate your hint. A USP is already on my "should do" list but the last power outage here was 4 years ago and for an 8 bay NAS + media server with an i5 I will probably need a rather large one, so I never really put that plan into reality. But perhaps I should re-prioritize.

Cheers

NicoLaz

Ok if you see it again and you have the emby server log, please attach it. Thanks !

Link to comment
Share on other sites

  • 1 year later...

 

Old thread but this is still an issue for me as my storage is not online all the time. I have learned (the hard way) to boot up my media storage if i need to update/reboot my emby server but occasionally i forget and as a result i have to rescrape my whole library again. I also created rules in my virtual environment not to boot up my embyserver after a poweroutage becuse my storage will not boot up.Could you maybe implement a check or something to verify if a medialibrary is responding or not? If it's not responding make an alarm/error instead of deleting everything?

On the upside my library is getting easier to rescrape as I'm fixing all the filenaming and missing info everytime this happens :)

(I'm running emby server on a ubuntu server with nfs access to my media library)

Edited by Tratten
Link to comment
Share on other sites

Gilgamesh_48
15 minutes ago, Tratten said:

 

Old thread but this is still an issue for me as my storage is not online all the time. I have learned (the hard way) to boot up my media storage if i need to update/reboot my emby server but occasionally i forget and as a result i have to rescrape my whole library again. I also created rules in my virtual environment not to boot up my embyserver after a poweroutage becuse my storage will not boot up.Could you maybe implement a check or something to verify if a medialibrary is responding or not? If it's not responding make an alarm/error instead of deleting everything?

On the upside my library is getting easier to rescrape as I'm fixing all the filenaming and missing info everytime this happens :)

(I'm running emby server on a ubuntu server with nfs access to my media library)

You "might" be able to reduce or even eliminate that problem by storing all metadata with your media. (That is a library option)

I have two servers. One runs on the same machine as my storage but the other is on a different machine and I can boot the second server without the data storage machine on at all and i lose nothing and my server does not need to rescan at all. Also,since I use DrivePool to combine all my drives my main Emby server comes up well before the storage becomes available on a reboot.

I "think" there is something else going on if you lose all metadata from your server coming up before the media is available.

Link to comment
Share on other sites

Hi @Gilgamesh_48, appreciate the help and I activated the option to store metadata with my media.

I also looked more into my problem and I've verified that the emby server deletes my library when rebooted while storage is offline.
And this is working as it should because the mount folder always exists locally on the server but the service does not have any knowledge 
if it's mounted or not. It finds the folder which is empty and starts to delete metadata.

To resolve this I changed my setup for mounting my nfs shares from fstab to systemd. Doing this I could then modify the emby-server.service
startupfile so it requires the nfs shares to be mounted before starting the service. 


Downside to this is that I need to modify the startupfile after every embyserverupdate , but I'm ok with that!

Link to comment
Share on other sites

Hi ,yes ideally this shouldn't happen but I think there are still a couple corner cases we're still trying to chase down. Thanks,.

Link to comment
Share on other sites

While not a solution another thing you could do short term is setup a little script to copy the library.db file before starting Emby.  That way if something went wrong you could put a good copy back.  The combination of that plus storing the graphics and NFO file with the media would go a long way toward a quick recovery.

If your VM setup and disk is setup for snapshots that would be even more ideal as you could grab a snapshot before Emby starts. A proper VM environment setup for snapshots will only store what's changed from the last snapshot so they are quite fast and hardly take any space. (kind of like an incremental backup).

This really shouldn't happen however. If you could send me a PM with a bit of info on your setup and how you have these libs mounted I can see if I can reproduce this and pass that onto Luke so he can look at specifics.  I might also be able to recommend a different setup possibly that won't do this as well.

I know I've seen setups where people mount to folders which is logical but then the mount the parent folder in Emby.  The parent folder is always there but if the sub directories show up empty Emby will remove the items because they don't exist.  On the other hand setting Emby up to mount the actual directory works because it shows up unavailable so Emby won't touch it.  That kind of thing is a little tweak in setup with a big return on functionality. 

We would be happy to look at this for you, we just need some specifics.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...