Prevent automatic trash removal

[enqbcvqw 2]

Is this possible? I have a script that forces some scans via API. Apparently if Emby can't find the folder from the API request in the DB it'll scan the folder above. This is great until a failure causes a disconnection of the network drive, after which emby simply proceeds to delete all content in the library, so there go my 13,000 movies that need to be rescanned.

This should not happen, sh*t happens and networks fail, Emby shouldn't just assume that things it can't see are gone forever as this could be due to a network blip.

Is there a configuration option to prevent this behavior? If not, this REALLY should be something that's supported. It's literally a field in the DB to soft delete instead of flat out removing it the item.

[Guest]

Is this deleting the Movies on the hard drive or just within the Library and database?

EDIT: I could see how a database cleanup/vacuum should be an option when needed, but usually the file monitoring takes care of this area, and should be stopping during scans. I always thought it was what triggered such events. Hard deletion of files and folders should not be something Emby does in MY opinion.

Edited April 19, 2021 by Guest

[enqbcvqw 2]

It's not deleted from the drive but the library.

The current implementation is not suitable for cloud storage drives where there are usually API or transfer limits imposed and the mounts can either fail or maintenance can happen on them.

Removing stuff that has possibly taken weeks to scan at the first sight of the items being unavailable is a bad implementation overall, for anything, even local hard drives as that creates unnecessary requests to metadata providers.

Please implement a persistent trash that the user needs to empty if they want to, the way Plex does it.

[Luke 37254]

Hi, we don't do that or we don't intend to at least.

 

[enqbcvqw 2]

Why not? Do you feel that it's OK to delete all media and have to rescan all of it, including hitting metadata agents just because of a temporary network glitch? Why couldn't the system give unavailable media the benefit of the doubt that it _may_ come back?

At least make this optional, forcing this as a default behavior with no control is not the right approach. Anyone who's hosted data on a cloud drive or a remote location knows what a massive pain in the butt it is to have to scan tens of thousands of items, not only is it really slow but it can also stall due to API and download quotas or bandwidth quotas.

[Guest]

I think what he means is that EMBY does not intend to delete the entries.. In fact if I remember right the reason you mentioned above was the reason not delete, nut would trigger an error. It should only delete them if the library is removed or they are manually deleted and should show up in the log file as such as well as the entry deleted from the database.

[enqbcvqw 2]

Emby thinks they're deleted because they can't be found since the remote mount point disconnected. The point of this thread is to not make that default behavior.

This is incredibly easy to reproduce, just:

1. Create a library with some media on a fuse mount

2. Add a new item to that fuse mount

3. Trigger an API request to scan the new item (say /mnt/remote/movies/bleh (1999)/bleh (1999).mkv

4. Unmount the fuse mount to simulate an issue

Emby will look for the item's path in the DB, won't find it, will try to scan the path above instead (/mnt/remote/movies/) and since that path is now "empty" because the fuse mount disconnected, every single item will be deleted from the library.

[Guest]

I am going out on a limb here to say that this is enabled...

 

REAL-TIME FILE MONITORING

 

ERROR LOG DURING MANUAL SCAN

2021-04-19 18:16:00.671 Error App: Error in ffprobe
	*** Error Report ***
	Version: 4.6.0.39
	Command line: C:\Users\------\AppData\Roaming\Emby-Server\system\EmbyServer.dll
	Operating system: Microsoft Windows 10.0.19042
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: C:/Users/------/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: C:\Users\------\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\------\AppData\Roaming\Emby-Server\system
	System.Exception: System.Exception: ffprobe failed for file:"-:\Ministry\2007 - Rio Grande Dub Ya\Ministry - The Great Satan (What Would Satan Do! Mix).mp3" - streams and format are both null: 
{
	
	}
	
	
ffprobe version 4.3.0-emby_2021_02_27-g8f222573e9+634 Copyright (c) 2007-2021 the FFmpeg developers and softworkz for Emby LLC
	  built with gcc 9.2.0 (Rev2, Built by MSYS2 project)
	file:-:\Ministry\2007 - Rio Grande Dub Ya\Ministry - The Great Satan (What Would Satan Do! Mix).mp3: No such file or directory
	
	
	   at Emby.Server.MediaEncoding.Probing.MediaProbeManager.GetMediaInfoInternal(String inputPath, String primaryPath, MediaProtocol protocol, String userAgent, Boolean extractChapters, String probeSizeArgument, Boolean isAudio, Boolean forceEnableLogging, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.FFProbeAudioInfo.Probe[T](MetadataResult`1 itemResult, MetadataRefreshOptions options, LibraryOptions libraryOptions, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.RunCustomProvider(ICustomMetadataProvider`1 provider, MetadataResult`1 itemResult, String logName, MetadataRefreshOptions options, LibraryOptions libraryOptions, RefreshResult refreshResult, CancellationToken cancellationToken)
	Source: Emby.Server.MediaEncoding
	TargetSite: Void MoveNext()

 

 

Ministry no longer shows up in as the disconnection in the network came during a scan of the current library files. During the disconnection, all previous entries of artist and albums which existed with the library that it did not find ( after being detected ) were deleted/removed from the Library.

Scanning will delete a file usually if it has been detected as moved or no longer existing as well as with Real Time Monitoring. It also adds files in; that it has detected being written to the location. Unless it is not connected during the entire process, I would not expect it to delete the entry in the database like this..

 

LOG DELETION EVENT

2021-04-19 16:05:35.424 Info App: Removing item from database, Type: Folder, Name: Unknown name, Path: \\-----\Music, Id: 421627
2021-04-19 16:05:46.232 Info App: Deleting path C:\Emby\metadata\library\65\654195eae70f0ac1967aa0b143dfd682
2021-04-19 16:05:46.233 Info App: Deleting path C:\Emby\metadata\library\91\911971f5bc973d83317520afd21d5595
2021-04-19 16:05:46.236 Info App: Deleting path C:\Emby\metadata\library\aa\aa3760914a029a07f62b8aaa6a041d93
2021-04-19 16:05:46.236 Info App: Deleting path C:\Emby\metadata\library\38\389f0457bd4ca86463b474533440d33c
2021-04-19 16:05:46.237 Info App: Deleting path C:\Emby\metadata\library\cf\cf021008cf71a2d640baf16b7c58f468
2021-04-19 16:05:46.237 Info App: Deleting path C:\Emby\metadata\library\88\888caa79f097d67f0cfdd0ba0e39ec16
2021-04-19 16:05:46.238 Info App: Deleting path C:\Emby\metadata\library\b2\b2bb2b8e8d3d3499f1a1ce54b2f736d3
2021-04-19 16:05:46.238 Info App: Deleting path C:\Emby\metadata\library\4f\4f43a503a019876d46b702d6dd3f8944
2021-04-19 16:05:46.239 Info App: Deleting path C:\Emby\metadata\library\f6\f6585fa3a828342c044e4c179a014bf4

 

 

So an option to turn off deletion on rescan, could be an option.

Stopping this ability completely actually breaks EMBY's functionality when rescanning a directory with changes.. and will effect Real Time Monitoring. This will also create a bloated metadata folder, and database which would have to be manually maintained.

So an option to turn off deletion/removal on rescan, could be an option. This could be a good feature request.

 

Edited April 20, 2021 by Guest

[enqbcvqw 2]

13 hours ago, Hxemby001 said:

I am going out on a limb here to say that this is enabled...

 

REAL-TIME FILE MONITORING

 

ERROR LOG DURING MANUAL SCAN

2021-04-19 18:16:00.671 Error App: Error in ffprobe
	*** Error Report ***
	Version: 4.6.0.39
	Command line: C:\Users\------\AppData\Roaming\Emby-Server\system\EmbyServer.dll
	Operating system: Microsoft Windows 10.0.19042
	Framework: .NET Core 3.1.13
	OS/Process: x64/x64
	Runtime: C:/Users/------/AppData/Roaming/Emby-Server/system/System.Private.CoreLib.dll
	Processor count: 4
	Data path: C:\Users\------\AppData\Roaming\Emby-Server\programdata
	Application path: C:\Users\------\AppData\Roaming\Emby-Server\system
	System.Exception: System.Exception: ffprobe failed for file:"-:\Ministry\2007 - Rio Grande Dub Ya\Ministry - The Great Satan (What Would Satan Do! Mix).mp3" - streams and format are both null: 
{
	
	}
	
	
ffprobe version 4.3.0-emby_2021_02_27-g8f222573e9+634 Copyright (c) 2007-2021 the FFmpeg developers and softworkz for Emby LLC
	  built with gcc 9.2.0 (Rev2, Built by MSYS2 project)
	file:-:\Ministry\2007 - Rio Grande Dub Ya\Ministry - The Great Satan (What Would Satan Do! Mix).mp3: No such file or directory
	
	
	   at Emby.Server.MediaEncoding.Probing.MediaProbeManager.GetMediaInfoInternal(String inputPath, String primaryPath, MediaProtocol protocol, String userAgent, Boolean extractChapters, String probeSizeArgument, Boolean isAudio, Boolean forceEnableLogging, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.MediaInfo.FFProbeAudioInfo.Probe[T](MetadataResult`1 itemResult, MetadataRefreshOptions options, LibraryOptions libraryOptions, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Manager.MetadataService`2.RunCustomProvider(ICustomMetadataProvider`1 provider, MetadataResult`1 itemResult, String logName, MetadataRefreshOptions options, LibraryOptions libraryOptions, RefreshResult refreshResult, CancellationToken cancellationToken)
	Source: Emby.Server.MediaEncoding
	TargetSite: Void MoveNext()

 

 

Ministry no longer shows up in as the disconnection in the network came during a scan of the current library files. During the disconnection, all previous entries of artist and albums which existed with the library that it did not find ( after being detected ) were deleted/removed from the Library.

Scanning will delete a file usually if it has been detected as moved or no longer existing as well as with Real Time Monitoring. It also adds files in; that it has detected being written to the location. Unless it is not connected during the entire process, I would not expect it to delete the entry in the database like this..

 

LOG DELETION EVENT

2021-04-19 16:05:35.424 Info App: Removing item from database, Type: Folder, Name: Unknown name, Path: \\-----\Music, Id: 421627
2021-04-19 16:05:46.232 Info App: Deleting path C:\Emby\metadata\library\65\654195eae70f0ac1967aa0b143dfd682
2021-04-19 16:05:46.233 Info App: Deleting path C:\Emby\metadata\library\91\911971f5bc973d83317520afd21d5595
2021-04-19 16:05:46.236 Info App: Deleting path C:\Emby\metadata\library\aa\aa3760914a029a07f62b8aaa6a041d93
2021-04-19 16:05:46.236 Info App: Deleting path C:\Emby\metadata\library\38\389f0457bd4ca86463b474533440d33c
2021-04-19 16:05:46.237 Info App: Deleting path C:\Emby\metadata\library\cf\cf021008cf71a2d640baf16b7c58f468
2021-04-19 16:05:46.237 Info App: Deleting path C:\Emby\metadata\library\88\888caa79f097d67f0cfdd0ba0e39ec16
2021-04-19 16:05:46.238 Info App: Deleting path C:\Emby\metadata\library\b2\b2bb2b8e8d3d3499f1a1ce54b2f736d3
2021-04-19 16:05:46.238 Info App: Deleting path C:\Emby\metadata\library\4f\4f43a503a019876d46b702d6dd3f8944
2021-04-19 16:05:46.239 Info App: Deleting path C:\Emby\metadata\library\f6\f6585fa3a828342c044e4c179a014bf4

 

 

So an option to turn off deletion on rescan, could be an option.

Stopping this ability completely actually breaks EMBY's functionality when rescanning a directory with changes.. and will effect Real Time Monitoring. This will also create a bloated metadata folder, and database which would have to be manually maintained.

So an option to turn off deletion/removal on rescan, could be an option. This could be a good feature request.

 

Real time monitoring is enabled but it doesn't do anything because the library is in an underlying fuse mount where inotify doesn't work. The changed content scans are triggered externally via API.

Glad that we agree that providing the option would be good, especially if you're dealing with unstable mounts or ones affected by network or connection issues. Nuking an entire library that might have taken weeks to scan over a slow or throttled connection because of a temporary network blip is an infuriating experience.

[Guest]

Yes.. I have the same situation occur.. as will most people here because they use NAS or data servers connected via network. In fact the log examples I posted, are from an event that occurred before this post... And I have issues with this problem in the past.

[Luke 37254]

Hi, yes please attach the emby server log from when this happens. thanks !

[shpankey 7]

I know this is old, but this is something I find frustrating too. On rare occasions my power will go out and my RAID server doesn't stay online but my Server (PC) does due to my UPS system. Emby detects I lost all my media and deletes everything. When my RAID box comes back online, EMBY detects new media and re-adds everything like the first time. Not only does this use a ton of CPU/Power to re-index everything, it takes a very long time and then my Recently Added is FUBAR'd.

 

Please add the ability to not delete trash automatically like Plex does to prevent this behavior

Edited May 5 by shpankey

[Luke 37254]

On 5/4/2024 at 9:08 PM, shpankey said:

I know this is old, but this is something I find frustrating too. On rare occasions my power will go out and my RAID server doesn't stay online but my Server (PC) does due to my UPS system. Emby detects I lost all my media and deletes everything. When my RAID box comes back online, EMBY detects new media and re-adds everything like the first time. Not only does this use a ton of CPU/Power to re-index everything, it takes a very long time and then my Recently Added is FUBAR'd.

 

Please add the ability to not delete trash automatically like Plex does to prevent this behavior

Hi @shpankey please attach the emby server log from when this happens. thanks !