Jump to content


Photo

Emby Just Deleted ALL of my Movies


  • This topic is locked This topic is locked
198 replies to this topic

#1 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 02 March 2019 - 01:50 PM

Ok, I am VERY, VERY upset to say the least.  Emby just deleted all the movies off my unRAID server!!!!!

 

I am Premiere Lifetime Member and I went into Emby Version 4.0.2.0 and went into Movies in my web browser.  In looking at the list of movies I had seen a lot of cover art was not showing up for some reason.  So I went back one screen and I clicked on the 3 dots and selected Refresh Metadata and selected to REPLACE ALL METADATA and checked the box to Replace all the Images. At that point it just went through and deleted everything!  GONE!  Nothing!  All deleted! 

 

Example from the log...

2019-03-02 12:25:01.032 Info HttpServer: HTTP POST http://192.168.1.200:8096/emby/Items/f137a2dd21bbc1b99aa5c0f6bf02a805/Refresh?Recursive=true&ImageRefreshMode=FullRefresh&MetadataRefreshMode=FullRefresh&ReplaceAllImages=true&ReplaceAllMetadata=true. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
2019-03-02 12:25:01.033 Info HttpServer: HTTP Response 204 to 192.168.1.100. Time: 1ms. http://192.168.1.200:8096/emby/Items/f137a2dd21bbc1b99aa5c0f6bf02a805/Refresh?Recursive=true&ImageRefreshMode=FullRefresh&MetadataRefreshMode=FullRefresh&ReplaceAllImages=true&ReplaceAllMetadata=true
2019-03-02 12:25:01.139 Error App: Error validating images for /config/root/default/Movies
*** Error Report ***
Version: 4.0.2.0
Command line: /system/EmbyServer.dll -programdata /config -ffdetect /bin/ffdetect -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
Operating system: Unix 4.18.20.0
64-Bit OS: True
64-Bit Process: True
User Interactive: True
Processor count: 4
Program data path: /config
Application directory: /system
System.IO.DirectoryNotFoundException: System.IO.DirectoryNotFoundException: Could not find a part of the path '/mnt/user/DVD'.
   at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
   at System.IO.Enumeration.FileSystemEnumerator`1..ctor(String directory, EnumerationOptions options)
   at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options)
   at System.IO.Enumeration.FileSystemEnumerableFactory.FileInfos(String directory, String expression, EnumerationOptions options)
   at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFiles(String path, String[] extensions, Boolean enableCaseSensitiveExtensions, Boolean recursive)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToArray()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at MediaBrowser.LocalMetadata.Images.LocalImageProvider.GetImages(BaseItem item, IEnumerable`1 paths, Boolean arePathsInMediaFolders, IDirectoryService directoryService)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at MediaBrowser.Providers.Manager.ItemImageProvider.ValidateImages(BaseItem item, IEnumerable`1 providers, IDirectoryService directoryService)
   at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)
Source: System.IO.FileSystem
TargetSite: IntPtr CreateDirectoryHandle(System.String, Boolean)
   at System.IO.Enumeration.FileSystemEnumerator`1.CreateDirectoryHandle(String path, Boolean ignoreNotFound)
   at System.IO.Enumeration.FileSystemEnumerator`1..ctor(String directory, EnumerationOptions options)
   at System.IO.Enumeration.FileSystemEnumerable`1..ctor(String directory, FindTransform transform, EnumerationOptions options)
   at System.IO.Enumeration.FileSystemEnumerableFactory.FileInfos(String directory, String expression, EnumerationOptions options)
   at System.IO.DirectoryInfo.InternalEnumerateInfos(String path, String searchPattern, SearchTarget searchTarget, EnumerationOptions options)
   at Emby.Server.Implementations.IO.ManagedFileSystem.GetFiles(String path, String[] extensions, Boolean enableCaseSensitiveExtensions, Boolean recursive)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToArray()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at MediaBrowser.LocalMetadata.Images.LocalImageProvider.GetImages(BaseItem item, IEnumerable`1 paths, Boolean arePathsInMediaFolders, IDirectoryService directoryService)
   at System.Linq.Enumerable.SelectManySingleSelectorIterator`2.ToList()
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at MediaBrowser.Providers.Manager.ItemImageProvider.ValidateImages(BaseItem item, IEnumerable`1 providers, IDirectoryService directoryService)
   at MediaBrowser.Providers.Manager.MetadataService`2.RefreshMetadata(BaseItem item, MetadataRefreshOptions refreshOptions, CancellationToken cancellationToken)

2019-03-02 12:25:01.233 Info App: Deleting item, Type: Movie, Name: Peter Gabriel: Still Growing Up Live and Unwrapped, Path: /mnt/user/Movies/Peter Gabriel - Still Growing Up, Id: 6
2019-03-02 12:25:01.257 Info App: Deleting item, Type: Movie, Name: A Perfect Getaway, Path: /mnt/user/Movies/A Perfect Getaway, Id: 18592
2019-03-02 12:25:01.275 Info App: Deleting item, Type: Movie, Name: Angels & Demons, Path: /mnt/user/Movies/Angels and Demons, Id: 18593
2019-03-02 12:25:01.328 Info App: Deleting item, Type: Movie, Name: August Rush, Path: /mnt/user/Movies/August Rush, Id: 18594
2019-03-02 12:25:01.346 Info App: Deleting item, Type: Movie, Name: Brides Maids, Path: /mnt/user/Movies/Brides Maids, Id: 18595
2019-03-02 12:25:01.361 Info App: Deleting item, Type: Movie, Name: The Descendants, Path: /mnt/user/Movies/The Descendants, Id: 18596
2019-03-02 12:25:01.378 Info App: Deleting item, Type: Movie, Name: I Don't Know How She Does It, Path: /mnt/user/Movies/I Don't Know How She Does It, Id: 18597

..............

 

This was on a UnRAID system running the current Docker.   It's GONE!  No change of recovery!  

Attached Files


Edited by dbott, 02 March 2019 - 04:17 PM.

  • Abobader, TeamB and DougParker1992 like this

#2 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 02 March 2019 - 01:51 PM

I seem to have posted this in the TEST area.  Mods, please move this to the needed area. Thank you.



#3 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 152201 posts
  • Local time: 09:03 AM

Posted 02 March 2019 - 04:08 PM

Hi there, can you please attach the complete emby server log? Thanks !



#4 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 02 March 2019 - 04:15 PM

Hello... Here are the last two.  I tried to recover, so I dismounted the array thus the new one and the older one with all the deletes.

 

 


Edited by dbott, 04 March 2019 - 08:21 PM.


#5 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 152201 posts
  • Local time: 09:03 AM

Posted 02 March 2019 - 04:27 PM

Ok it looks like you did actually use the delete media function, and for whatever reason it may have gone up an extra level when deleting the movie folder. I will have to add more logging to confirm this. I believe we have noticed this issue being associated with the database upgrade from 3.5, but it does not occur on newly created libraries. It is already resolved for the next release so that it will not happen again. I apologize for the disruption. 



#6 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 02 March 2019 - 04:33 PM

Look, nothing for nothing, but this just WIPED OUT my entire library.  Disruption?  This is a total muck-up of that makes for a HUGH impact!  This kind of issue this should never be allowed to happen at all as you are well aware.  But I trusted Emby, So I bought Emby with a lifetime license, and now Emby totally lost my ENTIRE collection! 

 

Thanks for apologizing, but come on Luke. This was major bad to the point you should right now be posting a BIG WARNING to everyone!!!


  • b0dyr0ck2006 likes this

#7 speechles OFFLINE  

speechles

    Advanced Member

  • App Developer
  • 5953 posts
  • Local time: 06:03 AM

Posted 03 March 2019 - 05:45 PM

What about your users. Imagine children now ask where are my disney movies??? ...and you have to tell them gone...

 

giphy.gif

 

It isn't a good idea to piss off children. If this happened to me I wouldn't be able to compose a post as sober as the OP has. I applaud him on that. I would've blown a gasket and shot through the roof.


Edited by speechles, 03 March 2019 - 05:45 PM.

  • skidmarks, Spaceboy, TeamB and 4 others like this

#8 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 03 March 2019 - 06:05 PM

This should be moved to the main General Server forum for more visibility.

 

so does that mean everyone that updated from 3.5 to 4.x is vulnerable to this?

 

What about now, this current release, should I be turning off write access for the user account I run Emby under to my media shares?

 

Seriously if all my media got deleted I would flip a table.

https://giphy.com/explore/table-flip

 

 

What about your users. Imagine children now ask where are my disney movies??? ...and you have to tell them gone...

 

giphy.gif

 

It isn't a good idea to piss off children. If this happened to me I wouldn't be able to compose a post as sober as the OP has. I applaud him on that. I would've blown a gasket and shot through the roof.

 

My fear is that I don't know if someone has something turned on to run a refresh or something every X days that it does not do the same script thus wiping them out.  (I have no idea...Just was a bad thought.)

 

It is amazing to say the least that this even happened, that it could do such a thing COULD HAPPEN with a software such as this point in the development. 

 

Things happen, I get that, but this really was bad to me. I lost 17TB of data.  Others could loose MUCH more.  I know they can't fix my issue at this point, so not much I can do other than accept the apology for the "Disruption".  This was not a "Disruption" however, it is a total loss. I don't backup to another server...I use unRAID for protection from a drive failure, but a delete like this is a delete. No recourse. (I don't back up my entire library as until now, I have never had the need.)  I never, ever, would expect a program such as Emby to do something like this at this point in the game.  Someone really muckled up to say the least.

 

And yeah, to NOT warn everyone and pull the release is just beyond me!  I think I am actually more upset about that actually for they are aware of it and confirmed it.... Yet NO MENTION OF IT!!! 



#9 Angelblue05 OFFLINE  

Angelblue05

    Advanced Member

  • Developers
  • 12738 posts
  • Local time: 07:03 AM

Posted 03 March 2019 - 06:29 PM

And remember its those kids who will be responsible for you in your later years.

When you get dumped in a shitty retirement home with no internet as you kids are leaving they will turn to you and say "This is for when you deleted all my Disney moves when I was 6", kids remember everything!

 

So the question is, am I safe?

Guessing that you are safe, if you don't use the delete function.

 

Ok it looks like you did actually use the delete media function, and for whatever reason it may have gone up an extra level when deleting the movie folder.

 

I think there's a thread somewhere with many suggestions regarding the delete option.

 

IMO, at the very least, Emby should have the option to define a folder as an alternative to the recycle bin. Until then, I refuse to use Emby to delete anything, since I don't have a backup in place.


Edited by Angelblue05, 03 March 2019 - 06:29 PM.

  • CBers, skidmarks, Spaceboy and 3 others like this

#10 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 03 March 2019 - 06:34 PM

Guessing that you are safe, if you don't use the delete function.

 

 

I think there's a thread somewhere with many suggestions regarding the delete option.

 

IMO, at the very least, Emby should have the option to define a folder as an alternative to the recycle bin. Until then, I refuse to use Emby to delete anything, since I don't have a backup in place.

 

I DID NOT CHOOSE DELETE!!!!  IT WAS AN REFRESH that did this!



#11 Angelblue05 OFFLINE  

Angelblue05

    Advanced Member

  • Developers
  • 12738 posts
  • Local time: 07:03 AM

Posted 03 March 2019 - 06:46 PM

I DID NOT CHOOSE DELETE!!!!  IT WAS AN REFRESH that did this!

 

I was just quoting what Luke said he ended seeing as the root issue. I'm sorry this happened to you, sir. I have no doubt they will do (or did) everything in their power to make sure this does not happen anymore moving forward.

 

@null_pointer

That's why I quoted Luke. He seemed to think that was the issue in the end and hasn't posted here* since. But OP says differently. So avoid deleting and refreshing until we know more.


Edited by Angelblue05, 03 March 2019 - 06:52 PM.

  • TeamB likes this

#12 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 152201 posts
  • Local time: 09:03 AM

Posted 04 March 2019 - 12:02 AM

Media items keep track of whether they own their whole folder or are in a mixed folder with other content. That's what determines if the delete function will go up a level when deleting. This happened in a situation when it shouldn't have due to a problem with the database upgrade from 3.5. Thanks.



#13 chowbok OFFLINE  

chowbok

    Advanced Member

  • Members
  • 146 posts
  • Local time: 07:03 AM

Posted 04 March 2019 - 12:37 AM

This is why I set the movie directories to be owned chowbok:emby and permed 1775, with the movie files themselves owned by me. That way Emby can still save images and metadata to the movie directories, but can't actually delete the movies if it loses its mind like it apparently has here. 

That won't help Windows folks, but I'm sure there's an equivalent in Windows perms. Something like "give the Emby user write permission but not delete permisson to these folders".

 


  • levander likes this

#14 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 04 March 2019 - 11:37 AM

Digging a little deeper in your logs

I can find a bunch of deletes that look like manual deletes of movies

Line 1359: 2019-03-02 08:38:18.139 Info HttpServer: HTTP DELETE http://192.168.1.200:8096/emby/Items/103130. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
Line 1360: 2019-03-02 08:38:18.146 Info App: Deleting item, Type: Movie, Name: A Simple Favor, Path: /mnt/user/Movies/Simple Favor, A (2018)/A Simple Favor (2018).mkv, Id: 103130

Line 1471: 2019-03-02 08:39:17.416 Info HttpServer: HTTP DELETE http://192.168.1.200:8096/emby/Items/103502. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
2019-03-02 08:39:17.416 Info App: Deleting item, Type: Movie, Name: 1517 to Paris, The (2018), Path: /mnt/user/Movies/1517 to Paris, The (2018)/The 1517 to Paris (2018).mkv, Id: 103502

Line 3167: 2019-03-02 12:08:01.315 Info HttpServer: HTTP DELETE http://192.168.1.200:8096/emby/Items/103506. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
Line 3168: 2019-03-02 12:08:01.316 Info App: Deleting item, Type: Movie, Name: The Humanity Bureau, Path: /mnt/user/Movies/Humanity Bureau, The (2017)/The Humanity Bureau (2017).mkv, Id: 103506

Around 2019-03-02 08:30 AM ish

A Simple Favor (2018)

The 1517 to Paris (2018)

 

Around 2019-03-02 12:10 PM ish

The Humanity Bureau (2017)

One Under the Sun (2017)

Finding Your Feet (2017)

911 (2017)

12 Years a Slave

The 13th Warrior

27 Dresses

The 33

About Time

 

Can you confirm you did in fact delete the above manually?

 

At this point it all looks good....Then BAM!!!

 

 

 

CORRECT....I did those manually.  I was doing some clean up and then wanted to get fresh cover art and metadata. As you can see, I also posted a screen shot of the option I was using that did it. 

Attached Files



#15 ebr ONLINE  

ebr

    Chief Bottle Washer

  • Administrators
  • 50647 posts
  • Local time: 09:03 AM

Posted 04 March 2019 - 11:47 AM

CORRECT....I did those manually.  I was doing some clean up and then wanted to get fresh cover art and metadata. As you can see, I also posted a screen shot of the option I was using that did it. 

 

Right, and what we're saying is it was one of those delete operations you performed that ended up deleting everything - due to bad data from a conversion problem.

 

The refresh all metadata did not cause the deletions - it just made it so you could see that everything was gone.

 

We sincerely apologize for this situation.  It is a possible consequence of allowing delete capability.  One piece of bad data can be disastrous and this is why we are always hesitant with requests for easier deletion in the app.

 

One thing we can do to help will be to re-design the process to show you if not exactly what, then at least a count of the number of items that will be deleted so that you can see that something may be wrong before you complete the operation.


  • Angelblue05 likes this

#16 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 04 March 2019 - 12:35 PM

I disagree!!!!!  You can CLEARLY SEE the delations happened when I did the refresh.  Each manual delete was just that, a PROPER delete.  ALL MY FILES where still there after the deletes.  (I was actually looking at physical directories to see if clean up happened as it should.) It was not until later I did the refresh where it REMOVED THEM ALL!!!!

 

It if happen the way you say, when I went to manually delete the second one it would have, should have, then told me it did not even exist to delete.  But nope, in the log you can see it did the delete just was it was instructed...One choice at a time.

 

THIS IS NOT SOMETHING I DID.


Edited by dbott, 04 March 2019 - 12:38 PM.


#17 Angelblue05 OFFLINE  

Angelblue05

    Advanced Member

  • Developers
  • 12738 posts
  • Local time: 07:03 AM

Posted 04 March 2019 - 12:39 PM

One thing we can do to help will be to re-design the process to show you if not exactly what, then at least a count of the number of items that will be deleted so that you can see that something may be wrong before you complete the operation.

Thank you for clarifying how this scenario occurred.

How would your suggestion translate to other clients deletion trigger? It’s a webclient-only solution?

Edited by Angelblue05, 04 March 2019 - 12:39 PM.


#18 speechles OFFLINE  

speechles

    Advanced Member

  • App Developer
  • 5953 posts
  • Local time: 06:03 AM

Posted 04 March 2019 - 04:00 PM

@null_pointer I can fill in the blanks. I now understand what happened. The delete happens removes the file. Then the refresh itself will remove reference in DB and images/metadata. It is this part that if told incorrectly how to traverse. Say a movie it knows is in the movie folder. To delete all in folder leaves an empty folder. It has to traverse back one step. This traversal back might be happening incorrectly and somehow gets back to root of the movie folder. Now it is dangerous and will go through systematically removing everything. I can see how migration from one server version to another with a DB can cause this. There must be better checks in place to prevent this. I would suggest if this were ever possible to happen that the library.db not be ported but be wiped out and recreated. Sometimes it is better this way. This can never happen again. This breaks trust. This causes emotional pain. Both of those can put a company into the ground.

 

There is ease of use and then ease of abuse. If a library DB update the server makes can make any part of the ease of use become ease of abuse there must be sanity checks around every situation of abuse. In that way you prevent a user on your system from doing the very same thing (delete all your shit again later just to be an ass) once you let them know this occured. People can be bastards once they know how to hurt you. Even friends can be bastards just to get even in their mind. Friends you think are friends on your Emby server. Equate users on an Emby server to everyday people in real life. A certain percent have evil intentions.

 

@Luke Please do something to prevent this from ever happening again in the future ever again. It cannot. It must not. There must be sanity here. Please ;)


Edited by speechles, 04 March 2019 - 04:22 PM.


#19 neik ONLINE  

neik

    Advanced Member

  • Members
  • 1024 posts
  • Local time: 02:03 PM

Posted 04 March 2019 - 04:17 PM

Would a workaround be to disallow all users to delete anything on the server?

When I delete something I like to go old fashion and use a file manager.



#20 dbott OFFLINE  

dbott

    Advanced Member

  • Members
  • 46 posts
  • Local time: 08:03 AM

Posted 04 March 2019 - 04:26 PM

Why has no warning been posted?  I can only guess you don't want people to panic and thus leave Emby seeing that something like this can and DID happen with your software.  I think you need to give notice seeing YOU ARE FULLY AWARE that there is an issue.  Now if this happens to another and another and another then where will you all be?

 

I lost 17TB...Others likely have MUCH MORE to loose.  But I have lost stuff that I can never find again. 

 

Will this same thing happen if I was to remove a TV show?  Would ALL my TV shows be wiped out? 

 

If I were to remove a Photo...Same?  Can you image people loosing photos!?!?!?!?

 

See how bad this can really be? 

 

Again...You KNOW ABOUT IT. You need to act to MAYBE save someone else as they may not be as nice about it as I have been. (1st world problem...But still a problem!)


  • Diedrich likes this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users