Completed BUG: emby for kodi will delete your entire library

[ToBeFrank 1]

I discovered the hard way today that the emby for kodi plugin will delete your entire library. Fortunately, my library was on a NAS with a good snapshot schedule so I got it all back. I'm not exactly sure of scenario that caused it since I didn't notice my library was gone until afterwards, but I think this it:

 

Set up emby for kodi and have it sync to your emby library.

Set up the emby user to allow deletes.

Set up emby for kodi to skip emby delete confirmation.

Reset the local kodi database in the emby for kodi plugin settings (I'm not sure if resetting the emby for kodi settings is also required)

 

Kodi 15.2

Emby for Kodi 2.1.3

 

EDIT: forgot to add versions

Edited February 16, 2016 by ToBeFrank

[Angelblue05 4130]

Thanks for reporting. Sorry this happened to you.

 

I'm looking at how this could have possibly happened, because a reset of local database doesn't trigger the Video Library On.Remove.

[ToBeFrank 1]

It's possible I may have hit on an emby bug. I moved a directory and synced the emby library also. Like I said, I'm not entirely certain what triggered it, but I definitely didn't do any deletes and only the actual video files (mkv, mpg, etc) in my library were deleted.

[xnappo 1593]

Yeah, I should have mentioned this possibility when we removed the delete confirmation.

 

I put it in there exactly because of this, but I had convinced myself it wouldn't happen anymore. 

 

I think this will happen if Kodi tries the scan the library (which shouldn't usually happen).

Edited February 16, 2016 by xnappo

[Angelblue05 4130]

Yeah, but even if it's scanning the library, we set everything as "unmonitored" as far as Kodi is concerned for the scans of emby content...

 

@@ToBeFrank do you still have the old Kodi log?

 

@@xnappo ok well, since it's such a big risk to remove the confirmation, I will remove the option. It didn't think such a thing was possible.

Edited February 16, 2016 by Angelblue05

[Angelblue05 4130]

@@ToBeFrank

 

Can you tell me, are you using playback mode set to Addon or Native (direct paths)?

Can you confirm your sources does not include a real path to your content on your NAS? Is it possible your content was made unavailable at one point, and Kodi scanned that source and saw everything was missing, then it triggered on.Remove for all the content? Can you share your advancedsettings.xml - maybe you have an automatic clean library setting enabled?

Edited February 16, 2016 by Angelblue05

[ToBeFrank 1]

Please don't remove the setting to turn off the confirmation.

 

I found where it occurred in a crashlog. It is attached. I think this is some sort of race condition. I had moved a directory full of test patterns (probably 1000 files or so) from my tv directory to the videos directory (in ssh on the server). I then had emby sync it's library so it would see the patterns had moved. It looks like during that sync the Emby for Kodi plugin sync kicked off, and that's when it deleted everything.

kodi_crashlog_20160215113522.log

advancedsettings.xml

Edited February 16, 2016 by ToBeFrank

[ToBeFrank 1]

Can you confirm your sources does not include a real path to your content on your NAS?

 

 

I'm not exactly sure what you mean by this, but I wanted to note that I have playback mode set to addon, but I have to have emby set up to redirect paths to the nfs location of the files accessible by the kodi box. Otherwise, emby for kodi complains that it can't direct play because it can't find the file (but then it proceeds to play just fine).

[Angelblue05 4130]

This seemed to have triggered the issue:

11:33:40 T:140136452945664   ERROR: Failed to open(//tv/Test Patterns/ColorHCFR Windows/Near Black/) opendir call failed with "NFS: Lookup of /tv/Test Patterns/ColorHCFR Windows/Near Black failed with NFS3ERR_NOENT(-2)"
11:33:40 T:140136452945664   ERROR: GetDirectory - Error getting nfs://freenas//mnt/storage/media/tv/Test Patterns/ColorHCFR Windows/Near Black/
11:33:40 T:140136452945664   ERROR: Failed to open(//tv/Test Patterns/ColorHCFR Windows/25% Grayscale/) opendir call failed with "NFS: Lookup of /tv/Test Patterns/ColorHCFR Windows/25% Grayscale failed with NFS3ERR_NOENT(-2)"
11:33:40 T:140136452945664   ERROR: GetDirectory - Error getting nfs://freenas//mnt/storage/media/tv/Test Patterns/ColorHCFR Windows/25% Grayscale/
11:33:40 T:140136452945664   ERROR: Failed to open(//tv/Test Patterns/CalMAN Windows/) opendir call failed with "NFS: Lookup of /tv/Test Patterns/CalMAN Windows failed with NFS3ERR_NOENT(-2)"
11:33:40 T:140136452945664   ERROR: GetDirectory - Error getting nfs://freenas//mnt/storage/media/tv/Test Patterns/CalMAN Windows/
11:33:40 T:140136452945664   ERROR: Failed to open(//tv/Test Patterns/Basic Settings/) opendir call failed with "NFS: Lookup of /tv/Test Patterns/Basic Settings failed with NFS3ERR_NOENT(-2)"
11:33:40 T:140136452945664   ERROR: GetDirectory - Error getting nfs://freenas//mnt/storage/media/tv/Test Patterns/Basic Settings/
11:33:40 T:140136452945664   ERROR: Failed to open(//tv/Test Patterns/ColorHCFR APL/) opendir call failed with "NFS: Lookup of /tv/Test Patterns/ColorHCFR APL failed with NFS3ERR_NOENT(-2)"
11:33:40 T:140136452945664   ERROR: GetDirectory - Error getting nfs://freenas//mnt/storage/media/tv/Test Patterns/ColorHCFR APL/

Then Kodi ran a clean database task because of it. That's not from Emby for Kodi, however because we also monitor on.Remove for the context menu, it deleted all your items. I don't know how we can fix this without making the prompt a necessary thing. Go to videos > files > Do you have any entries that are not named Emby (those are automatically created by the add-on)? Let me know.

Edited February 16, 2016 by Angelblue05

[xnappo 1593]

Yep, that will do it.  If we leave the option we need  to add 'use at your own risk' or something.

[Angelblue05 4130]

Yeah, well it says Caution! next to the setting now (I just need to push to github, but I like use at your own risk)

Edited February 16, 2016 by Angelblue05

[ToBeFrank 1]

Go to videos > files > Do you have any entries that are not named Emby (those are automatically created by the add-on)? Let me know.

 

 

Yes, I had tv, videos, and movies. I'm guessing they are left over from when I was going directly to my NAS over NFS. I've now removed those. It sounds like I wouldn't have hit this issue if they weren't there. Perhaps part of the install instructions should be to remove any old sources?

[Angelblue05 4130]

Yes, indeed, that's what messed up with your setup. I will add that info when we rework the initial setup. I had a couple of ideas in mind.

 

Thanks for confirming and again, sorry that this happened. We will add more warnings around this setting, for future users.

[Angelblue05 4130]

I do not think it's necessarily about the sources. I think it's because he had old sources and Kodi looked at them, and was unable to find the content. But to remove sources, it runs a clean database task which is different from the update library task. I assume Kodi saw the plugin paths, and proceeded to delete the entries, because they are not valid paths (direct paths). You could probably cause the same problem by running the clean database task under settings > Videos.

Edited February 16, 2016 by Angelblue05

[Angelblue05 4130]

Can you explain? @ I'll wait until I wake up to push to stable. So if you want to test or whatever else, let me know.

 

If you check his kodi log, you might find something I'm missing. I'm just deducing, because the timestamps are way too close together to be a coincidence.

Edited February 16, 2016 by Angelblue05

[Angelblue05 4130]

@ I do not understand your setting part. It's not clear.

 

Perhaps a setting that enables addon deleting that is disabled by default as well as even though there is a prompt some people might not know that a clean DB will request server deletes.

 

Let me know what I should add and I'll add it. Thanks for testing and confirming what I thought happened.

Edited February 16, 2016 by Angelblue05

[Angelblue05 4130]

Maybe to disable the emby delete, you need to be also using direct paths? This way, we eliminate that risk and people can understand why we made it this way?

[denethor 90]

This is reminded me one of my request. There must be a setting on server file system with xml files to completely disable deletes. (Not just a check box in WEB UI) Yes I know people are using deletes especially for TV Series Episodes. But at least there should be a option for user who does not. I'm not giving delete permissions to any of my users including my own admin account. But you can always assing that permission as long as it is available in UI.

 

I can see 3 major risks with this:

 

1. Server admin account can be compromised because of a weak password or bug in server and some can assign delete permission to compromised user account and delete you files.

2. There may be a bug/attack on emby connect. (remember what happened to plex forums)

3. Server admin can make a mistake and give delete permission to one of the server user who don't know what to do with that kind of power

 

Bottom line is I don' t want to see "Allow Media Deletion" check box in server management UI. I want to explicitly disable that UI element directly via server configuration XML file within the server file system. You can keep the current behaviour as a default. But for opt-in users it would be great to have an option to disable deletes entirely. For example server owners who add <disabledeletesfromUI>true</disabledeletesfromUI> to settings.xml can feel more comfortable.

Edited February 17, 2016 by denethor

[Angelblue05 4130]

Not really the same... Or I'm missing the point of what you posted. We are not discussing the ability to delete, but simply the confirmation prompt from the add-on when deleting an item. The issue is that Kodi already presents a dialog, therefore you basically have to confirm twice, which can be annoying.

 

@

 

I think we have two choices here:

We could always remove the VideoLibrary.on.remove section. Force users who want to use the option update to Kodi Isengard and use the context menu. Downside is that it's a bit more tedious to click thru the menus (1-2 extra clicks). We could always make an entry point and people could overwrite the delete key in their keymap, so they can still use Delete on content (not tested, but I assume it's possible..?). 

 

We can keep the skip confirmation, keep the on.remove section. But the skip confirmation option is only made available when Playback mode set to Native (direct paths) - paths are considered valid and the issue is non-existant.

Edited February 17, 2016 by Angelblue05

[Angelblue05 4130]

Alright,  I will make the adjustment and release stable. I didn't want to create stable with the possibility of wiping someone's entire library by accident (user error).

Edited February 17, 2016 by Angelblue05

[xnappo 1593]

We can't make a direct Emby Delete context menu entry?  I know it isn't as good as 'del key' but it is closer.

[Angelblue05 4130]

I made a simple change, just to get stable out faster. We can tweak it in the next beta.

 

I removed the on.remove section, added the skip deletion to the context menu option. This should be good enough for the majority of our users. At least there's no risk of wiping entire libraries anymore.

[denethor 90]

Not really the same... Or I'm missing the point of what you posted. We are not discussing the ability to delete...

Yes I got your point. I just wanted to share my ideas regarding to delete capability of server in general.

I just don't feel comfortable with that ability in UI