fbartels 9 Posted January 9, 2017 Share Posted January 9, 2017 Hi, little backstory: Over the weekend I upgraded the server Emby is running from Ubuntu 14.04 to 16.04. Unfortunately after the upgrade Emby mixed up the configured users so I ended up without a user with admin permissions in Emby and therefore decided to delete /var/lib/emby-server and start fresh. Setup wise I have one system (Synology NAS) providing the files and another system actually running Emby. It happens every now and then (every day if you don't remove the "clean database" from sheduled tasks, that Emby decided to clean out my library, when he can't find the media files (because the NAS is turned off). While it makes sense to remove an item from the database, that cannot be found anymore Emby should become suspicious of every item in the library vanished. Feature suggestion: if a mere ls on the folder of the library already does not give any results, skip clean for the library. 1 Link to comment Share on other sites More sharing options...
tocks 4 Posted January 9, 2017 Share Posted January 9, 2017 +1 Link to comment Share on other sites More sharing options...
CharleyVarrick 277 Posted January 9, 2017 Share Posted January 9, 2017 Something similar happened to me a while back and I wished a failsafe feature would have kicked in Link to comment Share on other sites More sharing options...
mastrmind11 717 Posted January 9, 2017 Share Posted January 9, 2017 https://emby.media/community/index.php?/topic/39767-library-disappeared-due-to-failed-mount/ https://emby.media/community/index.php?/topic/42630-server-ping-nas-offset/ 1 Link to comment Share on other sites More sharing options...
CBers 6779 Posted January 9, 2017 Share Posted January 9, 2017 How is Emby expected to keep things "online" if you go and turn the storage off If you don't turn the NAS off, then Emby won't "delete" your content. If you want to continue turning it off, then close Emby before you power off, then restart Emby after powering the NAS back on. Link to comment Share on other sites More sharing options...
CharleyVarrick 277 Posted January 9, 2017 Share Posted January 9, 2017 (edited) The source could become offline or otherwise unreachable without the user's knowledge. A simple "are you sure you want to delete a gazillion media items" could avoid lost hours (if not days) of rebuilding because a kid unplugged a wire somewhere in the house. Edited January 9, 2017 by jlr19 Link to comment Share on other sites More sharing options...
Latchmor 584 Posted January 9, 2017 Share Posted January 9, 2017 (edited) It's not the clean database that's doing it (well, not only) the Library Scan will remove items if media is offline. I've had to disable all Library Scan schedules. I manually scan when I know my NAS is online. For my situation the NAS goes off nearly every night and back on when I return from work. I'm managing it fine now by only scanning when online so it's no big deal to me. However, I can see the appeal of a process where Emby goes "Woah, I'm going to remove an entire library here, maybe I'll stop..." but I've no idea how easily you would achieve that. Edit: I should add I'm pretty sure this situation was handled better before the 'separate settings per library' were introduced. I used to have scheduled scans and scan on application startup on. When Emby auto updated through the day and scanned. My library was still there after a quick scan after putting my NAS online. Edited January 9, 2017 by Latchmor 1 Link to comment Share on other sites More sharing options...
CharleyVarrick 277 Posted January 9, 2017 Share Posted January 9, 2017 The 2 links from mastrmind11 (post# 4) are both MUST read on this subject Link to comment Share on other sites More sharing options...
fbartels 9 Posted January 10, 2017 Author Share Posted January 10, 2017 It's not the clean database that's doing it (well, not only) the Library Scan will remove items if media is offline. I've had to disable all Library Scan schedules. I manually scan when I know my NAS is online. Ah, yes thats the beauty of having to redo all sheduled tasks, "scan library" is now also set to the usual on time of my nas. Link to comment Share on other sites More sharing options...
Luke 37155 Posted January 10, 2017 Share Posted January 10, 2017 Thanks, this is being reviewed for the next release of Emby Server. Link to comment Share on other sites More sharing options...
fbartels 9 Posted January 11, 2017 Author Share Posted January 11, 2017 Thanks you Luke Link to comment Share on other sites More sharing options...
mastrmind11 717 Posted January 11, 2017 Share Posted January 11, 2017 Thanks Luke. Link to comment Share on other sites More sharing options...
guimello 1 Posted October 8, 2017 Share Posted October 8, 2017 has a failsafe been added for this yet? Link to comment Share on other sites More sharing options...
darkassassin07 432 Posted October 8, 2017 Share Posted October 8, 2017 (edited) If you guys are still looking for a good way to implement this, what about a toggle in each libraries configuration 'allow item removal during scheduled scans' When enabled, scans work as they do now, but disabled they can only add new media/metadata to the library. This would not effect scans manually triggered. (odds are if you are scanning manually, you know your media is available to scan) It would give people another option besides trying to schedule your scans during known downtime. That may not be possible with storage that sleeps unless being accessed, or offline sporadically. Im not sure how others have their libraries setup, but media is rarely removed from my libraries (at least movies/shows) and when it is, its done manually where i rescan manually after anyway. The other side of this is how to handle media that is apart of the library but not available. Two options I can think of are either not displaying them at all, though that would require frequent monitoring of all media items to see if they are available or not; or leaving them visible untill a user attempts to stream unavailable content then notify them 'this content is currently unavailable' Edited October 8, 2017 by darkassassin07 Link to comment Share on other sites More sharing options...
Luke 37155 Posted October 8, 2017 Share Posted October 8, 2017 has a failsafe been added for this yet? Yes, are you experiencing a problem? If so, can you please describe it? Thanks ! Link to comment Share on other sites More sharing options...
m33e 2 Posted February 9, 2018 Share Posted February 9, 2018 Yes, are you experiencing a problem? If so, can you please describe it? Thanks ! I'm experiencing this issue as well. I have some older media on a network share of a pc that I do not use frequently at all. So I only boot it up on demand, and then it would be nice if Emby didn't have to scan this share again and again, after the media becomes unavailable & deleted over and over again. What kind of failsafe is there? Maybe my setup doesnt't trigger the implementation? I actually only have one library called "Movies" added in Emby. In this library, I added multiple folders, all from different network shares of different machines. Link to comment Share on other sites More sharing options...
otispresley 81 Posted February 10, 2018 Share Posted February 10, 2018 I have had this happen before. Mine runs on Ubuntu 16.04.3 inside a Hyper-V Virtual Machine. My libraries are via CIFS mounts to storage on the host machine with an Internal vSwitch connecting them. For a while, there was a CIFS problem in Ubuntu where sometimes mounts would experience I/O issues and unmount. If I didn't catch it quickly, or I was away from home, then Emby would delete all my metadata, resulting in loss of watched status, recently added, etc. I would say that the Library Scan should trigger a check first to see if the root folder of the Library path is available. If not, then do not scan that library and log that the path was unavailable. Link to comment Share on other sites More sharing options...
Luke 37155 Posted February 10, 2018 Share Posted February 10, 2018 Can you please discuss an example and attach the information requested in how to report a problem? thanks. Link to comment Share on other sites More sharing options...
ebr 14947 Posted February 10, 2018 Share Posted February 10, 2018 It is really best not to trouble-shoot inside of a completed feature request topic (we might lose track of it) so, if you don't mind, it would be great if you open a new topic in the server forum with the information requested above. Thanks! Link to comment Share on other sites More sharing options...
otispresley 81 Posted February 11, 2018 Share Posted February 11, 2018 (edited) Can you please discuss an example and attach the information requested in how to report a problem? thanks. @@Luke, I won't have logs or anything at this point. The CIFS issue seems to have been resolved in Ubuntu some time ago. Since then, I have started over so I could use the .Net Core version of Emby. I only wanted to say that I have experienced a loss of metadata before so you know this problem is not isolated to a few users or use cases. A little more detail about my Emby library paths though is that I have my single CIFS mount for Emby in /etc/fstab so it is mounted at boot and before Emby loads. There are 2 other mounts as well, one for Nextcloud users and the other for access to another share. The mount used for the Emby library is mounted RW as the emby user (gid, uid) as follows: //<IP_Address>/Media /media/Media cifs credentials=/home/<USER>/.smbpass,uid=999,gid=999,sec=ntlm,vers=3.0 0 0 The media is actually stored on the Windows Server 2016 host that the Hyper-V VM running Emby is hosted on, so I have an Internal vSwitch for communication between the VM's and the host only. The drive hosting the media is a drive pool made up of all internal 3.5" disks using Stablebit DrivePool software. I do not have logs, but please let me know if you want further details about my setup and/or configuration. Edited February 11, 2018 by otispresley Link to comment Share on other sites More sharing options...
Luke 37155 Posted February 12, 2018 Share Posted February 12, 2018 Ok thanks for that info. It would be best to try and capture it next time it happens so that we can look at the specifics. Thanks. Link to comment Share on other sites More sharing options...
BiTeTHeBiGG 13 Posted June 26, 2023 Share Posted June 26, 2023 (edited) Hi @Luke, I can provide logs when the software deleted the entries on my side. I'm experiencing network loss as of a faulty network wall plug and one of my server mess up my whole database. The second server (testing purpose) did not scan at that time and is healthy. Not only all Items were removed but on readding something is broken as only 60% were newly added back and on specific one (more for sure there) only added back 28 episodes out of 368. I will attach later the log when the software is having error and deleting it. Let me just obfuscate a little. Not all should know my IPs and paths . Best regards BiTeTHeBiGG EDIT: Uploaded obfuscated log parts Debug-obfuscated.txt and as you can see all libraries are messed up. Here just a small screenshot of the anime library. I have to manually identify all the grayed items as they were not added correct back. this is a big bit of frustration. Edited June 26, 2023 by BiTeTHeBiGG added screenshot Link to comment Share on other sites More sharing options...
Luke 37155 Posted June 28, 2023 Share Posted June 28, 2023 On 6/26/2023 at 11:31 AM, BiTeTHeBiGG said: Hi @Luke, I can provide logs when the software deleted the entries on my side. I'm experiencing network loss as of a faulty network wall plug and one of my server mess up my whole database. The second server (testing purpose) did not scan at that time and is healthy. Not only all Items were removed but on readding something is broken as only 60% were newly added back and on specific one (more for sure there) only added back 28 episodes out of 368. I will attach later the log when the software is having error and deleting it. Let me just obfuscate a little. Not all should know my IPs and paths . Best regards BiTeTHeBiGG EDIT: Uploaded obfuscated log parts Debug-obfuscated.txt 121.16 kB · 0 downloads and as you can see all libraries are messed up. Here just a small screenshot of the anime library. I have to manually identify all the grayed items as they were not added correct back. this is a big bit of frustration. OK, regarding content getting cleared out of the database, I think this should be improved in the upcoming 4.8 server release. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now