Jump to content

Emby behaving strangely after backup/restore


Grimmer

Recommended Posts

Running DSM 6...

My media storage volume was getting full so I acquired a larger USB HDD and used it to create a fresh backup of the entire system (including the Emby server files).  Then deleted the volume and storage pool.  Since I already have regularly scheduled backups running, I figured it would be an easy interim solution to convert the Raid 1 (SHR) to Raid 0 in order to get more capacity.  After creating the storage pool and a new volume (ext4), I performed a restore of the entire system.  After it completed, I installed the latest Emby server (4.6.4.0).  It was up and running as it had been before with no titles missing.

Here are my issues:

1) After the restore when I added new media I realized that Emby is no longer scanning for new media when it is added.  It did scan the next morning at a seemingly random time (I couldn't find any setting where the scheduled scan task time is listed for comparison). I have unchecked and rechecked the option to 'scan immediately' in more than one library but none of the libraries are scanning immediately (the scan that happened overnight was more than 10 hours after the media was added).  I was suspicious that the Emby Server's user ID may have changed even though it still shows 'emby'; however, I have files that are set such that only the owner (emby) can access them and this new installation of Emby can read them just fine.  Question: From what I've read on this forum, the filesystem ext4 is compatible with immediate scanning, is this confirmed to be correct?  Perhaps there is a legacy configuration that needs to be removed and recreated?  Anyone know what file they would be stored in?

2) The emby server is set to store the NFO files in the media folders.  Previously when I added a tag to a title by editing the metadata through the emby interface, a line <tag>...<</tag> would be added to the NFO.  These are no longer appearing. I even added the tag manually to a title, but it still came up without the tag in the metadata.  Is Emby ignoring the NFO file?  Have the tags been moved somewhere outside of the NFO files in this latest release?  I thought that maybe a corrupt configuration from the restore activities might have confused the server into storing the metadata directly in the database, but when the files were actually scanned it did create NFO files in the media folder.

I have had less than satisfactory results using the manual 'scan media files' option... mostly because it would overwrite my customized images and parental ratings.  These could be locked against change; however, there is no option to blanket lock all titles and even using the metadata manager to do them one by one is way too much.  Unless the manual 'scan media files' routine has been improved, I would much rather never use it on the entire library.

Any thoughts? Suggestions?

Link to comment
Share on other sites

One other thing I just remembered regarding the NFO files.  I notice that titles with more than one version are getting a unique NFO for each version.  I attempted to restrict viewing of the unrated version while leaving the theatrical version available to all.  I tried the metadata manager but there was no way to edit the different versions. I also tried adding the <tag> ... </tag> line to the unrated version's NFO file.  However, either this is not an intended function, or the server is ignoring the NFO files that are in the media folders.  I never tried this before the backup / restore, so I don't know if it never worked or if it is broken.

Link to comment
Share on other sites

Hi, If you wiped the volume you likely have to adjust the INOTIFY events properties again on the Synology as it's set to a low number and most people blow past it and have to increase the number of events.  This is how Emby Server knows a file was changed because it receives this INOTIFY event from the NAS OS.
https://docs.syncthing.net/users/faq.html#inotify-limits

I'm running 4.7.0.11 on my Synology NAS and just tested add a tag to a movie and it did update the NFO file properly.

Have you checked to make sure you have writing NFO files enabled for this library?

I'm not sure I follow you on the manual scan media files.  Unless a file date/time was updated on the file Emby won't touch it.

Yes if you want to edit metadata of multi-version movies you need to split them apart first.  You can find a button to split them on the detail page of the movie right at the top of the Media Info Section.  Once done editing you can join them back together again.

  • Like 1
Link to comment
Share on other sites

Thanks for the information.  You mentioned that I need to adjust the INOTIFY events property again.  However, I have never adjusted it before .  Perhaps it was coincidental that the number of folders just happen to become 8193+ just after the system restore?  Based on the instructions in the link you provided I added the adjustment line to my sysctl.conf file and rebooted.  But when I went back to check it, I can see that it did not change anything:

user@host:~$ cat /etc/sysctl.conf
kernel.panic = 3
net.core.somaxconn = 65535
net.ipv4.tcp_tw_reuse  = 1
fs.inotify.max_user_watches=204800

user@host:~$ cat /proc/sys/fs/inotify/max_user_watches
8192
 

I also put the 204800 manually into /proc/sys/fs/inotify/max_user_watches:

user@host:~$ cat /proc/sys/fs/inotify/max_user_watches
204800

and executed: sysctl -p

The max_user_watches now anwsers back as 204800.

user@host:~/bin$ sysctl fs.inotify
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 204800

After which, I added a new media file to the Emby library files.  After waiting for 15 minutes, the new media had not been detected or added to the Emby database and no NFO file was created.

I have spent several hours trying to find out if the needed INOTIFY watches have been added by emby, and whether or not there are any existing watches at all.  A couple of ideas for extracting this information that I found on the internet all returned the count of watches as '0'.  Is it possible that no watches exist on the system at all? Or could these methods simply be failing?

Is there a way to see if INOTIFY events are actually firing to determine if they are broken or if emby is not receiving them?  Is there a way to force emby to remove and recreate the watches it normally implements?

Link to comment
Share on other sites

5 minutes ago, Luke said:

@Grimmer has this helped?

Thanks for noticing and taking a look.  I was in the process of responding when you did so I went ahead and posted what I had so far...

One additional thing I tried, was to see if the INOTIFY watches would be recreated if I edited the library and removed the files location then added it back.  Just to see if it would recreate the INOTIFY watch for that library.  This action forced a library scan, but did not work to find a media file added to it afterwards.  It has been almost 10 minutes and no NFO files have appeared.

Edited by Grimmer
Link to comment
Share on other sites

FrostByte

Did you also change "instances"?  You have to change both watches and instances.  Looking at your post above it appears instances is still at the default of 128.  Try changing it to 10000.  

You should see the inotify errors in your server logs shortly after adding the new media if that is the problem.

Link to comment
Share on other sites

I had not attempted to change the "instances', since it wasn't mentioned. I added it to the sysctl.conf file in the same manner that I added the max_user_watches and executed a sudo sysctl -p.

I now have:

user@host:~/bin$ sysctl fs.inotify
fs.inotify.max_queued_events = 16384
fs.inotify.max_user_instances = 10000
fs.inotify.max_user_watches = 204800

However, after that was done, I touched the test media file to change its filedate. That was 10 minutes ago and it hasn't been scanned by Emby. Do I need to stop and restart Emby?

Edited by Grimmer
Link to comment
Share on other sites

Sorry, forgot to ask if you meant the Emby server logs, or the DSM server logs...

Do I need to enable debug logging within Emby to catch those types of messages, or should they be there now?

Link to comment
Share on other sites

FrostByte

I don't think debugging needs to be on.  If it is a problem with INOTIFY you should be able to search the Emby server log for the word INOTIFY and see the errors.  It will say something about too many user instances or user watches 

I'm not sure just changing the date will trigger RTM though.  You may need to remove, rescan, and then add the file back into your media folder and rescan again.  Or, just add something else :)

Link to comment
Share on other sites

FrostByte

If it's still failing though see if you can post at least 20 minutes of Emby server log after adding something to your media library.

Link to comment
Share on other sites

I found these at the end of the messages log immediately after logging into DSM.

2021-09-15T13:14:13-06:00 CORTEX_CINEMA ore.Package.Server_1_check: pkgcurltool.cpp:280 Failed to curl perform, code=7, err=Error
2021-09-15T13:14:13-06:00 CORTEX_CINEMA ore.Package.Server_1_check: pkgcurltool.cpp:453 Failed to download https://synology.emby.media, code=7, err=Error
2021-09-15T13:14:13-06:00 CORTEX_CINEMA ore.Package.Server_1_check: pkgcurltool.cpp:280 Failed to curl perform, code=7, err=Error
2021-09-15T13:14:13-06:00 CORTEX_CINEMA ore.Package.Server_1_check: pkgcurltool.cpp:453 Failed to download https://synology.emby.media, code=7, err=Error

Link to comment
Share on other sites

Running this search:

user@host:/var/log$ ls | xargs sudo cat | grep 'inotify'

Only yielded results from the command line history logs.  Let me know if there is something else I should look for and if there is a specific log I should look into.

So far, it is still the case where the only time Emby scans media files is when manually told to do so, or on the 'every 12 hours' default scheduled task.

Link to comment
Share on other sites

Doing a quick search for Cartoon doesn't find it.

Just to verify this is stored on a Synology pool and not a network share correct?

Link to comment
Share on other sites

Emby is not reacting in any way to new media being added; however, it finds and processes the new media if a manual library scan is initiated or if the default scheduled task to scan is triggered. 

I noticed in the log that is says "watching...." library directories. But it doesn't appear to actually be able to see anything that is going on. 

Link to comment
Share on other sites

Happy2Play

The library is being monitored but is not seeing any changes at least in that log.

2021-09-15 13:44:46.841 Info LibraryMonitor: Watching directory /volume1/MyEmbyMedia/Cartoons

 

  • Like 1
Link to comment
Share on other sites

Happy2Play

But the thing is users are only seeing issues in 4.7 not 4.6 with RTM minus inotfify.  So there would have to be a specific environment issue going on here to me.

Link to comment
Share on other sites

I tried updating the DSM 'firmware' and then uninstalling and reinstalling Emby Server with a fresh download. But still not real-time-monitoring. 

Link to comment
Share on other sites

What if I uninstall Emby, then delete the user emby and reinstall? 

The full system restore brought in the previous user information and Emby configuration files. Perhaps there is a mismatch between old emby user ID and the new emby user / installation? However, there are files in the libraries that are owned by Emby and cannot be read by the group or others. And Emby reads them just fine.

Any other 'environment' issues I can look at? 

Edited by Grimmer
Link to comment
Share on other sites

Have you restarted the NAS after making changes to the startup?

If not that's the first thing to do just to be on the safe side.

If not working we likely need to try testing inotify events outside of Emby using something like inotify-tools from the SynoCommunity.

Link to comment
Share on other sites

I have rebooted it after changes. Still no RTM going on. 

However, the sysctl changes did not persist after the reboot. The sysctl.conf still has the added lines but the command sysctl fs.inotify outputs the original values (128, 8192). I haven't tried the sudo sysctl - p yet to see if it updates per the sysctl.conf file. I wonder if there is a separate file used on startup, maybe some type of synosysctl.conf or something... 

Link to comment
Share on other sites

One other question... As I understand it, inotify is built into the system, but whenever I search for information I always get a ton of hits that include Syncthing. Is Syncthing required? If so, I note that there is no 'sc-syncthing' user on the system. Need to add? I haven't installed a syncthing package before though, so either it came along with DSM or some other package may have included it. 

Link to comment
Share on other sites

Happy2Play
2 hours ago, Grimmer said:

I have rebooted it after changes. Still no RTM going on. 

However, the sysctl changes did not persist after the reboot. The sysctl.conf still has the added lines but the command sysctl fs.inotify outputs the original values (128, 8192). I haven't tried the sudo sysctl - p yet to see if it updates per the sysctl.conf file. I wonder if there is a separate file used on startup, maybe some type of synosysctl.conf or something... 

I would not worry about this unless you have inotify errors in your log.  And to my knowledge you can not edit the file and have it persist a reboot.  That is why a task has to be made.

Real time monitoring seems broken - Page 9 - Emby Server - Emby Community

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...