Emby server runs crazy every day 0730 - nothing is scheduled.

[TMCsw 247]

1 hour ago, requa3r0 said:

I realize that, and its a really bad way of checking. 

@Lukehelp me out here...a file representing a title in the library should not be heedlessly re-scanned just because of a timestamp change.

How do you know the time stamp was the only thing that changed?

try this 

echo mymovie1.mkv@my.addr1.com >movie1.strm
echo mymovie2.mkv@my.addr2.com >movie2.strm
touch -t 0101010000.00 movie{1,2}.strm
echo 1 
ls -l movie*
echo 2
echo mymovie2.mkv@my.addr2.com >movie1.strm
ls -l movie*
echo 3
touch -t 1112010000.00 movie1.strm
touch -t 2112010000.00 movie1.strm
ls -l movie*
touch -t 1112010000.00 movie1.strm

So based on that how would emby actually know what file is different (remember emby does not see how any of how this info changed, just that some have!) 

 As you have only partly figured out rsync doesn't need to change the file time (but using a checksum (in this case with tiny .strm (text) files won't take long but generating checksums for MB/GB/TB video files will take some time..... there are better options for 'rsync' to avoid this.

Here are 2 samples of the way I use it that doesn’t cause me any problems…:

rsync -Prompth <source> <destination>
rsync -rompth --info=progress2 --delete-before <source> <destination>

see you redacted it but: Don’t bite the hand that feeds you! He is trying to help you!, it just may take a few posts…

 

[Luke 42077]

There are definitely some small optimizations we can make. Currently the server keeps a blacklist of file extensions to ignore with the realtime monitor, which is why it is not ignoring this inatgz extension. We can improve this to make it a whitelist of only the extensions that the server cares about.

I don't think that will change much for you because you can see in your example there are events firing for both .strm and .inatgz. But I think in general this will benefit Emby users on the whole.

[Q-Droid 989]

7 hours ago, Luke said:

There are definitely some small optimizations we can make. Currently the server keeps a blacklist of file extensions to ignore with the realtime monitor, which is why it is not ignoring this inatgz extension. We can improve this to make it a whitelist of only the extensions that the server cares about.

I don't think that will change much for you because you can see in your example there are events firing for both .strm and .inatgz. But I think in general this will benefit Emby users on the whole.

Do you save a hash sum of files now when they're scanned? This might help with detection of changed files when the only difference between them is a time stamp.

 

 

Edited July 25, 2025 by Q-Droid

[Lessaj 467]

10 hours ago, Luke said:

Currently the server keeps a blacklist of file extensions to ignore with the realtime monitor, which is why it is not ignoring this inatgz extension.

The extension that rsync will use as it's still transferring is a random string, but I do believe that the file names will also always start with a period so that they're "hidden" until the transfer completes. For strm files this should really be instant because they're tiny but larger files may not.

[Luke 42077]

4 hours ago, Q-Droid said:

Do you save a hash sum of files now when they're scanned? This might help with detection of changed files when the only difference between them is a time stamp.

 

 

No but we are only listening to last modified change, not every single attribute.

[Q-Droid 989]

52 minutes ago, Luke said:

No but we are only listening to last modified change, not every single attribute.

Does this mean a scan/refresh will be triggered even if the only modification is date/time and nothing else or are other checks done for the event?

 

[Luke 42077]

38 minutes ago, Q-Droid said:

Does this mean a scan/refresh will be triggered even if the only modification is date/time and nothing else or are other checks done for the event?

 

Yes but a hashing technique would have to open up the file and read the contents anyway so the savings would not be significant. And when the file does change it would create more overhead.

[Q-Droid 989]

16 minutes ago, Luke said:

Yes but a hashing technique would have to open up the file and read the contents anyway so the savings would not be significant. And when the file does change it would create more overhead.

I guess that depends on what you do now which is unknown to outsiders. If you can't determine that a file of the same name and size has actually changed because of a new modified date/time then how much work and time goes into processing said file? If the time spent scanning/refreshing is negligible then I agree that it might not be worth it. Also for cases when a file is simply renamed. Are these treated as new files and scanned/refreshed as well?

The subject comes up often enough in the forum. That Emby gets bogged down with scans that might be doing more work than needed.

Hashing with a fast algorithm to generate a fingerprint of the files could be used to reduce overhead in a few scenarios. Moving a file doesn't change it and neither do copying or renaming. Multiple copies with same or different names in a library would have the same hash value. If the time changes but not the size can you be sure it wasn't modified? Is it worth knowing?

On the efficiency side up to the first 1MB (maybe more or even less) of each file could be used to generate a hash with virtually zero (a theoretical) chance of collisions. No need to hash sum the full length of multi-GB files.

 

[Luke 42077]

I would imagine there must be some specific cases where an optimization could be had by ignoring if the byte size hasn't changed. I'm just not sure what those are yet and how best to avoid false skips.