Jump to content

How does emby track media files?


K22R8CT

Recommended Posts

K22R8CT

I figured it'd use a consistent unique identifier like tmdb-id for movie libraries or tvdb-id for tv, but apparently not because I'm getting "New content added" alerts when existing media files, already identified by emby, are upgraded or renamed.

Link to comment
Share on other sites

Hi, new content notifications are generated when there is a new file, not when an existing file is changed. 

Link to comment
Share on other sites

K22R8CT

That's how it used to work but it changed in the last version or two.

e.g. this file was initially added at ~7:30AM then upgraded 3 times. Each time I received a notification email from Emby, which I turn into Mac notifications (last image.)

image.png.261da4c9162ed8cbb5a8c90402be1a21.png

image.thumb.png.0330b617396899be8928d81668d61853.png

image.png.9d6531164121033a7b3f3a007d297e10.png

Link to comment
Share on other sites

The behavior did not change. The file name must have changed from what it was previously, or maybe it was first deleted and then there was a small but large enough gap of time for Emby Server to clear it out of the database and then add it all over again.

Link to comment
Share on other sites

K22R8CT

That's definitely the sequence but I believe radarr waits until the old file's deleted and new one copied before telling emby to refresh. 

In case it's helpful here's a log excerpt from the initial addition:

2022-07-14 07:29:52.590 Info LibraryMonitor: Movies (/media/Movies) will be refreshed.
2022-07-14 07:29:56.407 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/media/Movies/Lightyear (2022)/Lightyear (2022).mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-07-14 07:29:56.488 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-07-14 07:29:56.511 Info App: MovieDbProvider: Finding id for item: Lightyear
2022-07-14 07:29:56.511 Info HttpClient: GET https://api.themoviedb.org/3/search/movie?api_key=?..f6bd687ffa63cd282b6ff2c6877f2669?..&query=Lightyear&language=en&year=2022
2022-07-14 07:29:56.811 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=?..f6bd687ffa63cd282b6ff2c6877f2669?..&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers&language=en&include_image_language=en,null
2022-07-14 07:29:56.842 Info HttpClient: GET https://private.omdbapi.com?apikey=?..fe53f97e?..&i=tt10298810&plot=short&tomatoes=true&r=json
2022-07-14 07:29:57.111 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=?..f6bd687ffa63cd282b6ff2c6877f2669?..&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers
2022-07-14 07:29:57.147 Info HttpClient: GET https://image.tmdb.org/t/p/original/vpILbP9eOQEtdQgl4vgjZUNY07r.jpg
2022-07-14 07:29:57.271 Info HttpClient: GET https://image.tmdb.org/t/p/original/xNXUa1fvqyilZJYjVuSpEH2Y1dZ.png
2022-07-14 07:29:57.307 Info HttpClient: GET https://image.tmdb.org/t/p/original/tqUD26YGjKmFqOJAgbNBah1gX0N.jpg
2022-07-14 07:29:57.348 Info HttpClient: GET https://webservice.fanart.tv/v3/movies/718789?api_key=?..5c6b04c68e904cfed1e6cbc9a9e683d4?..
2022-07-14 07:29:58.060 Info App: ProcessRun 'quick-extract-imageseries' Execute: /bin/ffmpeg -f matroska -threads 1 -skip_interval 10 -copyts -i file:"/media/Movies/Lightyear (2022)/Lightyear (2022).mkv" -an -sn -vf "scale=w=320:h=134" -vsync cfr -r 0.1 -f image2 "/config/cach
e/temp/d2cf93075953427a8f2e3e05cda49ac2/img_%05d.jpg"
2022-07-14 07:29:59.009 Info App: Sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server
2022-07-14 07:29:59.057 Info App: Completed sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server

And one of upgrades:

2022-07-14 11:40:33.358 Info LibraryMonitor: Movies (/media/Movies) will be refreshed.
2022-07-14 11:40:34.684 Info App: Removing item from database, Type: Movie, Name: Lightyear, Path: /media/Movies/Lightyear (2022)/Lightyear (2022).mp4, Id: 122693
2022-07-14 11:40:34.723 Info App: Deleting path /config/metadata/library/e0/e09cbc1d210fcabd2e6cb5e92cc31aeb
2022-07-14 11:40:37.037 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/media/Movies/Lightyear (2022)/Lightyear (2022).mkv" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-07-14 11:40:37.073 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-07-14 11:40:37.099 Info HttpClient: GET https://image.tmdb.org/t/p/original/vpILbP9eOQEtdQgl4vgjZUNY07r.jpg
2022-07-14 11:40:37.255 Info HttpClient: GET https://image.tmdb.org/t/p/original/xNXUa1fvqyilZJYjVuSpEH2Y1dZ.png
2022-07-14 11:40:37.290 Info HttpClient: GET https://image.tmdb.org/t/p/original/tqUD26YGjKmFqOJAgbNBah1gX0N.jpg
2022-07-14 11:40:39.469 Info LibraryMonitor: Lightyear (/media/Movies/Lightyear (2022)/Lightyear (2022).mkv) will be refreshed.
2022-07-14 11:40:39.729 Info App: Sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server

In each iteration it changed from mp4 to mkv or vice versa.

Would reversing the order so the new file's added to the DB before the old one's deleted resolve this?

It might also fix the problem of date-added being reset in these cases (which I'm using NFO files to work around.)

Link to comment
Share on other sites

K22R8CT

Right but is that ideal?

The second scan detected a file with (I assume) the same unique ID as the one it just deleted.

Shouldn't it be aware of that or maybe perform additions before deletions so on upgrade the unique ID persists in the library?

Link to comment
Share on other sites

16 hours ago, Anon28109 said:

or maybe perform additions before deletions so on upgrade the unique ID persists in the library?

Hi.  Well, if we did that,  your new movie would get removed....

The impact here is only on the notification, correct?  All other associated data is maintained...?

Link to comment
Share on other sites

18 hours ago, Anon28109 said:

Right but is that ideal?

The second scan detected a file with (I assume) the same unique ID as the one it just deleted.

Shouldn't it be aware of that or maybe perform additions before deletions so on upgrade the unique ID persists in the library?

Everything happens all at once. If the file is just replaced, then it will not be removed and re-added to the library. I think the situation is exactly what I outlined earlier - there is a gap of time between the removal and the new file coming in, and you may need to disable the server realtime monitor to avoid picking those events up.

Link to comment
Share on other sites

I think nfo's are the best way to solve this. It will have the date added value. Then we can have the notification system ignore new items that have date added values that are in the past.

In fact, come to think of it, I feel like we should probably do that anyway, independent of this issue. But of course if your external application is deleting your nfo, then that's another story.

Link to comment
Share on other sites

K22R8CT
21 hours ago, Luke said:

I think the situation is exactly what I outlined earlier - there is a gap of time between the removal and the new file coming in

Just ran a test to try to rule out the gap.

Shut down embyserver, then:

mv 'Lightyear (2022).mp4' 'Lightyear (2022).m4v'
touch 'Lightyear (2022).m4v'

Start it, same behavior - new content notification.

2022-07-17 07:39:59.973 Info App: Removing item from database, Type: Movie, Name: Lightyear, Path: /media/Movies/Lightyear (2022)/Lightyear (2022).mp4, Id: 122695
2022-07-17 07:40:00.013 Info App: Deleting path /config/metadata/library/e0/e09cbc1d210fcabd2e6cb5e92cc31aeb
...
2022-07-17 07:40:03.426 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/media/Movies/Lightyear (2022)/Lightyear (2022).m4v" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-07-17 07:40:03.523 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-07-17 07:40:03.527 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=‌f6bd687ffa63cd6ff2c6877f2669‌&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers&language=en&include_image_language=en,null
2022-07-17 07:40:03.656 Info HttpClient: GET https://private.omdbapi.com?apikey=‌fe539297e‌&i=tt10298810&plot=short&tomatoes=true&r=json
2022-07-17 07:40:03.826 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=‌f6bd687ffa63cd282b6ff2c6877f2669‌&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers
2022-07-17 07:40:03.857 Info HttpClient: GET https://image.tmdb.org/t/p/original/vpILbP9eOQEtdQgl4vgjZUNY07r.jpg
2022-07-17 07:40:04.092 Info HttpClient: GET https://image.tmdb.org/t/p/original/xNXUa1fvqyilZJYjVuSpEH2Y1dZ.png
2022-07-17 07:40:04.166 Info HttpClient: GET https://image.tmdb.org/t/p/original/oYRETudDuaySUf6RvQvy4f947Q9.jpg
2022-07-17 07:40:04.275 Info HttpClient: GET https://webservice.fanart.tv/v3/movies/718789?api_key=‌5c6b04dc68e9fed1e6cbc9a9e683d4‌
2022-07-17 07:40:04.546 Info HttpClient: GET https://assets.fanart.tv/fanart/movies/718789/moviethumb/lightyear-62c518601eb80.jpg
2022-07-17 07:40:05.022 Info App: Sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server
2022-07-17 07:40:05.659 Info App: Completed sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server

 

Edited by Anon28109
Link to comment
Share on other sites

K22R8CT
21 hours ago, ebr said:

The impact here is only on the notification, correct?  All other associated data is maintained...?

No, in fact if I weren't using NFOs, date-added would be reset. (I'm only using them to workaround it.)

Until recently it also appeared to workaround this 'new content notification on upgrade' problem.

Link to comment
Share on other sites

6 hours ago, Anon28109 said:

Just ran a test to try to rule out the gap.

Shut down embyserver, then:

mv 'Lightyear (2022).mp4' 'Lightyear (2022).m4v'
touch 'Lightyear (2022).m4v'

Start it, same behavior - new content notification.

2022-07-17 07:39:59.973 Info App: Removing item from database, Type: Movie, Name: Lightyear, Path: /media/Movies/Lightyear (2022)/Lightyear (2022).mp4, Id: 122695
2022-07-17 07:40:00.013 Info App: Deleting path /config/metadata/library/e0/e09cbc1d210fcabd2e6cb5e92cc31aeb
...
2022-07-17 07:40:03.426 Info MediaProbeManager: ProcessRun 'ffprobe' Execute: /bin/ffprobe -i file:"/media/Movies/Lightyear (2022)/Lightyear (2022).m4v" -threads 0 -v info -print_format json -show_streams -show_chapters -show_format -show_data
2022-07-17 07:40:03.523 Info MediaProbeManager: ProcessRun 'ffprobe' Process exited with code 0 - Succeeded
2022-07-17 07:40:03.527 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=‌f6bd687ffa63cd6ff2c6877f2669‌&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers&language=en&include_image_language=en,null
2022-07-17 07:40:03.656 Info HttpClient: GET https://private.omdbapi.com?apikey=‌fe539297e‌&i=tt10298810&plot=short&tomatoes=true&r=json
2022-07-17 07:40:03.826 Info HttpClient: GET https://api.themoviedb.org/3/movie/718789?api_key=‌f6bd687ffa63cd282b6ff2c6877f2669‌&append_to_response=alternative_titles,reviews,casts,releases,images,keywords,trailers
2022-07-17 07:40:03.857 Info HttpClient: GET https://image.tmdb.org/t/p/original/vpILbP9eOQEtdQgl4vgjZUNY07r.jpg
2022-07-17 07:40:04.092 Info HttpClient: GET https://image.tmdb.org/t/p/original/xNXUa1fvqyilZJYjVuSpEH2Y1dZ.png
2022-07-17 07:40:04.166 Info HttpClient: GET https://image.tmdb.org/t/p/original/oYRETudDuaySUf6RvQvy4f947Q9.jpg
2022-07-17 07:40:04.275 Info HttpClient: GET https://webservice.fanart.tv/v3/movies/718789?api_key=‌5c6b04dc68e9fed1e6cbc9a9e683d4‌
2022-07-17 07:40:04.546 Info HttpClient: GET https://assets.fanart.tv/fanart/movies/718789/moviethumb/lightyear-62c518601eb80.jpg
2022-07-17 07:40:05.022 Info App: Sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server
2022-07-17 07:40:05.659 Info App: Completed sending email alerts@pi.lan with subject Emby: Lightyear has been added to Emby Server

 

Even if there is no gap, I've already mentioned multiple times that different file names will cause them to be different items in the database, thus a removal and an addition happening at the same time. The gap would just be one possible explanation if the file names happened to be exactly the same.

Nfo is your best bet to preserve data in these situations.

Link to comment
Share on other sites

K22R8CT

Right, that's why I asked about a consistent unique identifier.

I did a test where I disabled NFO read/write for the library, marked the media item as played, changed the file's extension, then rescanned.

Emby remembered the played state but (due to the name change) reset date-added.

So different properties of the same media item are keyed to different things, which will inevitably result in inconsistencies.

 

Regardless I now have radarr (which distinguishes between 'new' and 'upgraded') send the alerts, so no problem closing this.

 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...