Missing watch state on most movies

[Sarge88 9]

So I just noticed that another of my movies have lost the played icon on them. After looking around the forums I tried restoring to the oldest backup I had using the user state restore option not the full restore and it didn't seem to fix it. From my understanding the watch state is saved to the users database so I was expecting this to fix the issue. I also realized that several of my 1080p versions were just gone. I'm not sure when it happened i just became aware of it after the play state stuff.

I did remove 2 plug-ins a couple days ago (auto box sets, auto movie version collections) and then today I added a list sync for radarr adding any missing 1080p versions of 4k movies I had which ended up being more than expected and ones I was certain I already had 1080p version. That's wwn I noticed the missing 1080p and then the watch states.

 

Is there something I can look for in the logs that might point me in the direction of wat caused some movies to go missing and maybe find out wat happened to my watch status in the process?

[Sarge88 9]

TV shows also aren't effected. And I just did a full restore from the oldest backup I have and still no luck. I'm also not the only one who is missing the movie watch states.

[Luke 42077]

 Hi, what you can look for is if metadata changed on any shows or movies. Watch data gets tied to the external ids (movie db, tvdb, imdb, etc), so if any of these values change on a movie or show then that would effectively clear the watch data as well.

[Sarge88 9]

13 minutes ago, Luke said:

 Hi, what you can look for is if metadata changed on any shows or movies. Watch data gets tied to the external ids (movie db, tvdb, imdb, etc), so if any of these values change on a movie or show then that would effectively clear the watch data as well.

Alright IL take a look. Also do you know if I can view the raw dB file to see if the watch status id's are still there and not somehow cleared? I have hthe playback reporting plug-ins that allows me to query the data too. 

Also I did do a metadata refresh but the provider ids shouldn't have changed.

Edited September 5, 2023 by Sarge88

[Sarge88 9]

OK so I was able to look at the user watchvstatus from the db in a json format. From the looks of it it should be working but it's not. For example I picked a random entry that had played:true and grabbed the key which turned out to be the themoviedb id. Looked it up in the emby metamanager to be sure the id was there, which it was. However the movie still showed as unplayed.unplanned. pics added for ref

[Sarge88 9]

after some more digging in the DB and the JSON from the backup plugin i think i may have found the issue. for some reason my UserDataKey has changed and thus was given a new Id in the DB's UserDataKeys2 Table. So my account(and only my account now it seems) has somehow had its Id changed and now has a userId of 5 which only has 2258 records while my original userid of 1 has the correct amount of 5894 records(ie playcount:1).

any thoughts on how to fix this? its a bit of an odd issue that im not aware wat may have caused it.

[Luke 42077]

A change of external ids for a movie or show is what would cause it to get a new UserDataKey. That could mean new external id added, one removed, or one changed. Any of those three.

[Sarge88 9]

4 minutes ago, Luke said:

A change of external ids for a movie or show is what would cause it to get a new UserDataKey. That could mean new external id added, one removed, or one changed. Any of those three.

Isn't the userdatakey unique to the user? Like the user only has one? I guess I'm trying to figure out how to get my watch status back. Should I just manually change my users userdatakey in the db to the old one and then delete the new key and it's affiliated connections? I have yet to find where the user ID are stored but I could also just bulk change the old id to the new id within the db but I'd have to first clear the new IDs play status so not get duplicates.

Idk what's the best way to resolve this do you think? Without going thru and manually retagging the played movies. 

[Luke 42077]

No, the user data key id is for each item in the MediaItems table. It's specific to the media, not the user.

Try joining the MediaItems table with the UserDatas table on UserDataKeyId.

[Luke 42077]

2 minutes ago, Sarge88 said:

Isn't the userdatakey unique to the user? Like the user only has one? I guess I'm trying to figure out how to get my watch status back. Should I just manually change my users userdatakey in the db to the old one and then delete the new key and it's affiliated connections? I have yet to find where the user ID are stored but I could also just bulk change the old id to the new id within the db but I'd have to first clear the new IDs play status so not get duplicates.

Idk what's the best way to resolve this do you think? Without going thru and manually retagging the played movies. 

To find out why the user data key id changed for a given movie, you'd have to have copies of the database file before and after. Do you have that?

[Sarge88 9]

1 hour ago, Luke said:

To find out why the user data key id changed for a given movie, you'd have to have copies of the database file before and after. Do you have that?

I have the last 14 days using the backup plugin. I checked my userdata folder for my name and the tables were roughly the same(ie not missing a bunch of movie play stats). But like I mentioned before I did do a full restore from the oldest backup I had and it didn't change anything.

I have been trying to search thru the db for some ref of the new id and when it was created but if it's setup the way u say it prob won't be there. 

[Sarge88 9]

1 hour ago, Luke said:

To find out why the user data key id changed for a given movie, you'd have to have copies of the database file before and after. Do you have that?

Also the fact that it seemed to have happened within the day and only effected the movies I have to assume u were right about the refreshed metadata causing the issue.

I also have full backups of my appdata folder every few days. So maybe I could do a complete restore using my tar backup of emby folder. However the fact that refreshing the metadata may have caused the issue worries me for future use. 

[Luke 42077]

Quote

I also have full backups of my appdata folder every few days. So maybe I could do a complete restore using my tar backup of emby folder. However the fact that refreshing the metadata may have caused the issue worries me for future use. 

Once you have an imdb and moviedb id filled in for your movies, then at that point there's generally nothing to worry about. It's only when one of them is missing and then gets filled in later that this might happen.

[Sarge88 9]

3 hours ago, Luke said:

Once you have an imdb and moviedb id filled in for your movies, then at that point there's generally nothing to worry about. It's only when one of them is missing and then gets filled in later that this might happen.

So reverting back to a old full backup wont resolve the issue if I have to refresh the metadata again then? That's not wat I want. 

[Luke 42077]

A backup of what?

[Sarge88 9]

1 minute ago, Luke said:

A backup of what?

To revert the metadata refresh I did. I have unRAID making backups of everything in the appdata folder. So I could revert to the most recent and get my watch status fixed. However doing so won't fix the issue where I'd lose it all again if I did the metadata refresh. If that is actually wat caused the issue of course. 

[Sarge88 9]

Also why it would seem to effect me and no one else. Atleast noone else has seemed to complain and I had a friend look at his and he didn't notice any movies missing the state

[Luke 42077]

Actually this does come up once in a while. Let's get to the bottom of this first. What are the external id's for dark tower before and after?

[Sarge88 9]

2 hours ago, Luke said:

Actually this does come up once in a while. Let's get to the bottom of this first. What are the external id's for dark tower before and after?

Before metadata refresh
ProviderIds:Tmdb=353491|Imdb=tt1648190
After metadata refresh
ProviderIds:Imdb=tt1648190|Tmdb=353491 

besides them being reversed, the PresentationUniqueKey's and UserDataKeyId's were also diff. oh and the Id is diff. which may also explains why its not showing as played if the user play stats use the id.

Edited September 6, 2023 by Sarge88

[Sarge88 9]

I can confirm that the refresh metadata is wat caused the playstats to all disappear. I reloaded a full non plugin backup and all my playstats were there, then did a refresh missing metadata sweep on movies and most of my playstats were gone.

[Sarge88 9]

Also it seems scan library files did the same thing. prob because scan files also includes a metadata refresh.

[Sarge88 9]

On 9/5/2023 at 9:31 PM, Luke said:

Actually this does come up once in a while. Let's get to the bottom of this first. What are the external id's for dark tower before and after?

Any thoughts? I'm afraid Todo a refresh metadata for my TV shows now. Did all the work to sync watch status from plex for all my users to now have to prob do it again. 

[Luke 42077]

What version of the server are you on?