Jump to content


Photo

Retrieving Changeset when client returns online (MediaImport)


  • Please log in to reply
16 replies to this topic

#1 LongMan OFFLINE  

LongMan

    Advanced Member

  • Members
  • 169 posts
  • Local time: 06:08 AM

Posted 05 February 2020 - 01:08 PM

@Luke can you shed any light on what should be done here. Montellese is implementing only syncing changeset using Server Data and data from Kodi Companion and ran into a snag. 

 

The request was meant for Angelbiue05 but she seems to be busy. Any help wpuld be appreciated.

 

 

From Montellese

 

  • working
    • added items are correctly retrieved from Emby directly
    • items with changed metadata (e.g. title or plot) are correctly retrieved from Emby directly
    • removed items are correctly retrieved from Kodi companion (not supported by Emby)
  • not working
    • items with changed user data (watched / unwatched, resume point, ...) are not correctly retrieved from Emby directly. They simply don't appear in the list.

@Angelblue05: is this supposed to work or am I doing something wrong? I read in the changelog of Kodi companion that UserDataChanged isn't populated anymore because it is supported by Emby directly. I can confirm that UserDataChanged is empty but Emby doesn't list the items as changed either.

 

 



#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 05 February 2020 - 01:10 PM

Can you give an example? Thanks.



#3 LongMan OFFLINE  

LongMan

    Advanced Member

  • Members
  • 169 posts
  • Local time: 06:08 AM

Posted 05 February 2020 - 09:50 PM

I asked him to provide some details.

 

I thinks he is looking for the Emby endpoint from which to obtain the data for a device that offline (wacthed/unwatched and resume points). He didn't find it in Kodi Companion.



#4 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 06 February 2020 - 09:39 AM

Hey Luke

 

I'm trying to use the MinDateLastSaved and MinDateLastSavedForUser arguments in the GET /Items API endpoint. In general it works great for getting only the items that changed since a specific time. But it does not seem to return items for which only the userdata (playcount, played, playback position) has changed.

 

To test it I did the following:

  1. I removed an existing movie from the library.
  2. I added a new movie to the library.
  3. I changed the plot of an existing movie in the library.
  4. I marked a movie as played in the library.
  5. I marked a movie as unplayed in the library.
  6. I started playback of a movie in the library and stopped halfway through.

When calling GET /Items with both MinDateLastSaved and MinDateLastSavedForUser set to a date before I executed action #1 I get the following items in the response:

  • #2 as a new movie
  • #3 as a changed movie

Obviously I don't get the removed movie but that is covered by the Kodi companion plugin.

But surprisingly I also don't get movies #4, #5 and #6 even though some of their (user)data have clearly changed.

 

Is this expected?



#5 LongMan OFFLINE  

LongMan

    Advanced Member

  • Members
  • 169 posts
  • Local time: 06:08 AM

Posted 07 February 2020 - 10:26 PM

@ Luke,

Is the details provided above enough to give us guidance?



#6 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 12 February 2020 - 03:55 PM

How did you put the values on the query string? Are they UTC? Did you try going a full day backwards as a test?



#7 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 12 February 2020 - 04:27 PM

The timestamps are in UTC. I can still make the exact same request and the result still doesn't contain the movies for which I changed the watched state / resume point.

 

This is the (redacted) request:

http://<IP>:<Port>/emby/Users/<UserId>/Items?api_key=<Token>&Recursive=true&Fields=PremiereDate%2CProductionYear%2CPath%2CSortName%2COriginalTitle%2CDateCreated%2CCommunityRating%2CVoteCount%2COfficialRating%2COverview%2CShortOverview%2CTaglines%2CGenres%2CStudios%2CProductionLocations%2CProviderIds%2CTags%2CPeople%2CRole%2CMediaStreams&ExcludeLocationTypes=Virtual%2COffline&Limit=100&MinDateLastSaved=2020-02-02T19%3A31%3A00%2B00%3A00&MinDateLastSavedForUser=2020-02-02T19%3A31%3A00%2B00%3A00&IncludeItemTypes=Movie&ParentId=<ParentId>&StartIndex=0


#8 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 12 February 2020 - 09:58 PM

Are you on Emby Server 4.3.1+?



#9 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 13 February 2020 - 01:47 AM

4.3.1



#10 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 13 February 2020 - 02:16 AM

can you use a sqlite viewer to open up your library.db file and look at the userdatas table? order it by playstatelastmodified. do you see your played items there?



#11 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 13 February 2020 - 02:19 AM

I already tried that with sqliteadmin but it says it can't open a database with a closed dataset. I tried with Emby server running and stopped. What sqlite viewer are you using?



#12 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 13 February 2020 - 01:50 PM

On Windows DB Browser for Sqlite, but yes you need to stop emby server while you do this.


  • LongMan likes this

#13 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 13 February 2020 - 05:19 PM

Yup that worked. I checked the UserDatas table and the three movies that I manually modified have the proper states (one unplayed, one played and one with a resume position and their PlaystateLastModified is 1580675670, 1580675794 and 1580675906 which correspond to 2020-02-02 20:34:30, 20:36:34 and 20:38:26 all of which are newer than the MinDateLastSavedForUser I set to "2020-02-02T19:31:00+00:00" which is 1580671860.

 

If I manually run the follow SQL query I get all 5 expected UserDatas entries:

SELECT * FROM UserDatas WHERE PlaystateLastModified > 1580671860

  • LongMan likes this

#14 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 21 February 2020 - 07:12 PM

OK let's get to the bottom of this. We need to know if the db query is using the date and returning the wrong result, or if it's not using the date at all, perhaps due to a parsing error at the api level.

 

Can you enable debug logging under the logs section? Then shutdown the server, and can you get to the server's /programdata/config/system.xml file and set LogAllQueryTimes to true?

 

After doing this the db query will show up in the log. That should give us our answer. Thanks.



#15 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 21 February 2020 - 07:20 PM

2020-02-22 00:16:37.578 Info HttpServer: HTTP GET http://localhost:8097/emby/Users/<UserId>/Items?Recursive=true&Fields=PremiereDate,ProductionYear,Path,SortName,OriginalTitle,DateCreated,CommunityRating,VoteCount,OfficialRating,Overview,ShortOverview,Taglines,Genres,Studios,ProductionLocations,ProviderIds,Tags,People,Role,MediaStreams&ExcludeLocationTypes=Virtual,Offline&Limit=100&MinDateLastSaved=2020-02-02T19:31:00+00:00&MinDateLastSavedForUser=2020-02-02T19:31:00+00:00&IncludeItemTypes=Movie&ParentId=f137a2dd21bbc1b99aa5c0f6bf02a805&StartIndex=0. UserAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36
2020-02-22 00:16:37.587 Debug SqliteItemRepository: GetItems query time: 1ms. Query: select A.Id,A.CommunityRating,A.Name,A.Path,A.PremiereDate,A.Overview,A.ProductionYear,A.OfficialRating,A.ForcedSortName,A.RunTimeTicks,A.Size,A.Container,A.DateCreated,A.guid,A.ParentId,A.OriginalTitle,A.Tagline,A.ProviderIds,A.Images,A.ProductionLocations,A.TotalBitrate,UserDatas.Rating,UserDatas.IsFavorite,UserDatas.Played,UserDatas.PlayCount,UserDatas.PlaybackPositionTicks,UserDatas.LastPlayedDateInt,UserDatas.AudioStreamIndex,UserDatas.SubtitleStreamIndex from MediaItems A left join UserDatas on A.UserDataKeyId=UserDatas.UserDataKeyId And (UserDatas.UserId=@UserId) where type=@type AND (DateLastSaved not null and DateLastSaved>=@MinDateLastSaved) AND ((DateLastSaved not null and DateLastSaved>=@MinDateLastSavedForUser) or (RatingLastModified not null and RatingLastModified>=@MinDateLastSavedForUser) or (PlaystateLastModified not null and PlaystateLastModified>=@MinDateLastSavedForUser)) AND TopParentId=@TopParentId Group by PresentationUniqueKey LIMIT 100
2020-02-22 00:16:37.589 Debug SqliteItemRepository: GetItems-TotalRecordCount query time: 1ms. Query:  select count (distinct PresentationUniqueKey) from MediaItems A left join UserDatas on A.UserDataKeyId=UserDatas.UserDataKeyId And (UserDatas.UserId=@UserId) where type=@type AND (DateLastSaved not null and DateLastSaved>=@MinDateLastSaved) AND ((DateLastSaved not null and DateLastSaved>=@MinDateLastSavedForUser) or (RatingLastModified not null and RatingLastModified>=@MinDateLastSavedForUser) or (PlaystateLastModified not null and PlaystateLastModified>=@MinDateLastSavedForUser)) AND TopParentId=@TopParentId


#16 Montellese OFFLINE  

Montellese

    Newbie

  • Members
  • 9 posts
  • Local time: 07:08 AM

Posted 21 February 2020 - 07:30 PM

Looking at the query i think the problem is that the WHERE condition is an AND combination of MinDateLastSaved and MinDateLastSavedForUser. So if I only change the userdata of an item the part filtering for MinDateLastSavedForUser matches these items but then it is AND-combined with MinDateLastSaved for the item which does not match because the "normal metadata" of the item hasn't been updated. If I remove MinDateLastSaved from the query I get all the changed items even the ones where I e.g. only changed the plot.

 

That doesn't seem very intuitive to me but it looks like I can get what I want by only passing MinDateLastSavedForUser and omitting MinDateLastSaved.



#17 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 149390 posts
  • Local time: 01:08 AM

Posted 21 February 2020 - 07:42 PM

Ok yea they're just independent params that don't check each other to see if specified. Glad we finally chased it down. Thanks for helping out.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users