BaseItem and UserData and Caching

[rhodges 39]

I was looking into the best way to deal with pulling user data for larger lists and got to digging into the Emby source.

 

I see in SqliteItemRepository, GetItemList, it calls GetFinalColumnsToSelect, and that method will add UserDataDb.UserData.UserId and the others to the list of the columns, but there is nothiing in GetItem(reader), nor are there columns in BaseItem to put them.

 

It seems that if we are pulling the data (when EnableJoinUserData) it would be good to return that up to the caller in BaseItem somewhere since we have already pulled it from the data store.

 

On a side note, I stumbled on this because I was loading all the user data UserDataManager.GetAllUserData(User.Id).ToList() and then when I converted from the MB.TV.Episode, I would use the GetItemDataKeys method to find the UserData in the list. It worked, but even that was very slow. For the show, Stargate Atlantis, on my computer, it would take 2.5 seconds. I noticed this was because Episode.GetItemDataKeys called to Episode.Series.GetItemDataKeys. For every Episode, it would load the Series information again just to pull the data keys. I hacked my code to duplicate the GetItemDataKeys functionality but cache the series. This of course significantly decreased my calls to the datastore. I went from 2.5 seconds to sub-second. (like 0.3 or something crazy).

 

This is what led me to looking at it and think, if we are pulling that in GetFinalColumnsToSelect, why not expose it in BaseItem. Maybe I'm missing something in that thought process?

 

What about, caching objects per request. That might be a bigger pain because then you have to worry about stale data. I think it if it was just done per request, maybe that would minimize the changes of having stale data?

[Luke 37188]

Currently at the api level the caller supplies the fields they want, and library items and user data are passed out of the api as one object. the library database is currently only using the user data for filtering, and it's the api layer along with DtoService that merges it all for return. The joining on the userdata table for filtering is relatively new and the next evolutionary step here is probably removing that middle internal layer and getting everything out of the database directly, including support for optional fields so that we can just select only the columns that we need. So yes, your suggestion is exactly where we want to go.