Jump to content


Photo

Emby4 Internet Channel plugin loads every video from API


  • Please log in to reply
4 replies to this topic

#1 billg OFFLINE  

billg

    Newbie

  • Members
  • 2 posts
  • Local time: 10:32 AM

Posted 17 September 2019 - 11:51 AM

I maintain a personal plugin which queries a remote API for videos by tag. The API can return several hundred (query count dependent upon plugin config) videos per tag and over one hundred tags.
 
Under 3.5.x the plugin would query the API in an on-demand fashion and only for the selected folder tag object only with direct user interaction. On 4.2.x (and possibly earlier although untested by myself) the plugin now queries the API for all tags and then all videos contained within that tag. Further it performs an automatic refresh per the "Refresh Internet Channels" task. The remote API frequently changes videos per tag which leads to massive local Emby DB churn plus a never ending remote API query/refresh cycle. As my plugin defaults to 40 videos per tag I now observe, under 4.2.x, it loading well over 4000 videos upon first install and then every 12 hours when the "Refresh Internet Channels" task fires it checks the status of those first 4000 videos and updates as needed.
 
Simplified example layout from Emby home screen-
Plugin folder
-Recent folder
--Video 1
--Video 2
--Video 3
--Video 4
--Video 5
--Video 6
--Video 7
-Tags folder
--Tag 1
---Video 1
---Video 2
---Video 3
---Video 4
---Video 5
---Video 6
--Tag 2
---Video 1
---Video 2
---Video 3
--Tag 3
---Video 1
---Video 2
---Video 3
---Video 4
--Tag 4
---Video 1
---Video 2
---Video 3
---Video 4
---Video 5
---Video 6
--Tag 5
---Video 1
---Video 2
---Video 2
Simplified example navigation-
  • 3.5.x
    • User enters the Plugin folder.
    • User enters the Tags folder.
      • The plugin queries the API for all tags available and then creates a folder item for each tag.
    • User enters the Tag 2 folder.
      • The plugin queries the API for all videos, up to plugin config count, and then creates a video item per result.
    • User plays back Video 2.
      • The plugin loads the video from the remote server and begins streaming playback.
    • User goes back and enters the Tag 5 folder.
      • The plugin queries the API for all videos, up to plugin config count, and then creates a video item per result.
    • User plays back Video 1.
      • The plugin loads the video from the remote server and begins streaming playback.
  • 4.2.x
    • Emby fires a refresh event and the plugin queries the API for all tags available and then creates a folder item for each tag.
    • The plugin queries the API for all videos, up to plugin config count, and then creates a video item per result for the Tag 1 folder.
    • This repeats for all remaining tag folders.
    • The scheduled task runs to refresh Internet Content.
    • User enters the Plugin folder.
    • User enters the Tags folder.
      • All tags are already present.
    • User enters the Tag 2 folder.
      • All videos are already present.
    • User plays back Video 2.
      • The plugin loads the video from the remote server and begins streaming playback.
    • User goes back and enters the Tag 5 folder.
      • Video content is out of date as the plugin is still refreshing all video content from the Tag 4 folder.
    • User plays back Video 1.
      • The plugin fails to load the video as it is no longer available despite displaying as so.
 
From a UX view the 4.2.x method is preferred as everything is pre-populated and available for immediate use. The downside, however, is that the "Refresh Internet Channels" task loads everything which leads to occasional API bans including content the user may never interact with.
 
Is there any documentation available on how to update the plugin for this new 4.x behavior which both unsubscribes the plugin from the "Refresh Internet Channels" task as well as allows the plugin to function only in an on-demand method?
 


#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 137605 posts
  • Local time: 11:32 AM

Posted 17 September 2019 - 04:43 PM

Hi, the reason for this is that by doing it on demand, any stalls coming from the channel plugin will hold up the user interface. This was a big problem for us because it was a constant source of troubleshooting.

 

Let me give an example. Consider something like the Next Pvr plugin channel. If the user shuts off their next pvr server, requests from the plugin will fail, and in some environments this may require having to wait a full 20-30 seconds for the requests to time out. This holds up the Emby user interface and leads to loading spinners, incomplete screens, etc.

 

Now, I'd be OK with this if users realized where the issue is. The problem is they don't, and they constantly come to us complaining of problems with Emby, Emby doesn't work, fix it, etc.

 

Eventually it just got to be too much and that's why I changed it to a background refresh.



#3 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 137605 posts
  • Local time: 11:32 AM

Posted 17 September 2019 - 04:43 PM

And additionally you can't integrate with other emby features such as sorting, filtering, etc if the content is not already in the database.



#4 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 137605 posts
  • Local time: 11:32 AM

Posted 17 September 2019 - 04:45 PM

 

 

allows the plugin to function only in an on-demand method?

 

There isn't, sorry. We just don't have the bandwidth to support the channels framework operating in two different modes, and I'm not sure we want to anyway. Changing this created a domino effect that led to more simplifications, performance improvements, etc. I would not want to give those up at this stage.



#5 billg OFFLINE  

billg

    Newbie

  • Members
  • 2 posts
  • Local time: 10:32 AM

Posted 18 September 2019 - 03:03 PM

Understood and that was my assumption that it was done for UX reasons.
 
The remote video content is not required to be in the local DB at all times for all videos and, due to update churn on the remote site, is preferred not to be so until a particular tag is selected. I struggle with a reason to preload 4000+ Internet streaming videos into the local DB when their churn rate is >100/day and less than 200 random videos will be watched during a month.
 
Is there any published plugin developer documentation available beyond "please refer to plugin X for guidance"? Swagger, from what I've seen, is geared towards API developers vs plugin developers.
 
You've given me enough to think about on how to re-architect the plugin to work in the 4.2.x line. I have a feeling I'll dump tags for now and instead rely on daily/recent.
 
Thank you for your replies.





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users