OpenSubtitles API request limit

[matejz 0]

Hey there,

 

does Emby server take into account the OpenSubtitles API requests limit (not subtitles download limit) when downloading subtitles as a task?

 

I'm a VIP member at OpenSubtitles and when I run the subtitles download task, after a few 10s of seconds, I start getting TooManyRequests HTTPExceptions.

2019-02-06 23:38:22.195 Info HttpClient: POST https://api.opensubtitles.org/xml-rpc
2019-02-06 23:38:22.209 Error SubtitleManager: Error downloading subtitles from Open Subtitles
	*** Error Report ***
	Version: 3.5.3.0
	Command line: /opt/emby-server/system/EmbyServer.dll -programdata /media-ssd/emby/ -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-rpm_{version}_x86_64.rpm
	Operating system: Unix 3.10.0.957
	64-Bit OS: True
	64-Bit Process: True
	User Interactive: True
	Processor count: 8
	Program data path: /media/emby/
	Application directory: /opt/emby-server/system
	MediaBrowser.Model.Net.HttpException: TooManyRequests
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod)
	   at OpenSubtitles.OpenSubtitleDownloader.SearchSubtitlesAsync(OpenSubtitleQuery[] parameters, CancellationToken cancellationToken)
	   at OpenSubtitles.OpenSubtitleDownloader.Search(SubtitleSearchRequest request, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Subtitles.SubtitleManager.SearchSubtitles(SubtitleSearchRequest request, CancellationToken cancellationToken)
	MediaBrowser.Model.Net.HttpException
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
	   at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod)
	   at OpenSubtitles.OpenSubtitleDownloader.SearchSubtitlesAsync(OpenSubtitleQuery[] parameters, CancellationToken cancellationToken)
	   at OpenSubtitles.OpenSubtitleDownloader.Search(SubtitleSearchRequest request, CancellationToken cancellationToken)
	   at MediaBrowser.Providers.Subtitles.SubtitleManager.SearchSubtitles(SubtitleSearchRequest request, CancellationToken cancellationToken)

According to OpenSubtitles API documentation, there is a limit of 40 HTTP requests per 10 seconds per IP address, even for VIP users.

 

More info can be found on https://forum.opensubtitles.org/viewtopic.php?f=8&t=16072

 

There is also a special XML RPC endpoint for VIP users with a more stable and faster server. More info in http://trac.opensubtitles.org/projects/opensubtitles/wiki/XMLRPC#LogIn

 

I'm using Emby Server 3.5 on a CentOS 7 linux.

 

Thanks, Matej

[Happy2Play 9780]

Your issue will still be you have to upgrade to v4 in order to get a updated plugin.  But have no idea if this issue still exists in plugin v1.0.11.0.

[Luke 42078]

We can look at incorporating the special server url for vip users.

[matejz 0]

Yea, I know I'll have to upgrade. I have upgrade planned, I'm just a little hassitated, since I'm also using Kodi and I'm not sure how stable new Emby is with latest Kodi.

 

Luke, do you know if the latest opensubtitles plugin in v4 support API request throttling? I think the special URL also has the same limits as the "standard" one.

Edited February 7, 2019 by matejz

[Luke 42078]

No, but once we switch to the special url we'll re-evaluate and see where we stand. thanks.

[matejz 0]

Great! 

[kikinjo 281]

Same issue here, vip user and timeout every now and than.

[matejz 0]

I think we are all hit by this API req limits.

 

Luke, when download subtitles task is running, does it search for subtitles for every movie/tv show, even if it already has subtitles?

[Luke 42078]

 

 

Luke, when download subtitles task is running, does it search for subtitles for every movie/tv show, even if it already has subtitles?

 

No.

[Luke 42078]

I've published an update to the OpenSubtitles plugin which now uses the special api for VIP members. Please try that. Thanks.

[matejz 0]

What is the plugin version? I'm still seeing 1.0.2.0 as the latest. Do I need to upgrade to Emby 4.0?

[Luke 42078]

Yes you'll need to update.

[kikinjo 281]

Cheers Luke, will test it today.

[kikinjo 281]

Latest beta, latest Opensubtitles plugin, checked VIP member

 

log:

 

https://pastebin.com/dAeqfbkk

[matejz 0]

The TooManyRequests problem is due to OpenSubtitles limiting number of API requests. It looks like this limit is there for VIP users as well.

[kikinjo 281]

Its working fine if u do manual search for subtitles (from movie for example). req limit pops up in log if i start scheduled task for download subs for whole library.

[matejz 0]

Yes, problems with req limit is only present when running as a task.

[Luke 42078]

Thanks for the info.