Jump to content
registor

Possibility to Use a Proxy for Scrapers

Recommended Posts

registor
Posted (edited)

Hi there, I've installed Emby server on my QNAP TS-231P2, it worked well. However, recently due to the internet blockage, when Emby tried to pull metadata from TMDB, it always got a time-out. So I'm thinking of using a proxy to solve the problem. Unfortunately, I didn't find any HTTP proxy settings in Emby control panel. Therefore I decided to install shadowsocks and privoxy to turn Socks5 proxy into a HTTP one. Too bad the privoxy rules just didn't work. Even after I added HTTP_proxy and HTTPS_proxy into the environment variables, and the command like curl or wget worked fine for api.themoviedb.org, the traffic from Emby seemed not to pass through the proxy and got time-out again. 

I wonder if there is any other means to solve the problem, either from a system level or software level. Also, I'd like to know if you are adding proxy settings in the future versions of Emby. Thanks for your help in advance. Any suggestions are welcome.

 

A piece of log is attached to show the typical situation.

 

2020-04-20 19:58:13.506 Info HttpClient: GET https://api.themoviedb.org/3/movie/342588?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers&language=zh-CN&include_image_language=zh-CN,zh,null,en
2020-04-20 19:58:33.521 Error HttpClient: Connection to https://api.themoviedb.org/3/movie/342588?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers&language=zh-CN&include_image_language=zh-CN,zh,null,en timed out
2020-04-20 19:58:33.590 Error ProviderManager: Error searching
    *** Error Report ***
    Version: 4.3.1.0
    Command line: /share/CACHEDEV1_DATA/.qpkg/EmbyServer/system/EmbyServer.dll -programdata /share/CACHEDEV1_DATA/.qpkg/EmbyServer/programdata -ffdetect /share/CACHEDEV1_DATA/.qpkg/EmbyServer/bin/ffdetect -ffmpeg /share/CACHEDEV1_DATA/.qpkg/EmbyServer/bin/ffmpeg -ffprobe /share/CACHEDEV1_DATA/.qpkg/EmbyServer/bin/ffprobe -defaultdirectory /share/CACHEDEV1_DATA -updatepackage emby-server-qnap_{version}_arm-x41.qpkg -noautorunwebapp
    Operating system: Unix 4.2.8.0
    64-Bit OS: False
    64-Bit Process: False
    User Interactive: True
    Runtime: file:///share/CACHEDEV1_DATA/.qpkg/EmbyServer/system/System.Private.CoreLib.dll
    Processor count: 4
    Program data path: /share/CACHEDEV1_DATA/.qpkg/EmbyServer/programdata
    Application directory: /share/CACHEDEV1_DATA/.qpkg/EmbyServer/system
    MediaBrowser.Model.Net.HttpException: MediaBrowser.Model.Net.HttpException: Connection to https://api.themoviedb.org/3/movie/342588?api_key=f6bd687ffa63cd282b6ff2c6877f2669&append_to_response=casts,releases,images,keywords,trailers&language=zh-CN&include_image_language=zh-CN,zh,null,en timed out ---> System.OperationCanceledException: The operation was canceled.
     at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
     at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
     at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
     --- End of inner exception stack trace ---
     at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
     at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsync(HttpRequestOptions options, String httpMethod)
     at MovieDb.MovieDbProvider.GetMovieDbResponse(HttpRequestOptions options)
     at MovieDb.MovieDbProvider.FetchMainResult(String id, Boolean isTmdbId, String language, String country, CancellationToken cancellationToken)
     at MovieDb.MovieDbProvider.DownloadMovieInfo(String id, String preferredMetadataLanguage, String preferredMetadataCountry, CancellationToken cancellationToken)
     at MovieDb.MovieDbProvider.GetMovieSearchResults(ItemLookupInfo searchInfo, CancellationToken cancellationToken)
     at MediaBrowser.Providers.Manager.ProviderManager.GetSearchResults[TLookupType](IRemoteSearchProvider`1 provider, TLookupType searchInfo, CancellationToken cancellationToken)
     at MediaBrowser.Providers.Manager.ProviderManager.GetRemoteSearchResults[TItemType,TLookupType](RemoteSearchQuery`1 searchInfo, BaseItem referenceItem, CancellationToken cancellationToken)
    Source: Emby.Server.Implementations
    TargetSite: Void MoveNext()
    InnerException: System.OperationCanceledException: The operation was canceled.
    Source: System.Net.Http
    TargetSite: Void HandleFinishSendAsyncError(System.Exception, System.Threading.CancellationTokenSource)
     at System.Net.Http.HttpClient.HandleFinishSendAsyncError(Exception e, CancellationTokenSource cts)
     at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
     at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)

Edited by registor

Share this post


Link to post
Share on other sites
Luke

HI there, yes it's possible for the future. thanks.

Share this post


Link to post
Share on other sites
registor

HI there, yes it's possible for the future. thanks.

Thanks for your reply! Is there any solutions to this problem for now?

Share this post


Link to post
Share on other sites
Luke

I guess you'd have to use a vpn.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...