Fetching metadata via system proxy

[n00b_extrodinaire 0]

Hi all

 

Literally just finished a docker install of Emby, and at the moment everything seems to be working nicely, BUT, I cannot retrieve metadata. I am behind a proxy, and have no VPN/other way of connecting to the internet. I have a system proxy configured, but it seems Emby is not taking it into account. It may be that it is because Emby is not aware of it via Docker container, and I am not sure if adding it via an environment variable will help?

 

In the past, with Java based products, I was able to add "-Dhttp.proxy" option to the launching parameters, and it would also help. Is there some similar way in doing it with Emby?

 

I will install Emby as a "raw" server on my Fedora machine, if the Metadata will be pulled via proxy. I just used Docker to see whether or not the product works, and it looks nice.

 

Any ideas would be greatly appreciated!

[Luke 39631]

Hi, have you tried turning off the proxy?

[n00b_extrodinaire 0]

Hi, have you tried turning off the proxy?

I can't. It is the only way I am able to access the internet. This was kind of the question, sorry if I did not make it clear enough.

 

The question basically is, if I have no other choice, how can I make Emby honour the system proxy settings?

[Luke 39631]

How did you configure the network on the docker container? I would try host networking mode.

[mastrmind11 722]

what is a "system proxy"?

[n00b_extrodinaire 0]

what is a "system proxy"?

In Linux you can configure a proxy for all type of protocols, http, https, ftp, socks etc. This means that all apps honouring the default network setrings, would use the system proxy to access the internet. However, not all systems honour this, like wget would sometimes need a proxy config in /etc/wgetrc, or curl needs /etc/curlrc. So not all apps honour system proxy, but most do, and hence the hope that Emby does.

 

@@Luke Honestly, not sure. I pulled the default docker image, and only changed the volumes and id of user and group. Could not find additional details about host vs bridge etc documentation. Sorry if I was careless and missed it. Could you point me in the direction of where to look?

 

Do you perhaps have any idea whether the code would/should honour system proxies?

 

Thabks for all the replies guys! Really appreciate it!

[Luke 39631]

It is a setting on the docker container

[n00b_extrodinaire 0]

Will do that in the morning and report back. Thanks.

Edited October 12, 2018 by n00b_extrodinaire

[n00b_extrodinaire 0]

Nope, still nothing.

 

 

2018-10-12 07:54:24.394 Error HttpClient: Error getting response from https://api.github.com/repos/MediaBrowser/Emby.Releases/releases
    *** Error Report ***
    Version: 3.5.3.0
    Command line: /system/EmbyServer.dll -programdata /config -ffmpeg /bin/ffmpeg -ffprobe /bin/ffprobe -restartexitcode 3
    Operating system: Unix 4.16.11.100
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Processor count: 4
    Program data path: /config
    Application directory: /system
    System.Net.Http.HttpRequestException: Network is unreachable ---> System.Net.Sockets.SocketException: Network is unreachable
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
    System.Net.Http.HttpRequestException
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
    InnerException: System.Net.Sockets.SocketException
    System.Net.Sockets.SocketException (101): Network is unreachable
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

 

Any other ideas? If this is the case, it will not help installing it straight onto my OS either.

[Luke 39631]

Did you start the server after setting the networking mode to host?

[n00b_extrodinaire 0]

Did you start the server after setting the networking mode to host?

Yes, I restarted the docker container?

 

I stopped the container, changed the networking to host with the docker run -d --net=host command. Does that not automatically restart the server? Or is there some cache I have to clear?  After starting the container, I went to the server's web page, and forced a metadata refresh to initiate a web lookup, and it seems in the logs that even the updating does not work due to network unreachable.

 

Is there something else I should do after starting up the container for it to read new --net=host setting?

[mastrmind11 722]

which proxy are you using?  post your config.  Can you reach anything outside the network from within the container?  ssh into the container and try pinging some stuff.  

docker exec -i -t container_name /bin/bash

[n00b_extrodinaire 0]

 

which proxy are you using?  post your config.  Can you reach anything outside the network from within the container?  ssh into the container and try pinging some stuff.  

docker exec -i -t container_name /bin/bash

 

That command did not work. But docker exec -i -t container_name sh did. Due to the nature of the network setup, Pinging stuff like google.com from bot the host and the container will not work. So that test does not resolv anything really. However, I did confirm that from inside the container, everything reachable by the host, is reachable by the container.

 

Post config of what? Emby? I have very little configured.

 

Regarding the proxy I use, it is a self setup squid.

 

Like I asked in the first post, I have quite a fair bit of experience about running application behind a proxy server. I know what to do in cases like curl, or wget. But then one also gets the options like in Java applications where, if the Java application is executed with a list of options in the form of -Doption.name=XXX, I am capable of adding my proxy options to the end of that list of options. My question is, is there a similar option list for Emby? Lets forget about the docker container for now. Lets pretend I install it onto my Fedora system directly. Is there a file where I could try and make the application use the squid as a proxy to the internet by forcing it? Or does such a file not exist?

 

Thanks for the help so far!

Edited October 12, 2018 by n00b_extrodinaire

[mastrmind11 722]

That command did not work. But docker exec -i -t container_name sh did. Due to the nature of the network setup, Pinging stuff like google.com from bot the host and the container will not work. So that test does not resolv anything really. However, I did confirm that from inside the container, everything reachable by the host, is reachable by the container.

 

Post config of what? Emby? I have very little configured.

 

Regarding the proxy I use, it is a self setup squid.

 

Like I asked in the first post, I have quite a fair bit of experience about running application behind a proxy server. I know what to do in cases like curl, or wget. But then one also gets the options like in Java applications where, if the Java application is executed with a list of options in the form of -Doption.name=XXX, I am capable of adding my proxy options to the end of that list of options. My question is, is there a similar option list for Emby? Lets forget about the docker container for now. Lets pretend I install it onto my Fedora system directly. Is there a file where I could try and make the application use the squid as a proxy to the internet by forcing it? Or does such a file not exist?

 

Thanks for the help so far!

look in the /etc/default/docker file.  There'a an http_proxy line that is commented out by default.  Uncomment it and add your proxy info.

 

you'll have to restart your containers for the change to take effect.

Edited October 12, 2018 by mastrmind11

[n00b_extrodinaire 0]

Excellent news! Will have a look! Thank you soooooo much. Will report back tomorrow! Sorry for delays, will let you know!

[n00b_extrodinaire 0]

Upon research it seems the /etc/default/docker file is only to make docker process itself run commands via proxy.

 

As I have asked before: Is there any parameters I can add to the launch of Emby as I would do with Java applications, or not? Forget about docker for the moment. What would I do if I installed the RPM directly on my system and the metadata is not begin fetched?

 

Thanks again.

[Luke 39631]

We don't have any such params because I would think it would already honor this.

[n00b_extrodinaire 0]

We don't have any such params because I would think it would already honor this.

It would seem it doesn't, which is a pity. Thanks for all the answers, I appreciate it. Unfortunately it is a deal breaker. But thanks again to who have replied.

[Luke 39631]

I would try the native package.

[n00b_extrodinaire 0]

Nope, still nothing. Updates fail, and found this in the log file:

 

 

2018-10-16 12:03:56.939 Info HttpClient: GET https://api.github.com/repos/MediaBrowser/Emby.Releases/releases
2018-10-16 12:03:56.940 Info HttpClient: GET https://www.mb3admin.com/admin/service/EmbyPackages.json
2018-10-16 12:03:56.961 Error HttpClient: Error getting response from https://www.mb3admin.com/admin/service/EmbyPackages.json
    *** Error Report ***
    Version: 3.5.3.0
    Command line: /opt/emby-server/system/EmbyServer.dll -programdata /var/lib/emby -ffmpeg /opt/emby-server/bin/ffmpeg -ffprobe /opt/emby-server/bin/ffprobe -restartexitcode 3 -updatepackage emby-server-rpm_{version}_x86_64.rpm
    Operating system: Unix 4.16.11.100
    64-Bit OS: True
    64-Bit Process: True
    User Interactive: True
    Processor count: 4
    Program data path: /var/lib/emby
    Application directory: /opt/emby-server/system
    System.Net.Http.HttpRequestException: Network is unreachable ---> System.Net.Sockets.SocketException: Network is unreachable
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
    System.Net.Http.HttpRequestException
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.CreateConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.WaitForCreatedConnectionAsync(ValueTask`1 creationTask)
       at System.Threading.Tasks.ValueTask`1.get_Result()
       at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.DecompressionHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
       at Emby.Server.Implementations.HttpClientManager.CoreHttpClientManager.SendAsyncInternal(HttpRequestOptions options, String httpMethod)
    InnerException: System.Net.Sockets.SocketException
    System.Net.Sockets.SocketException (101): Network is unreachable
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
       at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)

 

So yeah, native doesn't work either. Thanks again guys. It is a pity, seems like nice software, but yeah, not gonna work. Thanks again for the patience.