chromecast fails due to private IP

[ssube 0]

I have an Emby server running as a Kubernetes container. For most purposes it works fine, and behaves as a normal Docker container, except for the network: each container is assigned a private IP in the cluster's CIDR when it starts or restarts. This is a similar issue to https://emby.media/community/index.php?/topic/56771-chromecast-address/, but router configuration or port forwarding are not possible here, thanks to the unpredictable and ephemeral container IP.

 

Every app that allows you to specify the server address works perfectly (web app on Chrome, native app on Android and Xbox, etc). However, Chromecast is broken: it stays at the "Ready to cast" screen and never plays media, nor do I see the Chromecast in the Emby server's log (attached). Casting was one of my main reasons for buying Premiere, so this is a bit of a problem.

 

Looking at the server settings, it seems that Emby is using the cluster's private, ephemeral IP to connect instead of the DNS entry used by other devices.

Running on http port 8096.
In-Home (LAN) access: http://10.5.0.244:8096
Remote (WAN) access: http://media.home.holdmyran.ch:80

The 10.5/16 and 10.6/16 ranges are not routable from outside of the k8s cluster - my LAN uses 10.2/16 - and so one node runs an nginx ingress controller as a reverse proxy. The media.home.holdmyran.ch:80 address is correct and is what I use in the working apps.

 

Forwarding from the proxy to Emby works perfectly, as do apps where the server address is explicitly set. The nginx forwarding is set up via:

kubectl describe svc emby
Name:                     emby
Namespace:                default
Labels:                   k8s-app=emby
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"k8s-app":"emby"},"name":"emby","namespace":"default"},"spec":{...
Selector:                 k8s-app=emby
Type:                     NodePort
IP:                       10.6.119.245
Port:                     http  9180/TCP
TargetPort:               http/TCP
NodePort:                 http  32284/TCP
Endpoints:                10.5.0.244:8096
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>


kubectl describe ingress emby
Name:             emby
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
TLS:
  media-home-holdmyran-ch-tls terminates media.home.holdmyran.ch
Rules:
  Host                     Path  Backends
  ----                     ----  --------
  media.home.holdmyran.ch  
                           /   emby:http (<none>)
Annotations:
  certmanager.k8s.io/acme-challenge-type:            http01
  certmanager.k8s.io/cluster-issuer:                 letsencrypt-production
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"certmanager.k8s.io/acme-challenge-type":"http01","certmanager.k8s.io/cluster-issuer":"letsencrypt-production","nginx.ingress.kubernetes.io/ssl-redirect":"false"},"labels":{"k8s-app":"emby"},"name":"emby","namespace":"default"},"spec":{"rules":[{"host":"media.home.holdmyran.ch","http":{"paths":[{"backend":{"serviceName":"emby","servicePort":"http"},"path":"/"}]}}],"tls":[{"hosts":["media.home.holdmyran.ch"],"secretName":"media-home-holdmyran-ch-tls"}]}}

  nginx.ingress.kubernetes.io/ssl-redirect:  false
Events:                                      <none>

Adding to my confusion, the Emby logs show a combination of the remote address and local port even on successful (Chrome) requests:

2019-05-05 13:18:40.477 Info HttpServer: HTTP GET http://media.home.holdmyran.ch:8096/emby/Users/ac832bece75e4fe29af3a4c7b0b42952. UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
2019-05-05 13:18:40.478 Info HttpServer: HTTP Response 200 to 10.2.1.193. Time: 1ms. http://media.home.holdmyran.ch:8096/emby/Users/ac832bece75e4fe29af3a4c7b0b42952
2019-05-05 13:18:40.480 Info HttpServer: HTTP GET http://media.home.holdmyran.ch:8096/emby/Users/ac832bece75e4fe29af3a4c7b0b42952. UserAgent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36
2019-05-05 13:18:40.482 Info HttpServer: HTTP Response 200 to 10.2.1.193. Time: 1ms. http://media.home.holdmyran.ch:8096/emby/Users/ac832bece75e4fe29af3a4c7b0b42952

Is there a way to convince Chromecast to use the correct (remote) address as well?

 

 

Server log and pod description attached, k8s resource YAMLs are all in https://github.com/ssube/k8s-shards/tree/master/roles/apps/emby/templates

emby-20190505.txt

emby-pod.txt

Edited May 5, 2019 by ssube

[Luke 37057]

Hi, does the emby server dashboard display the correct remote address?

[ssube 0]

Yes, I configured the external domain and public port, and the remote address displayed is correct and accessible anywhere on the LAN (10.2/16).

Edited May 5, 2019 by ssube

[Luke 37057]

Is the local address displayed correctly as well?

[ssube 0]

No, the local address seems to be the problem:

 In-Home (LAN) access: http://10.5.0.244:8096 

Which is a private IP that is not visible by most devices on the LAN and changes somewhat regularly.

[fuzzthekingoftrees 10]

I think the chromecast always uses Google DNS regardless of how you have your network setup. If the DNS can't be resolved externally then the chromecast probably falls back to the LAN address which won't work for you. If you block the chromecast from accessing 8.8.8.8 and 8.8.4.4 on your firewall then I think it will fall back to the DNS provided by DHCP.

[Luke 37057]

No, the local address seems to be the problem:

 In-Home (LAN) access: http://10.5.0.244:8096 

Which is a private IP that is not visible by most devices on the LAN and changes somewhat regularly.

 

So it's visible to the casting device, but not the chromecast receiver?

[ssube 0]

If by "casting device" you mean the browser/phone I'm starting to cast from, then no.

 

The

10.5.0.244

address is only visible to the Emby, in this situation, and both the browser and chromecast use (or should be using) the

media.home.holdmyran.ch

address.

 

For most purposes, my Emby container is on its own network (10.5), which the other devices (on 10.2) cannot see. I want them all to use the remote WAN address, which is served by the DNS server on the 10.2 network (set over DHCP).

[Luke 37057]

@@ssube is the browser using the address that you want? have you tried by external ip instead of host name, in case chromecast may not be able to resolve the ip address? 

[ssube 0]

Sorry, been moving recently.

 

The browser is using the correct address, which is an nginx virtual host, so forcing it to use the IP might be tricky. It does seem like the Chromecast is unable to resolve, so blocking Google DNS is worth a try. I will look into both and report back soon.

[Luke 37057]

Sorry, been moving recently.

 

The browser is using the correct address, which is an nginx virtual host, so forcing it to use the IP might be tricky. It does seem like the Chromecast is unable to resolve, so blocking Google DNS is worth a try. I will look into both and report back soon.

 

@@ssube how did this go?

[Hoptional 57]

Did you ever resolve your issue? I have the same problem: Chromecast is trying to connect via the local IP, instead of the remote address.

[Luke 37057]

Is this in the web app?

[Luke 37057]

Nevermind, duplicate posting of: https://emby.media/community/index.php?/topic/81737-app-tries-to-cast-to-chromecast-using-the-servers-local-instead-of-the-remote-address/?view=getnewpost&fromsearch=1

[ssube 0]

I had it working once or twice, but was never able to reliably fix this. Tested a few different DNS configurations, but launching it from the app always failed. I got it to work once or twice using the web player, but only when chrome decided to use content optimization; not sure how that changed things.