Jump to content
ramonrue

ANSWERED Local Network on Kubernetes

Recommended Posts

ramonrue

Hi guys,

 

So I've seen that quite a few are already running Emby in a Kubernetes Cluster.

My setup currently looks like this:

- One Kubernetes Node (I'm using K8s because I'm familiar with it and I like writing deployment-manifests :-) )

- A standard router from my ISP, doing port-forwards on HTTP & HTTPS

- MetalLB assigning virtual IPs for loadbalancers (L2)

- Nginx Ingress using a LoadBalancer, so it gets a virtual IP. uses Service with `externalTrafficPolicy: local`.

- Emby using an Ingress-Resource, HTTPS, letsencrypt certificates.

 

My issue though is that it seems like Emby considers everything to be the "local" network.

From skimming through the logs:

2019-11-05 19:40:11.692 Info HttpServer: HTTP GET http://<mydomain>:8096/emby/Users/<key>. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.2 Safari/605.1.15
2019-11-05 19:40:11.695 Info HttpServer: HTTP Response 200 to 127.0.0.1. Time: 3ms. http://<mydomain>:8096/emby/Users/<key>
A few things:
My Emby instance is not reachable under http://<mydomain>:8096, but rather on https://<mydomain>:443 (through the Ingress).
 
It seems like Emby responds to the request to localhost??
This would explain why Emby considers everything to be local network.
Anybody an idea why?
 
Or what did you guys set to actually pass through the Source IP?
Like mentioned before, I'm already using `externalTrafficPolicy: Local` for the nginx-ingress service.
 
Also, for reference, the deployment yaml:
apiVersion: app/v1
kind: Deployment
metadata:
  name: emby-server
  labels:
    app: emby
spec:
  replicas: 1
  selector:
    matchLabels:
      app: emby
  template:
    metadata:
      labels:
        run: emby-server
        app: emby
    spec:
      containers:
      - name: emby-server
        image: emby/embyserver:latest
        env:
          - name: "UID"
            value: "1030" # emby user
          - name: "GID"
            value: "1111" # share group
        ports:
        - containerPort: 8096
          name: emby-http
        volumeMounts:
        - mountPath: /config
          name: emby-config
        - mountPath: /mnt/media
          name: emby-media
      volumes:
      - name: emby-media
        hostPath:
          type: Directory
          path: /media
      - name: emby-config
        hostPath:
          type: DirectoryOrCreate
          path: /mnt/cluster/emby

Total noob with emby, deployed it a couple of days ago. 

 

Any help would be appreciated.

Share this post


Link to post
Share on other sites
Luke

Hi, this is because Emby is seeing 127.0.0.1 as the remote ip of the client device.

 

I see you're using Nginx. I would suggest comparing your nginx setup to @@pir8radio.

 

Please let us know if this helps. Thanks.

Share this post


Link to post
Share on other sites
ramonrue

Thanks for the reply.

 

Not quite sure how it is with nginx ingress, as it already is configured as a reverse proxy.

you can find all config options for the ingress here: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

there's no such thing as the X-Real-IP. I suppose this is because that's what it's already doing.

I now only added the following annotations to my ingress:

   nginx.ingress.kubernetes.io/use-forwarded-headers: "false"

   nginx.ingress.kubernetes.io/forwarded-for-header: "true"

 

 

However, this did not fix my issue.
 
What is weird is that emby sees the localhost-IP as the remote IP. It is weird because the Emby Pod (Container) has its own IP address, which is different from the nginx IP address.
 
I'm sure I'm not the first one trying to set up emby on kubernetes, using Ingress :-)

Share this post


Link to post
Share on other sites
pir8radio

 

Thanks for the reply.

 

Not quite sure how it is with nginx ingress, as it already is configured as a reverse proxy.

you can find all config options for the ingress here: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

there's no such thing as the X-Real-IP. I suppose this is because that's what it's already doing.

I now only added the following annotations to my ingress:

   nginx.ingress.kubernetes.io/use-forwarded-headers: "false"

   nginx.ingress.kubernetes.io/forwarded-for-header: "true"

 

 

However, this did not fix my issue.
 
What is weird is that emby sees the localhost-IP as the remote IP. It is weird because the Emby Pod (Container) has its own IP address, which is different from the nginx IP address.
 
I'm sure I'm not the first one trying to set up emby on kubernetes, using Ingress :-)

 

 

it looks like ingress just builds the nginx configs for you...  you will need to take it up with them if they don't support additional X headers like x forwarded for and x real ip. NGINX supports this standard header.   It's needed for proxied services behind nginx to know who is connecting to them.   I say loose ingress it doesn't seem to provide anything on it's own that nginx doesn't already do including load balancing.   otherwise open a ticket with ingress/Kubernetes.

Share this post


Link to post
Share on other sites
ramonrue

Alright, figured it out!

 

Had to add the following to nginx-ingress's configMap:

data:  
  use-proxy-protocol: "true"
  proxy-real-ip-cidr: "0.0.0.0/32"
  use-forwarded-headers: "false"

thanks for the hints.

 

EDIT: actually, what's needed is only the `use-proxy-protocol`, as MetalLB seems to use that.

Edited by ramonrue
  • Like 1

Share this post


Link to post
Share on other sites
Luke

That's great, thanks for the feedback !

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...