Jump to content


Photo

Local Network on Kubernetes

kubernetes network linux docker

Best Answer ramonrue , 09 November 2019 - 01:04 PM

Alright, figured it out!

 

Had to add the following to nginx-ingress's configMap:

data:  
  use-proxy-protocol: "true"
  proxy-real-ip-cidr: "0.0.0.0/32"
  use-forwarded-headers: "false"

thanks for the hints.

 

EDIT: actually, what's needed is only the `use-proxy-protocol`, as MetalLB seems to use that.

Go to the full post


  • Please log in to reply
5 replies to this topic

#1 ramonrue OFFLINE  

ramonrue

    Member

  • Members
  • 13 posts
  • Local time: 02:55 PM

Posted 05 November 2019 - 04:23 PM

Hi guys,

 

So I've seen that quite a few are already running Emby in a Kubernetes Cluster.

My setup currently looks like this:

- One Kubernetes Node (I'm using K8s because I'm familiar with it and I like writing deployment-manifests :-) )

- A standard router from my ISP, doing port-forwards on HTTP & HTTPS

- MetalLB assigning virtual IPs for loadbalancers (L2)

- Nginx Ingress using a LoadBalancer, so it gets a virtual IP. uses Service with `externalTrafficPolicy: local`.

- Emby using an Ingress-Resource, HTTPS, letsencrypt certificates.

 

My issue though is that it seems like Emby considers everything to be the "local" network.

From skimming through the logs:

2019-11-05 19:40:11.692 Info HttpServer: HTTP GET http://<mydomain>:8096/emby/Users/<key>. UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.2 Safari/605.1.15
2019-11-05 19:40:11.695 Info HttpServer: HTTP Response 200 to 127.0.0.1. Time: 3ms. http://<mydomain>:8096/emby/Users/<key>
A few things:
My Emby instance is not reachable under http://<mydomain>:8096, but rather on https://<mydomain>:443 (through the Ingress).
 
It seems like Emby responds to the request to localhost??
This would explain why Emby considers everything to be local network.
Anybody an idea why?
 
Or what did you guys set to actually pass through the Source IP?
Like mentioned before, I'm already using `externalTrafficPolicy: Local` for the nginx-ingress service.
 
Also, for reference, the deployment yaml:
apiVersion: app/v1
kind: Deployment
metadata:
  name: emby-server
  labels:
    app: emby
spec:
  replicas: 1
  selector:
    matchLabels:
      app: emby
  template:
    metadata:
      labels:
        run: emby-server
        app: emby
    spec:
      containers:
      - name: emby-server
        image: emby/embyserver:latest
        env:
          - name: "UID"
            value: "1030" # emby user
          - name: "GID"
            value: "1111" # share group
        ports:
        - containerPort: 8096
          name: emby-http
        volumeMounts:
        - mountPath: /config
          name: emby-config
        - mountPath: /mnt/media
          name: emby-media
      volumes:
      - name: emby-media
        hostPath:
          type: Directory
          path: /media
      - name: emby-config
        hostPath:
          type: DirectoryOrCreate
          path: /mnt/cluster/emby

Total noob with emby, deployed it a couple of days ago. 

 

Any help would be appreciated.



#2 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 145887 posts
  • Local time: 08:55 AM

Posted 08 November 2019 - 12:40 AM

Hi, this is because Emby is seeing 127.0.0.1 as the remote ip of the client device.

 

I see you're using Nginx. I would suggest comparing your nginx setup to @pir8radio.

 

Please let us know if this helps. Thanks.



#3 ramonrue OFFLINE  

ramonrue

    Member

  • Members
  • 13 posts
  • Local time: 02:55 PM

Posted 09 November 2019 - 04:58 AM

Thanks for the reply.

 

Not quite sure how it is with nginx ingress, as it already is configured as a reverse proxy.

you can find all config options for the ingress here: https://kubernetes.g...on/annotations/

there's no such thing as the X-Real-IP. I suppose this is because that's what it's already doing.

I now only added the following annotations to my ingress:

   nginx.ingress.kubernetes.io/use-forwarded-headers: "false"

   nginx.ingress.kubernetes.io/forwarded-for-header: "true"

 

 

However, this did not fix my issue.
 
What is weird is that emby sees the localhost-IP as the remote IP. It is weird because the Emby Pod (Container) has its own IP address, which is different from the nginx IP address.
 
I'm sure I'm not the first one trying to set up emby on kubernetes, using Ingress :-)


#4 pir8radio OFFLINE  

pir8radio

    NGINX

  • Members
  • 2999 posts
  • Local time: 07:55 AM
  • LocationChicago

Posted 09 November 2019 - 12:15 PM

 

Thanks for the reply.

 

Not quite sure how it is with nginx ingress, as it already is configured as a reverse proxy.

you can find all config options for the ingress here: https://kubernetes.g...on/annotations/

there's no such thing as the X-Real-IP. I suppose this is because that's what it's already doing.

I now only added the following annotations to my ingress:

   nginx.ingress.kubernetes.io/use-forwarded-headers: "false"

   nginx.ingress.kubernetes.io/forwarded-for-header: "true"

 

 

However, this did not fix my issue.
 
What is weird is that emby sees the localhost-IP as the remote IP. It is weird because the Emby Pod (Container) has its own IP address, which is different from the nginx IP address.
 
I'm sure I'm not the first one trying to set up emby on kubernetes, using Ingress :-)

 

 

it looks like ingress just builds the nginx configs for you...  you will need to take it up with them if they don't support additional X headers like x forwarded for and x real ip. NGINX supports this standard header.   It's needed for proxied services behind nginx to know who is connecting to them.   I say loose ingress it doesn't seem to provide anything on it's own that nginx doesn't already do including load balancing.   otherwise open a ticket with ingress/Kubernetes.



#5 ramonrue OFFLINE  

ramonrue

    Member

  • Members
  • 13 posts
  • Local time: 02:55 PM

Posted 09 November 2019 - 01:04 PM   Best Answer

Alright, figured it out!

 

Had to add the following to nginx-ingress's configMap:

data:  
  use-proxy-protocol: "true"
  proxy-real-ip-cidr: "0.0.0.0/32"
  use-forwarded-headers: "false"

thanks for the hints.

 

EDIT: actually, what's needed is only the `use-proxy-protocol`, as MetalLB seems to use that.


Edited by ramonrue, 14 November 2019 - 03:46 AM.

  • Luke likes this

#6 Luke OFFLINE  

Luke

    System Architect

  • Administrators
  • 145887 posts
  • Local time: 08:55 AM

Posted 09 November 2019 - 03:22 PM

That's great, thanks for the feedback !







Also tagged with one or more of these keywords: kubernetes, network, linux, docker

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users