How can I configure the ingress-nginx controller to work with Cloudflare and Digital Ocean Load Balancer?

6.1k views Asked by At

I have tried the answers in this question. This is my current configuration:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    helm.sh/chart: ingress-nginx-2.13.0
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 0.35.0
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  use-proxy-protocol: 'true'
  enable-real-ip: "true"
  proxy-real-ip-cidr: "173.245.48.0/20,173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32"
  # use-forwarded-headers: "true"
  # compute-full-forwarded-for: "true"
  # forwarded-for-header: "Cf-Connecting-Ip"
  # forwarded-for-header: "X-Original-Forwarded-For"
  server-snippet: |
    real_ip_header CF-Connecting-IP;

And none of the configuration I have tried is actually giving the originating ip as the real ip.

Before I applied the configuration, I was getting:

Host: example.com
X-Request-ID: deadcafe
X-Real-IP: 162.158.X.X (A Cloudflare IP)
X-Forwarded-For: 162.158.X.X (Same as above)
X-Forwarded-Proto: https
X-Forwarded-Host: example.com
X-Forwarded-Port: 80
X-Scheme: https
X-Original-Forwarded-For: <The Originating IP that I want>
Accept-Encoding: gzip
CF-IPCountry: IN
CF-RAY: cafedeed
CF-Visitor: {"scheme":"https"}
user-agent: Mozilla/5.0 
accept-language: en-US,en;q=0.5
referer: https://pv-hr.jptec.in/
upgrade-insecure-requests: 1
cookie: __cfduid=012dadfad
CF-Request-ID: 01234faddad
CF-Connecting-IP: <The Originating IP that I want>
CDN-Loop: cloudflare

After applying the config map, the headers are:

Host: example.com
X-Request-ID: 0123fda
X-Real-IP: 10.X.X.X (An IP that matches the private ip of the Digital Ocean droplets in the vpc, so guessing its the load balancer)
X-Forwarded-For: 10.X.X.X (Same as above)
X-Forwarded-Proto: http
X-Forwarded-Host: example.com
X-Forwarded-Port: 80
X-Scheme: http
X-Original-Forwarded-For: <Originating IP>
Accept-Encoding: gzip
CF-IPCountry: US
CF-RAY: 5005deeb
CF-Visitor: {"scheme":"https"}
accept: /
user-agent: Mozilla/5.0
CF-Request-ID: 1EE7af
CF-Connecting-IP: <Originating IP>
CDN-Loop: cloudflare

So the only change after the configuration is that the real-ip now points to some internal resource on the Digital Ocean vpc. I haven't been able to track that down but I am guessing its the load balancer. I am confident that it is a DO resource because it matches the ip of the kubernetes nodes. So, I am not really sure why this is happening and what I should be doing to get the originating ip as the real ip.

2

There are 2 answers

2
Cameron Munroe On BEST ANSWER

The problem you are facing is here:

proxy-real-ip-cidr: "173.245.48.0/20,173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/12,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32"

However, the traffic being seen is coming from your DO LB instead 10.x.x.x. This is causing it to be ignored for this rule.

I did the following to get it functional:

apiVersion: v1
data:
  enable-real-ip: "true"
  server-snippet: |
    real_ip_header CF-Connecting-IP;
kind: ConfigMap
metadata:
[...]

Security Notice: This will apply to all traffic even if it didn't originate from Cloudflare itself. As such, someone could spoof the headers on the request to impersonate another IP address.

0
SergkeiM On

I was having the same issue and the above-accepted answer didn't help me.

This what I did to solve the issue:

#1 - In ingress-nginx-controller I've added the below annotation:

# The below will instruct DO LOadBalancer to pass CloudFlare Ips to ingress instead of DO LoadBalancer IPs
service.beta.kubernetes.io/do-loadbalancer-enable-proxy-protocol: 'true'

#2 - in Config Maps for ingress-nginx-controller I've added the below in data: here we set as well use-proxy-protocol adding set_real_ip_from (ClouDFlares Ips) server-snippet could be as below or #real_ip_header X-Forwarded-For; see for more

Plese use this link to get updated cloudflare Ips list

data:
  use-proxy-protocol: 'true'
  server-snippet: 'real_ip_header CF-Connecting-IP;'
  set_real_ip_from: '173.245.48.0/20,103.21.244.0/22,103.22.200.0/22,103.31.4.0/22,141.101.64.0/18,108.162.192.0/18,190.93.240.0/20,188.114.96.0/20,197.234.240.0/22,198.41.128.0/17,162.158.0.0/15,104.16.0.0/13,104.24.0.0/14,172.64.0.0/13,131.0.72.0/22,2400:cb00::/32,2606:4700::/32,2803:f800::/32,2405:b500::/32,2405:8100::/32,2a06:98c0::/29,2c0f:f248::/32'