has_many :codes

Getting the user’s real IP with haproxy ingress behind Cloudflare

Published  

These days I’m using the open source version of haproxy as ingress controller for Kubernetes over other popular options like Nginx and Traefik. haproxy is typically faster, and fixes for me a problem I had with Nginx: whenever Nginx reloads its configuration (for example, when my app creates an ingress resource for a custom domain), it cuts web sockets connections. Both Traefik and haproxy have a more dynamic reconfiguration so that problem doesn’t occur with either of these other ingress controllers. I prefer haproxy because of the performance and because I had some problems with Traefik when installed with the official Helm chart - it installs an older version of Traefik, among other things.

One issue I had with haproxy was that since I am using Cloudflare, all the IPs logged in haproxy’s access logs were Cloudflare’s IPs, instead the actual IPs of the users. Cloudflare sends the real IP with a CF-Connecting-IP header with each request, so we can use that header in our apps to identify the user’s IP correctly. We can easily do that with haproxy as well, so to fix the IPs shown in the access logs.

The first step is to create a config map in the namespace of the ingress controller, that includes the IPv4 and IPv6 ranges used by Cloudflare, which you can find here:

apiVersion: v1
data:
  cf-ips: |-
    173.245.48.0/20
    103.21.244.0/22
    103.22.200.0/22
    103.31.4.0/22
    141.101.64.0/18
    108.162.192.0/18
    190.93.240.0/20
    188.114.96.0/20
    197.234.240.0/22
    198.41.128.0/17
    162.158.0.0/15
    104.16.0.0/12
    172.64.0.0/13
    131.0.72.0/22
    2400:cb00::/32
    2606:4700::/32
    2803:f800::/32
    2405:b500::/32
    2405:8100::/32
    2a06:98c0::/29
    2c0f:f248::/32
kind: ConfigMap
metadata:
  name: cf-ips
  namespace: haproxy-ingress

I’ve installed haproxy in the haproxy-ingress namespace, so if you have installed it in a different namespace make sure to change that in the YAML manifest.

Next, you need to edit the haproxy deployment and mount that config map to a path in the controller’s container:

spec:
  template:
    spec:
      containers:
      - image: quay.io/jcmoraisjr/haproxy-ingress:v0.7.2
        name: haproxy-ingress
        volumeMounts:
        - mountPath: /etc/haproxy/cf-ips
          name: cf-ips
      volumes:
      - configMap:
          defaultMode: 256
          items:
          - key: cf-ips
            path: cf-ips
          name: cf-ips
          optional: false
        name: cf-ips

The final step is to add a simple annotation to each ingress resource that instructs haproxy to get the real IP from the CF-Connecting-IP header if the request originates from Cloudflare:

  annotations:
    ingress.kubernetes.io/config-backend: |
      acl from_cf src -f /etc/haproxy/cf-ips/cf-ips
      http-request set-src req.hdr(CF-Connecting-IP) if from_cf

That’s it! It’s very simple and fixes an annoying issue. Hope it helps.

© Vito Botta