Policy based routing setup for HAproxy airtel interface

Network Overview

1. ISP Connections

  • Airtel ISP:
    • Local network: 192.168.1.0/24
    • Public IP: Configured to forward traffic to 192.168.1.2 via NAT.
  • ACT ISP:
    • Local network: 172.21.0.0/21

2. HAProxy Server

  • Acts as the central point of connection for both ISPs.
  • Responsible for load balancing and routing requests.

Setup Details

1. Airtel NAT Configuration

  • Airtel ISP routes all requests from its public IP to 192.168.1.2 on the Airtel local network (192.168.1.0/24).

2. Network Diagram

graph TD
    Internet_Airtel["Airtel ISP, 36.255.252.149"] -->|NAT| Airtel_Network["192.168.1.0/24"]
    Internet_ACT["ACT ISP, 183.82.7.33"] --> ACT_Network["172.21.0.0/21"]

    Airtel_Network -->|testbr: 192.168.1.2| HAProxy["HAProxy Node"]
    ACT_Network -->|virbr20: 172.21.0.20| HAProxy

    subgraph HAProxy_Node
        HAProxy_Service_Airtel["HAProxy Service (192.168.1.2)"]
    end

    HAProxy_Service_Airtel -->|Traffic via Airtel| Internet

Implementing policy-based routing

Step 1: Create a New Routing Table

To define a custom routing table for HAProxy, edit the /etc/iproute2/rt_tables file:

sudo vi /etc/iproute2/rt_tables

Add the following line:

200    haproxy_route

Next, add a default route in the haproxy_route table:

sudo ip route add default via 192.168.1.1 dev eno2 table haproxy_route

Step 2: Verify Route Entry

To verify the new route entry:

sudo ip route show table haproxy_route

Step 3: Create Policy Routing Rule

Add a routing rule to ensure packets from 192.168.1.2 (HAProxy) use the newly created haproxy_route table:

sudo ip rule add from 192.168.1.2 table haproxy_route

To verify the rule:

sudo ip rule show

Step 4: Testing and Validation

After making the changes, restart HAProxy or reload its configuration. Then, monitor network traffic to confirm that responses from HAProxy (192.168.1.2) are routed correctly via 192.168.1.1 through the eno2 interface.

Step 5: Persistence Across Reboots

To make these settings persistent after a reboot:

Script: in /opt/scripts/haproxy-route-rules-out.sh file

#!/bin/bash

# Check if the interface has an IP address
IP_ADDR=$(ip addr show dev testbr | grep 'inet ' | awk '{print $2}')
if [ -n "$IP_ADDR" ]; then
  echo "Applying policy rules with IP $IP_ADDR" >> /var/log/policy-rules.log

  # Check if the route is already present
  if ! ip route show table haproxy_route | grep -q "^default via 192.168.1.1 dev testbr"; then
    ip route add default via 192.168.1.1 dev testbr table haproxy_route
    echo "Route added: default via 192.168.1.1 dev testbr table haproxy_route" >> /var/log/policy-rules.log
  else
    echo "Route already present: default via 192.168.1.1 dev testbr table haproxy_route" >> /var/log/policy-rules.log
  fi

  # Check if the rule is already present
  if ! ip rule show | grep -q "from 192.168.1.2 lookup haproxy_route"; then
    ip rule add from 192.168.1.2 table haproxy_route
    echo "Rule added: from 192.168.1.2 table haproxy_route" >> /var/log/policy-rules.log
  else
    echo "Rule already present: from 192.168.1.2 table haproxy_route" >> /var/log/policy-rules.log
  fi
fi

Configure /etc/systemd/system/haproxy-interface\@eno2.service file with below contents:

[Unit]
Description=Run script when eno2 is up
Wants=network-online.target
BindsTo=sys-devices-pci0000:00-0000:00:01.1-0000:01:00.1-net-eno2.device
After=network-online.target
Requires=systemd-networkd.socket
After=sys-devices-pci0000:00-0000:00:01.1-0000:01:00.1-net-eno2.device
After=network-online.target

[Service]
ExecStart=/bin/bash -c "/opt/scripts/haproxy-route-rules-out.sh"
RestartSec=5
Restart=always

[Install]
WantedBy=sys-devices-pci0000:00-0000:00:01.1-0000:01:00.1-net-eno2.device
root@syhydsrv001:~# cat /opt/scripts/haproxy-route-rules-out.sh

Enable and start the service:

systemctl enable --now haproxy-interface\@eno2.service

To check the logs:

tail -50f /var/log/policy-rules.log

Representation of above routing:

graph TD;
    A[HAProxy Server] -->|Source IP 192.168.1.2| B[Routing Rule];
    B --> C[Routing Table: haproxy_route];
    C -->|via 192.168.1.1| D[Interface eno2];
    D --> E[Destination ISP Network];
    F[Other Network Traffic] --> G[Default Routing Table];
    G --> H[Other Interfaces];

This diagram represents the routing flow for HAProxy traffic and how it is directed through the custom routing table, ensuring proper path selection for responses.