Haproxy-Configuration Setup

Prerequisite

Steps to Generate SSL Certificates Using Certbot

# For mdm.panchayatseva.com
sudo certbot certonly ---standalone d-d mdm.panchayatseva.com

# For orbit.panchayatseva.com
sudo certbot certonly --standalone -d orbit.panchayatseva.com

# For store.panchayatseva.com
sudo certbot certonly --standalone -d store.panchayatseva.com

Certificate File Structure

Certbot automatically places the certificate files in /etc/letsencrypt/live/<domain_name>/. For example:

  • For mdm.panchayatseva.com:

    /etc/letsencrypt/live/mdm.panchayatseva.com/
  ├── cert.pem         # The domain certificate
  ├── chain.pem        # The CA intermediate certificate
  ├── fullchain.pem    # Combination of cert.pem and chain.pem
  ├── privkey.pem      # Private key

  • For orbit.panchayatseva.com:

    /etc/letsencrypt/live/orbit.panchayatseva.com/
  ├── cert.pem
  ├── chain.pem
  ├── fullchain.pem
  ├── privkey.pem

  • For store.panchayatseva.com:

    /etc/letsencrypt/live/store.panchayatseva.com/
  ├── cert.pem
  ├── chain.pem
  ├── fullchain.pem
  ├── privkey.pem

Configure Certificates in HAProxy

For HAProxy, concatenate fullchain.pem and privkey.pem into a single .pem file for each domain:

# For mdm.panchayatseva.com
sudo cat /etc/letsencrypt/live/mdm.panchayatseva.com/fullchain.pem /etc/letsencrypt/live/mdm.panchayatseva.com/privkey.pem > /etc/haproxy/certs/mdm_panchayatseva_com.pem

# For orbit.panchayatseva.com
sudo cat /etc/letsencrypt/live/orbit.panchayatseva.com/fullchain.pem /etc/letsencrypt/live/orbit.panchayatseva.com/privkey.pem > /etc/haproxy/certs/orbit_panchayatseva_com.pem

# For store.panchayatseva.com
sudo cat /etc/letsencrypt/live/store.panchayatseva.com/fullchain.pem /etc/letsencrypt/live/store.panchayatseva.com/privkey.pem > /etc/haproxy/certs/store_panchayatseva_com.pem

Set Permissions

Ensure the HAProxy user has read access to the .pem files:

sudo chmod 600 /etc/haproxy/certs/*.pem
sudo chown haproxy:haproxy /etc/haproxy/certs/*.pem

PS_Orbit Configuration Overview

ACL (Access Control List)

  • ACL_psorbit_webservers: Matches requests for the domains orbit.panchayatseva.com and store.panchayatseva.com. 
    acl ACL_psorbit_webservers hdr(host) -i orbit.panchayatseva.com store.panchayatseva.com

Backend Routing

  • Requests matching ACL_psorbit_webservers are routed to the backend psorbit_webservers. 
    use_backend psorbit_webservers if ACL_psorbit_webservers

Backend Configuration: psorbit_webservers

Handles traffic for orbit.panchayatseva.com and store.panchayatseva.com.

backend psorbit_webservers
  timeout queue 25s
  balance leastconn
  option httpchk
  http-check send meth GET uri /apm/mon/health
  http-check expect status 200
  cookie pssborbitlb insert indirect nocache
  stick-table type ip size 1m expire 12h
  stick on cookie(psorbithz)
  server psorbit-node01 172.21.0.90:8182 maxconn 5000 check inter 15s fall 3 rise 3 cookie psorbit-node01 observe layer4 error-limit 9 on-error mark-down
  server psorbit-node02 172.21.0.91:8182 maxconn 5000 check inter 15s fall 3 rise 3 cookie psorbit-node02 observe layer4 error-limit 9 on-error mark-down
  server psorbit-node03 172.21.0.92:8182 maxconn 5000 check inter 15s fall 3 rise 3 cookie psorbit-node03 observe layer4 error-limit 9 on-error mark-down

Key Configurations and Their Purpose

Timeout Configuration

  • timeout queue 25s:
    Limits how long a request can wait in the backend’s queue before being dropped. Ensures requests are not stuck indefinitely when the backend servers are busy.

Load Balancing

  • balance leastconn:
    Distributes traffic to the server with the least number of active connections. Ensures even load distribution, especially when one server is less busy.

Health Checks

  • option httpchk:
    Enables health checks for backend servers to ensure availability.
  • http-check send meth GET uri /apm/mon/health:
    Performs an HTTP GET request to /apm/mon/health to verify server health.
  • http-check expect status 200:
    Marks the server as healthy only if the response status is 200 OK.

Session Persistence

  • cookie pssborbitlb insert indirect nocache:
    Adds a session cookie (pssborbitlb) to client responses, ensuring the client connects to the same backend server.
  • stick-table type ip size 1m expire 12h:
    Creates a stick table for session persistence using client IPs.
    • size 1m: Stores up to 1 million entries.
    • expire 12h: Entries expire after 12 hours.
  • stick on cookie(psorbithz):
    Uses the psorbithz cookie for sticky sessions.

Server Configuration

For each server (psorbit-node01, psorbit-node02, psorbit-node03):

  • maxconn 5000: Limits maximum simultaneous connections to 5000.
  • check: Enables health checks for the server.
  • inter 15s: Checks server health every 15 seconds.
  • fall 3: Marks the server as down after 3 consecutive failed health checks.
  • rise 3: Marks the server as up after 3 consecutive successful health checks.
  • cookie psorbit-node0X: Assigns a unique cookie for session persistence.
  • observe layer4: Monitors Layer 4 (TCP) for better efficiency.
  • error-limit 9: Marks the server down after 9 errors.
  • on-error mark-down: Marks the server as down in case of an error.

This configuration ensures efficient load balancing, health monitoring, and session persistence for the orbit.panchayatseva.com and store.panchayatseva.com domains.

MDM_SB_Node Configuration Overview

ACL (Access Control List)

  • ACL_mdm_sb_node: Matches requests for the host mdm.panchayatseva.com. 
    acl ACL_mdm_sb_node hdr(host) -i mdm.panchayatseva.com

Backend Routing

  • Requests matching ACL_mdm_sb_node are routed to the backend mdm_sb_node. 
    use_backend mdm_sb_node if ACL_mdm_sb_node

Backend Configuration: mdm_sb_node

Handles traffic for the mdm.panchayatseva.com domain.

backend mdm_sb_node
  mode http
  option tcp-check
  timeout check 30s
  server mdm_1 172.21.0.90:8382 check port 3306 port 9042 port 33145 port 8382 agent-check agent-addr 172.21.0.90 agent-port 9999 agent-send / agent-inter 60s inter 5s rise 1 fall 3
  server mdm_2 172.21.0.91:8382 check port 3306 port 9042 port 33145 port 8382 agent-check agent-addr 172.21.0.91 agent-port 9999 agent-send / agent-inter 60s inter 5s rise 1 fall 3 backup
  server mdm_3 172.21.0.92:8382 check port 3306 port 9042 port 33145 port 8382 agent-check agent-addr 172.21.0.92 agent-port 9999 agent-send / agent-inter 60s inter 5s rise 1 fall 3 backup

Key Configurations and Their Purpose

Mode

  • mode http:
    The backend operates in HTTP mode to handle web traffic for mdm.panchayatseva.com.

TCP Check

  • option tcp-check:
    Enables TCP-level health checks to verify connectivity to backend servers. This is a lower-level check compared to HTTP-based health checks, ensuring the server is reachable on specified ports.

Timeouts

  • timeout check 30s:
    Sets a 30-second timeout for health checks. If no response is received within this time, the check is marked as failed.

Server Configuration

Each server (mdm_1, mdm_2, mdm_3) has detailed health checks and failover settings:

Common Settings
  • check: Enables health checks on the backend server.
  • port: Monitors multiple critical ports (3306, 9042, 33145, 8382) to ensure availability.
  • agent-check: Adds an additional health-check mechanism through an agent running on the backend server.
    • agent-addr: Specifies the agent’s IP address.
    • agent-port 9999: Specifies the agent’s port for health checks.
    • agent-send /: Sends a basic request to the agent for health verification.
    • agent-inter 60s: Agent checks are performed every 60 seconds.
  • inter 5s: Health checks are conducted every 5 seconds.
  • rise 1: Marks the server as healthy after 1 successful health check.
  • fall 3: Marks the server as down after 3 consecutive failed health checks.
Primary Server
  • server mdm_1 172.21.0.90:8382:
    The primary backend server located at 172.21.0.90 on port 8382.
Backup Servers
  • backup:
    The servers mdm_2 (172.21.0.91:8382) and mdm_3 (172.21.0.92:8382) are designated as backup servers. They are used only if the primary server fails.

Purpose of This Configuration

  • Provides high availability for mdm.panchayatseva.com by routing traffic to a primary server (mdm_1) and using backup servers (mdm_2, mdm_3) in case of failure.
  • Monitors multiple critical ports (3306, 9042, 33145, 8382) to ensure all essential services on the backend servers are running.
  • Uses agents for additional health-check mechanisms, ensuring robust server monitoring and failover capabilities.