High Availability with CARP and VRRP

CARP (Common Address Redundancy Protocol) is an alternative protocol to VRRP (Virtual Router Redundancy Protocol), and the Cisco proprietary HSRP (Hot Standby Router Protocol).

BSD systems extensively use CARP while Linux systems prefer using VRRP.

The objective is to make two different gateways respond to the same IP. This is done by creating a Virtual IP, that is different than the IP used by both routers, to decide who will be active and who will be on standby.

In a network where two routers 10.0.0.2 and 10.0.0.3 communicate over CARP and define who will respond to the Virtual IP 10.0.0.1. When the active becomes inoperative the standby takes place.

ON PFSENSE WEB-UI

On the Master, go to Firewall > Virtual IPs:

Click in “+ Add“,
Select the “Type” as “CARP“,
Select the “Interface” of the LAN,
Define the Virtual IP address in “Address(es)“, for example 10.0.0.1 and mask /24,
Define the “Virtual IP Password“,
Leave the VHID as ‘1‘ for the first, but if you already have one virtual IP, choose another number,
The “Description” can be ‘LAN Default Gateway‘ for example,
Click on “Save“, then “Apply Changes“.

If you did the HV Sync [Read It], you don’t have to do the same at the Slave.

If the Virtual IP was created to the WAN interface, you will have to go to Firewall > NAT > Outbound:

Chose the “Outbound NAT Mode” for “Hybrid“,
Click on “Save“, then “Apply Changes“,
Click on “+ Add“,
In the table “Edit Advanced Outbound NAT Entry“, add in “Source” the Network address of the LAN, for example, 10.0.0.0/24,
At the table “Translation” > “Address” select the “WAN Virtual IP“,
Enter a “Description“, such as ‘WAN Virtual IP for NAT‘,
Click on “Save” and “Apply Changes“.

This will prevent the loss of the packets during the transition to the Slave.

ON LINUX

In Linux systems, VRRP can be accomplished using the tool KeepAliveD [link].

Installation in all redundant instances in the cluster.

sudo apt update
sudo apt install keepalived -y
sudo nano /etc/keepalived/keepalived.conf

In summary, by configuring all redundant instances in the pool as follows assuming the interface name is eth0.

Primary

vrrp_instance VirtualIP01 {
    state MASTER
    interface eth0
    virtual_router_id 999
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 7bd3a2aa-f49b-11ef-8b04-ff74caed956a
    }
    virtual_ipaddress {
        10.0.0.1
    }
}

Backup

vrrp_instance VirtualIP01 {
    state BACKUP
    interface eth0
    virtual_router_id 999
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 7bd3a2aa-f49b-11ef-8b04-ff74caed956a
    }
    virtual_ipaddress {
        10.0.0.1
    }
}

Configuration

VirtualIP01 is an arbitrary name for this setup.
- It helps to identify the purpose of the configuration block in the file.
MASTER/BACKUP represents their relationship at the initial state.
- Defines who has the authority to decide the active instance.
eth0 is the interface name in each instance.
- The interface names identify the hardware and are specific to the instance internals.
999 is a number that tells each other what virtual address(es) the help check refers to.
- Servers can have multiple relationships for different address(es).
90/100 is the priority set to each instance to define who has the greater to be active.
- In the event of multiple backups, the priority number makes more sense.

Enable the service in all instances.

sudo systemctl enable keepalived --now
sudo systemctl status keepalived
ip -4 -brief a

For a more refined prioritization taking into consideration the health of NGINX, define the master as follows.

vrrp_script Check_NGINX {
    script "/usr/bin/pgrep nginx"
    interval 2
    weight -30
}

vrrp_instance VirtualIP01 {
    state MASTER
    interface eth0
    virtual_router_id 999
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 7bd3a2aa-f49b-11ef-8b04-ff74caed956a
    }
    virtual_ipaddress {
        10.0.0.1
    }
    track_script {
        Check_NGINX
    }
}

Note: the script Check_NGINX will check if NGINX is running every 2 seconds. If it fails, the priority of the MASTER instance will drop by 30 (from 100 to 70), and the BACKUP instance, which has a priority of 90, might take over the virtual IP. It can also be set to check interface status (down/up) or a running process or manage routes based on status (with virtual_routes).

Reflections

Keep in mind that while the second server is the backup for the first server in one or more IPs, the first server can also be the backup for the second server in other VRRP relationships. Multiple servers can relate to each other in many ways at the same time.

VRRP by default does multicasting, which creates security concerns in a shared broadcast domain (not recommended). The usage of unicast peer(s) does not resolve the issue but gives it a bit more control over the communication. Replace the X with the IP of the peer instance.

    unicast_peer {
        10.0.0.X
    }