Skip to content

Commit

Permalink
feat(ns-ha): handle wg interfaces, ipsec interfaces, routes
Browse files Browse the repository at this point in the history
  • Loading branch information
gsanchietti committed Feb 14, 2025
1 parent 358c9b6 commit 6f4505a
Show file tree
Hide file tree
Showing 8 changed files with 251 additions and 1 deletion.
5 changes: 5 additions & 0 deletions packages/ns-ha/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ define Package/ns-ha/install
$(INSTALL_DIR) $(1)/usr/libexec
$(INSTALL_DIR) $(1)/etc/keepalived/scripts/
$(INSTALL_BIN) ./files/keepalived-config $(1)/usr/sbin
$(INSTALL_BIN) ./files/ns-ha-disable $(1)/usr/sbin
$(INSTALL_BIN) ./files/ns-ha-enable $(1)/usr/sbin
$(INSTALL_BIN) ./files/ns-ha-export $(1)/usr/sbin
$(INSTALL_BIN) ./files/ns-ha-import $(1)/usr/sbin
$(INSTALL_DATA) ./files/400-networks $(1)/etc/hotplug.d/keepalived
$(INSTALL_DATA) ./files/500-nathelpers $(1)/etc/hotplug.d/keepalived
$(INSTALL_DATA) ./files/500-netmap $(1)/etc/hotplug.d/keepalived
$(INSTALL_DATA) ./files/560-mac-binding $(1)/etc/hotplug.d/keepalived
Expand Down
32 changes: 32 additions & 0 deletions packages/ns-ha/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,13 @@ This package is a set of scripts to configure a high availability firewall.
Configured with keepalived, it will provide a failover mechanism between two nodes.

Requirements:
- 2 nodes with similar hardware
- nodes must be connected to the same LAN
- nodes must have a dedicated interface for the HA configuration
- nodes must have only one WAN interface configured with DHCP

## Configuration

The setup process will configure all the following:
- create a new firewall zone `ha`
- configure the HA interface, the one dedicated for the HA traffic
Expand Down Expand Up @@ -56,3 +59,32 @@ In this example:
/etc/init.d/firewall restart
/etc/init.d/keepalived restart
```

## How it works

The HA is always composed by two nodes: one is the master and the other is the backup.
All configuration must be node always on the master node.
The configuration is then automatically synchronized to the backup node.

The keepalived configuration uses a special crafted rsync script named `/etc/keepalived/scripts/ns-rsync.sh`.

The script is executed on the primary node, when it is master, at regular intervals and it will:
- export WireGuard interfaces, IPsec interfaces and routes to a special directory named `/etc/ha`
- synchronize all files listed inside by `sysupgrade -l` and all files added with the `add_sync_file` option from scripts inside `/etc/hotplug.d/keepalived` directory;
files are synchronized to backup node inside the directory `/usr/share/keepalived/rsync/`

The hotplug `keepalived` event is used to inform the system about changes in the keepalived status.

The event is triggered with an `ACTION` parameter that can be:

- `NOTIFY_SYNC`: the script is executed on the backup node, after a sync has been done and a listed file is changed
During this phase all directories (like `/etc/openvpn` and `/etc/ha`) are synched to the original position.
Also WireGuard interfaces, IPsec interfaces and routes are imported from the `/etc/ha` directory but in disabled state.

- `NOTIFY_MASTER`: the script can be executed both on the master and on the backup node:
- on the master node, after keepalived is started: this is the normal startup state
- on the backup node, after an switchover has been done: this is the failover state;
all WireGuard interfaces, IPsec interfaces and routes previously imported from the `/etc/ha` are enabled if they were enabled on the master node

- `NOTIFY_BACKUP`: the script is executed on the backup node, after keepalived is started or if the master returns up after a downtime
All non required services are disabled, including WireGuard interfaces, IPsec interfaces and routes.
19 changes: 19 additions & 0 deletions packages/ns-ha/files/400-network
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
#!/bin/sh

. /lib/functions/keepalived/hotplug.sh

set_service_name network_files

if [ "$ACTION" == "NOTIFY_MASTER" ]; then
if [ "$(/usr/libexec/rpcd/ns.ha call status | jq .role)" == "backup" ]; then
/usr/sbin/ns-ha-enable
fi
elif [ "$ACTION" == "NOTIFY_SYNC" ]; then
home=$(get_rsync_user_home)
rsync -avr $home/etc/ha/ /etc/ha/
/usr/sbin/ns-ha-import
elif [ "$ACTION" == "NOTIFY_BACKUP" ]; then
/usr/sbin/ns-ha-disable
fi

keepalived_hotplug
35 changes: 35 additions & 0 deletions packages/ns-ha/files/ns-ha-disable
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/usr/bin/python3

#
# Copyright (C) 2025 Nethesis S.r.l.
# SPDX-License-Identifier: GPL-2.0-only
#

import os
import json
import subprocess
from euci import EUci

out_dir = "/etc/ha"

def disable_interfaces(file):
u = EUci()
with open(os.path.join(out_dir, file), 'r') as f:
interfaces = json.load(f)
for interface in interfaces.keys():
u.set('network', interface, 'disabled', '1')
u.commit('network')

def disable_routes():
u = EUci()
with open(os.path.join(out_dir, 'routes'), 'r') as f:
routes = json.load(f)
for route in routes.keys():
u.set('network', route, 'disabled', '1')
u.commit('network')

if __name__ == "__main__":
disable_interfaces('wg_interfaces')
disable_interfaces('ipsec_interfaces')
disable_routes()
subprocess.run(["/sbin/reload_config"], capture_output=True)
37 changes: 37 additions & 0 deletions packages/ns-ha/files/ns-ha-enable
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
#!/usr/bin/python3

#
# Copyright (C) 2025 Nethesis S.r.l.
# SPDX-License-Identifier: GPL-2.0-only
#

import os
import json
import subprocess
from euci import EUci

out_dir = "/etc/ha"

def enable_interfaces(file):
u = EUci()
with open(os.path.join(out_dir, file), 'r') as f:
interfaces = json.load(f)
for interface, options in interfaces.items():
if options.get('disabled', '0') == '0':
u.set('network', interface, 'disabled', '0')
u.commit('network')

def enable_routes():
u = EUci()
with open(os.path.join(out_dir, 'routes'), 'r') as f:
routes = json.load(f)
for route, options in routes.items():
if options.get('disabled', '0') == '0':
u.set('network', route, 'disabled', '0')
u.commit('network')

if __name__ == "__main__":
enable_interfaces('wg_interfaces')
enable_interfaces('ipsec_interfaces')
enable_routes()
subprocess.run(["/sbin/reload_config"], capture_output=True)
67 changes: 67 additions & 0 deletions packages/ns-ha/files/ns-ha-export
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
#!/usr/bin/python3

#
# Copyright (C) 2025 Nethesis S.r.l.
# SPDX-License-Identifier: GPL-2.0-only
#

# Export the folloing network configuration to /etc/ha:
# - routes
# - ipsec interfaces
# - wireguard interfaces
# - wireguard peers
# This configuration will be imported as disabled on the backup node

import os
import json
from euci import EUci
from nethsec import utils

out_dir = "/etc/ha"

def export_routes():
routes = {}
u = EUci()
for route in utils.get_all_by_type(u, 'network', 'route'):
routes[route] = u.get_all('network', route)

with open(os.path.join(out_dir, 'routes'), 'w') as f:
json.dump(routes, f)

def export_ipsec_interfaces():
ipsec_interfaces = {}
u = EUci()
for interface in utils.get_all_by_type(u, 'network', 'interface'):
if interface.startswith('ipsec'):
ipsec_interfaces[interface] = u.get_all('network', interface)

with open(os.path.join(out_dir, 'ipsec_interfaces'), 'w') as f:
json.dump(ipsec_interfaces, f)

def export_wireguard_interfaces():
wireguard_interfaces = {}
u = EUci()
for interface in utils.get_all_by_type(u, 'network', 'interface'):
if interface.startswith('wg'):
wireguard_interfaces[interface] = u.get_all('network', interface)

with open(os.path.join(out_dir, 'wg_interfaces'), 'w') as f:
json.dump(wireguard_interfaces, f)

def export_wireguard_peers():
wireguard_peers = {}
u = EUci()
for section in u.get_all('network'):
if u.get('network', section).startswith('wireguard_'):
wireguard_peers[section] = u.get_all('network', section)

with open(os.path.join(out_dir, 'wg_peers'), 'w') as f:
json.dump(wireguard_peers, f)


if __name__ == '__main__':
os.makedirs(out_dir, exist_ok=True)
export_routes()
export_ipsec_interfaces()
export_wireguard_interfaces()
export_wireguard_peers()
53 changes: 53 additions & 0 deletions packages/ns-ha/files/ns-ha-import
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
#!/usr/bin/python3

#
# Copyright (C) 2025 Nethesis S.r.l.
# SPDX-License-Identifier: GPL-2.0-only
#

# Import the network configuration exported by the master node but in a disabled state

import os
import json
from euci import EUci

out_dir = "/etc/ha"

def import_interfaces(file):
u = EUci()
with open(os.path.join(out_dir, file), 'r') as f:
interfaces = json.load(f)
for interface, options in interfaces.items():
u.set('network', interface, 'interface')
for opt in options:
u.set('network', interface, opt, options[opt])
u.set('network', interface, 'disabled', '1')
u.commit('network')

def import_wireguard_peers():
u = EUci()
with open(os.path.join(out_dir, 'wg_peers'), 'r') as f:
peers = json.load(f)
for section, options in peers.items():
stype = "wireguard_"+section.split("_")[0]
u.set('network', section, stype)
for opt in options:
u.set('network', section, opt, options[opt])
u.commit('network')

def import_routes():
u = EUci()
with open(os.path.join(out_dir, 'routes'), 'r') as f:
routes = json.load(f)
for section, options in routes.items():
u.set('network', section, 'route')
for opt in options:
u.set('network', section, opt, options[opt])
u.set('network', section, 'disabled', '1')
u.commit('network')

if __name__ == "__main__":
import_interfaces('wg_interfaces')
import_wireguard_peers()
import_interfaces('ipsec_interfaces')
import_routes()
4 changes: 3 additions & 1 deletion packages/ns-ha/files/ns-rsync.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ ha_sync_send() {
local address ssh_key ssh_port sync_list sync_dir sync_file count
local ssh_options ssh_remote dirs_list files_list
local changelog="/tmp/changelog"
local ha_export="/etc/ha"

config_get address "$cfg" address
[ -z "$address" ] && return 0
Expand Down Expand Up @@ -80,7 +81,7 @@ ha_sync_send() {
fi

# shellcheck disable=SC2086
rsync -a --relative ${files_list} ${changelog} -e "ssh $ssh_options" --rsync-path="sudo rsync" "$ssh_remote":"$sync_dir" || {
rsync -a --relative ${files_list} ${ha_export} ${changelog} -e "ssh $ssh_options" --rsync-path="sudo rsync" "$ssh_remote":"$sync_dir" || {
log_err "rsync transfer failed for $address"
update_last_sync_time "$cfg"
update_last_sync_status "$cfg" "Rsync Transfer Failed"
Expand Down Expand Up @@ -151,6 +152,7 @@ main() {
return 1
fi

/usr/sbin/ns-ha-export
config_load keepalived
config_foreach ha_sync vrrp_instance

Expand Down

0 comments on commit 6f4505a

Please sign in to comment.