A com­mon way to es­tab­lish an IPsec tun­nel on Linux is to use an IKE dae­mon, like the one from the strongSwan project, with a min­i­mal con­fig­u­ration:¹

conn V2-1
  left        = 2001:db8:1::1
  leftsubnet  = 2001:db8:a1::/64
  right       = 2001:db8:2::1
  rightsubnet = 2001:db8:a2::/64
  authby      = psk
  auto        = route

The same con­fig­u­ra­tion can be used on both sides. Each side will fig­ure out if it is “left” or “right.” The IPsec site-to-site tun­nel end­points are 2001:​db8:1::1 and 2001:​db8:2::1. The pro­tected sub­nets are 2001:​db8:​a1::/64 and 2001:​db8:​a2::/64. As a re­sult, strongSwan con­fig­ures the fol­low­ing poli­cies in the kernel:

$ ip xfrm policy
src 2001:db8:a1::/64 dst 2001:db8:a2::/64
        dir out priority 399999 ptype main
        tmpl src 2001:db8:1::1 dst 2001:db8:2::1
                proto esp reqid 4 mode tunnel
src 2001:db8:a2::/64 dst 2001:db8:a1::/64
        dir fwd priority 399999 ptype main
        tmpl src 2001:db8:2::1 dst 2001:db8:1::1
                proto esp reqid 4 mode tunnel
src 2001:db8:a2::/64 dst 2001:db8:a1::/64
        dir in priority 399999 ptype main
        tmpl src 2001:db8:2::1 dst 2001:db8:1::1
                proto esp reqid 4 mode tunnel
[…]

This kind of IPsec tun­nel is a pol­icy-based VPN: en­cap­su­la­tion and de­cap­su­la­tion are gov­erned by these poli­cies. Each of them con­tains the fol­low­ing elements:

• a di­rec­tion (out, in or fwd²);
• a se­lec­tor (source sub­net, des­ti­na­tion sub­net, pro­to­col, ports);
• a mode (transport or tunnel);
• an en­cap­su­la­tion pro­to­col (esp or ah); and
• the end­point source and des­ti­na­tion ad­dresses.

When a match­ing pol­icy is found, the ker­nel will look for a cor­re­spond­ing se­cu­rity as­so­ci­a­tion (us­ing reqid and the end­point source and des­ti­na­tion ad­dresses):

$ ip xfrm state
src 2001:db8:1::1 dst 2001:db8:2::1
        proto esp spi 0xc1890b6e reqid 4 mode tunnel
        replay-window 0 flag af-unspec
        auth-trunc hmac(sha256) 0x5b68[…]8ba2904 128
        enc cbc(aes) 0x8e0e377ad8fd91e8553648340ff0fa06
        anti-replay context: seq 0x0, oseq 0x0, bitmap 0x00000000
[…]

If no se­cu­rity as­so­ci­a­tion is found, the packet is put on hold and the IKE dae­mon is asked to ne­go­ti­ate an ap­pro­pri­ate one. Oth­er­wise, the packet is en­cap­su­lated. The re­ceiv­ing end iden­ti­fies the ap­pro­pri­ate se­cu­rity as­so­ci­a­tion us­ing the SPI in the header. Two se­cu­rity as­so­ci­a­tions are needed to es­tab­lish a bidi­rec­tional tunnel:

$ tcpdump -pni eth0 -c2 -s0 esp
13:07:30.871150 IP6 2001:db8:1::1 > 2001:db8:2::1: ESP(spi=0xc1890b6e,seq=0x222)
13:07:30.872297 IP6 2001:db8:2::1 > 2001:db8:1::1: ESP(spi=0xcf2426b6,seq=0x204)

All IPsec im­ple­men­ta­tions are com­pat­i­ble with pol­icy-based VPNs. How­ever, some con­fig­u­ra­tions are dif­fi­cult to im­ple­ment. For ex­am­ple, con­sider the fol­low­ing propo­si­tion for re­dun­dant site-to-site VPNs:

A pos­si­ble con­fig­u­ra­tion be­tween V1-1 and V2-1 could be:

conn V1-1-to-V2-1
  left        = 2001:db8:1::1
  leftsubnet  = 2001:db8:a1::/64,2001:db8:a6::cc:1/128,2001:db8:a6::cc:5/128
  right       = 2001:db8:2::1
  rightsubnet = 2001:db8:a2::/64,2001:db8:a6::/64,2001:db8:a8::/64
  authby      = psk
  keyexchange = ikev2
  auto        = route

Each time a sub­net is mod­i­fied on one site, the con­fig­u­ra­tions need to be up­dated on all sites. More­over, over­lap­ping sub­nets (2001:​db8:​a6::/64 on one side and 2001:​db8:​a6::​cc:​1/128 at the other) can also be prob­lematic.

The al­ter­na­tive is to use route-based VPNs: any packet travers­ing a pseudo-in­ter­face will be en­cap­su­lated us­ing a se­cu­rity pol­icy bound to the in­ter­face. This brings two features:

1. Rout­ing dae­mons can be used to dis­trib­ute routes to be pro­tected by the VPN. This de­creases the ad­min­is­tra­tive bur­den when many sub­nets are present on each side.
2. En­cap­su­la­tion and de­cap­su­la­tion can be ex­e­cuted in a dif­fer­ent rout­ing in­stance or names­pace. This en­ables a clean sep­a­ra­tion be­tween a pri­vate rout­ing in­stance (where VPN users are) and a pub­lic rout­ing in­stance (where VPN end­points are).

Route-based VPN on Juniper#

Be­fore look­ing at how to achieve that on Linux, let’s have a look at the way it works with a Junos-based plat­form (like a Ju­niper vSRX). This plat­form has a long-stand­ing his­tory of sup­port­ing route-based VPNs (a fea­ture al­ready present in the Netscreen ISG platform).

Let’s as­sume we want to con­fig­ure the IPsec VPN from V3-2 to V1-1. First, we need to con­fig­ure the tun­nel in­ter­face and bind it to the “pri­vate” rout­ing in­stance con­tain­ing only in­ter­nal routes (with IPv4, they would have been RFC 1918 routes):

interfaces {
    st0 {
        unit 1 {
            family inet6 {
                address 2001:db8:ff::7/127;
            }
        }
    }
}
routing-instances {
    private {
        instance-type virtual-router;
        interface st0.1;
    }
}

The sec­ond step is to con­fig­ure the VPN:

security {
    /* Phase 1 configuration */
    ike {
        proposal IKE-P1 {
            authentication-method pre-shared-keys;
            dh-group group20;
            encryption-algorithm aes-256-gcm;
        }
        policy IKE-V1-1 {
            mode main;
            proposals IKE-P1;
            pre-shared-key ascii-text "d8bdRxaY22oH1j89Z2nATeYyrXfP9ga6xC5mi0RG1uc";
        }
        gateway GW-V1-1 {
            ike-policy IKE-V1-1;
            address 2001:db8:1::1;
            external-interface lo0.1;
            general-ikeid;
            version v2-only;
        }
    }
    /* Phase 2 configuration */
    ipsec {
        proposal ESP-P2 {
            protocol esp;
            encryption-algorithm aes-256-gcm;
        }
        policy IPSEC-V1-1 {
            perfect-forward-secrecy keys group20;
            proposals ESP-P2;
        }
        vpn VPN-V1-1 {
            bind-interface st0.1;
            df-bit copy;
            ike {
                gateway GW-V1-1;
                ipsec-policy IPSEC-V1-1;
            }
            establish-tunnels on-traffic;
        }
    }
}

We get a route-based VPN be­cause we bind the st0.1 in­ter­face to the VPN-V1-1 VPN. Once the VPN is up, any packet en­ter­ing st0.1 will be en­cap­su­lated and sent to the 2001:​db8:1::1 endpoint.

The last step is to con­fig­ure BGP in the “pri­vate” rout­ing in­stance to ex­change routes with the re­mote site:

routing-instances {
    private {
        routing-options {
            router-id 1.0.3.2;
            maximum-paths 16;
        }
        protocols {
            bgp {
                preference 140;
                log-updown;
                group v4-VPN {
                    type external;
                    local-as 65003;
                    hold-time 6;
                    neighbor 2001:db8:ff::6 peer-as 65001;
                    multipath;
                    export [ NEXT-HOP-SELF OUR-ROUTES NOTHING ];
                }
            }
        }
    }
}

The ex­port fil­ter OUR-ROUTES needs to se­lect the routes to be ad­ver­tised to the other peers. For example:

policy-options {
    policy-statement OUR-ROUTES {
        term 10 {
            from {
                protocol ospf3;
                route-type internal;
            }
            then {
                metric 0;
                accept;
            }
        }
    }
}

The con­fig­u­ra­tion needs to be re­peated for the other peers. The com­plete ver­sion is avail­able on GitHub. Once the BGP ses­sions are up, we start learn­ing routes from the other sites. For ex­am­ple, here is the route for 2001:​db8:​a1::/64:

> show route 2001:db8:a1::/64 protocol bgp table private.inet6.0 best-path

private.inet6.0: 15 destinations, 19 routes (15 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

2001:db8:a1::/64   *[BGP/140] 01:12:32, localpref 100, from 2001:db8:ff::6
                      AS path: 65001 I, validation-state: unverified
                      to 2001:db8:ff::6 via st0.1
                    > to 2001:db8:ff::14 via st0.2

It was learned both from V1-1 (through st0.1) and V1-2 (through st0.2). The route is part of the private rout­ing in­stance but en­cap­su­lated pack­ets are sent/received in the public rout­ing in­stance. No route-leak­ing is needed for this con­fig­u­ra­tion. The VPN can­not be used as a gate­way from in­ter­nal hosts to ex­ter­nal hosts (or vice-versa). This could also have been done with Junos’ se­cu­rity poli­cies (state­ful fire­wall rules) but do­ing the sep­a­ra­tion with rout­ing in­stances also en­sures routes from dif­fer­ent do­mains are not mixed and a sim­ple pol­icy mis­con­fig­u­ra­tion will not lead to a disaster.

Route-based VPN on Linux#

Start­ing from Linux 3.15, a sim­i­lar con­fig­u­ra­tion is pos­si­ble with the help of a vir­tual tun­nel in­ter­face.³ First, we cre­ate the “pri­vate” namespace:

# ip netns add private
# ip netns exec private sysctl -qw net.ipv6.conf.all.forwarding=1

Any “pri­vate” in­ter­face needs to be moved to this names­pace (no IP is con­fig­ured as we can use IPv6 link-lo­cal ad­dresses):

# ip link set netns private dev eth1
# ip link set netns private dev eth2
# ip netns exec private ip link set up dev eth1
# ip netns exec private ip link set up dev eth2

Then, we cre­ate vti6, a tun­nel in­ter­face (sim­i­lar to st0.1 in the Junos example):

# ip -6 tunnel add vti6 \
>  mode vti6 \
>  local 2001:db8:1::1 \
>  remote 2001:db8:3::2 \
>  key 6
# ip link set netns private dev vti6
# ip netns exec private ip addr add 2001:db8:ff::6/127 dev vti6
# ip netns exec private sysctl -qw net.ipv4.conf.vti6.disable_policy=1
# ip netns exec private ip link set vti6 mtu 1500
# ip netns exec private ip link set vti6 up

The tun­nel in­ter­face is cre­ated in the ini­tial names­pace and moved to the “pri­vate” one. It will re­mem­ber its orig­i­nal names­pace where it will process en­cap­su­lated pack­ets. Any packet en­ter­ing the in­ter­face will tem­po­rar­ily get a fire­wall mark of 6 that will be used only to match the ap­pro­pri­ate IPsec pol­icy⁴ be­low. The ker­nel sets a low MTU on the in­ter­face to han­dle any pos­si­ble com­bi­na­tion of ci­phers and pro­to­cols. We set it to 1500 and let PMTUD do its work.

We can then con­fig­ure strongSwan:⁵

conn V3-2
  left        = 2001:db8:1::1
  leftsubnet  = ::/0
  right       = 2001:db8:3::2
  rightsubnet = ::/0
  authby      = psk
  mark        = 6
  auto        = route
  keyexchange = ikev2
  keyingtries = %forever
  ike         = aes256gcm16-prfsha384-ecp384!
  esp         = aes256gcm16-prfsha384-ecp384!
  mobike      = no

The IKE dae­mon con­fig­ures the fol­low­ing poli­cies in the kernel:

$ ip xfrm policy
src ::/0 dst ::/0
        dir out priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:1::1 dst 2001:db8:3::2
                proto esp reqid 1 mode tunnel
src ::/0 dst ::/0
        dir fwd priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:3::2 dst 2001:db8:1::1
                proto esp reqid 1 mode tunnel
src ::/0 dst ::/0
        dir in priority 399999 ptype main
        mark 0x6/0xffffffff
        tmpl src 2001:db8:3::2 dst 2001:db8:1::1
                proto esp reqid 1 mode tunnel
[…]

These poli­cies are used for any source or des­ti­na­tion as long as the fire­wall mark is equal to 6, which matches the mark con­fig­ured for the tun­nel in­terface.

The last step is to con­fig­ure BGP to ex­change routes. We can use BIRD for this:

router id 1.0.1.1;
protocol device {
   scan time 10;
}
protocol kernel {
   persist;
   learn;
   import all;
   export all;
   merge paths yes;
}
protocol bgp IBGP_V3_2 {
   local 2001:db8:ff::6 as 65001;
   neighbor 2001:db8:ff::7 as 65003;
   import all;
   export where ifname ~ "eth*";
   preference 160;
   hold time 6;
}

Once BIRD is started in the “pri­vate” names­pace, we can check routes are learned correctly:

$ ip netns exec private ip -6 route show 2001:db8:a3::/64
2001:db8:a3::/64 proto bird metric 1024
        nexthop via 2001:db8:ff::5  dev vti5 weight 1
        nexthop via 2001:db8:ff::7  dev vti6 weight 1

The above route was learned from both V3-1 (through vti5) and V3-2 (through vti6). Like for the Junos ver­sion, there is no route-leak­ing be­tween the “pri­vate” names­pace and the ini­tial one. The VPN can­not be used as a gate­way be­tween the two names­paces, only for en­cap­su­la­tion. This also pre­vents a mis­con­fig­u­ra­tion (for ex­am­ple, IKE dae­mon not run­ning) from al­low­ing pack­ets to leave the pri­vate network.

As a bonus, un­en­crypted traf­fic can be ob­served with tcpdump on the tun­nel in­terface:

$ ip netns exec private tcpdump -pni vti6 icmp6
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vti6, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
20:51:15.258708 IP6 2001:db8:a1::1 > 2001:db8:a3::1: ICMP6, echo request, seq 69
20:51:15.260874 IP6 2001:db8:a3::1 > 2001:db8:a1::1: ICMP6, echo reply, seq 69

You can find all the con­fig­u­ra­tion files for this ex­am­ple on GitHub. The doc­u­men­ta­tion of strongSwan also fea­tures a page about route-based VPNs. It is pos­si­ble to re­place IPsec by Wire­Guard, a fast and mod­ern VPN im­ple­mentation.

1. https://strongswan.org/
2. https://libreswan.org/
3. https://github.com/vincentbernat/network-lab/blob/master/lab-routed-vpn/junos-V3-2.conf
4. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c6741fbed6dc0f183d26c4b6bca4517672f92e6c
5. https://docs.strongswan.org/docs/latest/config/proposals.html
6. https://docs.strongswan.org/docs/latest/howtos/securityRecommendations.html
7. https://bird.network.cz/
8. https://github.com/vincentbernat/network-lab/tree/master/lab-routed-vpn
9. https://docs.strongswan.org/docs/latest/features/routeBasedVpn.html
10. https://vincent.bernat.ch/en/blog/2018-route-based-vpn-wireguard
11. https://github.com/vincentbernat/network-lab/commit/a0b33a5619e32bcee10da7d9e669471681bff49c
12. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9e1437937807b0122e8da1ca8765be2adca9aee6
13. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0152eee6fc3b84298bb6a79961961734e8afa5b8