Network lab: site to site VPN

Vincent Bernat

The goal of this lab is to set up a site-to-site IPsec VPN. This lab is sim­i­lar to my first lab us­ing UML. The ma­jor dif­fer­ences are:

  • We only set up one VPN in­stead of two.
  • Static rout­ing is used in place of BGP for in­ter-site rout­ing. More­over, BIRD is used as a rout­ing daemon.
  • VPNs are us­ing an ex­ter­nal net­work for In­ter­net access.
  • Both in­ter­nal net­work and ex­ter­nal net­work are re­dun­dant us­ing one OSPF in­stance each.

The lab#

VPN lab
Lab topology for site-to-site VPN

The big picture#

The main point of this ar­chi­tec­ture is that you get two re­dun­dant net­works for each site. The ex­ter­nal ones are used when hosts need In­ter­net ac­cess. The in­ter­nal ones are used when they want to com­mu­ni­cate with each other. Be­cause we don’t want hosts from the first site to com­mu­ni­cate with hosts from the sec­ond site through the ex­ter­nal net­work, we ex­tend the in­ter­nal net­work with the help of a site-to-site IPsec VPN.

Each set of net­works is us­ing two layer-3 net­works on top of two layer-2 net­works. OSPF is used to pro­vide re­dun­dancy and ag­gre­ga­tion. For ex­am­ple, R1 has two routes through V1 to ac­cess R2 which is on the other site:

# ip route show 192.168.96.0/19
192.168.96.0/19  proto zebra  metric 10000
        nexthop via 192.168.1.100  dev eth0 weight 1
        nexthop via 192.168.2.100  dev eth1 weight 1

Setting up the lab#

First, grab the source of the lab:

$ git clone https://github.com/vincentbernat/network-lab.git
$ cd network-lab/lab-s2s-vpn

You may try to run ./setup di­rectly. It may be safer to build an im­age of De­bian in­side a di­rec­tory and use it as a base for the lab.

$ sudo debootstrap sid ../sid-chroot http://ftp.fr.debian.org/debian/
[…]
$ sudo chroot ../sid-chroot /bin/bash
# apt-get install iproute zsh rsyslog less makedev tcpdump
# apt-get install bird quagga racoon
# exit
$ ROOT=../sid-chroot ./setup

You still need a work­ing UML ker­nel. On Ubuntu, the one pro­vided in user-mode-linux does not con­tain the aufs.ko mod­ule. I did not find a sim­ple work­around. You can just grab the pack­age from De­bian and in­stall it on your Ubuntu with dpkg -i.

Use of BIRD#

With the pre­vi­ous lab, Quagga was used as an OSPF rout­ing dae­mon. We still use it on R1, R2, E1 and E2. How­ever, Quagga is not able to run mul­ti­ple in­stances of OSPF. BIRD can. There­fore, we use it on V1 and V2 be­cause we need two OSPF in­stances: one for the ex­ter­nal net­work and one for the in­ter­nal net­work. Since we also need ECMP, we need at least BIRD 1.3.0.

BIRD al­lows the def­i­ni­tion of mul­ti­ple rout­ing ta­bles. These ta­bles are in­de­pen­dent of the ker­nel rout­ing ta­bles. Each pro­to­col will main­tain its own set of routes and will al­low us to ex­port (or im­port) routes from one of the rout­ing ta­ble main­tained by BIRD. In our setup, we use four pro­to­cols and only one rout­ing ta­ble (the de­fault one).

BIRD routing tables
Route import/export between BIRD tables

The ospf EXTERNAL pro­to­col is one of the two OSPF in­stances we use on V1. We im­port all the routes learned through the OSPF process into the in­ter­nal BIRD rout­ing ta­ble. We don’t ex­port any route from the BIRD rout­ing ta­ble to this OSPF in­stance. Here is the con­fig­u­ra­tion of this protocol:

protocol ospf EXTERNAL {
   ecmp yes;
   import all;
   export none;
   area 0.0.0.0 {
      networks {
         203.0.113.0/26;
         203.0.113.64/26;
         203.0.113.150/32;
      };
      interface "eth2";
      interface "eth3";
      interface "lo" {
        stub yes;
      };
   };
}

203.0.113.150/32 is de­fined on the loop­back in­ter­face of V1. This is the IP of the lo­cal VPN endpoint.

The ospf INTERNAL pro­to­col is the sec­ond OSPF in­stance. We im­port all routes from this pro­to­col to the BIRD rout­ing ta­ble. We also want to ad­ver­tise the route to the re­mote site. This route is de­fined in the static STATIC pro­to­col and im­ported into BIRD rout­ing table.

protocol static STATIC {
   import all;
   export none;
   route 192.168.96.0/19 via "lo";
}

There­fore, ospf INTERNAL pro­to­col ex­port all routes orig­i­nat­ing from the static STATIC pro­to­col. This is done us­ing filters:

protocol ospf INTERNAL {
   ecmp yes;
   import all;
   export filter {
     if proto = "STATIC" then accept;
     reject;
   };
   area 0.0.0.0 {
      networks {
         192.168.1.0/24;
         192.168.2.0/24;
      };
      interface "eth0";
      interface "eth1";
   };
}

The kernel pro­to­col is man­ag­ing one ker­nel rout­ing ta­ble. By de­fault, it han­dles the main ta­ble. We don’t im­port any route from the ker­nel. How­ever, we copy routes that come from other pro­to­cols, ex­cept the static route be­cause we al­ready have a de­fault route and the static route is some kind of black­hole, we don’t want to use it for real routing.

protocol kernel {
   persist;
   import none;
   export filter {
     if proto = "EXTERNAL" then accept;
     if proto = "INTERNAL" then accept;
     reject;
   };
}

The fil­ter­ing mech­a­nism built in­side BIRD is pretty powerful.

Testing#

Af­ter start­ing the lab, you need to wait a few sec­onds. You should then be able to ping R2 from R1:

root@R1# ping -c3 -I 192.168.15.1 192.168.115.1
PING 192.168.115.1 (192.168.115.1) from 192.168.15.1 : 56(84) bytes of data.
Warning: time of day goes back (-5435us), taking countermeasures.
64 bytes from 192.168.115.1: icmp_req=1 ttl=62 time=0.921 ms
64 bytes from 192.168.115.1: icmp_req=2 ttl=62 time=0.569 ms
64 bytes from 192.168.115.1: icmp_req=3 ttl=62 time=0.698 ms
--- 192.168.115.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2009ms
rtt min/avg/max/mdev = 0.569/0.729/0.921/0.147 ms

On V1, you can check BIRD in­ter­nal rout­ing ta­ble us­ing birdc:

> show route
0.0.0.0/0          multipath [EXTERNAL 19:43] * E2 (150/10/1) [0.0.0.11]
        via 203.0.113.1 on eth2 weight 1
        via 203.0.113.65 on eth3 weight 1
192.168.96.0/19    dev lo [STATIC 19:42] * (200)
192.168.1.0/24     dev eth0 [INTERNAL 19:42] * I (150/10) [0.0.0.100]
192.168.2.0/24     dev eth1 [INTERNAL 19:42] * I (150/10) [0.0.0.100]
192.168.15.0/24    multipath [INTERNAL 19:43] * I (150/20) [0.0.0.10]
        via 192.168.1.10 on eth0 weight 1
        via 192.168.2.10 on eth1 weight 1
203.0.113.0/26     dev eth2 [EXTERNAL 19:42] * I (150/10) [0.0.0.100]
203.0.113.64/26    dev eth3 [EXTERNAL 19:42] * I (150/10) [0.0.0.100]
203.0.113.150/32   dev lo [EXTERNAL 19:42] * I (150/0) [0.0.0.100]

The ker­nel rout­ing ta­ble is sim­i­lar ex­cept it does not con­tain the static route.

Adding more redundancy#

The first goal of this lab was to test mul­ti­ple OSPF in­stances and ECMP with BIRD. It could be completed:

  • Adding a re­dun­dant VPN is easy: this is just a mat­ter of du­pli­cat­ing the setup of the ex­isting VPN.
  • Each router should be re­dun­dant by get­ting a sib­ling with the same con­fig­u­ra­tion. In the case of R1 and R2, a VIP could be shared be­tween the router and its sib­ling with VRRP to al­low hosts be­hind to have a sin­gle gate­way and not use a rout­ing protocol.
  • Be­cause of the use of static routes in­stead of BGP, there is noth­ing to de­tect a VPN fail­ure (which may be prob­lem­atic if we set up a re­dun­dant VPN). Some VPN are able to in­val­i­date a route when the as­so­ci­ated VPN is down. This is not our case. More­over, there may be some du­pli­ca­tion in rout­ing. This is not the case here be­cause each site has its own in­ter­nal network.