Technical Tip: Assigning VPNs to loopback interfaces prior to FortiOS v7.0.11

[sbrooks1]

Description	This article describes a solution to set up VPNs for loopback interfaces in FortiOS versions before v7.0.11. Interface-based IPsec was added in FortiOS v3.0. When this was first introduced, VRFs were not a feature, and the decision was made that if an IPsec interface is tied to a WAN interface, the route lookup for ESP in the IPsec layer could include the WAN interface index and therefore ensure that the route on that interface was found. This solution has proved successful for many customers. However, it does not work if the IPsec interface is bound to a loopback with the goal that routing finds the most appropriate WAN interface. As VRF0 by default is utilized by management traffic, a default static route is added to the route table, which will take precedence for encapsulated traffic bound to a loopback interface.
Scope	FortiGate.
Solution	Example: config system interface     edit "port2"         set vdom "root"         set vrf 30         set ip 192.168.1.2 255.255.255.0         set allowaccess ping         set type physical         set snmp-index 2     next     edit "port3"         set vdom "root"         set ip 10.254.3.254 255.255.255.0         set allowaccess ping         set type physical         set snmp-index 3     next       edit "port4"         set vdom "root"         set vrf 30         set ip 192.168.35.254 255.255.255.0         set allowaccess ping         set type physical         set snmp-index 4     next     edit "loopback1-vpn"         set vdom "root"         set vrf 30         set ip 10.206.0.1 255.255.255.0         set allowaccess ping         set type loopback         set snmp-index 15     next     edit "TEST_VPN"         set vdom "root"         set vrf 30         set type tunnel         set snmp-index 9         set interface "loopback1-vpn"     next config vpn ipsec phase1-interface     edit "TEST_VPN"         set interface "loopback1-vpn"         set peertype any         set net-device disable         set proposal aes128-sha256 aes256-sha256 aes128-sha1 aes256-sha1         set remote-gw 10.106.0.1         set psksecret ENC BS4N729Il+L3jr1AAZy56wP9IGgASw4AzAW3j7UnRnhFhHLOgBYK1uZxXBFafV6aGqj N/AoWzkgFO4NMGCPts7yyF4o61EXbimudHorOaXLIQ/qNzPXwZNGJKvADaFKUDAEBQGKT+K2sMlQwXi/SUn8iPGxrV7 xPmzrWC6rw0ViEEHQx00BhzvoQRmqS21vAiBIXTg==     next config router static     edit 1         set gateway 10.254.3.1         set device "port3"     next     edit 2         set dst 172.16.0.1 255.255.255.255         set device "TEST_VPN"     next     edit 4         set gateway 192.168.1.1         set distance 5         set device "port2"     next get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP        O - OSPF, IA - OSPF inter area        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2        E1 - OSPF external type 1, E2 - OSPF external type 2        i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area        * - candidate default Routing table for VRF=0 S*      0.0.0.0/0 [10/0] via 10.254.3.1, port3, [1/0]    <- Static route for VRF0. C       10.254.1.0/24 is directly connected, port1 C       10.254.3.0/24 is directly connected, port3 Routing table for VRF=30 S*      0.0.0.0/0 [5/0] via 192.168.1.1, port2, [1/0]    <- Static route for VRF30. C       10.206.0.0/24 is directly connected, loopback1-vpn S       172.16.0.1/32 [10/0] via TEST_VPN tunnel 10.106.0.1, [1/0] C       192.168.1.0/24 is directly connected, port2 C       192.168.35.0/24 is directly connected, port4 For source traffic from port4 in VRF30, the flow trace shows the route found via port3 in VRF0 instead of port2 in VRF30, even though the default in VRF30 has a lower administrative distance. diagnose debug flow filter addr 172.16.0.1 diagnose debug flow trace start 100 diagnose debug enable id=20085 trace_id=1 func=print_pkt_detail line=5871 msg="vd-root:30 received a packet(proto=6, 192.168.35.1:20504->172.16.0.1:443) tun_id=0.0.0.0 from port4. flag [S], seq 2733273465, ack 0, win 29200" id=20085 trace_id=1 func=init_ip_session_common line=6050 msg="allocate a new session-000004ad, tun_id=0.0.0.0" id=20085 trace_id=1 func=vf_ip_route_input_common line=2605 msg="find a route: flag=04000000 gw-10.106.0.1 via TEST_VPN" id=20085 trace_id=1 func=fw_forward_handler line=881 msg="Allowed by Policy-1:" id=20085 trace_id=1 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TEST_VPN, tun_id=0.0.0.0" id=20085 trace_id=1 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TEST_VPN" <- Should flow via Tunnel interface... id=20085 trace_id=1 func=esp_output4 line=844 msg="IPsec encrypt/auth" id=20085 trace_id=1 func=ipsec_output_finish line=544 msg="send to 10.254.3.1 via intf-port3" <- Instead, a route via port1 default GW is seen. To Stop Debug: diagnose debug disable diagnose debug reset Packet capture shows the SYN packet forwarded correctly to the VPN interface. diagnose sniffer packet any 'host 172.16.0.1' 4 0 Using Original Sniffing Mode interfaces=[any] filters=[host 172.16.0.1] 14.663333 port4 in 192.168.35.1.20540 -> 172.16.0.1.443: syn 4021015288 14.663789 TEST_VPN out 192.168.35.1.20540 -> 172.16.0.1.443: syn 4021015288 15.715398 port4 in 192.168.35.1.20540 -> 172.16.0.1.443: syn 4021015288 15.715481 TEST_VPN out 192.168.35.1.20540 -> 172.16.0.1.443: syn 4021015288 However, when checking which interface ESP traffic is initiated on, it appears to be forwarded via port3 instead of port2: diagnose sniffer packet any 'esp' 4 0 Using Original Sniffing Mode interfaces=[any] filters=[esp] 29.546111 port3 out 10.206.0.1 -> 10.106.0.1: ESP(spi=0x04705e16,seq=0x23) 30.594988 port3 out 10.206.0.1 -> 10.106.0.1: ESP(spi=0x04705e16,seq=0x24) Workaround: If the FortiGates are in a closed-off circuit and utilizing FortiManager as an FDS server, add static routes to the management IPs of the security fabric. get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP        O - OSPF, IA - OSPF inter area        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2        E1 - OSPF external type 1, E2 - OSPF external type 2        i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area        * - candidate default Routing table for VRF=0 S       <FGT HA>/32 [10/0] via 10.254.3.1, port3, [1/0]  <- Specific routes via port1 for HA management traffic. S       <FMG IP>/32 [10/0] via 10.254.3.1, port3, [1/0]  <- Specific routes via port1 for FortiManager management traffic. S       <FAZ IP>/32 [10/0] via 10.254.3.1, port3, [1/0]  <- Specific routes via port1 for FortiAnalyzer management traffic. C       10.254.1.0/24 is directly connected, port1 C       10.254.3.0/24 is directly connected, port3 Routing table for VRF=30 S*      0.0.0.0/0 [5/0] via 192.168.1.1, port2, [1/0] C       10.206.0.0/24 is directly connected, loopback1-vpn S       172.16.0.1/32 [10/0] via TEST_VPN tunnel 10.106.0.1, [1/0] C       192.168.1.0/24 is directly connected, port2 C       192.168.35.0/24 is directly connected, port4 Creating a static route for the VPN remote gateway forces the ESP traffic to take the desired interface, however, this will need to be done for each VPN tunnel, so at scale, this configuration becomes more complex. get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP        O - OSPF, IA - OSPF inter area        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2        E1 - OSPF external type 1, E2 - OSPF external type 2        i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area        * - candidate default Routing table for VRF=0 S*      0.0.0.0/0 [10/0] via 10.254.3.1, port3, [1/0] C       10.254.1.0/24 is directly connected, port1 C       10.254.3.0/24 is directly connected, port3 Routing table for VRF=30 S*      0.0.0.0/0 [5/0] via 192.168.1.1, port2, [1/0] S       10.106.0.1/32 [10/0] via 192.168.1.1, port2, [1/0]    <- Specific route for VPN remote gateway IP. C       10.206.0.0/24 is directly connected, loopback1-vpn S       172.16.0.1/32 [10/0] via TEST_VPN tunnel 10.106.0.1, [1/0] C       192.168.1.0/24 is directly connected, port2 C       192.168.35.0/24 is directly connected, port4 The above 2 workarounds will result in the following flow: diagnose debug flow filter addr 172.16.0.1 diagnose debug flow trace start 100 diagnose debug enable id=20085 trace_id=36 func=print_pkt_detail line=5871 msg="vd-root:30 received a packet(proto=6, 192.168.35.1:20618->172.16.0.1:443) tun_id=0.0.0.0 from port4. flag [S], seq 3084778468, ack 0, win 29200" id=20085 trace_id=36 func=init_ip_session_common line=6050 msg="allocate a new session-000017cd, tun_id=0.0.0.0" id=20085 trace_id=36 func=vf_ip_route_input_common line=2605 msg="find a route: flag=04000000 gw-10.106.0.1 via TEST_VPN" id=20085 trace_id=36 func=fw_forward_handler line=881 msg="Allowed by Policy-1:" id=20085 trace_id=36 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TEST_VPN, tun_id=0.0.0.0" id=20085 trace_id=36 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TEST_VPN" id=20085 trace_id=36 func=esp_output4 line=844 msg="IPsec encrypt/auth" id=20085 trace_id=36 func=ipsec_output_finish line=544 msg="send to 192.168.1.1 via intf-port2"    <- Traffic now exits via port2. To stop the debug: diagnose debug disable diagnose debug reset diagnose sniffer packet any 'esp' 4 0 Using Original Sniffing Mode interfaces=[any] filters=[esp] 14.199569 port2 out 10.206.0.1 -> 10.106.0.1: ESP(spi=0x04705e16,seq=0x2c) 15.255492 port2 out 10.206.0.1 -> 10.106.0.1: ESP(spi=0x04705e16,seq=0x2d) Use physical interfaces instead of loopback interfaces to terminate VPNs. Upgrade to firmware version 7.0.11. This default behavior has changed after v7.0.11, and the outbound ESP traffic for a VPN terminating on a loopback interface will use the route table from the assigned VRF.