VRRP - track remote ipsec tunnel gateway - kernel route always there

Question

Hello everybody,

I try to use VRRP tracking (vrdst command) to automatically make a VRRP failover when the IPSec VPN (interface mode) is down:

    edit "port2"
        set vdom "root"
        set ip 192.168.0.11 255.255.255.240
        set allowaccess ping
        set bfd enable
        set type physical
        set vrrp-virtual-mac enable
        config vrrp
            edit 5
                set vrip 192.168.0.10
                set priority 255
                set vrdst 192.168.255.2
            next
        end
        set snmp-index 2
    next

VRRP failover works successfully if I configure vrdst with an IP for which the FG does not have a route, e.g. 8.8.8.8.

But when I specify the IP of the tunnel interface remote-gw (local IP: 192.168.255.1, remote IP: 192.168.255.2) VRRP failover does not happen. It seems the local and remote tunnel IP is kept always in the kernel routing table.

Local (192.168.255.1) and remote (192.168.255.2) tunnel IP in "normal" routing table when IPSec VPN is down:

P-dcA-fw1 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

C 1.1.1.0/30 is directly connected, port1
O E2 10.0.0.0/24 [110/10] via 192.168.0.12, port2, 00:03:53
O E2 10.0.1.0/24 [110/10] via 192.168.0.12, port2, 00:03:53
O 172.16.0.0/25 [110/202] via 192.168.0.12, port2, 00:03:54
S 172.16.20.0/24 [10/0] via 192.168.0.1, port2
S 172.16.21.0/24 [10/0] via 192.168.0.1, port2
C 192.168.0.0/28 is directly connected, port2
C 192.168.0.10/32 is directly connected, port2
C 192.168.99.0/24 is directly connected, port10
S 192.168.100.0/24 [10/0] via 192.168.0.1, port2
C 192.168.255.1/32 is directly connected, To-C-dcA-fw1
C 192.168.255.2/32 is directly connected, To-C-dcA-fw1
O 192.168.255.5/32 [110/1] via 192.168.0.12, port2, 00:03:54
O 192.168.255.6/32 [110/201] via 192.168.0.12, port2, 00:03:54

Now with IPSec VPN down:

P-dcA-fw1 # get vpn ipsec tunnel summary
'To-C-dcA-fw1' 1.1.1.2:0 selectors(total,up): 1/0 rx(pkt,err): 0/0 tx(pkt,err): 0/0

Tunnel interface is up although IPSec VPN is down:

P-dcA-fw1 # get sys int
...
== [ To-C-dcA-fw1 ]
name: To-C-dcA-fw1 ip: 192.168.255.1 255.255.255.255 status: up netbios-forward: disable type: tunnel netflow-sampler: disable sflow-sampler: disable scan-botnet-connections: disable explicit-web-proxy: disable explicit-ftp-proxy: disable wccp: disable

No local or remote tunnel IP in "normal" routing table when IPSec VPN is down:

P-dcA-fw1 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

C 1.1.1.0/30 is directly connected, port1
O E2 10.0.0.0/24 [110/10] via 192.168.0.12, port2, 00:01:06
O E2 10.0.1.0/24 [110/10] via 192.168.0.12, port2, 00:01:06
O 172.16.0.0/25 [110/202] via 192.168.0.12, port2, 00:01:07
O 172.16.0.80/32 [110/202] via 192.168.0.12, port2, 00:01:07
S 172.16.20.0/24 [10/0] via 192.168.0.1, port2
S 172.16.21.0/24 [10/0] via 192.168.0.1, port2
C 192.168.0.0/28 is directly connected, port2
C 192.168.0.10/32 is directly connected, port2
C 192.168.99.0/24 is directly connected, port10
S 192.168.100.0/24 [10/0] via 192.168.0.1, port2
O 192.168.255.5/32 [110/1] via 192.168.0.12, port2, 00:01:07
O 192.168.255.6/32 [110/201] via 192.168.0.12, port2, 00:01:07

But 192.168.255.1 and 192.168.255.2 are still in the kernel routing table:

P-dcA-fw1 # get router info kernel
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->1.1.1.0/32 pref=1.1.1.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->1.1.1.1/32 pref=1.1.1.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->1.1.1.3/32 pref=1.1.1.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.0.0/32 pref=192.168.0.11 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.0.10/32 pref=192.168.0.10 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.0.11/32 pref=192.168.0.11 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.0.15/32 pref=192.168.0.11 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.99.0/32 pref=192.168.99.11 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.99.11/32 pref=192.168.99.11 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.99.255/32 pref=192.168.99.11 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.255.1/32 pref=192.168.255.1 gwy=0.0.0.0 dev=15(To-C-dcA-fw1)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->1.1.1.0/30 pref=1.1.1.1 gwy=0.0.0.0 dev=3(port1)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.0.0.0/24 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.0.1.0/24 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->172.16.0.0/25 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->172.16.0.80/32 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->172.16.20.0/24 pref=0.0.0.0 gwy=192.168.0.1 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->172.16.21.0/24 pref=0.0.0.0 gwy=192.168.0.1 dev=4(port2)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.0.0/28 pref=192.168.0.11 gwy=0.0.0.0 dev=4(port2)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.99.0/24 pref=192.168.99.11 gwy=0.0.0.0 dev=12(port10)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->192.168.100.0/24 pref=0.0.0.0 gwy=192.168.0.1 dev=4(port2)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->192.168.255.2/32 pref=192.168.255.1 gwy=0.0.0.0 dev=15(To-C-dcA-fw1)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->192.168.255.5/32 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->192.168.255.6/32 pref=0.0.0.0 gwy=192.168.0.12 dev=4(port2)

And here the difference between a traceroute for a "fake" IP with no route vs a traceroute for the remote tunnel IP:

P-dcA-fw1 # execute traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 32 hops max, 3 probe packets per hop, 84 byte packets
 1 traceroute: sendto: Network is unreachable
traceroute: wrote 8.8.8.8 84 chars, ret=-1
 *traceroute: sendto: Network is unreachable
traceroute: wrote 8.8.8.8 84 chars, ret=-1
 *^Ctraceroute: sendto: Network is unreachable
traceroute: wrote 8.8.8.8 84 chars, ret=-1
 *

P-dcA-fw1 # execute traceroute 192.168.255.2
traceroute to 192.168.255.2 (192.168.255.2), 32 hops max, 3 probe packets per hop, 84 byte packets
 1 * * *
 2 * * *^C
 3 *
P-dcA-fw1 #

Note: add-gw-route (phase1-interfae) is by default "disable" - so no effect there.

Is there any possibility to either

- remove the route for the remote tunnel IP from the kernel routing table or

- bring down the tunnel interface when the IPSec VPN is down?

Any help is greatly appreciated!

Kind regards,

Lukas

Toshi_Esumi · Answer

For your specific VRRP behavior question, you probably need to open a case at TAC to get the right answer. But we won't use VRRP or HSRP (if Cisco to Cisco) to fail-over the route from via one IPSec tunnel to via another path when the IPSec goes down. I see you're using OSPF between devices. You can set up a routing protocol like OSPF over IPSec for the routes to fail over to another path. As long as the FG or Eth interface of the FG is not dead, all packets would hit the FG first (because it's the master among the VRRP members) then just forward to the alternative routes in the "normal" routing table to get to the destinations.

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded