I've got a pair of FG-200B running v4.3.18 in A-P HA mode. Each cluster member is at a different location, HA links are across a dedicated line. On each site, there is one Cisco access router (19xx) in front of the FGT providing WAN access. These routers form a VRRP pair. (No VRRP for the FGTs as config sync is requested.)
Now, when the WAN line on one site closes down the routers fail over in about 15 s. But, as the link status of the FGT WAN port does not change, the FGTs do not fail over. So I configured a pingserver (gwdetect) on the FGT which is the next hop router.
That doesn't work as expected though. When one WAN line is down, the FGT still can reach the next hop router because the Ciscos have failed over, providing internet access across the HA link line. That's a catch22 I guess.
One solution would be that the router, when detecting it has to fail over, pulls it's port to the FGT down. FGT would sense a link failure and fail over as well.
Question now is: how is that configured on a Cisco router? Is it common, or arcane? Or do you have other suggestions how to synchronize the VRRP failover with a HA failover?
The pingserver will always use the FGT's WAN port, following the default route, so no choice here. To clarify I've redrawn the picture.
The switch on the front and back of the dedicated line is in fact a switch module inserted into the router, 4 ports. One is used for the line, two for redundant HA connections (not drawn here), one is unused.
As you can see when the right router fails over all internet bound traffic is led across the dedicated line and routed by the left router. But the firewall on the right still is working and will not notice that it's WAN connection has switched sides.
I asked the Cisco supporter of the ISP if we could block ICMP on that blue line, by static filter or ACL. He denied this.
So a second (weird) idea I had was to put another FGT in Transparent Mode into the blue line, blocking ICMP. To achieve device failover for this also, I'd configure an extra VDOM on the main FGT, or rather on both because of HA.
But this is clumsy and need a lot of documentation.
Thus my search for something in Cicso IOS like there is in FortiOS, pulling down an interface (link) if a failover is situation detected. In IOS, it's called 'tracking', and I bet there is something like this already.
that's right, you can specify egress port and IP for 'gwdetect'. I wanted to say that in this case there is no other way as to use the WAN port to monitor an offsite host.
As you can see only in the switch there are 2 ways out, with the default route pointing to the virtual VRRP IP address. Which may be on the right (per default) or on the left (if VRRP failover has occurred).
Am I too optimistic that the tracking on the Cisco router can kick off a link-down on a router port?
Maybe a silly idea, but since you do not want cross-site traffic at all, why even use VRRP? What if you were to configure both routers with the same inside IP address and remove the VRRP vlan altogether over the leased line, then directly connect the FG's to the routers. When a router goes down, the link on the FG goes down and a failover occurs. Use gwdetect to failover the cluster when there is no internet connectivity beyond the local router. The gwdetect result from the previous master should not be present on the new master FG.
The problem with that might be with the ARP table and the same mac_addr. If the 2nd router sends a or uses Gratuitous ARP some type of proxy-ARP or you could set the same MAC_addr on the interface Gi x/x interface and then use a "static arp" entry in the fortiOS config, then I could see this working as suggested.
Does the OP have fully manage of the ciscos? Can they do proxy-arp or have gratuitous arp intervals or "ip arp incomplete" retry functions?
Unfortunately, I do not manage the routers. They are CPE routers of the ISP who provides the lines. Probably I am already daring to search for a solution which is not 100% their standard setup ("150.000 customers in the world use the standard setup").
@MrSinners: took me a while to understand your proposal. The point is the WAN IP address which, with VRRP, is transfered over to the new master router after a failure (link or access). As there are dozens of VPN tunnels terminating on that public WAN IP this setup helps to keep up the lines. ARP may be a topic but a small one, it only changes the time span until the surrounding switches have picked up the new MAC. In fact, I think the VRRP is set up with both virtual IP and virtual MAC - it doesn't harm.
I'll have to read into the Cisco IOS manual(s) to check which actions tracking can trigger. I'm looking for a link-down in case the tracking fails, as simple as that. Ken, any pointers to online Reference manuals I need to check? IOS version is (with '???') v12 or such, probably the current one for the 19XX line of routers.