Synchronizing FGT HA with Cisco VRRP

[ede_pfau]

hello all,

I've got a pair of FG-200B running v4.3.18 in A-P HA mode. Each cluster member is at a different location, HA links are across a dedicated line. On each site, there is one Cisco access router (19xx) in front of the FGT providing WAN access. These routers form a VRRP pair. (No VRRP for the FGTs as config sync is requested.)

Now, when the WAN line on one site closes down the routers fail over in about 15 s. But, as the link status of the FGT WAN port does not change, the FGTs do not fail over. So I configured a pingserver (gwdetect) on the FGT which is the next hop router.

That doesn't work as expected though. When one WAN line is down, the FGT still can reach the next hop router because the Ciscos have failed over, providing internet access across the HA link line. That's a catch22 I guess.

One solution would be that the router, when detecting it has to fail over, pulls it's port to the FGT down. FGT would sense a link failure and fail over as well.

Question now is: how is that configured on a Cisco router? Is it common, or arcane? Or do you have other suggestions how to synchronize the VRRP failover with a HA failover?

Any input dearly appreciated.

[pcraponi]

Ede,

Have you looked at http://kb.fortinet.com/kb/documentLink.do?externalID=FD35173 ?

Fortigate will faillover if the gwdetect fails. When you configure "pingserver-monitor-interface", FortiOS will use this interface to reach the gwdetect instead HA link.

Regards,

Paulo R, NSE8

[ede_pfau]

Paulo,

thanks for your hint.

The pingserver will always use the FGT's WAN port, following the default route, so no choice here. To clarify I've redrawn the picture.

The switch on the front and back of the dedicated line is in fact a switch module inserted into the router, 4 ports. One is used for the line, two for redundant HA connections (not drawn here), one is unused.

As you can see when the right router fails over all internet bound traffic is led across the dedicated line and routed by the left router. But the firewall on the right still is working and will not notice that it's WAN connection has switched sides.

I asked the Cisco supporter of the ISP if we could block ICMP on that blue line, by static filter or ACL. He denied this.

So a second (weird) idea I had was to put another FGT in Transparent Mode into the blue line, blocking ICMP. To achieve device failover for this also, I'd configure an extra VDOM on the main FGT, or rather on both because of HA.

But this is clumsy and need a lot of documentation.

Thus my search for something in Cicso IOS like there is in FortiOS, pulling down an interface (link) if a failover is situation detected. In IOS, it's called 'tracking', and I bet there is something like this already.

[emnoc]

The pingserver will always use the FGT's WAN port, following the default route, so no choice here. To clarify I've redrawn the picture.

Hmm. if we are talking link-monitors than you  specify  the port and  source-ip . This doesn't really offer anything for the OP but figure I would correct that statement.

[ede_pfau]

hi Ken,

that's right, you can specify egress port and IP for 'gwdetect'. I wanted to say that in this case there is no other way as to use the WAN port to monitor an offsite host.

As you can see only in the switch there are 2 ways out, with the default route pointing to the virtual VRRP IP address. Which may be on the right (per default) or on the left (if VRRP failover has occurred).

Am I too optimistic that the tracking on the Cisco router can kick off a link-down on a router port?

[emnoc]

This would be a good test-case to see what happens, I never seen a setup trigger by let another redundancy detection. I see split-brians becoming a issues if  the HA link fails. 

1: I wonder if a link-mon from the left to the right and thru the local cisco GW-self-address would be ideal.

2: I wonder if a link-mon from the left to the right and thru the local cisco GW-vrrp-adresss would be ideal in  this situation & how would the behavior work.

Either way this would be interesting to see how/what the OP comes up with ;)

[MrSinners]

Maybe a silly idea, but since you do not want cross-site traffic at all, why even use VRRP? What if you were to configure both routers with the same inside IP address and remove the VRRP vlan altogether over the leased line, then directly connect the FG's to the routers. When a router goes down, the link on the FG goes down and a failover occurs. Use gwdetect to failover the cluster when there is no internet connectivity beyond the local router. The gwdetect result from the previous master should not be present on the new master FG.

[emnoc]

The problem with that might be with  the  ARP table and the same  mac_addr. If the  2nd router sends  a  or uses  Gratuitous ARP some type of  proxy-ARP or you could set the same  MAC_addr on the  interface Gi x/x interface and then use  a "static arp" entry in the fortiOS config, then  I could see this  working as suggested.

Does the OP have fully manage of the ciscos? Can they do proxy-arp or  have gratuitous arp intervals  or "ip arp incomplete" retry  functions?

[ede_pfau]

Unfortunately, I do not manage the routers. They are CPE routers of the ISP who provides the lines. Probably I am already daring to search for a solution which is not 100% their standard setup ("150.000 customers in the world use the standard setup").

@MrSinners: took me a while to understand your proposal. The point is the WAN IP address which, with VRRP, is transfered over to the new master router after a failure (link or access). As there are dozens of VPN tunnels terminating on that public WAN IP this setup helps to keep up the lines. ARP may be a topic but a small one, it only changes the time span until the surrounding switches have picked up the new MAC. In fact, I think the VRRP is set up with both virtual IP and virtual MAC - it doesn't harm.

I'll have to read into the Cisco IOS manual(s) to check which actions tracking can trigger. I'm looking for a link-down in case the tracking fails, as simple as that. Ken, any pointers to online Reference manuals I need to check? IOS version is (with '???') v12 or such, probably the current one for the 19XX line of routers.

[MrSinners · Accepted solution]

I already had a feeling that was the main reason for VRRP, the WAN side of the routers.. Maybe you can have a look at: https://supportforums.cisco.com/discussion/10794236/shut-interface-if-no-ping-response-using-ip-sla-...

They combine IP SLA tracking with an EEM script to bring an interface down. Pay extra attention to posts 2 and 3, if you want to use this it requires some editing for your environment.