Skip to main content
ede_pfau
SuperUser
ede_pfau
SuperUser
August 12, 2016
Solved

HA over VLAN to remote FGT?

  • August 12, 2016
  • 2 replies
  • 34046 views

hello all,

 

I'm planning to place the slave unit of a Fortigate HA cluster into a remote location. There is a leased line (layer 2) for the HA connect. Can anybody confirm that I can run the HA traffic across a VLAN between the access switches on each side of the line?

 

I know that HA traffic uses a non-standard ethertype, and I've tested that HA traffic is transfered unchanged over that line. But now there will be VRRP traffic between 2 routers on this line as well, and I'd like to isolate the HA traffic on a VLAN of it's own.

 

There is the option to enable authentication and encryption of the HA traffic but this will cost performance. Though it will isolate the traffic I guess.

 

I appreciate any advice, esp. from someone who has already separated a HA cluster geographically.

Best answer by MrSinners

"Latency is important" did not fully bring my point across, 15 ms is more than enough, even 100 ms would do. Depending on the setup of the customer and the quality of the leased line, a situation could occur in which some heartbeat packets are not send out quickly enough or some are missed by the other node and an active-active split brain situation occurs, which causes all traffic to be dropped. This could happen because of:

- congestion on the leased line

- other provider issues or maintenance

- being targeted by a ddos attack

- a higher amount of incoming/outgoing traffic than expected

- inspecting more traffic than anticipated or the unit can handle, causing high CPU load which might prevent handling of the HB packets

- the amount of sessions being synced between the units and whether sessions-less sessions are synced (udp and icmp)

 

When one of these points occurs some traffic will be affected but not all of it, but when HB packets are missed and a split brain situation happens all traffic is pretty much over until the nodes see each other again and the cluster is restored. The chances of this actually happening is very low. Things to look out for is the System/HA logging and look for "HB interface lost" messages. Depending on the cause of these issues, different solutions might apply. However, if you want the cluster to be more lenient when missing some HB packets, fine tuning is possible of the following settings in the "config system ha" configuration:

hb-lost-threshold <threshold_integer>      default value = 6 (which allows 5 packets to be missed before the HB interface is marked as "lost", at the 6th missed HB packet the interface is marked as "lost")

hb-interval <interval_integer>                   default value = 2 (which makes it 200 ms)

 

We can calculate the time in which the FortiGate marks a HB interface as lost by combining these values: 6 x 200 ms = 1 second and 200 ms. Depending on timing this can be slightly less or higher. Only change these values after investigating HB interface lost messages and you are certain this is the right thing to do, as this can be caused by other factors (e.g. the patch cable to the switch could be broken)

 

More information at http://kb.fortinet.com/kb/documentLink.do?externalID=10043

2 replies

ede_pfau
SuperUser
ede_pfauAuthor
SuperUser
August 16, 2016

*bump*

Nobody ever put the members of an HA cluster into 2 different locations? Never used a VLAN for the HA link?

 

Thanks in advance for any advice on these.

pcraponi
pcraponi
New Member
August 16, 2016

Hi Ede,

I did it several times. Without any problem..

 

FGT_A ----------- MPLS ------------ FGT_B

 

We always use a dedicated L2 VLAN (access) for the heartbeat link.

You need be attention on the latency between the units. If you have more than 100ms, you will faillover the HA. The HA timers can be configured via CLI if is your case.

 

On some Fortinet products (FortiWan) when I do this scenario, we need disable the STP in switch's too...

 

Regards,

Paulo R, NSE8

ede_pfau
SuperUser
ede_pfauAuthor
SuperUser
August 16, 2016

Thank you Paulo, valueable infos on this. I hope I'll have control over the switches, they come as part of the dedicated line (from Colt).

I'll check up on the HA parameters to handle the higher latency. A ping will show me what to expect.

 

Thanks again!

MikePruett
MikePruett
New Member
August 16, 2016

As long as the circuit is fast enough (low latency) you should be fine

ede_pfau
SuperUser
ede_pfauAuthor
SuperUser
August 17, 2016

Latency wasn't on my radar yet, good point.

I was wildly guessing whether non-standard ethertype traffic can be tagged in a VLAN without trouble. Although, both cluster members will only see untagged traffic as the switches at the head and tail of the line will tag/untag.

James_Ndefo
James_Ndefo
New Member
August 22, 2016

I don't see why not unless the HA traffic is all on some proprietary Fortinet protocol(now i'm playing). One main thing to look at as alluded to by mike is the link speed. definitely want to factor that in.

 

Should be essentially a physical link between 2 fortigates as you would do if they were side by side, just running across a distance on same subnet. Mind letting us know how that goes?

 

 

Terms & ConditionsAccessibility statement