vrrp between fortinet

[pbarbieri]

Hello Guys,

a question: I have two fortinet working in VRRP each fortinet is connected to different switch trunked with lacp. The configuration of two firewall is fine but they are at the same time master . They exchange some messages between them 224.0.0.8 (protocol 112) but nothing they remain master both. I tried to ping from one switch the physical ip address of the other but nothing . From each switch we can see only the virtual IP that is the virtual ip of the local firewall . The switches are trunked well.

I wonder if It is mandatory to use with VRRP proxy-arp configuration because I  suspect that arp do not resolve the physical IP of the firewalls connected to the other switch.

Configuration:

switch 1 connected to firewall 1

switch 2 connected to firewall 2

switch 1 and switch 2 trunked 

I checked with get info router vrrp of both everything is fine but do not resolve the arbitrage who is master who is slave, probably because they do not see each other due to some arp problem. 

What is the role of proxy-arp in vrrp in this case they can work without?

Thanks!!!

[gfleming · Accepted solution]

OK it looks like RCC_LAN is a VLAN interface. Can you confirm by doing " show full system interface RCC_LAN | grep type". 

If so, this VLAN interface needs to be attached to a physical interface, such as the redundant interface. 

Can you shed more light on any of the other interfaces on the FortiGate? Is there more L3 interfaces between FGT and SW? What is the nature of the "trunk" interface between the FortiGates? How does it work and how is it connected?

Now, with that said, you probably do not need the VLAN interface unless you need trunking with tagged VLAN access to a downstream switch. Given what I see here there is no tagged traffic between the FGT and the switch. If RCC_LAN is the only L3 interface on the FGT, then I suggest moving the IP configuration, including VRRP, into the Redundant interface config. Or, at the very least, assigning the RCC_LAN interface to the "SW1-SW2" interface. 

config system interface
  edit RCC_LAN
    set interface "SW1-SW2"
  end

Given your use case, I would suggest leveraging FortiGate HA with SD-WAN to satisfy your requirements.

With FortiGates in HA, you only have one configuration to manage. With SD-WAN you can put both your WAN interfaces (and GRE tunnels) into SD-WAN interface and do your load balancing/failover using SD-WAN. There would be no duplication of multicast traffic as there is only one logical device on the network.

[gfleming]

More details on your topology are needed. How are the two switches connected together? Are they using MC-LAG or some other mechanism for sharing state? Or are they simply using a trunk port between them?

The fact that you can't ping from one FGT to the other FGT tells me your downstream network topology is not adequate to support VRRP communication. You need both FGTs to be on the same broadcast domain for VRRP comms to work (224.x.X.X multicast addresses will not route between segments).

[pbarbieri]

Thank you very much for your support Graham!

The two switches are trunked with lacp l3_l4 . I have annexed a picture:

port 1 and 2 are in redundant configuration, only one is active (this case port 1 for fw1 and port 2 for fw 2)

-port 1 and 2 (same vlan) of firewall 1 have a physical ip and a virtual ip

-port 1 and 2 (same vlan) of firewall 2 have a different physical ip and same virtual ip

-same broadcast domain

If I remove the cable that connect the port 2 of firewall 2 with switch 2  (active in this case) o I will be able to ping from the switch 2 the physical IP address of firewall 1  this means that the switches are well trunked connected.

the vrrp has been created between port 1-2 fw1 and port 1-2 fw2  (between active port)

It seems that this configuration of VRRP with redundant port create some issue or some spanning tree issue or i need to remove the trunk between fw1 and fw2 ?

thanks.

[gfleming]

This is an interesting-looking topology. Before delving into the VRRP issues can I ask:

- Why aren't you using FortiGate HA here? It will simplify your configuration and most likely give you the same, or better functionality.

And please confirm you are using FortiGate "Redundant" interface type for port 1 and port 2? If so you need to configure the VRRP under the redundant interface. Are you configuring VRRP under the physical ports, port1 and port2? If they are bundled in redundant interface, please configure VRRP there.

Also have you considered just creating LACP and have FW1 port1 and port2 in LACP connecting to SW1 and FW port1 and port2 in LACP connecting to SW2? 

[pbarbieri]

hello Graham, 

Yes I confirm everything about you wrote, I use fortigate redundant interface for port 1 and port 2 , I have configured VRRP under the redundant interface and they are bundled in redundant interface.  I can tell you that even if the fortigate  consider the backup port as a backup ,not active, it is not disabled because I see the backup port led blinking and I am sure that  the setting of  the interfaces of my firewall 600D is well set. Only disabled the backup port the  led are switched off. I don't want that this could be the reason of some spanning tree reaction,  ports could be a backup for fortinet but not for the switches. 

Regarding your question why I didnt use the HA. Two reason I have inherit this architecture including the configuration and second could I use HA even if the two firewall are completely different in terms of configuration and addressing? i don't want to create a clone of the first firewall but simply move the control to the second one in case of fault of the first one. Yes I confirm that this architecture suffers of something evil mistake  but still I couldnt  reach to understand possibly with your support and I appreciate  that you use your skill and time for this issue.

[gfleming]

OK I think next best step would be for you to show the configuration of the redundant interfaces. Can you please paste output of "show system interface <redundant_int_name>" for both FW1 and FW2 here?

Also please note that when interfaces are configured in redundant bundle, the layer 1 will continue to function. That is, you will see LED on the port.

OK so the two Firewalls have different configurations? In which way to they differ? If they are different then I dont understand how you could leverage VRRP. Leveraging VRRP typically means each router in the VRRP group would be configured to forward traffic in the same or very similar manner. This is why I believe you could leverage FortiGate HA instead. But please share more details on how the Firewall configurations are different between each other and I can advise further.

[pbarbieri]

Hello Graham, thank you for your useful info!

I annexed a section of the configuration about the two firewall about port redundant, and vrrp 

FIREWALL 1
----------
edit "SW1-SW2"
set vdom "root"
set type redundant
set member "port1" “port2”
set role lan
set snmp-index 23
next
edit "RCC_LAN"
set vdom "root"
set ip 10.140.1.132 255.255.255.0
set allowaccess ping https ssh
set vrrp-virtual-mac enable
config vrrp
edit 140
set vrgrp 360
set vrip 10.140.1.131
set priority 200
set adv-interval 1
set start-time 5
set preempt enable
set status enable
next
end

FIREWALL 2
-------------
edit "SW1-SW2"
set vdom "root"
set type redundant
set member "port2" “port1”
set role lan
set snmp-index 23
next
edit "RCC_LAN"
set vdom "root"
set ip 10.140.1.133 255.255.255.0
set allowaccess ping https ssh
set vrrp-virtual-mac enable
config vrrp
edit 140
set vrgrp 360
set vrip 10.140.1.131
set priority 50
set adv-interval 1
set start-time 8
set preempt enable
set status enable
next
end

config system ha
set override disable
end

--------------

The two firewall are connected to different routers that create a redundant path (one primary and one secondary),  The clients connected to the switches will use the secondary firewall in case of fault of the first one following a different way to reach the remote system and have different GRE tunnels inside. I'd like to move to HA I am agree is more efficient  and more suitable respect to the VRRP. Traffic is mainly multicast and remove VRRP and send simultaneously same traffic to both firewall means duplicate traffic  that will have as result to receive duplicate multicast traffic to the remote server unless to implement some priority mechanism inside the firewalls, VRRP could be one of these.

[gfleming · Accepted solution]

OK it looks like RCC_LAN is a VLAN interface. Can you confirm by doing " show full system interface RCC_LAN | grep type". 

If so, this VLAN interface needs to be attached to a physical interface, such as the redundant interface. 

Can you shed more light on any of the other interfaces on the FortiGate? Is there more L3 interfaces between FGT and SW? What is the nature of the "trunk" interface between the FortiGates? How does it work and how is it connected?

Now, with that said, you probably do not need the VLAN interface unless you need trunking with tagged VLAN access to a downstream switch. Given what I see here there is no tagged traffic between the FGT and the switch. If RCC_LAN is the only L3 interface on the FGT, then I suggest moving the IP configuration, including VRRP, into the Redundant interface config. Or, at the very least, assigning the RCC_LAN interface to the "SW1-SW2" interface. 

config system interface
  edit RCC_LAN
    set interface "SW1-SW2"
  end

Given your use case, I would suggest leveraging FortiGate HA with SD-WAN to satisfy your requirements.

With FortiGates in HA, you only have one configuration to manage. With SD-WAN you can put both your WAN interfaces (and GRE tunnels) into SD-WAN interface and do your load balancing/failover using SD-WAN. There would be no duplication of multicast traffic as there is only one logical device on the network.

[pbarbieri]

Thanks Graham to dedicate your time for this analysis I have appreciate it!

For the SW the port 1 and 2 of firewall are tagged and no any other interface between SW and FW

I forgot to send all configuration section and the interface SW1-SW2 is already inside the RCC_LAN definition:

edit "RCC_LAN"

        set vdom "root"

        set ip 10.140.1.133 255.255.255.0

        set allowaccess ping https ssh

        set vrrp-virtual-mac enable   

        config vrrp

            edit 140

                set vrgrp 360

                set vrip 10.140.1.131

                set priority 50

                set adv-interval 1

                set start-time 8

                set preempt enable

                set status enable

            next

        end

        set role lan

        set snmp-index 24

        set mtu-override enable

        set mtu 1300

        set interface "SW1-SW2"

        set vlanid 101

    next

I think that FW trunk has not reason to remain and this could be a further problem

I wish to study your advice to work with SD-WAN this could remove all issues to let this architecture to work!  I will update about the progress

Cheers

[gfleming]

Excellent! In the near-term I think we need to examine your Switch config to get VRRP working for you properly. Can you post configuration output for the switchports that connect to each FortiGate port as well as the configuration for the two switchports that are trunking the switches together?

And for future-state here are some good docs on HA and SD-WAN:

https://docs.fortinet.com/document/fortigate/7.0.6/administration-guide/889544/sd-wan-quick-start

https://docs.fortinet.com/document/fortigate/7.0.0/sd-wan-sd-branch-deployment-guide/643203/introduc...

https://docs.fortinet.com/document/fortigate/7.0.6/administration-guide/666376/high-availability

https://docs.fortinet.com/document/fortigate/7.0.6/administration-guide/62403/fgcp