Losing ARP entry of default gateway when connecting Fortigate VLAN switch port to LAN

[Martin5]

Hi all,

To support my problem description, please see attached diagram below, which is a simplified overview of my network.

I am entirely new to Fortinet firewalls. I've always worked with Cisco firewalls but recently the company has decided to move away from Cisco and switch to Fortinet devices.

I am setting up a FortiGate 70F on the latest firmware 7.6.2. By default out of the box the Fortinet was configured with WAN1, WAN2 and DMZ ports configured as "Physical Interfaces". Port 1 through 5 were setup in VLAN Switch Mode, with VLAN 0 configured on the default 192.168.1.0/24 network.

My office network is on VLAN 70, subnet 10.70.70.0/24.

I broke up the VLAN Switch and removed port 2 through 5 from it so that they would turn into physical interfaces again. Only port 1 is still in the VLAN Switch mode. See below:

Port 5 is currently my initial admin access port while I configure the firewall (that's why its called "Initial"). Whenever I connect this interface to my access layer switch, I can access the firewall on its IP address. I configured a static route on the Fortigate to use 10.70.70.1 as the default gateway, which is my Cisco Firepower firewall.

Now for some to me unbeknownst reason, the second I connect port 1 of the Fortigate to my access switch, both my laptop and the Fortigate lose their ARP entry to 10.70.70.1 and can therefore no longer connect to the internet. When I disconnect port 1, it takes a couple of minutes for the devices to re-learn the MAC address of the Cisco gateway and then internet connection is restored.

I am trying to understand the following:

1. Why do I lose my ARP entry to the gateway on all of my devices connected to the same switch when I connect a VLAN Switch type fortigate port to my network?
2. What exactly is this VLAN Switch inside the Fortigate? I can setup ports as "Hardware Switch" and configure subinterfaces on their own SVI VLANs. I don't understand the difference between Hardware Switch, Software Switch and VLAN Switch ports. I tried looking this up in the Fortinet documentation but it's still not clear enough to me.

If someone could help me understand the behavior that I'm seeing and difference between the 3 modes, that would be greatly appreciated!

[sjoshi · Accepted solution]

You can refer below article to get more details on it.

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Hardware-switch-Software-switch-VLAN-switc...

https://docs.fortinet.com/document/fortigate/7.6.1/administration-guide/183531/virtual-vlan-switch

[dingjerry_FTNT · Accepted solution]

Hi @Martin5 ,

You can't leave Member as empty.  If this Hardware switch interface (internal) is not in use, you may delete it directly.

Otherwise, you have to back up the configuration, edit it using a text file editor, replace internal with internal4 or the interface you want to use, and delete the Hardware switch configuration.  Reload the FGT configuration which will force you to reboot the FGT.

[dingjerry_FTNT]

Hi @Martin5 ,

Here are the KB articles about Hardware/Software/VLAN switch on FortiGate:

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Hardware-switch-Software-switch-VLAN-switc...

https://community.fortinet.com/t5/Internal-Knowledge-Base-Articles/Technical-Tip-What-is-and-how-to-...

When you enable the VLAN switch, there is no Hardware switch anymore.  They can't coexist.

BTW, why do you assign VLAN 0 to the VLAN switch?  Maybe you can try another VLAN ID.

[Martin5]

Hi dingjerry,

I did not configure VLAN 0 on the VLAN switch. This is the out-of-the-box configuration and it was already there the first time I turned the device on. As far as I know a VLAN 0 doesn't exist so I don't really understand what it means.

So far my understanding of the VLANs configured on physical interfaces versus VLAN switch is this: on a physical interface, it acts as a layer 3 trunk port "router on a stick" principle. When turning ports on the Fortigate into VLAN Switch mode it becomes a layer 2 port as if it's an actual switch. Is that correct?

What I'm mostly interested in though is the issue with the loss of MAC address in my ARP cache to the gateway when I connect port 1 to my network. Port 1 is of type VLAN switch, but its configured in a different subnet (192.168.1.0/24). My LAN switch is in 10.70.70.0/24 and VLAN 70. Why do all my devices connected to this LAN switch lose their ARP entry to the default gateway when I connect port 1?

I will not be using this setup in production. I just want to understand why this behavior is happening from a learning perspective.

[sjoshi · Accepted solution]

You can refer below article to get more details on it.

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Hardware-switch-Software-switch-VLAN-switc...

https://docs.fortinet.com/document/fortigate/7.6.1/administration-guide/183531/virtual-vlan-switch

[Martin5]

I think I figured out the ARP problem. After connecting port 1 to my LAN switch again, I lost my ARP MAC entry to the gateway. When running Wireshark to see what is happning, I see a whole bunch of STP messages from the Fortinet.

As soon as I disabled STP on the Fortinet VLAN switch, after a minute or so my internet connection came back.

Our office is fairly small and we did not custom configure STP on our switches, so all of them are still on the default 32768 priority. I am guessing that the Fortinet has a lower MAC address and became the root bridge. But please correct me if I'm wrong.

Thanks.

[dingjerry_FTNT]

I think you are right.

[Martin5]

How do I delete all interfaces from a switch in the fortigate? I want to use all interfaces as regular standard layer 3 ports. Through the GUI it won't let me delete all interfaces. It wants me to have at least 1 port assigned to the switch.

How do I get Internal4 out of the switch as well?

[dingjerry_FTNT · Accepted solution]

Hi @Martin5 ,

You can't leave Member as empty.  If this Hardware switch interface (internal) is not in use, you may delete it directly.

Otherwise, you have to back up the configuration, edit it using a text file editor, replace internal with internal4 or the interface you want to use, and delete the Hardware switch configuration.  Reload the FGT configuration which will force you to reboot the FGT.

[Toshi_Esumi]

Really? It hasn't rebooted yet.

FortiGate-70F (internal) # show
config system virtual-switch
  edit "internal"
    set physical-switch "sw0"
  next
end

FortiGate-70F (internal) # end

FortiGate-70F (global) #
FortiGate-70F (global) # end

FortiGate-70F #

By the way, this is 7.2.10 with multi-vdom env.

Toshi

[dingjerry_FTNT · Accepted solution]

Sorry, my bad.  I just tested it on my FGT, yes, we can have members as empty.

So @Martin5 ,

What you can do:

config system virtual-switch

edit internal

config port

delete internal4

end

You should be able to remove the internal4 out of the Hardware switch.