FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
msanjaypadma
Staff
Staff
Article Id 332948
Description

 

This article describes how to block untagged traffic through ACL and how to block untagged or looped Layer 2 traffic that is received by a Layer 2 switch & high softirq for the core CPU.

 

Scope

 

FortiGate.

 

Solution

 

Consider the following network setup and configuration:

network Diagram ACL.PNG

 

FortiGate:

 

config system interface

    edit port1
        set vdom "root"
        set type physical
        set snmp-index 1
    next

    edit vlan10
        set ip 192.168.20.1 255.255.255.0
        set allowaccess ping fabric
        set device-identification enable
        set snmp-index 2
        set interface "fortilink"
        set vlanid 2
    next

end


On the switch connecting interface port1, a trunk port configuration will be applied, which will tag all VLAN traffic and transmit it to the FortiGate device with tagged packet information.


In certain scenarios, Layer 2 loops or switch issues can cause traffic to be looped and forwarded to the firewall on the physical interface with untagged packet information. This can potentially lead to CPU core spikes on the firewall.

 

Run 'get system performance output' to verify if the CPU is going high & in case of the broadcast/L2 loop coming from the switches, the softirq will go high. The output should look like below.

 

get sys perf status
CPU states: 0% user 0% system 0% nice 55% idle 0% iowait 0% irq 45% softirq
CPU0 states: 1% user 0% system 0% nice 57% idle 0% iowait 0% irq 42% softirq
CPU1 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU2 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU3 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU4 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU5 states: 0% user 0% system 0% nice 15% idle 0% iowait 0% irq 85% softirq
CPU6 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU7 states: 0% user 0% system 0% nice 0% idle 0% iowait 0% irq 100% softirq
CPU8 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU9 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU10 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU11 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU12 states: 2% user 0% system 0% nice 98% idle 0% iowait 0% irq 0% softirq
CPU13 states: 6% user 0% system 0% nice 94% idle 0% iowait 0% irq 0% softirq
CPU14 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq
CPU15 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq

 

If seeing softirq going high up to 100%, even for half of the core CPUs, understand that the packets are getting looped between the firewall & the switches.

 

To investigate this issue further, run 'dia netlink interface packet-rate' in the CLI, to see if receiving a high number of packets at the firewall interface, run this command 4-5 times an intervals of 2-3 seconds & verify the number of packets being received (TX-rate) at the firewall interface.

 

diagnose netlink interface packet-rate
Interface   RX-rate(per second) TX-rate(per second)
port1              600                             496504920
port_ha          25                                47
ha                  26                                 38

 

Collect the below sniffer output to identify what types of packets are coming to the firewall interface. For example, it could be ICMP, esp, or any other TCP/UDP packets.

 

SSH1:


diagnose sniffer packet (interface name) '' 6 2000 l 

 

In the case of ESP packets:


SSH2:


diagnose sniffer packet any 'esp' 6 2000 l

 

Sniffer output will capture 2000 packets; it is possible to tweak the packet size but be careful while running the sniffer in a CPU device.

 

Check on the switch side to know why they are forwarding a high number of packets to the firewall and ask them to rate-limit the packets at the Switch end or check if they are sending untagged/ legitimate traffic.

 

This is how untagged packets will look like in sniffer output, the tagged packets will have VLAN information.

 

2024-08-27 19:14:20.969882 port1-- x.x.x.x -> y.y.y.y: ESP(spi=0xdba59c0a,seq=0x61a)
2024-08-27 19:14:20.973437 port1-- x.x.x.x -> y.y.y.y: ESP(spi=0xdba59c0a,seq=0x61a)

 

In the case of ESP traffic, it will show the same seq numbers repeating for ESP multiple times.

 

Sniffer output will give us an idea, of whether it is a firewall or switch that is creating a loop. In case, find an issue with the FortiGate creating the loop, reach out to TAC to share all the given log output.

 

At the firewall end, it is possible to configure an Access Control List (ACL) on the physical interface to block if it is untagged traffic or if it is not legitimate.

 

Below is an example, of how ESP is blocked and IKE untagged VLAN ID packets received on port1 physical interface.

 

config firewall acl
    edit 1
        set interface port1
        set srcaddr "all"
        set dstaddr "all"
        set service "IKE" "ESP"
    next
end