Technical Tip: Understanding FGSP in Cloud Networks with UTM Firewall Policies

[Josh_M]

Description	This article describes the behavior and role of FGSP in Cloud Networks, specifically when using UTM Profiles (such as IPS, Antivirus, etc).
Scope	FortiGate, Azure, AWS, Google Cloud.
Solution	It is often necessary to leverage FGSP in cloud environments when deploying in an Active/Active type architecture, especially when using BGP, Load Balancers, or other technology that may result in Asymmetric routing. FGSP operates differently when UTM profiles are in use, meaning IPS, AntiVirus, Web Filtering, DNS Filter or DLP. When a UTM profile is in use, and return traffic is received on the other firewall, the traffic is sent over the FGSP link encapsulted in UDP, using UDP port 708. Important: Sessions will not be synced when UTM profiles are in Proxy Mode. Only sessions matching policies without UTM, or using UTM features in Flow-Mode are supported. When observed from Hub 2, using a 'diagnose sniffer packet', the packets will appear to be received and not forwarded. Packets being received on port2 but no forwarding traffic is matched This is because the traffic is being forwarded to the FGSP peer, but encapsulated within a UDP packet. So it does not match the filter. On the receiving side (Hub 1), the same 'diagnose sniffer packet' command looks like this: Return traffic packets appear to come in on port2 The packet is coming in via the FGSP heartbeat interface, in this example and most Cloud deployments will be port2 as well. The only noticeable difference here, compared to normal symmetric return traffic, is the traffic does not return on sriovslv and simply appears on port2. Observing Encapsulated Packets between FGSP Peers: To observe these packets as they traverse the FGSP peer it is necessary to capture the packets between FGSP and export them into Wireshark. Then remove the UDP headers to reveal the original return traffic packet. Once loaded into Wireshark, find the packets with a larger length - the length of general heartbeat packets is between 65 and 90 bytes (but is most often 76 bytes). Find one larger than this. Once identified, confirm that this is not simply a heartbeat/keepalive FGSP packet which can be achieved by observing some plaintext information about the packet such as an interface name. 'port2' in plaintext is visible 'Right-click' the packet data, select 'Copy as Hex Stream' and paste the result into this website: https://hpd.gasmi.net/ Packet is decoded to be 10.2.250.230 -> 10.2.250.229 which are the IP addresses of the interfaces FGSP is bound to. Because the packet is encapsulated, it is required to remove the UDP & IP headers to see the original packet. This is achieved by finding the start of the UDP payload by hovering the cursor over the bytes until 'UDP Payload' is visible. In this example the UDP payload begins at the highlighted byte ("14") It is then necessary to count 24 bytes beyond this, these 24 bytes are the UDP headers that need to be removed to see the encapsulated packet. In this example, the first 24 bytes of the UDP payload are: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 14 00 00 00 72 00 00 00 70 6F 72 74 32 00 00 00 60 29 25 82 67 7F 00 00 Removing this, and all preceding bytes, from the overall Hex Stream results in: 00 22 48 42 10 D7 7C 1E 52 66 0A AC 08 00 45 00 00 54 FF 30 00 00 40 01 C0 3C 0A 02 05 04 AC 1F 00 17 00 00 3B 7F 00 02 00 73 22 52 DF 67 00 00 00 00 01 7F 02 00 00 00 00 00 10 11 12 13 14 15 16 17 18 19 1A 1B 1C 1D 1E 1F 20 21 22 23 24 25 26 27 28 29 2A 2B 2C 2D 2E 2F 30 31 32 33 34 35 36 37 Which when pasted into https://hpd.gasmi.net/ results in the encapsulated packet being revealed: 10.2.5.4 -> 172.31.0.23 (ICMP Echo reply)

[rp1996]

Nice article!