FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
luisedopicari
Article Id 411313
Description By default, SD-WAN settings typically provide load balancing across overlays at the session level. However, there are scenarios where a single session needs to be distributed across all overlays. In such cases, session-based load balancing does not support per-packet distribution. This article explains how to configure SD-WAN on FortiGate to enable per-packet load balancing (Layer 3) instead of session-based load balancing
Scope

FortiGate Firmware: 7.4.8 build2795.

Solution

FortiGate's IPsec Aggregate interface provides per-packet load balancing and redundancy by combining multiple IPsec tunnels into a single logical interface. You can configure it with algorithms like Round-robin, L3 (based on IP addresses), L4 (based on TCP/UDP ports), or Redundant (using the first available tunnel) to distribute traffic and increase capacity or provide failover for site-to-site VPNs. The topology has one Spoke with three underlays and one HUB with one underlay. For the creation of IPsec tunnel (S2S), each tunnel has a loopback interface. The configuration shows the creation of one IPsec Aggregate interface with the algorithm Round-robin, for the three IPsec tunnels and the setup of SDWAN rule for this purpose.

 

1.jpg

 

2.jpg

 

Step 1: Configure Loopback Interfaces:

 

Spoke:

 

config system interface

    edit "loopback1"

        set vdom "root"

        set ip 10.208.111.17 255.255.255.255

        set allowaccess ping

        set type loopback

    next

    edit "loopback2"

        set vdom "root"

        set ip 10.208.112.17 255.255.255.255

        set allowaccess ping

        set type loopback

    next

    edit "loopback3"

        set vdom "root"

        set ip 10.208.113.17 255.255.255.255

        set allowaccess ping

        set type loopback

    next

end

 

Hub:

 

config system interface

    edit "loopback1"

        set vdom "root"

        set ip 10.99.111.4 255.255.255.255

        set allowaccess ping

        set type loopback

    next

    edit "loopback2"

        set vdom "root"

        set ip 10.99.111.5 255.255.255.255

        set allowaccess ping

        set type loopback

    next

    edit "loopback3"

        set vdom "root"

        set ip 10.99.111.6 255.255.255.255

        set allowaccess ping

        set type loopback

    next

    edit "SLA_LOOPB"

        set vdom "root"

        set ip 10.99.111.100 255.255.255.255

        set allowaccess ping

        set type loopback

    next

end

 

Step 2: Configure static routes.

 

Spoke: 

 

config router static

    edit 2

        set dst 10.99.111.4 255.255.255.255

        set gateway 10.45.3.1

        set device "port2"

        set comment "Loopback Network"

    next

    edit 3

        set dst 10.99.111.5 255.255.255.255

        set gateway 10.45.4.1

        set device "port3"

        set comment "Loopback Network"

    next

    edit 4

        set dst 10.99.111.6 255.255.255.255

        set gateway 10.45.5.1

        set device "port4"

        set comment "Loopback Network"

    next

    edit 5

        set dst 10.99.111.100 255.255.255.255

        set distance 1

        set sdwan-zone "OVERLAY"

        set comment "SLA Health Check"

    next

end

 

Hub:

 

config router static

    edit 2

        set dst 10.208.111.17 255.255.255.255

        set gateway 10.45.6.1

        set device "port2"

        set comment "Loopback Network"

    next

    edit 3

        set dst 10.208.112.17 255.255.255.255

        set gateway 10.45.6.1

        set device "port2"

        set comment "Loopback Network"

    next

    edit 4

        set dst 10.208.113.17 255.255.255.255

        set gateway 10.45.6.1

        set device "port2"

        set comment "Loopback Network"

    next

end

 

Step 3: Configure IPsec tunnels:

 

Spoke:

 

config vpn ipsec phase1-interface

    edit "C1VPN"

        set interface "loopback1"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 11

        set remote-gw 10.99.111.4

        set psksecret TEST

    next

    edit "C2VPN"

        set interface "loopback2"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 12

        set remote-gw 10.99.111.5

        set psksecret TEST

    next

    edit "C3VPN"

        set interface "loopback3"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 13

        set remote-gw 10.99.111.6

        set psksecret TEST

    next

end

config vpn ipsec phase2-interface

    edit "C1VPN"

        set phase1name "C1VPN"

        set auto-negotiate enable

    next

    edit "C2VPN"

        set phase1name "C2VPN"

        set auto-negotiate enable

    next

    edit "C3VPN"

        set phase1name "C3VPN"

        set auto-negotiate enable

    next

end

 

Hub:

 

config vpn ipsec phase1-interface

    edit "C1VPN"

        set interface "loopback1"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 11

        set remote-gw 10.208.111.17

        set psksecret TEST

    next

    edit "C2VPN"

        set interface "loopback2"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 12

        set remote-gw 10.208.112.17

        set psksecret TEST

    next

    edit "C3VPN"

        set interface "loopback3"

        set ike-version 2

        set peertype any

        set net-device disable

        set aggregate-member enable

        set proposal aes256-sha256

        set dpd on-idle

        set dhgrp 14

        set network-overlay enable

        set network-id 13

        set remote-gw 10.208.113.17

        set psksecret TEST

    next

end

config vpn ipsec phase2-interface

    edit "C1VPN"

        set phase1name "C1VPN"

        set auto-negotiate enable

    next

    edit "C2VPN"

        set phase1name "C2VPN"

        set auto-negotiate enable

    next

    edit "C3VPN"

        set phase1name "C3VPN"

        set auto-negotiate enable

    next

end

 

Step 3: Configure IPsec Interface Aggregate:

 

Spoke:

 

config system ipsec-aggregate

    edit "1IPSECAGG"

        set member "C1VPN" "C2VPN" "C3VPN"

        set algorithm round-robin

    next

end

config system interface

    edit "1IPSECAGG"

        set vdom "root"

        set ip 172.31.1.1 255.255.255.255

        set allowaccess ping

        set type tunnel

        set remote-ip 172.31.1.2 255.255.255.255

        set monitor-bandwidth enable

        set role wan

    next

end

 

Hub:

 

config system ipsec-aggregate

    edit "1IPSECAGG"

        set member "C1VPN" "C2VPN" "C3VPN"

        set algorithm round-robin

    next

end

config system interface

    edit "1IPSECAGG"

        set vdom "root"

        set ip 172.31.1.2 255.255.255.255

        set allowaccess ping

        set type tunnel

        set remote-ip 172.31.1.1 255.255.255.255

        set monitor-bandwidth enable

        set role wan

    next

end

 

Step 4: Configure iBGP.

 

Spoke:

 

config router bgp

    set as 65400

    set router-id 10.255.1.2

    set keepalive-timer 5

    set holdtime-timer 15

        config neighbor

            edit "172.31.1.2"

                set soft-reconfiguration enable

                set interface "1IPSECAGG "

                set remote-as 65400

                set update-source "1IPSECAGG "

            next

        end

        config network

            edit 1

                set prefix 10.46.0.0 255.255.255.0

            next

        end

    end

 

Hub:

 

config router bgp

    set as 65400

    set router-id 10.255.1.1

        config neighbor

            edit "172.31.1.1"

                set soft-reconfiguration enable

                set interface " 1IPSECAGG"

                set remote-as 65400

                set update-source " 1IPSECAGG"

                set route-reflector-client enable

            next

        end

        config network

            edit 1

                set prefix 10.45.0.0 255.255.255.0

            next

        end

    end

 

Step 5: Configure SD-WAN.

 

Spoke:

 

config system sdwan

    set status enable

        config zone

            edit "OVERLAY"

            next

        end

        config members

            edit 1

                set interface "1IPSECAGG"

                set zone "OVERLAY"

            next

        end

        config health-check

            edit "SLA_LOOPBACK"

                set server "10.99.111.100"

                set interval 1000

                set failtime 3

                set members 1

                    config sla

                        edit 1

                            set link-cost-factor latency

                            set latency-threshold 150

                        next

                    end

                next

            end

            config service

                edit 1

                    set name "ALL_TRAFFIC"

                    set mode sla

                    set dst "all"

                    set src "all"

                        config sla

                            edit " SLA_LOOPBACK "

                                set id 1

                            next

                        end

                    set priority-members 1

                next

            end

        end

 

Step 5: Configure Firewall Rules.

 

Spoke:

 

config firewall policy

    edit 2

        set name "IPSEC_LOOPB"

        set srcintf "port2" "port3" "port4" "loopback1" "loopback2" "loopback3"

        set dstintf "port2" "port3" "port4" "loopback1" "loopback2" "loopback3"

        set action accept

        set srcaddr "10.99.111.4" "10.99.111.5" "10.99.111.6" "loopback1" "loopback2" "loopback3"

        set dstaddr "10.99.111.4" "10.99.111.5" "10.99.111.6" "loopback1" "loopback2" "loopback3"

        set schedule "always"

        set service "IKE" "PING"

    next

    edit 3

        set name "OVERLAY_IPSECAGG"

        set srcintf "port5" "OVERLAY"

        set dstintf "port5" "OVERLAY"

        set action accept

        set srcaddr "LAN_SPOKE" "LAN_SERVERS"

        set dstaddr "LAN_SPOKE" "LAN_SERVERS"

        set schedule "always"

        set service "ALL"

    next

end

 

Hub:

 

config firewall policy

    edit 2

        set name "IPSEC_LOOPB"

        set srcintf "port2" "port3" "port4" "loopback1" "loopback2" "loopback3"

        set dstintf "port2" "port3" "port4" "loopback1" "loopback2" "loopback3"

        set action accept

        set srcaddr "10.99.111.4" "10.99.111.5" "10.99.111.6" "loopback1" "loopback2" "loopback3"

        set dstaddr "10.99.111.4" "10.99.111.5" "10.99.111.6" "loopback1" "loopback2" "loopback3"

        set schedule "always"

        set service "IKE" "PING"

    next

    edit 3

        set name "OVERLAY_IPSECAGG"

        set srcintf "port3" "1IPSECAGG"

        set dstintf "port3" "1IPSECAGG "

        set action accept

        set srcaddr "LAN_SPOKE" "LAN_SERVERS"

        set dstaddr "LAN_SPOKE" "LAN_SERVERS"

        set schedule "always"

        set service "ALL"

    next

end

 

Monitoring of per-packet distribution:

Figure 1: One TCP session is balanced by 3 IPsec tunnels (site to site).

 

3.jpg

 

Figure 2: One TCP session from host 10.46.0.9 to server 10.45.0.9.

 

4.jpg