Description
This article describes how to extend VLANs (VXLAN) over multiple WAN connections (SD-WAN).
Scope
FortiGate.
Solution
Virtual Extensible LAN (VXLAN) is a network virtualization technology used in large cloud computing deployments.
It encapsulates OSI Layer 2 Ethernet frames within Layer 3 IP packets using the standard destination port 4789.
SD-WAN allows for load balancing traffic between multiple WAN connections, thereby providing redundancy when one of the WAN connections is unavailable.
In this scenario, FortiGate has multiple WAN connections, which are configured under the SD-WAN interface.
Multiple VLANs (same at both locations) are configured.
VXLAN connects locations at Layer 2 over Layer 3.
Note: all interface IPs in this article are used as an example; in the real world, wan1, wan2, and Lo1 IP addresses should be routed on the internet (not private IPs).
- WAN Configuration.
As shown on the diagram, both FortiGates have multiple WAN connections, namely wan1 and wan2.
| FortiGate 1 | FortiGate 2 |
| config system interface edit "wan1" set vdom "root" set ip 11.11.11.11 255.255.255.0 set allowaccess ping fgfm set type physical set lldp-reception enable set role wan set snmp-index 1 next edit "wan2" set vdom "root" set ip 33.33.33.33 255.255.255.0 set allowaccess ping fgfm set type physical set lldp-reception enable set role wan set snmp-index 2 next end |
config system interface edit "wan1" set vdom "root" set ip 22.22.22.22 255.255.255.0 set allowaccess ping fgfm set type physical set lldp-reception enable set role wan set snmp-index 1 next edit "wan2" set vdom "root" set ip 44.44.44.44 255.255.255.0 set allowaccess ping fgfm set type physical set lldp-reception enable set role wan set snmp-index 2 next end |
- Loopback Interface.
A loopback interface is configured at each location (as stated before, the IP addresses on both sides need to be routed on the Internet).
| config system interface edit "Lo1" set vdom "root" set ip 1.1.1.1 255.255.255.255 set allowaccess ping https http set type loopback set role lan set snmp-index 11 next end |
config system interface edit "Lo1" set vdom "root" set ip 2.2.2.2 255.255.255.255 set allowaccess ping https http set type loopback set role lan set snmp-index 11 next end |
- VXLAN Interface.
A VXLAN interface is configured as a bound to the loopback interface.
The remote IP is configured under the system. VXLAN is the peer side loopback interface IP address.
| config system vxlan edit "vxlan1" set interface "Lo1" set vni 10 set remote-ip "2.2.2.2" next end |
config system vxlan edit "vxlan1" set interface "Lo1" set vni 10 set remote-ip "1.1.1.1" next end |
- SD-WAN Interface.
A SD-WAN interface is used under wan1 and wan2, defined as members.
| config system sdwan set status enable config members edit 1 set interface "wan1" set gateway 11.11.11.1 next edit 2 set interface "wan2" set gateway 33.33.33.3 next end end |
config system sdwan set status enable config members edit 1 set interface "wan1" set gateway 22.22.22.2 next edit 2 set interface "wan2" set gateway 44.44.44.4 next end end |
- Static Route.
A single static route is configured to send all traffic outside via the SD-WAN interface.
| config router static edit 1 set distance 1 set sdwan-zone "virtual-wan-link" next end |
config router static edit 1 set distance 1 set sdwan-zone "virtual-wan-link" next end |
- Switch Interface.
A software switch is configured with port1 and the VXLAN interface 'vxlan1'.
Port 1 is connected to an internal switch where multiple VLAN interfaces are configured.
In the configuration below, the 'set intra-switch-policy implicit' (default) command implicitly allows traffic between switch members port1 and VXLAN1.
| config system switch-interface edit "local1" set vdom "root" set member "port1" "vxlan1" next end |
config system switch-interface edit "local1" set vdom "root" set member "port1" "vxlan1" next end |
Note that if 'set intra-switch-policy explicit' is used, it will require additional firewall policies to allow traffic between port1 and VXLAN.
- Firewall Policy.
A firewall policy is needed to allow traffic between the SD-WAN and the loopback interfaces. Policies are with NAT disabled in order for traffic that comes from, in this example, 1.1.1.1, with destination 2.2.2.2 on port 4789, will not be NATted, will be routed, hence, the need for the IP addresses of loopback interfaces to be routed on the internet.
| config firewall policy edit 1 set srcintf "Lo1" set dstintf "virtual-wan-link" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ALL" next edit 2 set srcintf "virtual-wan-link" set dstintf "Lo1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ALL" next end |
config firewall policy edit 1 set srcintf "Lo1" set dstintf "virtual-wan-link" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ALL" next edit 2 set srcintf "virtual-wan-link" set dstintf "Lo1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ALL" next end |
- Testing.
When Host1 (VLAN10: 192.168.10.1/24) which is connected on switch1 tries to reach Host2 (VLAN10: 192.168.10.2/24) connected to switch2, the following behavior will be observed.
C:\Users\Host1>ping 192.168.10.2
Pinging 192.168.10.2 with 32 bytes of data:
Reply from 192.168.10.2: bytes=32 time=37ms TTL=50
Reply from 192.168.10.2: bytes=32 time=34ms TTL=50
Reply from 192.168.10.2: bytes=32 time=37ms TTL=50
Reply from 192.168.10.2: bytes=32 time=43ms TTL=50
Ping statistics for 192.168.10.2:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 34ms, Maximum = 43ms, Average = 37ms
The benefit of the above setup is that if wan1 goes unreachable, traffic between the hosts at both locations will flow over the wan2 connection.
The above scenario can also work without an SD-WAN interface.
Instead of an SD-WAN interface, static routes for each WAN interface should be configured.
