FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
DiegoBernardelli
Article Id 297665
Description This article describes how to set up a VXLAN over an IPsec scenario using switch interfaces to link local interfaces to VXLAN peers.
Scope

FortiGate v7.0and v7.2.

Solution

In this scenario, there are two remote locations with FortiGates connected to the internet with the need to extend the LAN broadcast domain (10.10.10.0/24) among the two sites. Additionally, thanks to the use of switch interfaces, the LAN segments will be also able to access the internet through the local FortiGate (this is the advantage of using switch interfaces over virtual wire pair).

 

vxlan.png

 

The configuration:

 

Notes FGT-A FGT-B
IPSec VPN configuration. config vpn ipsec phase1-interface
    edit "vpnIpsecVxlan"
        set interface "port1"
        set peertype any
        set net-device disable
        set proposal aes128-sha256
        set remote-gw <FGT-B public IP>
        set psksecret <PSK>
    next
end

config vpn ipsec phase2-interface
    edit "vpnIpsecVxlan"
        set phase1name "vpnIpsecVxlan"
        set proposal aes256-sha256
    next
end
config vpn ipsec phase1-interface
    edit "vpnIpsecVxlan"
        set interface "port1"
        set peertype any
        set net-device disable
        set proposal aes128-sha256
        set remote-gw <FGT-A public IP>
        set psksecret <PSK>
    next
end

config vpn ipsec phase2-interface
    edit "vpnIpsecVxlan"
        set phase1name "vpnIpsecVxlan"
        set proposal aes256-sha256
    next
end

Set IP addresses on IPsec tunnel interfaces.

These will be the IPs used to encapsulate the traffic with VXLAN.

config system interface
    edit "vpnIpsecVxlan"
        set vdom "root"
        set ip 10.0.0.1 255.255.255.255
        set type tunnel
        set remote-ip 10.0.0.2 255.255.255.0
        set interface "port1"
    next
end
config system interface
    edit "vpnIpsecVxlan"
        set vdom "root"
        set ip 10.0.0.2 255.255.255.255
        set type tunnel
        set remote-ip 10.0.0.1 255.255.255.0
        set interface "port1"
    next
end

Set up the VXLAN peer based on the IPsec tunnel interface.

 

This configuration will generate a VXLAN interface.

 

VNI must match

config system vxlan
    edit "vxlanInterface"
        set interface "vpnIpsecVxlan"
        set vni 1
        set remote-ip "10.0.0.2"
     next
end
config system vxlan
    edit "vxlanInterface"
        set interface "vpnIpsecVxlan"
        set vni 1
        set remote-ip "10.0.0.1"
    next
end

 

Create a switch interface to bridge the local LAN interface with the newly created VXLAN interface

 

** Implicit is the default config and it allows everything between port3 and the VLAN interface, set it to explicit to enforce firewall policies on this traffic and manage MTU as explained at the end of the table

 

config system switch-interface
    edit "switchVxlan"
        set vdom "root"
        set member "port3" "vxlanInterface"

        set intra-switch-policy implicit **
    next
end

config system switch-interface
    edit "switchVxlan"
        set vdom "root"
        set member "port3" "vxlanInterface"
    next
end

set an IP on the switch interface to use the local FortiGate as a gateway for the connected LAN segment.

 

This is not required to make VXLAN work.

config system interface
    edit "switchVxlan"
        set vdom "root"
        set ip 10.10.10.3 255.255.255.0
        set allowaccess ping
        set type switch
    next
end
 
Configure policy to allow VXLAN (udp 4789) on the IPsec tunnel. config firewall policy
    edit <n>
        set name "policy"
        set srcintf "vpnIpsecVxlan"
        set dstintf "vpnIpsecVxlan"
        set action accept
        set srcaddr "all"
        set dstaddr "all"
        set schedule "always"
        set service "vxlan_udp_4789"
        set logtraffic all
    next
config firewall policy
    edit <n>
        set name "policy"
        set srcintf "vpnIpsecVxlan"
        set dstintf "vpnIpsecVxlan"
        set action accept
        set srcaddr "all"
        set dstaddr "all"
        set schedule "always"
        set service "vxlan_udp_4789"
        set logtraffic all
    next

 

Test:

  1. ping from PC1 to PC2 captured by sniffing with level 6 traffic from 10.0.0.1 to 10.0.0.2:

 

ws_vxlan.png

 

  1. VXlan forwarding database:

 

FGT-A# diagnose sys vxlan fdb list vxlanInterface
mac=00:00:00:00:00:00 state=0x0082 remote_ip=10.0.0.2 port=4789 vni=1 ifindex=22
mac=00:69:72:6f:59:01 state=0x0002 remote_ip=10.0.0.2 port=4789 vni=1 ifindex=22

 

FGT-A# diagnose netlink brctl name host switchVxl

show bridge control interface switchVxlan host.
...
2 23 vxlanInterface 00:69:72:6f:59:01 120 Hit(120)

 

PC2 MAC address

 

ps2mac.PNG

 

 

  1. Internet access from PC1 locally:

 

port3 in 10.10.10.1 -> 8.8.8.8: icmp: echo request
switchVxlan in 10.10.10.1 -> 8.8.8.8: icmp: echo request
port1 out 10.5.26.176 -> 8.8.8.8: icmp: echo request
port1 in 8.8.8.8 -> 10.5.26.176: icmp: echo reply
switchVxlan out 8.8.8.8 -> 10.10.10.1: icmp: echo reply

port3 out 8.8.8.8 -> 10.10.10.1: icmp: echo reply

 

To avoid packet fragmentation, could be useful to use the option set intra-switch-policy explicit under switch interface configuration and add an ad hoc firewall policy, to filter traffic between switch-interface members, as explained here: 

Issues-with-PMTUD-and-VXLAN 

 

Note that:

  • If set service 'ALL' configuration is not used under the intra-switch interface traffic firewall policy, it is useful to permit ICMP beyond the desired traffic, to not discard ICMP probe packets.
  • ON VXLAN over IPSEC scenario, consider also IPsec overhead, as explained here: Troubleshooting-IPsec-VPN-tunnel-errors-with-large-packets 

 

Related document:

VXLAN over IPsec tunnel with virtual wire pair