Technical Tip: Types of design to avoid when users behind VXLAN require a gateway

[fwilliams]

Description	This article describes a design that is not suitable for a solution looking to provide service to users behind VXLAN beyond the local subnet e.g. Internet access. VXLAN is discussed in detail in rfc7348.
Scope	FortiOS (platforms running kernel version 3.2 and above support VXLAN).
Solution	The topology above depicts a solution where users behind VXLAN need access to other resources by breaking out through the local gateway, which was configured on port1 on FortiGate since it is not possible to assign an IP address to a VWP (which is the binding method used in the above design with 'internal port' & 'VXLAN interface' as members). The agenda when designing the above network is that traffic from siteB will flow through the VXLAN, hit the SiteA VXLAN interface, then flow to the internal port, down to port3 on the LAN switch, and finally travel up to SiteA port1 (through port4) where the Gateway IP address is configured. After, the traffic will break out to other internal subnets or the internet.  This design will not work: a ping to the Gateway IP from the PC or user behind siteB, for example, will respond briefly and stop (there may be 3 ping responses or fewer). The best design practice in cases like this is to use the Virtual Switch as 'internal port/s', 'VXLAN interface' the binding method, and assign the Gateway IP directly on the virtual switch interface. The following is a breakdown of the 2 ways the internal interface/s can be bound with the VXLAN: A. Virtual Wire Pair (VWP): this is one way to bind the internal port to the VXLAN interface. But the Gateway IP address assignment or routing to a different subnet is not provided with this method, as VWP is layer2 and only permits exactly 2 interfaces (the VXLAN and internal port, in this case) as members. Virtual Wire Pair configuration: config system virtual-wire-pair     edit "vwp"         set member "internal port" "vxlan"  <- Can accommodate exactly 2 interfaces.         set wildcard-vlan enable     next end B. Virtual Switch. This is the preferred binding method if the Gateway or routing to other networks needs to be configured for users behind the internal port/s or VXLAN (i.e. traffic from the remote side of the VXLAN). This way, users can access other resources residing on a different network, including the internet (the routing, NAT and Firewall policy check will be done on FortiGate-A, in this example). Virtual Switch configuration: config system switch-interface     edit "SW"         set vdom "root"         set member "internal port" "vxlan" "internal port2"…  <- Can accommodate more than 2 interfaces.         set intra-switch-policy {implicit | explicit}     next end With the virtual switch binding method, traffic flow will be similar to the following, as the Gateway IP is directly assigned on the virtual switch interface: For more informations on VxLAN, refer to: General VXLAN configuration and topologies  Technical Tip: How to setup a VXLAN over IPsec deployment