EVPN VXLAN overlays <=> VDOMs

[jakoandras]

I'd like to use EVPN VXLAN similarly to VLANs with VDOMs. Different VLANs on one physical port can be placed in different VDOMs. I'd like to do the same with VXLANs, something like this:

ngfw (root) # sh system evpn

config system evpn
    edit 702
        set rd "10.0.1.2:702"
        set import-rt "1:6006"
        set export-rt "1:6006"
        set ip-local-learning enable
        set arp-suppression enable
    next
    edit 703
        set rd "10.0.1.2:703"
        set import-rt "1:6012"
        set export-rt "1:6012"
        set ip-local-learning enable
        set arp-suppression enable
    next
end

ngfw (root) # sh system vxlan

config system vxlan
    edit "vxl702"
        set interface "lo0"
        set vni 6006
        set evpn-id 702
    next

    edit "vxl703"
        set interface "lo0"
        set vni 6012
        set evpn-id 703
    next

end

ngfw (root) # sh sys int vxl702
config system interface
    edit "vxl702"
        set vdom "tenant702"
        set type vxlan
        set interface "lo0"
    next
end

ngfw (root) # sh sys int vxl703
config system interface
    edit "vxl703"
        set vdom "tenant703"
        set type vxlan
        set interface "lo0"
    next
end

I tried this on a 1800F with 7.4.8 and 7.6.3. Trying to change the VDOM of the VXLAN interfaces from root to the tenant's VDOM gives the following error:

Virtual domain can not be changed for vxlan interface.
node_check_object fail! for vdom tenant702

value parse error before 'tenant702'
Command fail. Return code -651

So I'd like to have the underlay interface (lo0) doing VXLAN encapsulation in the root VDOM, while placing the overlay interfaces in other VDOMs. Is it possible?

Background:

In our network, tenants (not really tenants, they are the departments of our university) use VXLAN overlay networks, so that they can connect their internal networks to their VDOM.

We have a HA pair of 1800Fs, located in two buildings, connected to two different routers. Tenants have VXLANs with VTEPs a few routing hops away from the FortiGates. As I need to connect their VXLANs to their VDOMs, I must terminate their VXLANs on the two routers connected directly to the FortiGates, and use 802.1Q VLANs between the router and the adjacent FortiGate. This way these routers translate VNI to VLAN ID and vice versa.

It would be much easier if I wouldn't have to do anything with these VXLANs on the two routers, neither allocate the same VLAN ID on both routers to be used between the router and the firewall. These routers should only pass VXLAN encapsulated traffic as members of the underlay network, and the FortiGates could terminate the VXLAN inside the appropriate VDOMs.

Regards,

András

[Jean-Philippe_P]

Hello jakoandras, 

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible. 

Thanks, 

[Jean-Philippe_P]

Hello,

We are still looking for an answer to your question.

We will come back to you ASAP.

Thanks,

[Jean-Philippe_P]

Hello again!

I found this solution. Can you tell me if it helps, please?

In FortiGate, particularly with the 1800F series and the versions you mentioned (7.4.8 and 7.6.3), the handling of VXLAN interfaces and their association with VDOMs can be somewhat restrictive.

As you've discovered, VXLAN interfaces are typically tied to the root VDOM, and attempting to change their VDOM assignment results in an error. This limitation arises because VXLAN encapsulation and the associated configurations are often designed to be managed at a higher level (the root VDOM) to maintain consistency and simplify routing and bridging operations.

Current Limitations

1. VXLAN Interface Binding: The VXLAN interfaces must remain in the root VDOM. This is a design choice in FortiGate's architecture to ensure that VXLAN encapsulation is handled uniformly across the device.
   
   

2. Routing and Policies: Since the VXLAN interfaces cannot be moved to individual VDOMs, you may need to manage routing and security policies at the root VDOM level. This means that while the VXLAN traffic can be encapsulated and forwarded through the root VDOM, the actual policy enforcement for tenant traffic would need to be handled through appropriate routing and firewall policies.
   
   

Possible Workarounds

While direct assignment of VXLAN interfaces to specific VDOMs is not supported, you can consider the following approaches:

1. Routing Policies: Use routing policies in the root VDOM to direct traffic to the correct tenant VDOMs. You can create policies that route traffic based on the VXLAN Network Identifier (VNI) or other criteria, allowing you to effectively segregate tenant traffic while keeping the VXLAN interfaces in the root VDOM.
   
   

2. Subinterfaces with VLANs: If feasible, consider using VLANs for tenant segmentation instead of VXLANs. This would allow you to utilize the existing capability to assign VLAN subinterfaces to different VDOMs, which might simplify management and policy application.
   
   

3. EVPN and VXLAN Configuration: Continue to leverage EVPN for VXLAN overlay management. Ensure that your EVPN configuration is set up properly to facilitate the distribution of MAC addresses and IP information across the VXLAN segments.
   
   

4. Consult Fortinet Support: Since this involves specific configurations and potential limitations in the FortiGate OS, reaching out to Fortinet support or consulting the official documentation may provide additional insights or updates on feature enhancements in newer versions.
   
   

Conclusion

In summary, while it is not currently possible to assign VXLAN interfaces directly to tenant VDOMs in FortiGate, you can manage tenant traffic through routing policies and potentially consider alternative configurations such as VLANs. If your use case requires flexibility beyond what is currently supported, it may be beneficial to keep an eye on future FortiGate firmware updates or enhancements that could address these limitations.

[jakoandras]

Dear Jean-Philippe,

Thanks for your answer and the suggested workarounds. I hope to find time in the following days to check them, and I will get back to you afterwards.

Regards,

András