Problems with Windows Active Directory services through VXLAN over IPsec VPN

[aescobar]

Hello Community.

I would like to ask for your help with a Fortigate 200F (HA- A/A), with FortiOS 7.0.9.
I have an IPsec VPN with a customer, who has a Fortigate 400E on the same Firmware version. Through this IPsec VPN I am extending a vlan via VXLAN (10.100.0.0/23) (Vlan ID 1). The communication between the sites is correct for the computers that are not inside the Domain. This is evidenced, when entering the Domain (It is in Site B- Fortigate 400E), a computer that is in site A (Fortigate 200F). Windows authentication takes at least 30-45 minutes, they never update the GPOs and the client machine performance is super slow. When I remove the client machine from the domain, the machine starts to perform optimally. My suspicion is, that the fortigate has problems handling traffic from Kerberos, LDAP, NetBios, Etc. over VXLAN.

These are my VPN over VXLAN configurations:

config vpn ipsec phase1-interface
edit "VPN_SiteB"
set interface "port14"
set peertype any
set net-device disable
set proposal aes256-sha256
set remote-gw 185.20.X.X

set psksecret ENC PASSWRD
next
end

config vpn ipsec phase2-interface
edit "VPN_SiteB"
set phase1name "VPN_SiteB"
set proposal aes256-sha256
set auto-negotiate enable
next
end

config system switch-interface

edit "SW_VXLAN_1"
set vdom "root"
set member "port17" "VXLAN.1"
next

config system vxlan
edit "VXLAN.1"
set interface "VPN_SiteB"
set vni 1
set remote-ip "10.72.19.2"
next
end

config system interface

edit "VPN_SiteB"
set vdom "root"
set ip 10.72.19.1 255.255.255.255
set allowaccess ping
set type tunnel
set remote-ip 10.72.19.2 255.255.255.252
set interface "port14"
next

edit "SW_VXLAN_1"
set vdom "root"
set ip 10.100.0.253 255.255.254.0
set allowaccess ping
set type switch 
next

end

I appreciate your help.

[Anthony_E]

Hello,

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible.

Thanks,

[funkylicious]

Have you tried configuring MTU for the IPsec interface to a lower value like 1372 if not already ?

[aescobar]

Hi funkylicious.

I have changed the MTU size on the IPsec VPN interface of site A and site B to (set mtu-override enable, set mtu 1372), but the behaviour is the same. Even when client machines at Site A go to log in while inside the domain (Domain Controller at Site B) it takes 30 to 45 minutes. I have also disabled hardware acceleration within Phase 1 of the VPN at both sites and the behaviour is the same.

I look forward to any other ideas that can help me resolve this.

Thank you.

[funkylicious]

I know it's a long shot, but have you tried switching to a-p and see if the problem persists ?

[aescobar]

Thanks for the suggestion, I applied it last night on both Firewalls, setting the cluster to Active-Passive mode but the behavior is the same. I have also tried enabling the following with no result:

config vpn ipsec phase1-interface
edit VPN_SiteB
set ip-fragmentation pre-encapsulation
set npu-offload disable
next
end

I am still reviewing the issue

[aescobar · Accepted solution]

Solution that worked for me.

After reviewing the case with Fortinet's TAC, we found that the problem was indeed caused by the size of the MTUs. The Windows client machines were trying to send packets with an MTU greater than 1500 and in addition to the payload that is added in the VPN communication (Approx. 28 Bytes), the packet is too big. It was not possible to make the connection work, forcing the MTU Override on the IPsec VPN interfaces, neither on the WAN Interfaces where the VPN was travelling, nor on the LAN interfaces. It was only possible to make the communication work, disabling in both Fortigates "set honor-df disable" with which FortiGate is able to ignore the 'do not defragment' bit of a packet.

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Global-setting-honor-df-explained/ta-p/197...

Regards.

[Kangming]

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Setting-TCP-MSS-value/ta-p/194518

TCP MSS may be helpful.