Visitor III

Question

IPsec VPN - pkt-Size mismatch due to LDAP Group Overflow

Forum|Forum|8 months ago
September 24, 2025
3 replies
1190 views

Hello everyone,
we're currently experiencing issues with some users trying to connect to our IPsec VPN. The FortiClient times out during connection, even though the configuration and password are correct.
After investigation, I found that the root cause seems to be related to users who are members of a large number of LDAP groups. For those users, the FortiGate fails to process the authentication request and throws the following error:

diagnose debug application ike -1 diagnose debug application authd 60 diagnose debug application samld -1 diagnose debug application fnbamd -1 diagnose debug application eap_proxy -1 diagnose debug console timestamp enable diagnose debug enable

2025-09-19 11:57:45 1758275865.817240: 2025-09-19 11:57:45 EAP-TTLS/PAP: Correct user password 2025-09-19 11:57:45 1758275865.817297: 2025-09-19 11:57:45 EAP-TTLS: PHASE2_START -> SUCCESS 2025-09-19 11:57:45 1758275865.817396: 2025-09-19 11:57:45 EAP-TTLS: Derived key - hexdump(len=64): 2025-09-19 11:57:45 1758275865.817513: 2025-09-19 11:57:45 EAP: EAP entering state SELECT_ACTION 2025-09-19 11:57:45 1758275865.817569: 2025-09-19 11:57:45 EAP: getDecision: method succeeded -> SUCCESS 2025-09-19 11:57:45 1758275865.817625: 2025-09-19 11:57:45 EAP: EAP entering state SUCCESS 2025-09-19 11:57:45 1758275865.817681: 2025-09-19 11:57:45 EAP: Building EAP-Success (id=218) 2025-09-19 11:57:45 1758275865.817741: 2025-09-19 11:57:45 CTRL-EVENT-EAP-SUCCESS 00:00:00:00:00:00 2025-09-19 11:57:45 1758275865.817853: 2025-09-19 11:57:45 RADIUS SRV: Reply to 127.0.0.1:2480 2025-09-19 11:57:45 2025-09-19 11:57:45 [831] __rad_rxtx-fd 11, state 2(Challenged) 2025-09-19 11:57:45 RADIUS SRV: Removing completed session 0xf71 after timeout 2025-09-19 11:57:45 [833] __rad_rxtx-Stop rad conn timer. 2025-09-19 11:57:45 [883] __rad_rxtx- 2025-09-19 11:57:45 [432] __rad_udp_recv-Recved 8192 bytes. Buf sz 8192 2025-09-19 11:57:45 [1214] fnbamd_rad_validate_pkt-Size mismatch rx=8192 pkt=8492 2025-09-19 11:57:45 [908] __rad_rxtx-Error validating radius rsp 2025-09-19 11:57:45 [1031] __rad_error-Ret 5, st = 2. 2025-09-19 11:57:45 [301] fnbamd_radius_get_next_auth_prot-Next auth prot EAP 2025-09-19 11:57:45 [1080] __rad_error- 2025-09-19 11:57:45 [307] __rad_udp_close-closed. 2025-09-19 11:57:45 [967] __rad_conn_stop-Stop rad conn timer. 2025-09-19 11:57:45 [1301] fnbamd_rad_process-Result from radius svr 'EAP_PROXY' is 5, req 74036713144255 2025-09-19 11:57:45 [1503] fnbamd_rad_process-Challenged: 0, FTK_Challenge: 0, CHG_PWD: 0, Invaid_Digest: 0, No_Message_Authenticator_Attr: 0, State_Len: 0 2025-09-19 11:57:45 [2804] fnbamd_rad_result-Error (5) for req 74036713144255 2025-09-19 11:57:45 [133] fnbamd_comm_send_result-Not enough buffer for EAP message 2025-09-19 11:57:45 [239] fnbamd_comm_send_result-Sending result 5 (nid 0) for req 74036713144255, len=6688 2025-09-19 11:57:45 [600] destroy_auth_session-delete session 74036713144255 2025-09-19 11:57:45 [1342] fnbamd_rad_stop- 2025-09-19 11:57:45.819054 ike V=root:0:VPN_User:14365 EAP 74036713144255 result FNBAM_ERROR 2025-09-19 11:57:45.819084 ike V=root:0:VPN_User: EAP failed for user "user"

Environment:
FortiOS: 7.4.8
FortiClient: 7.4.3.1790

Configuration Overview:

config user group     edit "LDAP_VPN_USER"         set member "contoso-Domain-Controller"         config match             edit 1                 set server-name "contoso-Domain-Controller"                 set group-name "CN=GG_VPN_USER,OU=GG,OU=04_Groups,DC=contoso,DC=intern"             next         end     next end  config user ldap     edit "contoso-Domain-Controller"         set server "ip 1"         set secondary-server "ip 2"         set cnid "sAMAccountName"         set dn "dc=contoso,dc=intern"         set type regular         set username "contoso\\username"         set password ENC x     next end  config vpn ipsec phase1-interface     edit "VPN_User"         set eap enable         set eap-identity send-request         set authusrgrp "LDAP_VPN_USER"     next end

I already tried the steps in this Fortinet KB article, but unfortunately, reapplying the certificate settings didn’t resolve the issue.
https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-How-to-resolve-the-error-Not-enough-buffer/ta-p/407210

I also found a bug fix mentioned in FortiOS 7.6.1:
Bug ID 1023871
IPSec IKEv2 with SAML cannot match the Entra ID group during EAP due to a buffer size issue.

However, we are not using SAML or Entra ID in our setup.

My questions:
Is this a known bug affecting LDAP-based EAP authentication?
Is there a workaround to increase the buffer size or reduce the group data being sent?
Will Bug ID 1023871 potentially fix this issue even though it mentions SAML?

Any insights or suggestions would be greatly appreciated!

Anthony_E

Staff

Hello,

Thank you for using the Community Forum. I will seek to get you an answer or help. We will reply to this thread with an update as soon as possible.

Thanks,

Best Regards

Anthony_E

Staff

Hello,

We are still looking for someone to help you.

We will come back to you ASAP.

Thanks,

Best Regards

kaman

Staff

Hi mycoolusername,

Bug ID 1023871 is for IPSec IKEv2 with SAML - fail to match Entra ID group during EAP on 7.2.8 (7.4.3 works) not for the ldap. I couldn't find any bug regarding issue with the LDAP query size.

In general, LDAP can't talk to EAP Protocol which is responsible for carrying authentication data for IKEv2.

That's not an IKEv2 specific issue, it affects anything that uses EAP.

Since there is no defined RFC regarding this implementation for IKEv2 FortiOS uses a proprietary implementation to achieve LDAP auth for Dialup IKEv2. This is the reason why 3rd party clients will not be able to connect.

In this case you can use IKEv1 or check the guide below.

https://docs.fortinet.com/document/fortigate/7.6.4/administration-guide/442351/ldap-authentication-with-ikev2-using-udp-or-tcp-as-transport

https://docs.fortinet.com/document/forticlient/7.4.0/new-features/907253/eap-ttls-support-for-ipsec-vpn-7-4-3

You will need to use FortiClient 7.4.3 and newer version and you will need to modify XML file, please check the guide below:

https://community.fortinet.com/t5/FortiGate/Technical-Tip-How-to-enable-EAP-TTLS-for-IPSec-IKEv2-tunnels-in/ta-p/408602

If still having the same issue please collect the same debug logs and share with us.

diag debug reset
diagnose debug cons time en
diag debug application fnbamd -1
diagnose debug app eap_proxy -1
diag vpn ike log filter rem-addr4 <RemoteClientpublicIp>
diag debug app ike -1
diagnose debug enable

Regards,
Aman

mycoolusernameAuthor

Visitor III

Hello kaman,

thanks for the information. The VPN was already setup using the guides from the links in your post.
The issue still exists and here is the full debug log using your debug commands.
When removing some amount of groups from the user in the Active Directory, the user is able to connect to the VPN. Re-adding some amount of groups and he fails to connect again.

diag debug reset diagnose debug cons time en diag debug application fnbamd -1 diagnose debug app eap_proxy -1 diag vpn ike log filter rem-addr4 91.186.6x.xxx diag debug app ike -1 diagnose debug enable

I cant paste the full log in here, since the forum post only allows for max 150k characters.
Do you have a pastebin like services to post logs?

Sign up

Login with SSO

Login to the community

Login with SSO

Scanning file for viruses.

This file cannot be downloaded