Technical Tip: A possible Root Cause for RADIUS Authentication Failure on FortiGate

Problem:

When FortiGate is authenticates a Radius server on behalf of remote Radius users, the authentication fails, even though the settings are configured properly and the connection between the FortiGate and the Radius server is reachable.

On RADIUS Server logs, the following can be noticed for why the connection was denied:' Reason Code: 96
Reason: Authentication failed due to an EAP session timeout; the EAP session with the access client was incomplete.'

On FortiGate, on running the fnbamd debug command:

diagnose debug application fnbamd -1

diagnose debug enable

To stop the debug processes in the end, press 'Ctrl+C' and enter 'diagnose debug disable'.

The following debug errors can be noticed: the connection was denied.

[1898] fnbamd_radius_auth_validate_pkt-RADIUS resp code 3
[430] extract_chap_error-CHAP err: E=691 R=0 V=3
[1549] fnbamd_auth_handle_radius_result-->Result for radius svr 'GTIRadiusP1' 10.224.105.11(1) is 1
[209] fnbamd_comm_send_result-Sending result 1 (nid 0) for req 1504684073, len=2536
authenticate 'testuser' against 'mschap2' failed, assigned_rad_session_id=1504684073 session_timeout=0 secs idle_timeout=0 secs!

Solution:

After confirming the correct configuration and consuming all possible troubleshooting steps, the problem can be related to the MTU value, and it's fixed by adjusting the MTU value on the Radius NPS server to a value of 1200. After this change, radius authentication should start working. 

The following document discusses the issue: MTU for NPS radius and radius client

Related article: 

Troubleshooting Tip: RADIUS authentication troubleshooting

Troubleshooting Tip: Possible reason for RADIUS Reject code 3

Technical Tip: FortiGate RADIUS authentication not working with Microsoft NPS when using accented or UTF-8 special characters in passwords