FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Hsharma
Staff
Staff
Article Id 394064
Description This article describes how to troubleshoot an issue where SSL connection fails in FIPS-MODE due to the error 'encoding method rejected in fips-cc mode'.
Scope

FortiGate and FortiClient.

Solution

When FortiGate is running in FIPS Mode, SSL connection may not establish when using a tunnel mode FortiClient connection.

 

In the SSL VPN debug logs, the following output is seen.

 

diag debug disable

diag debug application sslvpn -1

diad debug enable

 

mt_web_auth_info_parser_common:533 no session id in auth info
2025-05-26 19:43:14 [3852:root:2348]rmt_web_get_access_cache:885 invalid cache, ret=4103
2025-05-26 19:43:14 [3852:root:2348]User Agent: FortiSSLVPN (Windows NT; SV1 [SV{v=02.01; f=07;}])
2025-05-26 19:43:14 [3852:root:2348]get_cust_page:125 saml_info 0
2025-05-26 19:43:14 [3852:root:2348]req: /remote/logincheck
2025-05-26 19:43:14 [3852:root:2348]Transfer-Encoding n/a
2025-05-26 19:43:14 [3852:root:2348]Content-Length 205
2025-05-26 19:43:14 [3852:root:2348]readPostEnter:19 Post Data length 205.
2025-05-26 19:43:14 [3852:root:2348]User Agent: FortiSSLVPN (Windows NT; SV1 [SV{v=02.01; f=07;}])
2025-05-26 19:43:14 [3852:root:2348]rmt_web_auth_info_parser_common:533 no session id in auth info
2025-05-26 19:43:14 [3852:root:2348]rmt_web_access_check:804 access failed, uri=[/remote/logincheck],ret=4103,
2025-05-26 19:43:14 [3852:root:2348]encoding method 0
2025-05-26 19:43:14 [3852:root:2348]encoding method rejected in fips-cc mode
2025-05-26 19:43:14 [3852:root:2348]could not decode 'enc' data properly.
2025-05-26 19:43:14 [3852:root:2348]User Agent: FortiSSLVPN (Windows NT; SV1 [SV{v=02.01; f=07;}])
2025-05-26 19:43:14 [3852:root:2348]Transfer-Encoding n/a
2025-05-26 19:43:14 [3852:root:2348]Content-Length 205
2025-05-26 19:43:34 [3852:root:2348]Timeout for connection 0x7fa8236800.

 

When the connection is tested with web mode, it works as expected.

 

The issue is with the FortiClient version which is not compatible with the FIPS configuration on firewall. 

 

FortiClient version 7.2.5, 7.4.1, or higher should be used with FortiGate with FIPS enabled.