Troubleshooting Tip: FortiClient TLS 'error 5029': failed to establish the VPN connection

[akumarr]

Description

This article describes how to rectify the 'failed to establish the VPN connection', '5029 error'.

Solution

While connecting the FortiClient, the following error may appear.

This error happens because of the TLS mismatch.

Go to Internet Explorer -> Settings -> Internet options -> Advanced, scroll down, and check the TLS version.

In the image above, only TLS 1.2 is selected on the client end while FortiGate does not support TLS 1.2. Check the output below.
Verify the validity of the TLS settings configured on the FortiGate end as well as the TLS settings on the client end.

sh ful
config vpn ssl settings
    set reqclientcert disable
    set ssl-max-proto-ver tls1-1
    set ssl-min-proto-ver tls1-0

Next, select TLS 1.1 and TLS 1.0 on the client machine end or change the TLS version to 1.2 on the FortiGate end.
Change the settings on the client machine end.
As soon as settings are changed, connecting the FortiClient will be possible.

Server Certificate.

If all step here has been followed by still getting the same error to connect, make sure to check the server certificate are set and not empty. This can be verified under SSL-VPN Setting -> Server.

Certificate: change it accordingly.

After the certificate has been set, it will be possible to connect to SSL-VPN.

Another possible reason for this error if the above steps did not help is if FortiGate uses a self-signed certificate as an SSL VPN server certificate and there is another firewall in between which performs certificate inspection.

                                                                        

In the following packet capture, the client sent an alert (Level: Fatal, Description: Illegal Parameter) after the 'Certificate, Server key Exchange, Server Hello Done'.

                                                                               

The solution is to either: 

1. Disable certificate inspection on intermediary firewall/s.
2. Use a trusted certificate signed by a public certificate authority for the SSL VPN server certificate on the FortiGate.