FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
MigenaM
Staff
Staff
Article Id 389059
Description This article describes the scenario where the STARTTLS connection cannot be established to the FortiMail Server when SSL/SSH deep-inspection is enabled on the firewall policy.
Scope FortiGate, FortiMail.
Solution

In this scenario, a FortiGate is placed before the FortiMail.

Some mail servers cannot establish a STARTTLS connection to this infrastructure.

 

SSL/SSH is enabled on the firewall policy, and this can also be configured with web server protection active, as seen below:

 

SSL-SSH web server protection.PNG

 

The errors seen on FortiMail are as follows:

'STARTTLS=server, error: accept failed=-1, reason=unknown, SSL_error=5, errno=0, retry=-1, relay=<mailserver> <IP address>'

 

And on FortiGate:

'Client connection failure due to incorrect SSL handshake message.(type:1)'

 

If the SSL/SSH profile is disabled on the firewall policy, the issue is no longer present.

Mostly, this error occurs for 2 reasons:

  • FortiMail is being inspected by an upstream firewall.
  • SSL/TLS negotiation is not correct.

 

The following can be done to mitigate the issue:

  • Under the SSL/SSH profile configured to use for this configuration:

 

config firewall ssl-ssh-profile
     edit "Clone of custom-deep-inspection"
         config ssl
             set min-allowed-ssl-version <----- Check here what the version is and allow the rest of the tls versions.

 

The options are as follows:

(ssl) # set min-allowed-ssl-version
        ssl-3.0 SSL 3.0.
        tls-1.0 TLS 1.0.
        tls-1.1 TLS 1.1.
        tls-1.2 TLS 1.2.
        tls-1.3 TLS 1.3.

 

  • This same change can be done on the FortiMail device as well under System Global:

 

config system global

    set ssl-versions ?

           tls1_0 TLS 1.0
           tls1_1 TLS 1.1
           tls1_2 TLS 1.2
           tls1_3 TLS 1.3

 

While the configuration and the certificate used on the FortiGate and FortiMail are the same, if the issue is only happening for some recipients, it might be that these senders do not match the settings configured on FortiGate, or are using lower TLS versions, so these changes might be helpful in this case.