Technical Tip : FortiClient valid phase1 hash algorithm for SmartCard RSA signature authentication

Description
Two IKE phase1 authentication methods can be used for IPSec VPN :

- pre-shared key
- RSA signature (rsa-sig)

The key pair (private key + certificate) used for rsa-sig authentication can be :

- stored on the FortiClient itself ('X509 Certificate')
- retrieved from a secured eToken repository ('SmartCard X509 Certificate')

The hash algorithm negotiated between two IKE peers is used to create a signed hash (HMAC) that is used for authentication.

FortiClient 3.0 and 4.0 are only able to use MD5 hash algorithm to create the HASH payload when SmartCard is used.

If SHA-1 is used, an erroneous HASH payload is generated and subsequently signed (SIG payload) by FortiClient.
This SIG payload is then sent to the remote peer which fails to process it.

If FortiOS is used as dialup server, then FortiOS IKE debug will report "signature verification failed" upon receipt of the erroneous SIG payload.
Scope
All FortiClient 3.0 releases
FortiClient 4.0, 4.1.1 to 4.1.3
Solution
Support for HMAC SHA-1 has been added in :

- FortiClient 4.1.4 and above
- FortiClient 4.2 and above


Only HMAC MD5 is supported for :

- FortiClient 3.0 (all MR)
- FortiClient 4.0 GA
- FortiClient 4.1 to 4.1.3

For these releases, only select MD5 as hash algorithm in phase1 proposals when SmartCard is used, as shown in the FortiClient screen-shot below :