| Description | This article analyzes some special cases where 'Signature verification failed' appears on IKE debug. |
| Scope | FortiGate, FortiClient, 3rd party dial up software. |
| Solution |
A brief explanation of how signature verification works, after encryption keys and algorithms are negotiated on the first 2 messages (IKE_SA_INIT) FortiGate will attempt to confirm the endpoint by validating the signature presented. This signature is in ASN1 format, \xFF padded and encrypted with the certificate private key. It is sent on the first authentication message (IKE_AUTH message ID 1) under 'Authentication' Payload (ID:39).
FortiGate uses the certificate public key to decrypt and retrieve the signature which it will cross validate. Under normal circumstances the error 'Signature verification failed' appears when a different signature algorithm is used, therefore the signature does not match. However there is a common case where the endpoint leaves the 'Authentication' Payload completely empty.
If this happens, 'Signature verification failed' will appear on IKE debug pointing to an issue of signing algorithm but the actual problem is that the signature is missing completely.
This can happen in some third party dial up software like StrongSwan on Linux, or when the certificate is pulled from USB key solutions like SafeNet. When the 'Signature verification failed' error appears on the IKE debug, it is recommended to decrypt the traffic and verify that the encrypted byte array is sent by the endpoint before starting any investigation on FortiGate. The following article explains how to do it: Technical Tip: How to decrypt IPSec Phase-2 (ISAKMP) packets IKEv2 |
