Description
This article describes issues that occur during VPN establishment due to 'signature verification failed' errors in IKE debug logs for an IKEv2 certificate based IPsec VPN.
Scope
All versions of FortiGate.
Solution
This error is seen if FortiGate is unable to validate the contents of an IKE AUTH payload from a peer (which is signed by the peer certificate) and is indicated as 'signature' in the IKE debugs on FortiGate. It generally suggests that there is a mismatch in the hash algorithm used for this signature generation.
Some vendors acquire this hash algorithm from the phase1 proposal being used. However, this is not the case with FortiOS. FortiGate does not derive this hash algorithm from the phase1 proposals and by default uses SHA-1 to avoid interoperability problems. So in some cases, the tunnel may fail to establish and return 'signature verification failed' errors if the sha1 phase1 proposal is not chosen (depending on whether the remote end derives the hash algorithm from the chosen proposals or not).
In such cases, if the remote gateway supports 'signature hash algorithm negotiation' (RFC 7427), explicitly set up the hash algorithms to be used on FortiGate:
config vpn ipsec phase1-interface
edit test_VPN
set digital-signature-auth enable
set signature-hash-alg ?
sha1 SHA1.
sha2-256 SHA2-256.
sha2-384 SHA2-384.
sha2-512 SHA2-512.
If enabled, it is possible to track the hash algorithm negotiation in the IKE debugs as well:
ike 1:1ad2b504c30cfbfc/0000000000000000:344: responder received SA_INIT msg
ike 1:1ad2b504c30cfbfc/0000000000000000:344: received notify type NAT_DETECTION_SOURCE_IP
ike 1:1ad2b504c30cfbfc/0000000000000000:344: received notify type NAT_DETECTION_DESTINATION_IP
ike 1:1ad2b504c30cfbfc/0000000000000000:344: received notify type FRAGMENTATION_SUPPORTED
ike 1:1ad2b504c30cfbfc/0000000000000000:344: received notify type SIGNATURE_HASH_ALGORITHMS ------------->
ike 1:1ad2b504c30cfbfc/0000000000000000:344: incoming proposal:
.
.
ike 1:1ad2b504c30cfbfc/0000000000000000:344: matched proposal id 2
ike 1:1ad2b504c30cfbfc/0000000000000000:344: proposal id = 2:
ike 1:1ad2b504c30cfbfc/0000000000000000:344: protocol = IKEv2:
ike 1:1ad2b504c30cfbfc/0000000000000000:344: encapsulation = IKEv2/none
ike 1:1ad2b504c30cfbfc/0000000000000000:344: type=ENCR, val=AES_CBC (key_len = 256)
ike 1:1ad2b504c30cfbfc/0000000000000000:344: type=INTEGR, val=AUTH_HMAC_SHA2_256_128
ike 1:1ad2b504c30cfbfc/0000000000000000:344: type=PRF, val=PRF_HMAC_SHA2_256
ike 1:1ad2b504c30cfbfc/0000000000000000:344: type=DH_GROUP, val=MODP2048.
ike 1:1ad2b504c30cfbfc/0000000000000000:344: lifetime=86400
ike 1:1ad2b504c30cfbfc/0000000000000000:344: SA proposal chosen, matched gateway test_VPN ----------->
.
.
ike 1:test_VPN:344: processing notify type SIGNATURE_HASH_ALGORITHMS
ike 1:test_VPN:344: hash algorithm 2 (sha2-256)
ike 1:test_VPN:344: hash algorithm 3 (sha2-384)
ike 1:test_VPN:344: hash algorithm 4 (sha2-512)
ike 1:test_VPN:344: matched hash algorithm 2 (sha2-256) --------------->
ike 1:test_VPN:344: responder preparing SA_INIT msg.
.
ke 1:test_VPN:344: Validating X.509 certificate
ike 1:test_VPN:344: peer cert, subject='10.5.52.59', issuer='testIssuingCA'
ike 1:test_VPN:344: peer CA cert, subject='testIssuingCA', issuer='testIssuingCA'
ike 1:test_VPN:344: peer ID verified
ike 1:test_VPN:344: building fnbam peer candidate list
ike 1:test_VPN:344: FNBAM_GROUP_NAME candidate 'peer_test_grp'
ike 1:test_VPN:344: certificate validation pending
ike 1:test_VPN:344: fnbam reply 'peer_test_grp'
ike 1:test_VPN:344: fnbam matched peergrp 'peer_test_grp'
ike 1:test_VPN:344: certificate validation complete
ike 1:test_VPN:344: certificate validation succeeded
ike 1:test_VPN:344: signature verification succeeded using sha256WithRSAEncryption, oid=1.2.840.113549.1.1.11
ike 1:test_VPN:344: auth verify done ----------------------------------->
ike 1:test_VPN:344: responder AUTH continuation
ike 1:test_VPN:344: authentication succeeded
ike 1:test_VPN:344: responder creating new child
From FortiOS v7.2.1 onwards, it is also possible to set up FortiGate to derive signature hash algorithms from the phase1 proposal matched when RSA is used. Do so with the following configuration:
config vpn ipsec phase1-interface
edit test_VPN
set rsa-signature-hash-override enable/disable (Disabled by default. When enabled, the signature hash algorithm is derived from the chosen phase1 proposal.)
NOTE: Settings 'rsa-signature-hash-override' and 'digital-signature-auth' are mutually exclusive. If one is enabled, the other is hidden.
