| The following error is noticed in the IKE debugs: 'mismatched DH group in KE payload'.
2025-06-10 12:43:14.462965 ike V=root:0:IPSecVPN:56: mismatched DH group in KE payload, selected 14, received 5
2025-06-10 12:43:14.462993 ike V=root:0:IPSecVPN:56: sending INVALID_KE notify
2025-06-10 12:43:14.463024 ike 0:IPSecVPN:56: out E16C2C2451F1195200000000000000002920222000000000000000260000000A00000011000E
2025-06-10 12:43:14.463100 ike V=root:0:IPSecVPN:56: sent IKE msg (INVALID_KE_PAYLOAD): xx.xx.xx.xx:500->yy.yy.yy.yy:1012, len=38, vrf=0, id=e16c2c2451f11952/000000000
0000000, oif=5
This error occurs when a negotiation failure happens for the DH-Group. In the above error, it can be concluded that FortiGate is receiving negotiation for DH-Group=5, but on FortiGate, DH-Group is set to 14.
Solution: - Verify if the DH group is the same on both peers.
- Verify if PFS is enabled on the phase2 selectors, are match on both the peers.
To change the DH-Group on FortiGate via CLI, the following commands can be used:
config vpn ipsec phase1-interface
edit "tunnel-name"
set dhgrp <DH number>
end
config vpn ipsec phase2-interface
edit "tunnel-name"
set dhgrp <DH number>
end
More information on how to check regarding the DH group can be found in this KB article: Technical Tip: How to check if Diffie-Hellman(DH) group is the same on both peer units. Execute the following debug commands to collect IKE logs:
diagnose vpn ike log-filter dst-addr4 xx.xx.xx.xx <----- Public IP of remote-end.
diagnose debug console timestamp enable
diagnose debug application ike -1
diagnose debug enable
Note:
Starting from v7.4.1, the 'diagnose vpn ike log-filter dst-addr4' command has been changed to 'diagnose vpn ike log filter rem-addr4'. Starting from v7.6.5 and v8.0.0, the default DH groups for Phase1 and Phase2 IPsec VPN tunnels will be updated from 14 and 5 to 14, 20, and 21. This is stated in the v7.6.5 release notes: Changes in default behavior.
This change is to further increase the cryptographic strength used for the tunnel, as DH group 5 is considered to be outdated.
There can be instances where the IKE debug may not output the exact error due to a DH group, such as :
ike V=root:0: comes x.x.x.x:500->y.y.y.y:500,ifindex=41,vrf=0,len=92....
ike V=root:0: IKEv1 exchange=Informational id=04dfe542abd0d7cf/810995424accf382:c51b4c9c len=92 vrf=0
...
ike V=root:0:IPsec_Tunnel :49825: notify msg received: NO-PROPOSAL-CHOSEN
ike V=root:0:IPsec_Tunnel :49825:: no matching IPsec SPI
If the IPsec tunnel is down after upgrading to v7.6.5, and it is verified that the DH group is mismatched, the solution is to change the DH group in the VPN settings, make sure both 2 sides have identical configurations. Either change back the FortiGate DH group to 14 and 5, or change the peer DH group to 14, 20, and 21 (If the FortiGate is using the default DH group value before upgrade).
Related articles:
Troubleshooting Tip: The IPsec VPN tunnel not coming up, with debug message 'ignoring IKE request, interface is administratively down' Troubleshooting Tip: Troubleshooting IPsec Site-to-Site Tunnel Connectivity | 
