Technical Tip: Users Fail to Connect to IPSec VPN with Error 'mismatched DH group in KE payload'

When multiple Diffie-Hellman (DH) groups are configured in the Phase 1 IPsec VPN settings, FortiGate may respond to FortiClient with an INVALID_KE_PAYLOAD, requesting a different DH group—even if the group received from FortiClient is configured in the Phase 1 settings.
This behavior has been observed only with dial-up IPsec VPN connections between FortiGate and FortiClient.

Example configuration:

config vpn ipsec phase1-interface
    edit "Dialup-Tunnel"
        set type dynamic
        set interface "port9"
        set ike-version 2
        set peertype one
        set net-device disable
        set mode-cfg enable
        set proposal aes128-sha256 aes256-sha256 aes128gcm-prfsha256 aes256gcm-prfsha384 chacha20poly1305-prfsha256
        set dhgrp 20 16 14 <-----
        set peerid "64112"
        set assign-ip-from dhcp
        set dns-mode auto
        set psksecret ENC b5O6****
        set dpd-retryinterval 60

    next
end

IKE debug logs:

ike V=root:0: comes 10.10.10.1:45420->10.10.10.3:500,ifindex=15,vrf=0,len=1072....
ike V=root:0: IKEv2 exchange=SA_INIT id=629d73743e3527e2/0000000000000000 len=1072
ike 0: in 629D73743E3527E2000000000000000021202208000000000000043022000198020000C8010100150300000C0100000D800E01000300000C0100000C800E01000300000C0100000D800E00C00300000C0100000C800E00C00300000C0100000D800E00800300000C0100000C800E0080030000080300000E030000080300000D030000080300000C030000080300000503000008030000080300000804000010030000080400001F030000080400000F030000080400000E030000080200000203000008020000040300000802000005030000080200000603000008020000070000000802000008000000CC02010014030000080100001C0300000C01000014800E01000300000C01000013800E01000300000C01000012800E01000300000C01000014800E00C00300000C01000013800E00C00300000C01000012800E00C00300000C01000014800E00800300000C01000013800E00800300000C01000012800E00800300000804000010030000080400001F030000080400000F030000080400000E0300000802000002030000080200000403000008020000050300000802000006030000080200000700000008020000082800020800100000294FDDB988773EF897A034B75F0A95B58C41AB14A382DC952A6369CF99989635FA8BB3876A36B3997DE8CF91C161639A545B1FA6E11850701DF82FC850242F829C376A9B78E664E24611123E3EC71332777D4E168D1B3CD5C51186DE04AF55EC51913BF9C492BD5161DE82DEACC94D95148486B9752EAA0CE05FD4359586C62C0DEE40E571B08CC97565F6084A2A9473C636A459DA73BB9ABCD6D0360C47A7BC1CBC75610A8B7F259F05DD0F652242208FA875C3F6036AD6C147D1AA1859C0A0AF1985D7761CA7F494BE08DC4B88A51DE027DE044EA3D1EFB939688C590A454EE66B429EF71FCF723CFA9278E118AC697B1EC34D68C626D001E84F7D63E338C17B1466BB404D0D8F3B4A9F007615D5EECBEA7CFAB0A24D7224796B48499F56C75A26AD8002D3EEC2C9515B87F350E5C7481E48E2E025EDDAB6997C6EA6AB88F8F8BBC75AD54DC5B1BB55AD5E256A0529ADFA8067ECA81BF26BF0666163DFA261F4B8B2E6E4565F8E7A9CF37A225F2F817C7E52531AB33E37D06C9A44535C3A852E2E9536EAC0F3A2BDB949F6A9333F0AC217F6AB1B73A2D6C21B43734DB04387BF02668FAD369AD7374FDDDAFDAF7617320D2FEDEA84A5497E80974DC44C094F476B12E3CD644F6459A0D63C61D33BF704EE746AC02079724BBE0D1430C795984DB36D8C1857C1BD9224D7838DC5E476A360E7A786E296A01C2F286B6DD2E5D629000024EAA141EA4574D39EC9AF68FDCF5316ADF90BB3411A571BFB3C08B78DB99DAF4F2900001C000040041FBFA38E38DF27226A335C983DF1154D9AD408E72900001C000040051A0E03AE074B91ACB9B5D3945598A2BCFFC317DE290000080000402E000000100000402F0001000200030004
...
ike V=root:0:629d73743e3527e2/0000000000000000:798927: incoming proposal:
ike V=root:0:629d73743e3527e2/0000000000000000:798927: proposal id = 1:
...
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=DH_GROUP, val=MODP2048.
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=DH_GROUP, val=MODP3072.
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=DH_GROUP, val=CURVE25519.
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=DH_GROUP, val=MODP4096.
...
ike V=root:0:629d73743e3527e2/0000000000000000:798927: matched proposal id 1
ike V=root:0:629d73743e3527e2/0000000000000000:798927: proposal id = 1:
ike V=root:0:629d73743e3527e2/0000000000000000:798927: protocol = IKEv2:
ike V=root:0:629d73743e3527e2/0000000000000000:798927: encapsulation = IKEv2/none
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=ENCR, val=AES_CBC (key_len = 128)
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=INTEGR, val=AUTH_HMAC_SHA2_256_128
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=PRF, val=PRF_HMAC_SHA2_256
ike V=root:0:629d73743e3527e2/0000000000000000:798927: type=DH_GROUP, val=MODP2048.
ike V=root:0:629d73743e3527e2/0000000000000000:798927: lifetime=86400
ike V=root:0:629d73743e3527e2/0000000000000000:798927: SA proposal chosen, matched gateway Dialup-Tunnel
ike V=root:0:Dialup-Tunnel:Dialup-Tunnel: created connection: 0xc645a40 15 10.10.10.3->10.10.10.1:45420.
...
ike V=root:0:Dialup-Tunnel:798927: mismatched DH group in KE payload, selected 14, received 16 <-----
ike V=root:0:Dialup-Tunnel:798927: sending INVALID_KE notify
ike 0:Dialup-Tunnel:798927: out 629D73743E3527E200000000000000002920222000000000000000260000000A00000011000E
ike V=root:0:Dialup-Tunnel:798927: sent IKE msg (INVALID_KE_PAYLOAD): 10.10.10.3:500->10.10.10.1:45420, len=38, vrf=0, id=629d73743e3527e2/0000000000000000, oif=15
ike V=root:0:Dialup-Tunnel: connection expiring due to phase1 down

This issue has been resolved in FortiOS version 8.0.0 (scheduled to be released in February 2026).
These timelines for firmware release are estimates and may be subject to change.

Workaround:

Configure only a single DH group in the Phase 1 IPsec settings.

General debug information required by FortiGate TAC for investigation:

• Debugs:

diagnose debug console timestamp enable

diagnose vpn ike log filter rem-addr4 10.189.0.182 
diagnose debug application ike -1
diagnose debug enable
<reproduce the issue>
diagnose debug reset

• TAC Report:

execute tac report

• Configuration file of the FortiGate.