Phase 1 will not come up Fortigate VM to Sonicw2ll TZ270

[nowarrantyrepair]

Fortigate VM to Sonicwall. The tunnel won't come up and the sonicwall is responding with Invalid Syntax. Everything is same on both ends. No idea why it will not come up.

Sonicwall is sending this

SENDING>>>> ISAKMP OAK IKE_SA_INIT (InitCookie:0x964d86bb85c7dd9f RespCookie:0x0000000000000000, MsgID: 0x0) (NOTIFY: Invalid KE Payload)

Fortigate VM Logs

ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike shrank heap by 159744 bytes
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:250: negotiation timeout, deleting
ike 0:Tunnel01: connection expiring due to phase1 down
ike 0:Tunnel01: deleting
ike 0:Tunnel01: deleted
ike 0:Tunnel01: schedule auto-negotiate
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: created connection: 0x111e10b0 3 10.1.101.4->23.24.216.31:500.
ike 0:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:500 negotiating
ike 0:Tunnel01: no suitable IKE_SA, queuing CHILD_SA request and initiating IKE_SA negotiation
ike 0:Tunnel01:252: generate DH public value request queued
ike 0:Tunnel01:252: out 5B9C6767256848B300000000000000002120220800000000000001C02200003800000034010100050300000C0100000C800E0080030000080200000203000008030000020300000804000005000000080400000E28000108000E000005A013F79A0875898D5997BBA9B1400BDB52FBF623CAFDEEE83CFB7879682450B3ED59C2966BA28046C7FA375028117D2CD85D33AA22CB9173DD6FCF7A5382AF072A044D199A59AE9E2D536D936A871AF543DEC293D6A012733A9060BA7E26497DA26254669448BA49F9762645AF3F2FD696E980F927735EF1BC541C328F9C0A2E9D24CTunnel01768EF2F1C882FE95AA99BAF9600F0917477505F6DA13D5820EA54E1E88CD802FF2E96012C03123DDE7DE291A5A5B1D64524839A509083CCE5397B2EB5D50908D9CBC4619AE81742A6EC7DBFD46920D51FAAF953C0B6961FF0E951B6EC744578D4B899968286F38B03FA77AF931D06D6C44669A286D2E13D8BB0ADE629000024C00B8C481311DBFFABCC689AD96E27068FE94D5B98072DFBD816EC4D9E9B29DB2900001C000040049311AA16A9E9533DC05E79295CCFC31B66972F842900001C000040054C9E35FD52699B872ADBB93CCFF1CFAB80AF9953000000080000402E
ike 0:Tunnel01:252: sent IKE msg (SA_INIT): 10.1.101.4:500->23.24.216.31:500, len=448, vrf=0, id=5b9c6767256848b3/0000000000000000
ike 0: comes 23.24.216.31:500->10.1.101.4:500,ifindex=3,vrf=0....
ike 0: IKEv2 exchange=SA_INIT_RESPONSE id=5b9c6767256848b3/0000000000000000 len=38
ike 0: in 5B9C6767256848B300000000000000002920222000000000000000260000000A000000110005
ike 0:Tunnel01:252: initiator received SA_INIT response
ike 0:Tunnel01:252: processing notify type INVALID_KE_PAYLOAD
ike 0:Tunnel01:252: generate DH public value request queued
ike 0:Tunnel01:252: out 5B9C6767256848B300000000000000002120220800000000000001802200003800000034010100050300000C0100000C800E0080030000080200000203000008030000020300000804000005000000080400000E280000C80005000007C4757DEE79DD642518000698C0B5CEF266C01AC835396C993969AC978476614D083B2A047D16FA055625540D5C7DE478EA5BF491C3D2777A5F15DD342E4BD263D8BEB1C247F1BB1E8F8061DFC19B5D6CF01E9C6419729E390B4AD553C4AB347B25AC52CF1B0F7B71447A27D3044564084E85B02BA3A9ABA7AB8C7Tunnel01CD37EF891B79DACC93D88D43EF7D69F4D49644B48F6DDBF366EF99AC5404106315C4764E55F3438ADFAAAA855883906E7C96018C95217B90DDF7374DB07AE984EF60B31290000248311DF08BE59AE0C91C5D841C285603FA6362F49D39587924F3577DB1BF6F4242900001C000040049311AA16A9E9533DC05E79295CCFC31B66972F842900001C000040054C9E35FD52699B872ADBB93CCFF1CFAB80AF9953000000080000402E
ike 0:Tunnel01:252: sent IKE msg (SA_INIT): 10.1.101.4:500->23.24.216.31:500, len=384, vrf=0, id=5b9c6767256848b3/0000000000000000
ike 0: comes 23.24.216.31:500->10.1.101.4:500,ifindex=3,vrf=0....
ike 0: IKEv2 exchange=SA_INIT_RESPONSE id=5b9c6767256848b3/0000000000000000 len=36
ike 0: in 5B9C6767256848B300000000000000002920222000000000000000240000000800000007
ike 0:Tunnel01:252: initiator received SA_INIT response
ike 0:Tunnel01:252: processing notify type INVALID_SYNTAX
ike 0:Tunnel01:252: malformed message
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:Tunnel01: IPsec SA connect 3 10.1.101.4->23.24.216.31:0
ike 0:Tunnel01:Tunnel01: using existing connection
ike 0:Tunnel01:Tunnel01: config found
ike 0:Tunnel01: request is on the queue
ike 0:Tunnel01:252: negotiation timeout, deleting
ike 0:Tunnel01: connection expiring due to phase1 down
ike 0:Tunnel01: deleting
ike 0:Tunnel01: deleted

[vponmuniraj]

Hi,

The "INVALID_KE_PAYLOAD" indicates a DH group mismatch. Please check the phase1 DH group on both sides. 

Regards,