Help
FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
nithincs
Staff
Staff

Created on ‎05-08-2020 11:54 PM Edited on ‎12-26-2024 11:17 PM By Community Manager

Article Id 193250

Technical Tip: How to check if Diffie-Hellman(DH) group is the same on both peer units

Description

 

This article describes that when the FortiGate is configured to establish IPsec VPN tunnel with remote peer, any mismatch in the IKE parameters will cause an immediate negotiation failure. Hence, the tunnel will not be established for both phase1 and phase2.

This article describes how to check if the DH group is the same in both peer units.

 

Scope

 

FortiGate.

Solution

 

Make sure the corresponding phase1 IKE Diffie-Hellman (DH) group is same as DH group set in FortiGate. When Perfect Forward Secrecy (PFS) is enabled on phase2, DH group also needs to match. The DH group selected in phase2 is used each time a phase2 SA is established or rekeyed, such that when phase2 lifetime expires and a new negotiation takes place.


These examples illustrate a failure due to DH group mismatch.

 

Phase 1:

 

ike 0: comes 10.40.16.57:500->10.40.16.20:500,ifindex=3....
ike 0: IKEv1 exchange=Aggressive id=bc55c602b3aa4243/0000000000000000 len=472
ike 0: in
ike 0::108: peer identifier IPV4_ADDR 10.40.16.57
ike 0:bc55c602b3aa4243/0000000000000000:108: incoming proposal:
ike 0:bc55c602b3aa4243/0000000000000000:108: proposal id = 0:
ike 0:bc55c602b3aa4243/0000000000000000:108:   protocol id = ISAKMP:
ike 0:bc55c602b3aa4243/0000000000000000:108:      trans_id = KEY_IKE.
ike 0:bc55c602b3aa4243/0000000000000000:108:      encapsulation = IKE/none
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_ENCRYPT_ALG, val=AES_CBC, key-len=128
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_HASH_ALG, val=SHA2_256.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=AUTH_METHOD, val=PRESHARED_KEY.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_GROUP, val=MODP1024.
ike 0:bc55c602b3aa4243/0000000000000000:108: ISAKMP SA lifetime=86400
ike 0:bc55c602b3aa4243/0000000000000000:108: my proposal, gw DR_port1_2:
ike 0:bc55c602b3aa4243/0000000000000000:108: proposal id = 1:
ike 0:bc55c602b3aa4243/0000000000000000:108:   protocol id = ISAKMP:
ike 0:bc55c602b3aa4243/0000000000000000:108:      trans_id = KEY_IKE.
ike 0:bc55c602b3aa4243/0000000000000000:108:      encapsulation = IKE/none
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_ENCRYPT_ALG, val=AES_CBC, key-len=128
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_HASH_ALG, val=SHA2_256.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=AUTH_METHOD, val=PRESHARED_KEY.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_GROUP, val=MODP2048.
ike 0:bc55c602b3aa4243/0000000000000000:108: ISAKMP SA lifetime=86400
ike 0:bc55c602b3aa4243/0000000000000000:108: proposal id = 1:
ike 0:bc55c602b3aa4243/0000000000000000:108:   protocol id = ISAKMP:
ike 0:bc55c602b3aa4243/0000000000000000:108:      trans_id = KEY_IKE.
ike 0:bc55c602b3aa4243/0000000000000000:108:      encapsulation = IKE/none
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_ENCRYPT_ALG, val=AES_CBC, key-len=128
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_HASH_ALG, val=SHA2_256.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=AUTH_METHOD, val=PRESHARED_KEY.
ike 0:bc55c602b3aa4243/0000000000000000:108:         type=OAKLEY_GROUP, val=MODP1536.
ike 0:bc55c602b3aa4243/0000000000000000:108: ISAKMP SA lifetime=86400
ike 0:bc55c602b3aa4243/0000000000000000:108: negotiation failure
ike Negotiate ISAKMP SA Error: ike 0:bc55c602b3aa4243/0000000000000000:108: no SA proposal chosen

 

In the above example, the remote peer Phase1 IKE Diffie-Hellman group is 2 and in the local firewall, it is 5 and 14. Changing the Phase1 IKE Diffie-Hellman group to 2 in the local firewall will result in successful phase1 negotiation.

 

ike 0:DR_port1_2:196: initiator: aggressive mode get 1st response...
ike 0:DR_port1_2:196: VID RFC 3947 4A131C81070358455C5728F20E95452F
ike 0:DR_port1_2:196: VID DPD AFCAD71368A1F1C96B8696FC77570100
ike 0:DR_port1_2:196: DPD negotiated
ike 0:DR_port1_2:196: VID FORTIGATE 8299031757A36082C6A621DE00000000
ike 0:DR_port1_2:196: peer is FortiGate/FortiOS (v0 b0)
ike 0:DR_port1_2:196: VID FRAGMENTATION 4048B7D56EBCE88525E7DE7F00D6C2D3
ike 0:DR_port1_2:196: VID FRAGMENTATION 4048B7D56EBCE88525E7DE7F00D6C2D3C0000000
ike 0:DR_port1_2:196: peer identifier IPV4_ADDR 10.40.16.57
ike 0:DR_port1_2:196: negotiation result
ike 0:DR_port1_2:196: proposal id = 1:
ike 0:DR_port1_2:196:   protocol id = ISAKMP:
ike 0:DR_port1_2:196:      trans_id = KEY_IKE.
ike 0:DR_port1_2:196:      encapsulation = IKE/none
ike 0:DR_port1_2:196:         type=OAKLEY_ENCRYPT_ALG, val=AES_CBC, key-len=128
ike 0:DR_port1_2:196:         type=OAKLEY_HASH_ALG, val=SHA2_256.
ike 0:DR_port1_2:196:         type=AUTH_METHOD, val=PRESHARED_KEY.
ike 0:DR_port1_2:196:         type=OAKLEY_GROUP, val=MODP1024.
ike 0:DR_port1_2:196: ISAKMP SA lifetime=86400
ike 0:DR_port1_2:196: received NAT-D payload type 20
ike 0:DR_port1_2:196: received NAT-D payload type 20
ike 0:DR_port1_2:196: selected NAT-T version: RFC 3947
ike 0:DR_port1_2:196: NAT not detected
ike 0:DR_port1_2:196: ISAKMP SA 3ffc2deda4ce3955/8d11eb06e27ffb35 key 16:D25A5D375448BDEE3423AA121B56980E
ike 0:DR_port1_2:196: PSK authentication succeeded
ike 0:DR_port1_2:196: authentication OK
ike 0:DR_port1_2:196: add INITIAL-CONTACT

 

Phase 2:
 
ike 57:FGT1:5682439: received create-child request

ike 57:FGT1:5682439: responder received CREATE_CHILD exchange

ike 57:FGT1:5682439: responder creating new child

ike 57:FGT1:5682439:8691164: peer proposal:

ike 57:FGT1:5682439:8691164: TSi_0 0:10.126.250.45-10.126.250.45:0

ike 57:FGT1:5682439:8691164: TSr_0 0:10.115.251.220-10.115.251.220:0

ike 57:FGT1:5682439:8691164: TSr_1 0:10.115.251.0-10.115.251.255:0

ike 57:FGT1:5682439:FGT2:8691164: comparing selectors

ike 57:FGT1:5682439:FGT2-2:8691164: comparing selectors

ike 57:FGT1:5682439:FGT2-3:8691164: comparing selectors

ike 57:FGT1:5682439:FGT2-4:8691164: comparing selectors

ike 57:FGT1:5682439:FGT2-4:8691164: matched by rfc-rule-2

ike 57:FGT1:5682439:FGT2-4:8691164: phase2 matched by subset

ike 57:FGT1:5682439:8691164: local narrowing exactly matches static selector

ike 57:FGT1:5682439:FGT2-4:8691164: accepted proposal:

ike 57:FGT1:5682439:FGT2-4:8691164: TSi_0 0:10.126.250.45-10.126.250.45:0

ike 57:FGT1:5682439:FGT2-4:8691164: TSr_0 0:10.115.251.0-10.115.251.255:0

ike 57:FGT1:5682439:FGT2-4:8691164: autokey

ike 57:FGT1:5682439:FGT2-4:8691164: incoming child SA proposal:

ike 57:FGT1:5682439:FGT2-4:8691164: proposal id = 1:

ike 57:FGT1:5682439:FGT2-4:8691164:   protocol = ESP:

ike 57:FGT1:5682439:FGT2-4:8691164:      encapsulation = TUNNEL

ike 57:FGT1:5682439:FGT2-4:8691164:         type=ENCR, val=AES_CBC (key_len = 256)

ike 57:FGT1:5682439:FGT2-4:8691164:         type=INTEGR, val=SHA256

ike 57:FGT1:5682439:FGT2-4:8691164:         type=INTEGR, val=SHA384

ike 57:FGT1:5682439:FGT2-4:8691164:         type=INTEGR, val=SHA512

ike 57:FGT1:5682439:FGT2-4:8691164:         type=DH_GROUP, val=ECP521

ike 57:FGT1:5682439:FGT2-4:8691164:         type=ESN, val=NO

ike 57:FGT1:5682439:FGT2-4:8691164: my proposal:

ike 57:FGT1:5682439:FGT2-4:8691164: proposal id = 1:

ike 57:FGT1:5682439:FGT2-4:8691164:   protocol = ESP:

ike 57:FGT1:5682439:FGT2-4:8691164:      encapsulation = TUNNEL

ike 57:FGT1:5682439:FGT2-4:8691164:         type=ENCR, val=AES_CBC (key_len = 256)

ike 57:FGT1:5682439:FGT2-4:8691164:         type=INTEGR, val=SHA512

ike 57:FGT1:5682439:FGT2-4:8691164:         type=DH_GROUP, val=ECP384

ike 57:FGT1:5682439:FGT2-4:8691164:         type=ESN, val=NO

ike 57:FGT1:5682439:FGT2-4:8691164: lifetime=28800

ike 57:FGT1:5682439:FGT2-4:8691164: no proposal chosen

 
In the example above for phase2, the remote peer Phase2 Diffie-Hellman group is 21 and in the local firewall, it is 20. Ensuring that corresponding DH group values configured in both Phase1 and Phase2 resolves this issue.
 
From RFC3526, RFC5903, and RFC7296 follows a mapping of supported DH Group to their respective OAKLEY_GROUP value.
 
DH Group 1: 768-bit MODP Group
DH Group 2: 1024-bit MODP Group
DH Group 5: 1536-bit MODP Group
DH Group 14: 2048-bit MODP Group
DH Group 15: 3072-bit MODP Group
DH Group 16: 4096-bit MODP Group
DH Group 17: 6144-bit MODP Group
DH Group 18: 8192-bit MODP Group
DH Group 19: 256-bit random ECP Group
DH Group 20: 384-bit random ECP Group
DH Group 21: 521-bit random ECP Group


To modify the DH group value, use the commands below:

 

config vpn ipsec phase1-interface
    edit "tunnel-name"
        set dhgrp <DH number>
    end

config vpn ipsec phase2-interface
    edit "tunnel-name"
        set dhgrp <DH number>
    end

 

OR

 

config vpn ipsec phase1
    edit "tunnel-name"
        set dhgrp <DH number>
    end

config vpn ipsec phase2
    edit "tunnel-name"
        set dhgrp <DH number>
    end

34273
Suggest New Article
Article Feedback
Broad. Integrated. Automated.

The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.

Social Media
Security Research
Company
News & Articles
Contact Us

Copyright 2025 Fortinet, Inc. All Rights Reserved.