Help
FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
cgustave
Staff
Staff

Created on ‎03-04-2011 08:19 AM Edited on ‎04-04-2022 12:49 AM By Anonymous

Article Id 189477

Technical Tip : Microsoft ISA server 2006 and FortiGate IPSec VPN configuration example

Purpose

The purpose of this technical note is to provide a simple example of an IPSec tunnel between a Microsoft ISA server 2006 running on a Microsoft Server 2003 system and a FortiGate firewall.  FortiGate configuration samples are provided for 4.1.9 (4.0 MR1 patch 9)  and 4.2.3 (4.0 MR2 patch) releases.

Although the document was written for 4.1/4.2, it also applies for 4.3/5.0 releases.

This example does not aim to provide the best security configuration but rather a simple configuration base.  Security enforcements can easily done as a second stage to implement tighter policies.


Scope

Microsoft ISA server 2006, VPN IPSec, no natting, no proxying.

FortiGate using IPSec interface mode (but policy mode is also possible)


Diagram
cgustave_33079_diagram.jpg

Expectations, Requirements
It is recommended to use the latest FortiGate patch release for the firmware version.  You may also want to use the latest service pack for your windows system and ISA server.
Configuration
Microsoft ISA server configuration:

cgustave_33079_VPN_remote_sites.jpg

 



cgustave_33079_VPN_remote_sites_general.jpgcgustave_33079_VPN_remote_sites_Addresses.jpgcgustave_33079_VPN_remote_sites_connection.jpgcgustave_33079_VPN_remote_sites_connection_Phase1.jpgcgustave_33079_VPN_remote_sites_connection_Phase2.jpgcgustave_33079_VPN_remote_sites_authentication.jpg



cgustave_33079_config_networks_networks.jpg
cgustave_33079_config_networks_internal_general.jpgcgustave_33079_config_networks_internal_addresses.jpgcgustave_33079_config_networks_internal_domains.jpgcgustave_33079_config_networks_internal_webbrowser.jpgcgustave_33079_config_networks_internal_autodiscovery.jpgcgustave_33079_config_networks_internal_firewallclients.jpgcgustave_33079_config_networks_internal_Webproxy.jpg

cgustave_33079_config_network_network_set.jpgcgustave_33079_config_network_network_rules.jpgcgustave_33079_config_networks_network_rules_Networkrelationship.jpg

cgustave_33079_config_networks_webChaining.jpg


cgustave_33079_firewall_policy.jpg
cgustave_33079_firewall_policy_general.jpgcgustave_33079_firewall_policy_action.jpgcgustave_33079_firewall_policy_protocol.jpgcgustave_33079_firewall_policy_from.jpgcgustave_33079_firewall_policy_to.jpgcgustave_33079_firewall_policy_users.jpgcgustave_33079_firewall_policy_Schedule.jpgcgustave_33079_firewall_policy_contenttype.jpg




FortiGate 4.1.9 configuration:

Phase 1:
cgustave_33079_FG4_1_9_ipsec_phase1.jpg

cgustave_33079_FG4_1_9_ipsec_phase1_details.jpg

Phase 2:
cgustave_33079_FG4_1_9_ipsec_phase2_details.jpg


Firewall Addresses:
cgustave_33079_FG4_1_9_firewall_address.jpg

Firewall Policies:
cgustave_33079_FG4_1_9_firewall_policies.jpg

VPN route:
cgustave_33079_FG4_1_9_router_static.jpg

Interfaces summary:
cgustave_33079_FG4_1_9_network_interfaces.jpg


IPSec Monitoring:
cgustave_33079_FG4_1_9_ipsec_monitor.jpg





FortiGate 4.2 configuration:

Phase1:
cgustave_33079_FG4_2_3_phase1_details.jpg


Phase2:
cgustave_33079_FG4_2_3_phase2_details.jpg



Firewall Address:
cgustave_33079_FG4_2_3_fw_address.jpg


Firewall Policies:
cgustave_33079_FG4_2_3_fw_policies.jpg


VPN Route:
cgustave_33079_FG4_2_3_routes.jpg


Interfaces Summary:
cgustave_33079_FG4_2_3_interface_sum.jpg

IPSec monitor:
cgustave_33079_FG4_2_3_ipsec_monitor.jpg

 


IMPORTANT NOTES:

  • ISA Server phase2 selector and IP ranges:
ISA server (in the version tested) only supports well subnet ranges starting from network address to broadcast. You should be using local range or distant range such as 10.100.0.0-10.100.0.255. Attempt to use 10.100.0.1-10.100.0.254 for instance would cause the ISA server to present unexpected phase2 selectors ending in selector mismatch. FortiGate 'diag debug application ike -1' debug output shows the selector send from the peer.

 

  • Dead Peer Detection (DPD):
ISA server (in the version tested) does not seem to support dead peer detection (DPD).


Verification



Troubleshooting
FortiGate 4.1.9 ike debug (for reference):

FG3600-4 # diag debug enable
FG3600-4 # diagnose debug application ike -1

FG3600-4 # ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Quick id=686acad5ee11e2db/b160c04e53d70b46:07e154e3 len=300
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:2:42: responder received first quick-mode message
ike 0:isa_server:2:42: peer proposal is: peer:10.100.0.0-10.100.1.255, me:10.102.0.0-10.102.1.255, ports=0/0, protocol=0/0
ike 0:isa_server:2:42: trying ph2_isa_server
ike 0:isa_server:2:ph2_isa_server:42: matched phase2
ike 0:isa_server:2:ph2_isa_server:42: autokey
ike 0:isa_server:2:ph2_isa_server:42: my proposal:
ike 0:isa_server:2:ph2_isa_server:42: proposal id = 1:
ike 0:isa_server:2:ph2_isa_server:42:   protocol id = IPSEC_ESP:
ike 0:isa_server:2:ph2_isa_server:42:      trans_id = ESP_3DES
ike 0:isa_server:2:ph2_isa_server:42:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:2:ph2_isa_server:42:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:2:ph2_isa_server:42: incoming proposal:
ike 0:isa_server:2:ph2_isa_server:42: proposal id = 1:
ike 0:isa_server:2:ph2_isa_server:42:   protocol id = IPSEC_ESP:
ike 0:isa_server:2:ph2_isa_server:42:      trans_id = ESP_3DES
ike 0:isa_server:2:ph2_isa_server:42:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:2:ph2_isa_server:42:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:2:ph2_isa_server:42: negotiation result
ike 0:isa_server:2:ph2_isa_server:42: proposal id = 1:
ike 0:isa_server:2:ph2_isa_server:42:   protocol id = IPSEC_ESP:
ike 0:isa_server:2:ph2_isa_server:42:      trans_id = ESP_3DES
ike 0:isa_server:2:ph2_isa_server:42:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:2:ph2_isa_server:42:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:2:ph2_isa_server:42: set pfs=1024
ike 0:isa_server:2:ph2_isa_server:42: using tunnel mode.
ike 0:isa_server:2:ph2_isa_server:42: add RESPONDER-LIFETIME 1800 seconds
ike 0:isa_server:2: sent IKE msg (quick_r1send): 172.31.208.225:500->192.168.183.82:500, len=324
ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Quick id=686acad5ee11e2db/b160c04e53d70b46:07e154e3 len=52
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:2:ph2_isa_server:42: SA life soft seconds=1750.
ike 0:isa_server:2:ph2_isa_server:42: SA life hard seconds=1800.
ike 0:isa_server:2:ph2_isa_server:42: IPsec SA selectors #src=1 #dst=1
ike 0:isa_server:2:ph2_isa_server:42: src 0 7 10.102.0.0-10.102.1.255
ike 0:isa_server:2:ph2_isa_server:42: dst 0 7 10.100.0.0-10.100.1.255
ike 0:isa_server:2:ph2_isa_server:42: add IPsec SA: SPIs=21a7a7f3/bcadbb57
ike 0:isa_server:2:ph2_isa_server:42: added IPsec SA: SPIs=21a7a7f3/bcadbb57
ike 0:isa_server:2:ph2_isa_server:42: sending SNMP tunnel UP trap



FortiGate 4.2.3 ike debug (for reference):

ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Identity Protection id=b8f7eb5f02e16f62/0000000000000000 len=168
ike 0:isa_server: new connection.
ike 0:isa_server:11: responder: main mode get 1st message...
ike 0:isa_server:11: VID MS NT5 ISAKMPOAKLEY 1E2B516905991C7D7C96FCBFB587E46100000004
ike 0:isa_server:11: VID FRAGMENTATION 4048B7D56EBCE88525E7DE7F00D6C2D3
ike 0:isa_server:11: VID draft-ietf-ipsec-nat-t-ike-02\n 90CB80913EBB696E086381B5EC427B1F
ike 0:isa_server:11: VID unknown (16): 26244D38EDDB61B3172A36E3D0CFB819
ike 0:isa_server:11: negotiation result
ike 0:isa_server:11: proposal id = 1:
ike 0:isa_server:11:   protocol id = ISAKMP:
ike 0:isa_server:11:      trans_id = KEY_IKE.
ike 0:isa_server:11:      encapsulation = IKE/none
ike 0:isa_server:11:         type=OAKLEY_ENCRYPT_ALG, val=3DES_CBC.
ike 0:isa_server:11:         type=OAKLEY_HASH_ALG, val=SHA.
ike 0:isa_server:11:         type=AUTH_METHOD, val=PRESHARED_KEY.
ike 0:isa_server:11:         type=OAKLEY_GROUP, val=1024.
ike 0:isa_server:11: ISKAMP SA lifetime=28800
ike 0:isa_server:11: selected NAT-T version: draft-ietf-ipsec-nat-t-ike-02\n
ike 0:isa_server:11: cookie b8f7eb5f02e16f62/0fae5657fc81d42f
ike 0:isa_server:11: sent IKE msg (ident_r1send): 172.31.208.225:500->192.168.183.82:500, len=124
ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Identity Protection id=b8f7eb5f02e16f62/0fae5657fc81d42f len=232
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:11: responder:main mode get 2nd message...
ike 0:isa_server:11: NAT not detected
ike 0:isa_server:11: sent IKE msg (ident_r2send): 172.31.208.225:500->192.168.183.82:500, len=228
ike 0:isa_server:11: ISAKMP SA b8f7eb5f02e16f62/0fae5657fc81d42f key 24:70B760E77CEA0329D408B04139EF40FD8A3B9C7A3323923D
ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Identity Protection id=b8f7eb5f02e16f62/0fae5657fc81d42f len=68
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:11: responder: main mode get 3rd message...
ike 0:isa_server:11: PSK authentication succeeded
ike 0:isa_server:11: authentication OK
ike 0:isa_server:11: sent IKE msg (ident_r3send): 172.31.208.225:500->192.168.183.82:500, len=68
ike 0:isa_server:11: established IKE SA b8f7eb5f02e16f62/0fae5657fc81d42f
ike 0:isa_server: DPD disabled, not negotiated
ike 0:isa_server:11: no pending Quick-Mode negotiations
ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Quick id=b8f7eb5f02e16f62/0fae5657fc81d42f:29293648 len=300
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:11:134: responder received first quick-mode message
ike 0:isa_server:11:134: peer proposal is: peer:0:10.100.0.0-10.100.1.255:0, me:0:10.102.0.0-10.102.1.255:0
ike 0:isa_server:11:134: trying ph2_isa_server
ike 0:isa_server:11:ph2_isa_server:134: matched phase2
ike 0:isa_server:11:ph2_isa_server:134: autokey
ike 0:isa_server:11:ph2_isa_server:134: my proposal:
ike 0:isa_server:11:ph2_isa_server:134: proposal id = 1:
ike 0:isa_server:11:ph2_isa_server:134:   protocol id = IPSEC_ESP:
ike 0:isa_server:11:ph2_isa_server:134:   PFS DH group = 2
ike 0:isa_server:11:ph2_isa_server:134:      trans_id = ESP_3DES
ike 0:isa_server:11:ph2_isa_server:134:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:11:ph2_isa_server:134:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:11:ph2_isa_server:134: incoming proposal:
ike 0:isa_server:11:ph2_isa_server:134: proposal id = 1:
ike 0:isa_server:11:ph2_isa_server:134:   protocol id = IPSEC_ESP:
ike 0:isa_server:11:ph2_isa_server:134:   PFS DH group = 2
ike 0:isa_server:11:ph2_isa_server:134:      trans_id = ESP_3DES
ike 0:isa_server:11:ph2_isa_server:134:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:11:ph2_isa_server:134:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:11:ph2_isa_server:134: negotiation result
ike 0:isa_server:11:ph2_isa_server:134: proposal id = 1:
ike 0:isa_server:11:ph2_isa_server:134:   protocol id = IPSEC_ESP:
ike 0:isa_server:11:ph2_isa_server:134:   PFS DH group = 2
ike 0:isa_server:11:ph2_isa_server:134:      trans_id = ESP_3DES
ike 0:isa_server:11:ph2_isa_server:134:      encapsulation = ENCAPSULATION_MODE_TUNNEL
ike 0:isa_server:11:ph2_isa_server:134:         type = AUTH_ALG, val=SHA1
ike 0:isa_server:11:ph2_isa_server:134: set pfs=1024
ike 0:isa_server:11:ph2_isa_server:134: using tunnel mode.
ike 0:isa_server:11:ph2_isa_server:134: add RESPONDER-LIFETIME 1800 seconds
ike 0:isa_server:11: sent IKE msg (quick_r1send): 172.31.208.225:500->192.168.183.82:500, len=324
ike 0: comes 192.168.183.82:500->172.31.208.225:500,ifindex=4....
ike 0: IKEv1 exchange=Quick id=b8f7eb5f02e16f62/0fae5657fc81d42f:29293648 len=52
ike 0: found isa_server 172.31.208.225 4 -> 192.168.183.82:500
ike 0:isa_server:11:ph2_isa_server:134: SA life soft seconds=1747.
ike 0:isa_server:11:ph2_isa_server:134: SA life hard seconds=1800.
ike 0:isa_server:11:ph2_isa_server:134: IPsec SA selectors #src=1 #dst=1
ike 0:isa_server:11:ph2_isa_server:134: src 0 7 0:10.102.0.0-10.102.1.255:0
ike 0:isa_server:11:ph2_isa_server:134: dst 0 7 0:10.100.0.0-10.100.1.255:0
ike 0:isa_server:11:ph2_isa_server:134: add IPsec SA: SPIs=9fe07abb/2851f139
ike 0:isa_server:11:ph2_isa_server:134: IPsec SA dec spi 9fe07abb key 24:A4F69816DD4A3B2958B09D33737DA82795EF55C13C47A353 auth 20:9FE3ED9F87A5CB611754FEADB1EA2337A19F98BE
ike 0:isa_server:11:ph2_isa_server:134: IPsec SA enc spi 2851f139 key 24:3E65F4DB34C2A484F3C557F4ABA10BE1FAE031932508A7EC auth 20:3991774C6C12E15EA3E8EC9B47C6D1310173035B
ike 0:isa_server:11:ph2_isa_server:134: added IPsec SA: SPIs=9fe07abb/2851f139
ike 0:isa_server:11:ph2_isa_server:134: sending SNMP tunnel UP trap





======================================================

This KB article should be maintained by: TAC/TAC-L3
Articles with very similar or duplicate content exist:  none
Content of this KB article could be integrated to another article:  none
Is this article relevant to currently supported product versions: yes
What currently supported versions is this article relevant to: 4.3/5.0
Is this article ONLY relevant to non-supported versions: no
If this article was written for an unsupported version, can it be modified/updated for a supported one:
=> document was written for 4.1 and 4.2 but still applies to 4.3/5.0 releases.
=> I have no plan to update this document for 4.3 or 5.0 screenshot.
=> If someone want to go for it. Feel free.
Is this topic already documented in TechDocs: no
Do you propose this article to be discontinued/moved to internal KB area: no
Article was rewritten, as a result of this evaluation: no
Changes done:
Other remarks and recommendations:
Date this article was evaluated: 2013-03-26
Evaluated by: cgustave

======================================================

9512
0 Kudos
Suggest New Article
Article Feedback
Contributors
Broad. Integrated. Automated.

The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.

Social Media
Security Research
Company
News & Articles
Contact Us

Copyright 2024 Fortinet, Inc. All Rights Reserved.