Help with OpenSwan to Fortigate IPsec VPN

looks good ( conf wise ). What' s actually your problem? What I would suggest; 1st capture IKE traffic to validate your sending and receiving ph1 packets e,g tcpdump -i " insert the interface" -vvv udp and port 500 if your you not seeing any traffic to the 2 end-points, I would suggest a top level firewall/acl is blocking traffic. I' ve been burned numerous times in regard to the above, where an external firewall/router was prevnting my IKE and ESP ( protocol 50 ) traffic. Also don' t forget any local ACL if your running a local filter like iptables. i' m assuming ClearOS supports iptables. If IKE is being present, I would inspect for esp traffic next e.g tcpdump -i " insert interface' -vvv proto 50 You can also inspect from the FGt with the diag sniffer packet commands. and validate correct packets are being sent to each en-points. 2nd, are you using certs or psk? if your using certs try a PSK and rekey the PSK on both openswan and fortigate . look at your /etc/ipsec.secrets file 3rd, make sure your FGt has matching quickmode src/dst selectors that matches your left/right subnets ( local/remote ) 4th When you bring up the tunnel, what does your ipsec status shows on openswan and what logs are generated on FGT and any logs on the openswan host? 5th Do you have any NAT issues and rely on NAT-t keepalives On openswan you will need the following in the ipsec.conf file config setup nat_traversa l= yes watch I now see that you do have enabled in yout cfg ;) Overall, Openswan is not that hard to setup and just review the tips/trick from above and ensure you are matching the correct phase172 proposal and ensure PFS is surely enabled. You have a host of means to debug and analyze any conditions from the linux host side of things. lastly; are you running phase1-2 in interface modes? if yes, ensure you have a static route for the remote-destination. And finally , good luck

Thank you for your help, emnoc. Feedback, in order, follows. Note: Public external-interface IPs changed to private ones (10.0.*.1), but otherwise this is exact. The problem seems to be that phase 2 is not completing. From " ipsec --auto status" on the OpenSwan device: 000 #1: " test" :500 STATE_MAIN_R3 (sent MR3, ISAKMP SA established); EVENT_SA_EXPIRE in 86188s; lastdpd=-1s(seq in:0 out:0); idle; import:not set 000 #3: " test" :500 STATE_QUICK_I2 (sent QI2, IPsec SA established); EVENT_SA_REPLACE_IF_USED in 85438s; newest IPSEC; eroute owner; isakmp#2; idle; import:admin initiate 000 #3: " test" esp.b1440bf7@10.0.2.1 esp.4e9a8de9@10.0.1.1 tun.0@10.0.2.1 tun.0@10.0.1.1 ref=0 refhim=4294901761 1. Sniffing with tshark on the OpenSwan device shows IKE is passing: [root@openswan ~]# tshark -i eth0 ' udp port 500' Capturing on eth0 0.000000 10.0.2.1 -> 10.0.1.1 ISAKMP Informational 0.000280 10.0.1.1 -> 10.0.2.1 ISAKMP Informational However, ESP is not. I assumed this was because phase 2 is not completing. Sniffs of ' ip proto 50' show no traffic. Sniffing at the Fortigate using " diag sniffer packet" produces matching output. 2. I' m using PSK. I' ve re-entered it repeatedly and I' m certain it is correct, although there seems to be no way to view the key on the Fortigate once saved. 3. I had not set quickmode selectors. I' ll follow this post with two more that have image attachments. Should I set the subnets, but leave the ports and protocol set to 0? 4. Logs as above. /var/log/messages on OpenSwan device show: Aug 29 23:11:09 gw2 ipsec__plutorun: 002 added connection description " test" Aug 29 23:11:12 gw2 ipsec__plutorun: 104 " test" #2: STATE_MAIN_I1: initiate 5. No NAT is in use. Similarly, I' ve ensured the iptables firewall on the OpenSwan device is totally down or clears esp, ah, and 500/UDP correctly. I' m familiar with sniffing on both devices, but don' t know how to view the VPN logs on the fortigate (except the not-very-helpful " Log Access" in the GUI), which shows: 2011-08-29 23:10:46 error ipsec 37130 negotiate progress IPsec phase 2 Can you tell me how to see more informational logs or debug output from the CLI on the fortigate? Thanks again, Tyler

Fortigate phase 1 was attached in the last message. Phase 2 is here.

Suggestions & a few comments; Curious as you have DF groups 2,5,14 set and then 5. Could you just select one for your ph-1 proposals and then identify this in your ipsec.conf e.g ike=aes256-sha1-modp1024! that would be for AES256 sha1 and DF-grp 2. I' m posting the listing for strongswan and DF-grps but for OpenSwan, they should be the same for both sides to reduce it mismatches on proposal offered. I' ve never been a big fan of multiple proposals personally, but too each his own http://wiki.strongswan.org/projects/strongswan/wiki/IKEv1CipherSuites To ease diagnostic, you could eliminate PFS and reduce the proposal options for both ends. And if that works, than you know for sure, it' s a proposal issues. On the ipsec logs, you can review the system logs if you set these up on the FGT and also running openswan in a verbose mode and rerun your " ipsec auto --up" and review any details that comes up. see this link for pluto log options to get an ideal http://wiki.openswan.org/index.php/Openswan/Plutodebug lastly, on the FGT you will need to eliminate the 0.0.0.0/0 src/dst selections and set your address to reflect the local ( left ) and remote ( right ) subnets or an exact reverse from the OpenSwan side. so I' m guessing from your cfg posted earlier; Source= 192.168.1.0/24 Dst= 192.168.71.0/24 Make a simple firewall rule to allow traffic from these 2 subnets ad then police and tighten it up after the VPN works. NOTE: It would help if you can post the details log mesages; here for review. or run a FGT side diag e.g diag debug app ike Keep us inform on whatever the outcome is if you don' t mind.

Thank you very much, emnoc. Making two changes fixed the problem: 1. Setting the internal LAN subnets on the Quick Mode Selector in the phase2 settings of the Fortigate. 2. Reducing the encryption proposals on both sides, and specifying the diffie-hellman setting on the OpenSwan side. Thank you very much for the link to the strongSwan wiki. The final ipsec.conf contains: auth=esp type=tunnel pfs=yes compress=yes ike=3des-sha1-modp1536! phase2=esp phase2alg=3des-sha1

Good job and enjoy. I played around this over 54-5 years at a place that relied heavily on linux based firewalls using both psk and certs. If you can do Open/Strong Swan VPN/IPSEC, than all other firewalls seems relatively simple ;)