GNS3 IPSEC faliover simulation FortiOS 5.6.4

[fslomka]

 Crosspost https://www.reddit.com/r/fortinet/comments/9cfksg/gns3_ipsec_faliover_simulation_fortios_564/

I am trying to simulate how a Fortigate would handle a IPSec failover and recover in GNS3.​Fortigate Config Customerhttps://nopaste.xyz/?770ad3dc22aee4ce#QKRgHKolvL6aHPe6Lmbpbu6YgiCAI/mlSdKXgoR7xjc=Fortigate Config Providerhttps://nopaste.xyz/?67a237612bf3ac23#Pai8maxDB+1HRIiFY132ZBnS1Oy4/jd6K+h1jwTfvps=​This is my setup​Fortigate Customer IPSec InternetB is set to Monitor IPSec InternetA so only one IPSec at a time is possible.​Fortigate Customer 192.168.0.1Fortigate Provider 192.168.1.1NAT is not used for VPNBlackhole routing enabledlink-monitor is used on VPN​​And this is what I do​I let InternetA fail, connection via InternetB to Provider will be established and ping will be working again after 70 sec.result --> Failover works without issues.​I will repair InternetA again.result --> Ping will not work anymore and diag debug flow shows(id143 where InternetA kicks in):​​id=20085 trace_id=142 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:635->192.168.1.1:2048) from port1. type=8, code=0, id=635, seq=151."id=20085 trace_id=142 func=resolve_ip_tuple_fast line=5386 msg="Find an existing session, id-00000322, original direction"id=20085 trace_id=142 func=ipsecdev_hard_start_xmit line=635 msg="enter IPsec interface-InternetB"id=20085 trace_id=142 func=esp_output4 line=892 msg="IPsec encrypt/auth"id=20085 trace_id=142 func=ipsec_output_finish line=527 msg="send to 192.168.122.216 via intf-port10"id=20085 trace_id=143 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:635->192.168.1.1:2048) from port1. type=8, code=0, id=635, seq=152."id=20085 trace_id=143 func=resolve_ip_tuple_fast line=5386 msg="Find an existing session, id-00000322, original direction"id=20085 trace_id=143 func=vf_ip_route_input_common line=2576 msg="find a route: flag=04000000 gw-192.168.122.1 via port10"id=20085 trace_id=143 func=fw_strict_dirty_session_check line=249 msg="SNAT mismatch policy 1 nat 1 ip 0.0.0.0, drop"id=20085 trace_id=144 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:635->192.168.1.1:2048) from port1. type=8, code=0, id=635, seq=153."id=20085 trace_id=144 func=init_ip_session_common line=5470 msg="allocate a new session-00000360"id=20085 trace_id=144 func=vf_ip_route_input_common line=2576 msg="find a route: flag=04000000 gw-192.168.122.1 via port10"id=20085 trace_id=144 func=fw_forward_handler line=743 msg="Allowed by Policy-1: SNAT"id=20085 trace_id=144 func=__ip_session_run_tuple line=3209 msg="SNAT 192.168.0.130->192.168.122.213:61051"​This two messages make me think:msg="SNAT mismatch policy 1 nat 1 ip 0.0.0.0, drop"msg="Allowed by Policy-1: SNAT"​Fortigate tries to establish a new session via port1 --> sd-wan​If I restart ping in this point of scenario this is what will happen(ping will work):​id=20085 trace_id=159 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:636->192.168.1.1:2048) from port1. type=8, code=0, id=636, seq=1."id=20085 trace_id=159 func=init_ip_session_common line=5470 msg="allocate a new session-00000362"id=20085 trace_id=159 func=vf_ip_route_input_common line=2576 msg="find a route: flag=04000000 gw-192.168.1.1 via InternetA"id=20085 trace_id=159 func=fw_forward_handler line=743 msg="Allowed by Policy-3:"id=20085 trace_id=159 func=ipsecdev_hard_start_xmit line=635 msg="enter IPsec interface-InternetA"id=20085 trace_id=159 func=esp_output4 line=892 msg="IPsec encrypt/auth"id=20085 trace_id=159 func=ipsec_output_finish line=527 msg="send to 192.168.122.216 via intf-port9"id=20085 trace_id=160 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:636->192.168.1.1:2048) from port1. type=8, code=0, id=636, seq=2."id=20085 trace_id=160 func=resolve_ip_tuple_fast line=5386 msg="Find an existing session, id-00000362, original direction"id=20085 trace_id=160 func=ipsecdev_hard_start_xmit line=635 msg="enter IPsec interface-InternetA"id=20085 trace_id=160 func=esp_output4 line=892 msg="IPsec encrypt/auth"id=20085 trace_id=160 func=ipsec_output_finish line=527 msg="send to 192.168.122.216 via intf-port9"id=20085 trace_id=161 func=print_pkt_detail line=5311 msg="vd-root received a packet(proto=1, 192.168.0.130:636->192.168.1.1:2048) from port1. type=8, code=0, id=636, seq=3."id=20085 trace_id=161 func=resolve_ip_tuple_fast line=5386 msg="Find an existing session, id-00000362, original direction"id=20085 trace_id=161 func=ipsecdev_hard_start_xmit line=635 msg="enter IPsec interface-InternetA"id=20085 trace_id=161 func=esp_output4 line=892 msg="IPsec encrypt/auth"id=20085 trace_id=161 func=ipsec_output_finish line=527 msg="send to 192.168.122.216 via intf-port9"​The new session got established via policy ID 3 which is port1 --> VPN1​In short termsAll OK > InternetA broken > everything works through InternetB > InternetA repaired again > ping through VPN1 will not work anymore unless restarted.

[fslomka]

I have found the solution. Instead of having 2 different priorities in my static route I put in different distanced. I am required to use the "monitor" command because I am allowed only to have one tunnel open at a time.   This is how a working routing looks tight now on customer. edit 2 set dst 192.168.1.0 255.255.255.0 set device "VPN1" next edit 3 set dst 192.168.1.0 255.255.255.0 set distance 11 set device "VPN2" next   It seems that using the priority is breaking something but distance is not