diag debug - Traffic looks flowing.. with tunnel DOWN!!!!!!!!!
I have 10 IPSEc VPNs, with 10 diff remote customers with 10 diff remote OS endpoints
Yesterday, one of the VPNs stoped working properly. In the remote endpoint, it was a MicroTik and even there, there are more than 10 VPNs.
In both cases (my fortigate and remote microtik) only ONE of the several existing VPNs are showing a weird behaviour
IN my case, i can show the remote VPN admin guy, that the traffic is coming, returning back , and more, using debug, i can show traffic flowing to remote endpoint VPN, and i can see the policy number
id=20085 trace_id=138 func=fw_forward_handler line=675 msg="Allowed by Policy-608:"
id=20085 trace_id=138 func=ipsecdev_hard_start_xmit line=121 msg="enter IPsec interface-Trustvpn"
IN the other site, he claims that he can´t see my traffic coming back, in fact, barely can see the traffic going, in his perspective looks like the traffic is goint to nowhere.. he can´t see traffic being sent inside the VPN
More bizarre: The message stating the policy number and showing that the packa has been delivered is shown EVEN WITH THE TUNNEL DOWN! Fortigate shows the tunnel up, but we´ve changed the password of my side and the tunnle never got down!
so, we did a reset, a flush,m re-cunched passwords again and yet, the tunnle gots UP in a matter of seconds, even a refresh in the GUI and the tunnel is there, up and running.
there is a source NAT, all traffic from remote endpoint, looks like a coming from a ip subnet, like an IP Pool in source VPN
with the tunnel DOWN:
the test, :
telnet> open 10.224.11.1 8070
Trying 10.224.11.1...
telnet: connect to address 10.224.11.1: Connection timed out
telnet> open 10.224.11.1 8070
Trying 10.224.11.1...
telnet: connect to address 10.224.11.1: Connection timed out
telnet>
tcpdump:
interfaces=[TrustDBAvpnNG]
filters=[port 8070]
pcap_lookupnet: TrustDBAvpnNG: no IPv4 address assigned
13.742825 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
14.742942 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
16.743190 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
20.743687 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
28.744194 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
44.746181 10.24.0.73.59854 -> 10.224.11.1.8070: syn 696877932
the DIAG:
id=20085 trace_id=138 func=init_ip_session_common line=4629 msg="allocate a new session-5c465ddb"
id=20085 trace_id=138 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-10.224.11.1 via
id=20085 trace_id=139 func=print_pkt_detail line=4478 msg="vd-root received a packet(proto=6, 10.24.0.73:60204-
id=20085 trace_id=139 func=resolve_ip_tuple_fast line=4539 msg="Find an existing session, id-5c465ddb, original direction"
id=20085 trace_id=139 func=ipsecdev_hard_start_xmit line=121 msg="enter IPsec interface-Trustvpn"
id=20085 trace_id=139 func=ipsec_output_finish line=232 msg="send to 202.92.192.122 via intf-port2"
id=20085 trace_id=139 func=esp_output4 line=897 msg="encrypting, and send to 171.251.81.31 with source 202.92.192.122"
Ghost jokes apart, how can i got down all IPSEC services, without rebooting the unit?
SHould i reboot the entire CLUSTER?
FOrtiOS 5.2.7