FortiGate "Remembers" Bad Routes

[Bill_Daly]

Fortigate 90D (v5.2.5)

I have two offices connected by a site-to-site IPSec VPN: 192.168.2.0/24 and 192.168.4.0/24.

If the VPN is down, any traffic sent to the other office gets routed to the Internet.

However, when the VPN comes back up, any IP's I tried to access while the VPN was down still get routed to the Internet while other IP's in the same subnet get routed correctly.

For example:

My computer is 192.168.2.99, my local Fortigate is 192.168.2.1 and I'm trying to access 192.168.4.5 and 192.168.4.50 on the other side of the VPN.  I pinged 192.168.4.50 while the VPN was down.  The VPN is back up when I run the two tracert commands below, one right after the other.  One gets correctly routed through the VPN and the other through the Internet:

C:\>tracert 192.168.4.5 Tracing route to 192.168.4.5 over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 192.168.2.1 2 * * * Request timed out. 3 44 ms 44 ms 49 ms 192.168.4.5 Trace complete. C:\>tracert 192.168.4.50 Tracing route to 192.168.4.50 over a maximum of 30 hops: 1 <1 ms <1 ms <1 ms 192.168.2.1 2 2 ms 1 ms 2 ms 70.46.74.1.nw.nuvox.net [70.46.74.1] 3 2 ms 2 ms 2 ms 216.199.88.109.nw.nuvox.net [216.199.88.109] 4 * * * Request timed out. 5 * * * Request timed out.

Somehow rebooting my computer seems to get the routing to 192.168.4.50 working again, but I don't see how my computer would affect the routing.

Can someone tell me what's going on?

[ede_pfau · Accepted solution]

I will try.

The routing decision is made very early when a packet arrives at the FGT. Once routing is determined, and a policy allows the traffic, a session is established on the FGT. Subsequent traffic of this session is not treated the same way (for efficiency) but offloaded to the network ASIC. Basically, the NP ASIC is programmed to forward packets from one port to another, pre-determined port judging by the session ID attached to the packets.

Now you can see that while the session is still alive, a change in port availability is not affecting the path the packets take. Session lifetime is something around 180 seconds or so, at a minimum. Only after the session has timed out due to lack of traffic, incoming packets will be decided upon anew.

Without this 'split' handling of traffic forwarding a FGT would not be able to control traffic in wirespeed.

To alleviate the dead alley effect you are describing there are (at least) 2 means:

- prevent traffic from and to private networks to establish a session to the WAN

- use of Dead Gateway Detection

The first method is technically done by creating black hole routes of very low priority. For all RFC1918 private network addresses you create a bh route (a route which destination is discarding all traffic) with priority 255. As long as there is any other route for this private network with a higher priority, like a route to the remote LAN via a VPN, traffic will use it; if the tunnel is down, the preferred route is deleted from the active routing table, and the 'shadowed', previously inactive blackhole route is inserted. All traffic to the VPN will now be discarded, and no session be established (!).

Thus, no data leakage to the WAN. And as soon as the tunnel goes up again, a new session will be established via the tunnel.

DGD will help FortiOS to determine that a port is down, by exchanging hello packets and checking for their echos, much like 'ping' (that's why DGD will not always work if the other side's router is from a different vendor). This way it can detect that a network path is down while the actual physical link still is up.

Some time ago I've posted a batch command file with blackhole routes to all known RFC1918 networks which you can download and import. Existing routes will not be affected. [link]https://forum.fortinet.com/FindPost/120872[/link]