Traffic takes wrong route - cache?

[sw2090]

I have the following constellation:

FGT 100E on site that has two ipsec tunnels to HQ.

It has two static routes for HQ Net over the tunnels with different prio (for Failover).

Also there is some vlan interfaces for special purposes on Site.

I now have a device in one vlan that wants to talk to a server at HQ.

It has all neccessary policies and routes on both sides. Just on the 100E on Site the order of the policies had been worong.

Due to that the internet policy for the vlan matched first and the policy to the server at HQ so never matched. Correct so far.

I changed the order of the policies so the other one can match first.

I also deleted every session of the device.

Thus flow debug shows me that the traffic still matches the wrong policy and gets routet to the internet instead to HQ.

Due to static routing the FGT should know it has to route traffic to HQ over the tunnel with lowest prio  first and higher prio second. It does that perfect except for that one device.

Is the routing cached somewhere idependent from sessions?

[lobstercreed]

Hey Sebastian,

Forgive me if I'm stating the obvious, but it seems like you had a routing problem and tried to solve it with firewall policy (unless you meant routing policy), which of course does not work but would explain the continued wrong behavior.

"Due to that the internet policy for the vlan matched first and the policy to the server at HQ so never matched. Correct so far. I changed the order of the policies so the other one can match first."

Routing comes before policy decisions in the firewall so the order of policies that have different egress interfaces doesn't matter in the first place.

Now, how to solve the routing problem is the question, but we'd need more info to try to help with that.

- Daniel

[sw2090]

Daniel,

usually I just create the policies to allow the traffic to flow. 

Routing is (except from WAN Interfaces) static here anyways.

And yes routing comes before policies. That's the point where it goes wrong way because due to whatever reason my FGT prefers to select the default route for that traffic even though there is static routes.

So lets say the device is 10.1.2.3 in 10.1.2.0/24.

The Server at HQ is 192.168.23.23/32.

FGT is 10.1.2.254 in the device subnet.

Then there is two ipsec tunnels to HQ.

For both there is a static route:

192.168.23.0/24 via tunnel1 prio 10, distance 10

192.168.23.0/24 via tunnel2 prio 20, distiance 20

(this is for failover and will switch to tunnel2 when tunnel1 is gone)

Also there is three default routes:

0.0.0.0/0.0.0.0 via wan1 with prio 1 and gw of wan1

0.0.0.0/0.0.0.0 via wan2 with prio 1 and gw of wan2

0.0.0.0/0.0.0.0 via wan3 with prio 1 and gw of wan3

Those are automatically created by SD-WAN and I cannot change them due to that.

SD-WAN handles which one of them to use when :)

Given this the FGT should select the route via tunnel1 (or tunnel2 if tunnel1 is gone) if there is traffic that wants to go to 192.168.23.0/24. 

Now I have traffic coming from 10.1.2.3 via the corresponding vlan interface and going to 192.168.23.23.

Why does the FGT select one of the default routes for this (an in consequence then only match the internet policy).

Usually Subnets the FGT knows either by interface (connected route) or static route come before the default route as in any other network device too.

I also openend a TAC Ticket on that as I have no more idea.

If you want me to provide more info let me know :)

[Benoit_Rech_FTNT]

Hello Sebastian,

When the default route was used instead of the most specific routes, are you sure that your VPN tunnels where up ? If this is not the case, then you can use "blackhole" routes with higher distance than the VPN to prevent the traffic to go through the default route when the VPN is not up.

Moreover, they are three commands to troubleshoot the routing on the FortiGate: * get router info routing-table all (see the routing table, with IPSec, dynamic protocols, ...) * get router info kernel (or diag ip route list) are the two commands to show the kernel routing table.  * diagnose ip rtcache list (display the route cache). If you need more help, please share the output of the three commands. Best regards, Benoit

[sw2090]

This was solved meanwhile. It was no fault of the FortiGate...it was due to other network missconfig that was not on the FGT ;)

So to say a "fatal adiministrator error" or a OSI Layer 8 Problem ;D

[Benoit_Rech_FTNT]

Good to see the problem is solved, that's the more important.