Policy Based Routing does not work as expected, fortigate 5.2.11

[zorro]

Hi everybody PBR on my Fortgate is not working as expected but rather kind of odd. I have FGT300D running firmware 5.2.11.

It's an outer/perimeter FW installation, one Internet breakout, couple of interface based IPSec VPN terminated. Routing is static only. That part works without any problem. In addition I have couple of PBR rules that route traffic sourcing from specific subnet to another specific route to an interface. That traffic is sent to a NetScaler SD-WAN box which is deployed virtually inline. Basically traffic sent by PBR rules is being encapsulated (gets new DST IP, SRC IP is now SD-WAN) and sent it back to same the interface of firewall, but then due to new source and destination IP in new IP header, it simply follows the routing table. Returning traffic is getting back to SD-WAN box the same way, after being decapsulated it’s sent back to firewall. Destination IP address in returning traffic is known to firewall and finds its way back to initial source. The problem is that this works only when the traffic is initiated from the local site where my firewall is. When one session is initiated from remote site, traffic does not come through. If I start pinging from a remote site it doesn’t go through, but if I start ping from local site at the same time, then suddenly, remote ping starts to get replies! Is this a bug or I lack some configuration?

[Baptiste]

did you try a specific PBR route for your Netscaler ?

you can also create a rule with Netscaler IP and dest. IP => stop policy processing. (put it at top level)

So packets from Netscaler are not sent anymore to itself

[zorro]

Hi Baptiste

Thank you for your reply :)

Not sure I understood what you meant with NetScaler sending packets to itself and how that could help firewall to do its job? Please could you explain it a bit more?

Here are my PBR policies:

config router policy     edit 10         set input-device "port1"         set src "172.14.192.0/255.255.252.0"         set dst "172.60.80.0/255.255.255.0"         set gateway 172.14.198.2     next     edit 11         set input-device "port1"         set src "172.14.192.0/255.255.252.0"         set dst "172.60.99.0/255.255.255.0"         set gateway 172.14.198.2     next end

Local internal subnet is 172.14.192.0/22

Remote subnets are 172.60.80.0/24 and 172.60.99.0/24

Local NetScaler SD-WAN sits on its own subnet 172.14.198.0/24 with IP address .2

Local interface on firewall connected to internal core switch, port1

Also when host from local subnet 172.14.192.0/22 sends ICMP packet to host subnet on remote site 172.60.80.0/24, the packet is by PBR sent to local NetScaler SD-WAN (172.14.198.2). NetScaler SD-WAN encapsulates the packet and the new packet has SRCIP 172.14.198.2 and destination is some public address of another NetScaler SD-WAN box on remote site, let's say 2.2.2.2. When such packet comes to firewall it goes out normally following the default route in routing table. There is one 1:1 NAT rule which translates SRC IP 172.14.198.2 to public routable IP, let's say 1.1.1.1, but that's not that important here.

Returnig packet has DST IP 1.1.1.1 (after NAT 172.14.198.2) and source 2.2.2.2. That is rather not problematic. I see that traffic coming back to NetScaler SD-WAN. NetScaler SD-WAN decapsulates that packet and sends it back to local host. That packet arrives to firewall with DST IP in subnet 172.14.192.0/22 and SRC IP from remote subnet 172.60.80.0/24. That part works perfectly when communication is initiated from local site.

But when remote host initiates communication and sends first ICMP packet, this packet arrives the local SD-WAN but firewall does NOT send it to local subnet!? Or it does not until I initiate ping from local to remote host.

I have also in routing table a route to 172.60.0.0/16 pointing to IPSec VPN to remote site, but I can't see how it could eventually interfere with more specific routes?

BR Z.

[emnoc]

The cli cmd diag debug flow  is your  best friend in this issue

1: I would analyze it

2: I would review the output especially any lines that says routes or policy or lookup

Ken

[zorro]

Tnx Ken

I will take a shot with this.

BR Z.

[UTM4thewin]

It could be an issue with RPF for the traffic originating from the remote site.  If you don't have a static or dynamic(rip,ospf or bgp) route in the routing table for 172.60.80.0/24 & 172.60.99.0/24 then the traffic originating from the remote site might be getting dropped because of the anti-spoofing.  The debug flow will show it if RPF is dropping the traffic. 

[mohamed_sabbah]

I came across this thread (which is little old) however I thought to add this comment in case it will help anyone reading the thread. In my opinion I can see that Outbound Ping is working because the SD-WAN box is configured properly to handle Outbound Many-to-One NAT (or what is known as PAT). So private IP addresses going outbound via the SD-WAN will have the SRC address translated to 1.1.1.1 (if my understanding of the setup is correct). However, we need to check the SD-WAN box for Inbound NAT. So if remote site (2.2.2.2) starts pinging (1.1.1.1) which is the SD-WAN box Public IP, we need NAT rules to translate the destination address to the range (172.14.198.x) which is the local subnet between SD-WAN box and FG firewall. So I would first investigate this Inbound NAT configuration on the SD-WAN box as most likely this is the place of fault. If that NAT is configured properly then it should have a corresponding VIP configured on FG to further translate the incoming traffic to other local subnets/hosts, with suitable inbound firewall policies to allow this traffic.