FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
rphulekar
Staff
Staff
Article
Description You may experience routing issues when some traffic is not using the routing table. There are possible causes of this issue.
Components
  • All FortiGate units running FortiOS 4.0 MR3, v5.0, v5.2, v5.4
Steps or Commands

rmetzger_13842_13842.jpg

When the system works expected, the routing table will look like the following output.

get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*       0.0.0.0/0 [10/0] via 192.168.183.254, port2
S       10.0.0.0/8 [10/0] via 192.168.183.254, port2
S       10.176.0.0/23 [10/0] is directly connected, to3600
C       10.177.0.0/23 is directly connected, port1
C       192.168.182.0/23 is directly connected, port2

Ping between PC-A and PC-B using the following diagnose command:.

diagnose debug flow filter proto 1

diagnose debug flow show console enable
show trace messages on console

diagnose debug flow trace start 2

id=20085 trace_id=10 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=10 msg="allocate a new session-0000030b"
id=20085 trace_id=10 msg="find a route: gw-10.176.0.23 via to3600"
id=20085 trace_id=10 msg="Allowed by Policy-2:"
id=20085 trace_id=10 msg="enter IPsec interface-to3600"
id=20085 trace_id=10 msg="encrypted, and send to 192.168.182.225 with source 192.168.183.37"
id=20085 trace_id=10 msg="send to 192.168.182.225 via intf-port2"
id=20085 trace_id=11 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=11 msg="Find an existing session, id-0000030b, original direction"
id=20085 trace_id=11 msg="enter IPsec interface-to3600"
id=20085 trace_id=11 msg="encrypted, and send to 192.168.182.225 with source 192.168.183.37"
id=20085 trace_id=11 msg="send to 192.168.182.225 via intf-port2"

Check the routecache

diagnose ip rtcache list

family=02 tab=254 vf=0 type=01 tos=0 flag=00000200
10.176.0.23@17(to3600)->10.177.0.21@9(port1) gwy=0.0.0.0 prefsrc=0.0.0.0
ci: ref=2 lastused=41 expire=0 err=00000000 used=0 br=0 pmtu=1500

family=02 tab=254 vf=0 type=01 tos=0 flag=00000200
10.177.0.21@9(port1)->10.176.0.23@17(to3600) gwy=0.0.0.0 prefsrc=0.0.0.0
ci: ref=2 lastused=111 expire=0 err=00000000 used=0 br=0 pmtu=1436

Check the session list

diagnose sys session list

session info: proto=1 proto_state=00 expire=55 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
bandwidth=0/sec guaranteed_bandwidth=0/sec traffic=0/sec prio=0 ha_id=0 hakey=44
tunnel=/
state=may_dirty npu
statistic(bytes/packets/err): org=60/1/0 reply=60/1/0 tuples=2
orgin->sink: org pre->post, reply pre->post dev=9->17/17->9 gwy=10.176.0.23/10.177.0.21
hook=pre dir=org act=noop 10.177.0.21:512->10.176.0.23:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.176.0.23:512->10.177.0.21:0(0.0.0.0:0)
misc=0 policy_id=2 auth_info=0 ids=0xf6f91e58 vd=0 serial=0000034c tos=ff/ff app=0

Compare to the output when the tunnel is down.

Routing table

get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*       0.0.0.0/0 [10/0] via 192.168.183.254, port2
S       10.0.0.0/8 [10/0] via 192.168.183.254, port2
C       10.177.0.0/23 is directly connected, port1
C       192.168.182.0/23 is directly connected, port2

Ping from PC-A to PC-B

diagnose debug flow trace start 2

id=20085 trace_id=12 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=12 msg="allocate a new session-00000433"
id=20085 trace_id=12 msg="find a route: gw-192.168.183.254 via port2"
id=20085 trace_id=12 msg="find SNAT: IP-192.168.183.37, port-55933"
id=20085 trace_id=12 msg="Allowed by Policy-1: SNAT"
id=20085 trace_id=12 msg="SNAT 10.177.0.21->192.168.183.37:55933"
id=20085 trace_id=13 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=13 msg="Find an existing session, id-00000433, original direction"
id=20085 trace_id=13 msg="SNAT 10.177.0.21->192.168.183.37:55933"

This is also normal as there is another route for 10.0.0.0/8 network using Port2. If the tunnel comes back and there are applications that keep trying to reach PC-B, then the traffic will not flow correctly.

get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*       0.0.0.0/0 [10/0] via 192.168.183.254, port2
S       10.0.0.0/8 [10/0] via 192.168.183.254, port2
S       10.176.0.0/23 [10/0] is directly connected, to3600
C       10.177.0.0/23 is directly connected, port1
C       192.168.182.0/23 is directly connected, port2

id=20085 trace_id=14 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=14 msg="Find an existing session, id-00000433, original direction"
id=20085 trace_id=14 msg="find a route: gw-192.168.183.254 via port2"
id=20085 trace_id=14 msg="find SNAT: IP-192.168.183.37, port-55936"
id=20085 trace_id=14 msg="SNAT 10.177.0.21->192.168.183.37:55933"
id=20085 trace_id=15 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=15 msg="Find an existing session, id-00000433, original direction"
id=20085 trace_id=15 msg="SNAT 10.177.0.21->192.168.183.37:55933"
diagnose debug flow trace start 2

Traffic uses Port2, but not the IPSEC interface. This is because the routecache that created when the IPSEC tunnel down is still active.

family=02 tab=254 vf=0 type=01 tos=0 flag=00000200
10.177.0.21@9(port1)->10.176.0.23@8(port2) gwy=192.168.183.254 prefsrc=0.0.0.0
ci: ref=2 lastused=94 expire=0 err=00000000 used=0 br=0 pmtu=1500

Solution

The solution here is to use a firewall policy. Add a deny policy for traffic using the Public Interface instead of IPSEC tunnel. For example:

 

config firewall policy
   edit 3
      set srcintf "port1"
      set dstintf "port2"
      set srcaddr "all"
      set dstaddr "RemoteLAN"
   set schedule "always"
      set service "ANY"
   next
   edit 1

diagnose debug flow trace start 2

id=20085 trace_id=20 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=20 msg="allocate a new session-00000538"
id=20085 trace_id=20 msg="find a route: gw-192.168.183.254 via port2"
id=20085 trace_id=20 msg="Denied by forward policy check"
id=20085 trace_id=21 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1."
id=20085 trace_id=21 msg="allocate a new session-0000053b"
id=20085 trace_id=21 msg="find a route: gw-192.168.183.254 via port2"
id=20085 trace_id=21 msg="Denied by forward policy check"

Use a blackhole route for network destinated over VPN tunnel with higher cost. This causes the traffic to be dropped when it hits the blackhole route.

get router info routing-table all

Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*       0.0.0.0/0 [10/0] via 192.168.183.254, port2
S       10.0.0.0/8 [10/0] via 192.168.183.254, port2
S       10.176.0.0/23 [20/0] is a summary, Null
C       10.177.0.0/23 is directly connected, port1
C       192.168.182.0/23 is directly connected, port2

config router static
   edit 1
      set device "port2"
      set gateway 192.168.183.254
   next
   edit 2
      set device "to3600"
      set dst 10.176.0.0 255.255.254.0
   next
   edit 3
      set device "port2"
      set dst 10.0.0.0 255.0.0.0
      set gateway 192.168.183.254
   next
   edit 4
      set blackhole enable
      set distance 20
      set dst 10.176.0.0 255.255.254.0
   next
end

diagnose debug flow trace start 2
id=20085 trace_id=22 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1." id=20085 trace_id=22 msg="allocate a new session-0000059a" id=20085 trace_id=23 msg="vd-root received a packet(proto=1, 10.177.0.21:512->10.176.0.23:8) from port1." id=20085 trace_id=23 msg="allocate a new session-0000059c"

When the tunnel comes back again, it starts using the route to the IPSEC interface not the NULL route (blackhole route).

get router info routing-table database Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP       O - OSPF, IA - OSPF inter area       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2       E1 - OSPF external type 1, E2 - OSPF external type 2       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area       > - selected route, * - FIB route, p - stale info
S       *> 0.0.0.0/0 [10/0] via 192.168.183.254, port2 S       *> 10.0.0.0/8 [10/0] via 192.168.183.254, port2 S       10.176.0.0/23 [20/0] is a summary, Null S       *> 10.176.0.0/23 [10/0] is directly connected, to3600 C       *> 10.177.0.0/23 is directly connected, port1 C       *> 192.168.182.0/23 is directly connected, port2


Contributors