GRE tunnel and policy routing (PBR)

[hubertzw]

Hi,

I'm trying to forward some traffic (PBR) via different interface (GRE Tunnel). When I have a default route via port1 (with better metric) it doesn't work:

Local-FortiGate # get router info routing-table all

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S* 0.0.0.0/0 [10/0] via 10.200.1.254, port1
C 10.0.1.0/24 is directly connected, port3
C 10.200.1.0/24 is directly connected, port1
C 10.200.2.0/24 is directly connected, port2


Local-FortiGate # get router info routing-table database

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       > - selected route, * - FIB route, p - stale info

S *> 0.0.0.0/0 [10/0] via 10.200.1.254, port1
S 0.0.0.0/0 [100/0] is directly connected, E-LD7
C *> 10.0.1.0/24 is directly connected, port3
C *> 10.200.1.0/24 is directly connected, port1
C *> 10.200.2.0/24 is directly connected, port2


Local-FortiGate #

The PBR is ignored:

Local-FortiGate # id=20085 trace_id=28 func=print_pkt_detail line=5517 msg="vd-root:0 received a packet(proto=1, 10.0.1.10:1->10.0.2.10:2048) from port3. type=8, code=0, id=1, seq=157."
id=20085 trace_id=28 func=init_ip_session_common line=5682 msg="allocate a new session-00001ad2"
id=20085 trace_id=28 func=iprope_dnat_check line=4942 msg="in-[port3], out-[]"
id=20085 trace_id=28 func=iprope_dnat_check line=4955 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=28 func=vf_ip_route_input_common line=2591 msg="find a route: flag=04000000 gw-10.200.1.254 via port1"
id=20085 trace_id=28 func=iprope_fwd_check line=726 msg="in-[port3], out-[port1], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"
id=20085 trace_id=28 func=__iprope_tree_check line=548 msg="gnum-100004, use addr/intf hash, len=2"
id=20085 trace_id=28 func=__iprope_check_one_policy line=1996 msg="checked gnum-100004 policy-1, ret-matched, act-accept"
id=20085 trace_id=28 func=__iprope_user_identity_check line=1806 msg="ret-matched"

When I remove the route via port1:

Local-FortiGate # get router info routing-table all

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S* 0.0.0.0/0 [100/0] is directly connected, E-LD7
C 10.0.1.0/24 is directly connected, port3
C 10.200.1.0/24 is directly connected, port1
C 10.200.2.0/24 is directly connected, port2


Local-FortiGate #

the PBR works fine:

id=20085 trace_id=27 func=print_pkt_detail line=5517 msg="vd-root:0 received a packet(proto=1, 10.0.1.10:1->10.0.2.10:2048) from port3. type=8, code=0, id=1, seq=153."
id=20085 trace_id=27 func=init_ip_session_common line=5682 msg="allocate a new session-0000194a"
id=20085 trace_id=27 func=iprope_dnat_check line=4942 msg="in-[port3], out-[]"
id=20085 trace_id=27 func=iprope_dnat_check line=4955 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=27 func=vf_ip_route_input_common line=2578 msg="Match policy routing: to 10.0.2.10 via ifindex-20"
id=20085 trace_id=27 func=vf_ip_route_input_common line=2591 msg="find a route: flag=04000000 gw-10.0.2.10 via E-LD7"
id=20085 trace_id=27 func=iprope_fwd_check line=726 msg="in-[port3], out-[E-LD7], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"
id=20085 trace_id=27 func=__iprope_tree_check line=548 msg="gnum-100004, use addr/intf hash, len=2"
id=20085 trace_id=27 func=__iprope_check_one_policy line=1996 msg="checked gnum-100004 policy-2, ret-matched, act-accept"
id=20085 trace_id=27 func=__iprope_user_identity_check line=1806 msg="ret-matched"

and traffic is sent via E-LD7 GRE tunnel. Any idea why?

[Toshi_Esumi · Accepted solution]

Don't send the distance higher for the 0/0 route into the tunnel. But set it 10 and also set the priority number higher(priority is lower) like 10. Then both show up in the routing-table then your PBR would work.

[hubertzw]

Much better now:

Local-FortiGate # get router info routing-table all

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S* 0.0.0.0/0 [10/0] via 10.200.1.254, port1
                  [10/0] is directly connected, E-LD7, [10/0]
C 10.0.1.0/24 is directly connected, port3
C 10.200.1.0/24 is directly connected, port1
C 10.200.2.0/24 is directly connected, port2


Local-FortiGate #

I modified the PBR a bit and only TCP should be matched:

config router policy
    edit 1
        set input-device "port3"
        set srcaddr "LOCAL_SUBNET"
        set dstaddr "REMOTE_SUBNET"
        set protocol 6
        set output-device "E-LD7"
    next
end

and when I tested it ICMP was sent via port1:

id=20085 trace_id=30 func=print_pkt_detail line=5517 msg="vd-root:0 received a packet(proto=1, 10.0.1.10:1->10.0.2.10:2048) from port3. type=8, code=0, id=1, seq=165."
id=20085 trace_id=30 func=init_ip_session_common line=5682 msg="allocate a new session-0000229e"
id=20085 trace_id=30 func=iprope_dnat_check line=4942 msg="in-[port3], out-[]"
id=20085 trace_id=30 func=iprope_dnat_check line=4955 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=30 func=vf_ip_route_input_common line=2591 msg="find a route: flag=04000000 gw-10.200.1.254 via port1"
id=20085 trace_id=30 func=iprope_fwd_check line=726 msg="in-[port3], out-[port1], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"

and TCP traffic via GRE:

id=20085 trace_id=31 func=print_pkt_detail line=5517 msg="vd-root:0 received a packet(proto=6, 10.0.1.10:50731->10.0.2.10:80) from port3. flag , seq 2551198907, ack 0, win 8192"
id=20085 trace_id=31 func=init_ip_session_common line=5682 msg="allocate a new session-000022b0"
id=20085 trace_id=31 func=iprope_dnat_check line=4942 msg="in-[port3], out-[]"
id=20085 trace_id=31 func=iprope_dnat_check line=4955 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=31 func=vf_ip_route_input_common line=2578 msg="Match policy routing: to 10.0.2.10 via ifindex-20"
id=20085 trace_id=31 func=vf_ip_route_input_common line=2591 msg="find a route: flag=04000000 gw-10.0.2.10 via E-LD7"
id=20085 trace_id=31 func=iprope_fwd_check line=726 msg="in-[port3], out-[E-LD7], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"

Thank you!