|
ADVPN is set up with SD-WAN and uses BGP as a routing protocol. For setup and topology, see Basic SD-WAN/ADVPN design | FortiGate
Upon running a debug flow, traffic is seen leaving the VPN to the HUB instead of the shortcut tunnel:
id=65308 trace_id=65 func=print_pkt_detail line=5870 msg="vd-root:0 received a packet(proto=1, 172.31.2.160:63903->10.6.100.1:2048) tun_id=0.0.0.0 from port1. type=8, code=0,
id=65308 trace_id=65 func=init_ip_session_common line=6055 msg="allocate a new session-023b0844"
id=65308 trace_id=65 func=__vf_ip_route_input_rcu line=1991 msg="find a route: flag=00000000 gw-X.X.X.X via ascftcvpn02"
id=65308 trace_id=65 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=46, len=2"
id=65308 trace_id=65 func=fw_forward_handler line=990 msg="Allowed by Policy-1:"
id=65308 trace_id=65 func=ip_session_confirm_final line=3111 msg="npu_state=0x1100, hook=4"
id=65308 trace_id=65 func=ids_receive line=431 msg="send to ips"
id=65308 trace_id=65 func=ipsecdev_hard_start_xmit line=662 msg="enter IPSec interface ascftcvpn02, tun_id=0.0.0.0"
id=65308 trace_id=65 func=_do_ipsecdev_hard_start_xmit line=222 msg="output to IPSec tunnel ascftcvpn02, tun_id=X.X.X.X, vrf 0"
Checking the routing table shows that there is an active route to the destination via a shortcut tunnel:
FGT01 # get router info routing-table details 10.6.100.1
Routing table for VRF=0
Routing entry for 10.6.100.0/24
Known via "bgp", distance 200, metric 0, best
Last update 16:02:32 ago
* vrf 0 10.1.251.7 priority 1 (recursive via ascftcvpn02 tunnel X.X.X.X)
* vrf 0 10.1.252.8 priority 1 (recursive is directly connected, ascftcvpn02_2)
* vrf 0 10.2.251.5 priority 1 (recursive via aschtovpn02 tunnel X.X.X.X)
Checking the session list reveals the session is not controlled and routed by SD-WAN rules, as it is missing the 'sdwan_service_id' flag:
session info: proto=1 proto_state=00 duration=7 expire=53 timeout=0 flags=00000000 socktype=0 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty ndr npu f00
statistic(bytes/packets/allow_err): org=168/2/1 reply=0/0/0 tuples=2
tx speed(Bps/kbps): 22/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=3->22/22->3 gwy=98.6.21.162/0.0.0.0
hook=pre dir=org act=noop 172.31.2.160:25->10.6.100.1:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.6.100.1:25->172.31.2.160:0(0.0.0.0:0)
misc=0 policy_id=1 pol_uuid_idx=15806 auth_info=0 chk_client_info=0 vd=0
serial=023b1a1a tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=80000000 ngfwid=n/a
npu_state=0x001108
npu info: flag=0x82/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason: redir-to-ips denied-by-nturbo
Furthermore, the SD-WAN service rule status does not include the destination subnet (in this case is 10.6.100.0/24) as a destination address.
FGT01 # diagnose sys sdwan service 2
Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(2), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(2):
1: Seq_num(5 ascftcvpn02), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(6 aschtovpn02), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(1):
10.6.240.0-10.6.240.255
This issue is caused by the missing of the remote subnet in an SD-WAN rule. To add the subnet to the SD-WAN rule, navigate to SD-WAN -> SD-WAN Rules and edit the dedicated rule for the VPN tunnel.
After modifying the SD-WAN rule, check the SD-WAN service rule status and confirm the remote subnet is added:
FGT01 # diagnose sys sdwan service 2
Service(2): Address Mode(IPV4) flags=0x200 use-shortcut-sla
Tie break: cfg
Gen(1), TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla), sla-compare-order
Members(2):
1: Seq_num(5 ascftcvpn02), alive, sla(0x1), gid(0), cfg_order(0), local cost(0), selected
2: Seq_num(6 aschtovpn02), alive, sla(0x1), gid(0), cfg_order(1), local cost(0), selected
Src address(1):
0.0.0.0-255.255.255.255
Dst address(2):
10.6.100.0-10.6.100.255
10.6.240.0-10.6.240.255
To verify this, use the command 'diagnose sys session list' and confirm traffic is routed based on SD-WAN rules.
session info: proto=1 proto_state=00 duration=27 expire=33 timeout=0 flags=00000000 socktype=0 sockport=0
state=log may_dirty ndr npu f00
statistic(bytes/packets/allow_err): org=32/1/1 reply=32/1/1 tuples=2
tx speed(Bps/kbps): 1/0 rx speed(Bps/kbps): 1/0
orgin->sink: org pre->post, reply pre->post dev=3->2759/2759->3 gwy=x.x.x.x
hook=pre dir=org act=noop 172.31.2.160:9998->10.6.240.1:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.6.240.1:9998->172.31.2.160:0(0.0.0.0:0)
misc=0 policy_id=1 pol_uuid_idx=15806 auth_info=0 chk_client_info=0 vd=0
sdwan_mbr_seq=1 sdwan_service_id=2 <- routing follows the SD-WAN rule number 2
no_ofld_reason: redir-to-ips denied-by-nturbo
Note:
From the debug flow output, it is possible to identify which SD-WAN service ID or policy route handled the traffic.
Related article:
Technical Tip: Understanding the Differences between Policy Routes, SD-WAN Rules, and ISDB Routes fo...
|
