SDWAN and sflow

[GabbaTech]

Hi,

Has anyone seen any odd behaviour with sflow when SD-WAN is configured?

A couple of 50E's running 6.2.3 with exactly the same config. SD-WAN interface comprising of wan1 external and two IPSec tunnels. SD-WAN rules have traffic for specific 10.0.0.0/24 range to go via the IPSec tunnels and everything else to go via the internet on wan1. The collector lives in Azure so needs to get there via the Internet. Some devices send sflow data out wan1 and some send it via the IPSec tunnels. Adding a static route to force traffic out via wan1 for the netflow collector to the devices that send via the IPSec tunnels works.

I see bug ID 627951 lists NTP and FSSO not following SD-WAN rules and I'm wondering if this is something similar.

Cheers,

 Gareth

[emnoc]

Did you set the sflow source address

e.g

config sys sflow

   set collector-ip x.x.x.x

   set collector port 5552

   set source-ip y.y.y.y

Ken Felix

[GabbaTech]

Tested with both the source-ip set and not set. Interestingly, the ones that are working don't have the source-ip set.

Here is one that is working. Debugging the session shows tunnel=/

device_1 # show system netflow config system netflow     set collector-ip 102.133.x.x     set collector-port 9996 end

device_1 # diagnose sys session filter dst 102.133.x.x device_1 # diagnose sys session list session info: proto=17 proto_state=00 duration=266148 expire=165 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=255/255 state=local nds netflow-origin netflow-reply statistic(bytes/packets/allow_err): org=6374232/28067/1 reply=0/0/0 tuples=2 tx speed(Bps/kbps): 21/0 rx speed(Bps/kbps): 0/0 orgin->sink: org out->post, reply pre->in dev=0->0/0->0 gwy=0.0.0.0/0.0.0.0 hook=out dir=org act=noop 102.134.x.x:1068->102.133.x.x:9996(0.0.0.0:0) hook=in dir=reply act=noop 102.133.x.x:9996->102.134.x.x:1068(0.0.0.0:0) misc=0 policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=00032e66 tos=ff/ff app_list=0 app=0 url_cat=0 rpdb_link_id = 00000000 ngfwid=n/a dd_type=0 dd_mode=0 total session 1 device_1 # get router info routing-table details Routing table for VRF=0 Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP        O - OSPF, IA - OSPF inter area        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2        E1 - OSPF external type 1, E2 - OSPF external type 2        i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area        * - candidate default S*      0.0.0.0/0 [1/0] via 102.134.x.x, wan1                   [1/0] via 172.16.190.2, HO_1                   [1/0] via 172.17.190.2, HO_2 S       10.10.10.0/24 [10/0] via 172.16.190.2, HO_1                       [10/0] via 172.17.190.2, HO_2 S       10.10.20.0/24 [10/0] via 172.16.190.2, HO_1                       [10/0] via 172.17.190.2, HO_2 S       10.10.21.0/24 [10/0] via 172.16.190.2, HO_1                       [10/0] via 172.17.190.2, HO_2 S       10.10.22.0/24 [10/0] via 172.16.190.2, HO_1                       [10/0] via 172.17.190.2, HO_2 C       10.10.190.0/24 is directly connected, lan C       102.134.x.x/30 is directly connected, wan1 C       172.16.190.1/32 is directly connected, HO_1 C       172.16.190.2/32 is directly connected, HO_1 C       172.17.190.1/32 is directly connected, HO_2 C       172.17.190.2/32 is directly connected, HO_2 C       172.18.190.1/32 is directly connected, Monitoring

Below one isn't working. Also note that tunnel HO_1 is down as per the routing table but it's still sending out via HO_1 according to the session list. In order to get netflow to go over the internet a static route needs to be added.

device_2 # show system netflow config system netflow     set collector-ip 102.133.x.x     set collector-port 9996 end device_2 # diagnose sys session filter dst 102.133.x.x device_2 # diagnose sys session list session info: proto=17 proto_state=00 duration=92197 expire=169 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4 origin-shaper= reply-shaper= per_ip_shaper= class_id=0 ha_id=0 policy_dir=0 tunnel=HO_1/ vlan_cos=255/255 state=log local nds netflow-origin netflow-reply statistic(bytes/packets/allow_err): org=8596348/15708/1 reply=0/0/0 tuples=2 tx speed(Bps/kbps): 90/0 rx speed(Bps/kbps): 0/0 orgin->sink: org out->post, reply pre->in dev=0->0/0->0 gwy=0.0.0.0/0.0.0.0 hook=out dir=org act=noop 102.134.x.x:4319->102.133.x.x:9996(0.0.0.0:0) hook=in dir=reply act=noop 102.133.x.x:9996->102.134.x.x:4319(0.0.0.0:0) misc=0 policy_id=0 auth_info=0 chk_client_info=0 vd=0 serial=0000f838 tos=ff/ff app_list=0 app=0 url_cat=0 rpdb_link_id = 00000000 ngfwid=n/a dd_type=0 dd_mode=0 total session 1

device_2 # get router info routing-table details Routing table for VRF=0 Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP        O - OSPF, IA - OSPF inter area        N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2        E1 - OSPF external type 1, E2 - OSPF external type 2        i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area        * - candidate default S*      0.0.0.0/0 [1/0] via 102.134.x.x, wan1                   [1/0] via 172.17.80.2, HO_2 S       10.10.10.0/24 [10/0] via 172.17.40.2, HO_2 S       10.10.20.0/24 [10/0] via 172.17.40.2, HO_2 S       10.10.21.0/24 [10/0] via 172.17.40.2, HO_2 S       10.10.22.0/24 [10/0] via 172.17.40.2, HO_2 C       10.10.40.0/24 is directly connected, lan S       102.133.x.x/32 [10/0] via 102.134.x.x, wan1 C       102.134.x.x/30 is directly connected, wan1 C       172.16.40.1/32 is directly connected, HO_1 C       172.16.40.2/32 is directly connected, HO_1 C       172.17.40.1/32 is directly connected, HO_2 C       172.17.40.2/32 is directly connected, HO_2 C       172.18.40.1/32 is directly connected, Monitoring

All are running

Version: FortiGate-50E v6.2.3,build1066,191218 (GA)