Issue with Cross-VLAN Communication over VXLAN/IPSEC between Two Sites

[5q46n2te8jPWJY]

Hello,

I am encountering an issue with my configuration that I am unable to resolve.

I have two sites connected with VXLAN over IPSEC. On each of my sites, I have two VLANs (VLAN 10 and VLAN 20).

Here are my tests:

• SITE A / VLAN 10 successfully pings SITE B / VLAN 10
• SITE A / VLAN 20 successfully pings SITE B / VLAN 20
• SITE A / VLAN 10 successfully pings SITE A / VLAN 20
• SITE A / VLAN 20 successfully pings SITE A / VLAN 10
• SITE B / VLAN 10 successfully pings SITE A / VLAN 10
• SITE B / VLAN 20 successfully pings SITE A / VLAN 20
• SITE B / VLAN 10 successfully pings SITE B / VLAN 20
• SITE B / VLAN 20 successfully pings SITE B / VLAN 10

However,

• SITE A / VLAN 10 cannot ping SITE B / VLAN 20
• SITE A / VLAN 20 cannot ping SITE B / VLAN 10
• SITE B / VLAN 10 cannot ping SITE A / VLAN 20
• SITE B / VLAN 20 cannot ping SITE A / VLAN 10

Do you have any idea why?

Thanks for your help!

[HarshChavda]

Hello @5q46n2te8jPWJY ,

Are you using 0.0.0.0/0 in phase2 selector of IPSEC tunnel, if not can you verify if these VLANs are included. We can also check with running a sniffer on both side.

Reference: https://community.fortinet.com/t5/FortiGate/Troubleshooting-Tip-Routing-Issue/ta-p/195727

[5q46n2te8jPWJY]

I confirm that I am indeed using the correct phase2 selector, as recommended, with 0.0.0.0/0.

[hazim]

Hello @5q46n2te8jPWJY 

You can run debug flow to see the traffic outgoing/incoming to verify:

https://community.fortinet.com/t5/FortiGate/Technical-Tip-Using-filters-to-review-traffic-traversing...

[5q46n2te8jPWJY]

.

[tachen]

How you gateway design each VLAN, do you use EVPN control plane?

FortiOS 7.4.5 not support IRB nor anycast gateway.

[5q46n2te8jPWJY]

I believe you might be mistaken. In the video here, it seems to work without needing that specific configuration like IRB or anycast gateway.

[5q46n2te8jPWJY]

@hazim 

When I run a ping from SITE A / VLAN 10 to SITE B / VLAN 20,

On site A Fortigate CLI, I see that :

diagnose debug flow filter clear
diagnose debug flow filter daddr 10.1.20.2
diagnose debug enable
diagnose debug flow trace start 5

id=65308 trace_id=137 func=print_pkt_detail line=5920 msg="vd-VDOM1:0 received a packet(proto=1, 10.1.10.1:16213->10.1.20.2:2048) tun_id=0.0.0.0 from VXLAN10-SW. type=8, code=0, id=16213, seq=1."
id=65308 trace_id=137 func=init_ip_session_common line=6110 msg="allocate a new session-210f0670"
id=65308 trace_id=137 func=__vf_ip_route_input_rcu line=1988 msg="find a route: flag=00000000 gw-0.0.0.0 via VXLAN20-SW"
id=65308 trace_id=137 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=5, len=9"
id=65308 trace_id=137 func=fw_forward_handler line=998 msg="Allowed by Policy-247:"
id=65308 trace_id=137 func=ip_session_confirm_final line=3128 msg="npu_state=0x100, hook=4"
id=65308 trace_id=138 func=print_pkt_detail line=5920 msg="vd-VDOM1:0 received a packet(proto=1, 10.1.10.1:16213->10.1.20.2:2048) tun_id=0.0.0.0 from VXLAN10-SW. type=8, code=0, id=16213, seq=2."
id=65308 trace_id=138 func=resolve_ip_tuple_fast line=6013 msg="Find an existing session, id-210f0670, original direction"
id=65308 trace_id=138 func=npu_handle_session44 line=1342 msg="Trying to offloading session from VXLAN10-SW to VXLAN20-SW, skb.npu_flag=00000400 ses.state=00000200 ses.npu_state=0x00000100"
id=65308 trace_id=138 func=fw_forward_dirty_handler line=444 msg="state=00000200, state2=00000000, npu_state=00000100"
id=65308 trace_id=139 func=print_pkt_detail line=5920 msg="vd-VDOM1:0 received a packet(proto=1, 10.1.10.1:16213->10.1.20.2:2048) tun_id=0.0.0.0 from VXLAN10-SW. type=8, code=0, id=16213, seq=3."
id=65308 trace_id=139 func=resolve_ip_tuple_fast line=6013 msg="Find an existing session, id-210f0670, original direction"
id=65308 trace_id=139 func=npu_handle_session44 line=1342 msg="Trying to offloading session from VXLAN10-SW to VXLAN20-SW, skb.npu_flag=00000400 ses.state=00000200 ses.npu_state=0x00000100"
id=65308 trace_id=139 func=fw_forward_dirty_handler line=444 msg="state=00000200, state2=00000000, npu_state=00000100"
id=65308 trace_id=140 func=print_pkt_detail line=5920 msg="vd-VDOM1:0 received a packet(proto=1, 10.1.10.1:16213->10.1.20.2:2048) tun_id=0.0.0.0 from VXLAN10-SW. type=8, code=0, id=16213, seq=4."
id=65308 trace_id=140 func=resolve_ip_tuple_fast line=6013 msg="Find an existing session, id-210f0670, original direction"
id=65308 trace_id=140 func=ipv4_fast_cb line=53 msg="enter fast path"
id=65308 trace_id=141 func=print_pkt_detail line=5920 msg="vd-VDOM1:0 received a packet(proto=1, 10.1.10.1:16213->10.1.20.2:2048) tun_id=0.0.0.0 from VXLAN10-SW. type=8, code=0, id=16213, seq=5."
id=65308 trace_id=141 func=resolve_ip_tuple_fast line=6013 msg="Find an existing session, id-210f0670, original direction"
id=65308 trace_id=141 func=ipv4_fast_cb line=53 msg="enter fast path"

 On site B Fortigate, with the same CLI, I see nothing...

Do you have an idea ?

[funkylicious]

I recreated a similar lab like in the video, hit the same issue where i couldnt ping from S1/A to S2/B and did a debug and looked similar like yours, did a diagnose netlink brctl list and diagnose netlink brctl name host <> to confirm that I can see the mac of host in S2/B in FGT-A and then the ping worked

[5q46n2te8jPWJY]

Which version do you use for your lab?