VPC1 (Sitio1) after pinging FW_Sitio1 and FW_Sitio2:
0c:6b:24:7c:7c:00 192.168.1.2 expires in 32 seconds
0c:6b:24:a1:87:00 192.168.1.1 expires in 66 seconds
About your questions...
I removed the SD-WAN for now, all I use now is mpls-sitioX. Why the SD-WAN? The customer has 4 sites with one or two paths to each other. To increase redundancy we will add an Internet VPN to these paths. So, one site can have from one up to three ways of reaching another site. In order to leave the ISPs outside the routing configuration (believe me, I have enough material for a complete season of "Tales from the Crypt" with this), we will use VPNs in the existing connections. Wrapping this up: the SD-WAN will have 3 VPNS.
1st) That drawing is a GNS3 environment, I'm not that skilled at drawing :)
2nd) What I learned from the packet captures is:
This is how a ping from PC1 to FW_Sitio2 looks like:
2020-04-16 17:57:21.953601 port1 in arp who-has 192.168.1.2 (ff:ff:ff:ff:ff:ff) tell 192.168.1.3
2020-04-16 17:57:21.953621 vx-lan1 out arp who-has 192.168.1.2 (ff:ff:ff:ff:ff:ff) tell 192.168.1.3
2020-04-16 17:57:21.953672 lan1 in arp who-has 192.168.1.2 (ff:ff:ff:ff:ff:ff) tell 192.168.1.3
2020-04-16 17:57:21.955282 vx-lan1 in arp reply 192.168.1.2 is-at c:6b:24:7c:7c:0
2020-04-16 17:57:21.955286 port1 out arp reply 192.168.1.2 is-at c:6b:24:7c:7c:0
2020-04-16 17:57:21.956365 port1 in 192.168.1.3 -> 192.168.1.2: icmp: echo request
2020-04-16 17:57:21.956370 vx-lan1 out 192.168.1.3 -> 192.168.1.2: icmp: echo request
When I captured pings (vxlan) from FW_Sitio1 to FW_Sitio2, I saw the packets going this path: FW_Sitio1/internal/port1 -> FW_Sitio1/internal/vx-lan1 -> encapsulation -> FW_Sitio1/internal/int-ext1 (vdom link) -> FW_Sitio1/root/int-ext0 (vdom link) -> FW_Sitio1/root/mpls-sitio2 (vpn).
When I captured pings from PC1 to FW_Sitio2's IP, I saw the same path, but they stop when arriving at int-ext0. They never go further.[/ul]
The captures are at https://pastebin.com/m31rkuvC, the TL;DR is: for ARP requests I see the packets leaving through the "mpls_sitio2" tunnel at the root VDOM. For ICMP packets they go up to "int-ext0" in the root VDOM, but not further.
The capture has FW_Sitio2's point of view at the end.
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.