Technical Tip: IP Routing flow process under FortiSwitch

[ehamud]

Description	This article describes how to check the IP routing flow process under FortiSwitch.
Scope	Any FortiSwitch that supports the IP routing feature.
Solution	When talking about forward all IP packets, it means IP routing from one interface to another. On this topology, PC1 is going to send IP packets to PC2, which has to be routed by the FortiSwitch-1 and FortiSwitch-2. To reach the subnet 192.168.4.x, it is necessary to set the IPs needed and create a static route back and forth. Here is the configuration: FortiSwitch-1: Configure the Internal interface IP, and I created a physical type interface that will be acting as a Gateway to the PC1: S108DVNEHL-----F # show system interface     config system interface         edit "mgmt"             set mode dhcp             set allowaccess ping https ssh             set type physical             set secondary-IP enable             set snmp-index 11             set defaultgw enable                 config secondaryip                     edit 1                         set ip 192.168.1.99 255.255.255.0                         set allowaccess ping https ssh                     next                 end             next                 edit "internal"                         set ip 192.168.13.1 255.255.255.240                         set allowaccess ping https ssh                         set type physical                         set snmp-index 10                         set macaddr 4a:ed:10:7f:00:01                     next                         edit "GWInt"                             set ip 192.168.3.254 255.255.255.0                             set allowaccess ping https http ssh radius-acct                             set type physical                             set l2-interface "port1"                             set snmp-index 14                         next                     end Configure static route: S108DVNEHLH-----F (static) # show     config router static         edit 1             set device "internal"             set dst 192.168.4.0 255.255.255.0             set gateway 192.168.13.2         next     end FortiSwitch-2: Is the same process as the number 1: S108DVAJT9-----D # show system interface     config system interface         edit "mgmt"             set mode dhcp             set allowaccess ping https ssh             set type physical             set secondary-IP enable             set snmp-index 11             set defaultgw enable                 config secondaryip                     edit 1                         set ip 192.168.1.99 255.255.255.0                         set allowaccess ping https ssh                     next                 end             next                 edit "internal"                     set ip 192.168.13.2 255.255.255.240                     set allowaccess ping https ssh                     set type physical                     set snmp-index 10                     set macaddr aa:81:8e:e7:a8:43                 next                     edit "GWInterf"                         set ip 192.168.4.254 255.255.255.0                         set allowaccess ping https http ssh radius-acct                         set type physical                         set l2-interface "port2"                         set snmp-index 14                     next                 end Static route: S108DVAJT9-----D (static) # show     config router static         edit 1             set device "internal"             set dst 0.0.0.0 0.0.0.0             set gateway 192.168.13.1         next     end Previously IP address has been set to PC1 and PC2 with its own Gateway. It is now possible to reach the network 192.168.4.1 and vice versa: PC1> ping 192.168.4.1 84 bytes from 192.168.4.1 icmp_seq=1 ttl=62 time=3.036 ms 84 bytes from 192.168.4.1 icmp_seq=2 ttl=62 time=1.857 ms 84 bytes from 192.168.4.1 icmp_seq=3 ttl=62 time=1.692 ms 84 bytes from 192.168.4.1 icmp_seq=4 ttl=62 time=4.323 ms 84 bytes from 192.168.4.1 icmp_seq=5 ttl=62 time=4.361 ms PC2> ping 192.168.3.1 84 bytes from 192.168.3.1 icmp_seq=1 ttl=62 time=2.032 ms 84 bytes from 192.168.3.1 icmp_seq=2 ttl=62 time=4.998 ms 84 bytes from 192.168.3.1 icmp_seq=3 ttl=62 time=4.398 ms 84 bytes from 192.168.3.1 icmp_seq=4 ttl=62 time=4.433 ms 84 bytes from 192.168.3.1 icmp_seq=5 ttl=62 time=4.613 ms PC1 creates an IP packet using its IP address 192.168.3.1 as the source. But using the IP 192.168.4.1 as the destination, PC1 has now a challenge to figure out evaluating if the destination is local or remote. In this case, PC1 is under the 192.168.3.1: 254 range and the IP 192.168.4.1 is not local. For that reason, it is necessary to use the default Gateway. Now a second challenge to PC1 is to know the destination MAC address from the default Gateway, and ARP table from PC1 shows the MAC address that belongs to the port1 Switch Gateway interface: PC1> arp 0c:4e:73:60:00:01  192.168.3.254 expires in 118 seconds The ethernet frame will be reflected like this under the FortiSwitch:     Source Destination Source Destination 00:50:79:66:68:00        0C:4E:73:60:00:01          192.168.3.1         192.168.4.1 If the FCS is not correct, it is dropped immediately, if the FCS is correct, the frame will be processed this way: The destination MAC address belongs to a multicast address that the Switch listens to. The destination MAC address belongs to the interface Switch. The destination MAC address belongs to the broadcast address from the subnet that the switch port interface is connected to. In this scenario,  the number 2 option is working for us using port1 from FortiSwitch-1. Also, another requirement to validate is the Header Checksum, if it does not have any error no issues here, otherwise it will be dropped. Remember the Ethernet frame has the following fields: Preamble SFD DESTINATION SOURCE TYPE IP PACKET FCS Now at this point, the IP packet field is needed for the flow process, this field is divided into the following fields: Version Header Length Type of service Total Length Identification IP Flags Fragment Offset Time to live 255 Protocol Header Checksum Source 192.168.3.1 Destination 192.168.4.1 IP Option Let´s check the routing table under FortiSwitch-1, this Switch knows how to reach 192.168.4.0/24 network using the next hop 192.168.13.2 using the internal interface: S108DVNEHL-----F # get router info routing-table Codes: K - kernel route, C - connected, S - static, R - RIP,        O - OSPF, I - IS-IS, B - BGP, T - Table,        > - selected route, * - FIB route, q - queued, r - rejected, b - backup, ^ - HW install failed        t - trapped, o - offload failure C>*  192.168.3.0/24 is directly connected, GWInt, 23:30:39 S>*  192.168.4.0/24 [10/0] via 192.168.13.2, internal, weight 1, 04:52:53 C>*  192.168.13.0/28 is directly connected, internal, 05:39:00 One field is in place, it is called TTL (Time to Live) which will be decreased by one, FortiSwitch-1 will do this by changing the IP header and calculating again the checksum. Let´s check the FortiSwitch-1 ARP table, it is doing a match with the IP internal from FortiSwitch-2, if the ARP is empty, FortiSwitch-1 will be doing an ARP request to find out the MAC address from 192.168.13.2: S108DVNEHL-----F # get system arp Address           Age(min)   Hardware Addr      Interface 192.168.13.2      1          aa:81:8e:e7:a8:43  internal 192.168.3.1       8          00:50:79:66:68:00  GWInt Now a net IP packet is encapsulated in a new Ethernet frame like this pointing to FortiSwitch-2: Source 4a:ed:10:7f:00:01 Destination aa:81:8e:e7:a8:43 Source 192.168.3.1 Destination 192.168.4.1 FortiSwitch-2 will be doing the same as FortiSwitch-1, check the FCS, de-encapsulate the IP packet, check the IP header checksum, and check the destination IP address. The routing table from FortiSwitch-2, 192.168.4.0 is directly connected to GWInterf the TTL of the IP packet will be reduced by ONE from 254 to 253, this is the ideal number, but the value can be chosen randomly by another one. S108DVAJT9-----D # get router info routing-table Codes: K - kernel route, C - connected, S - static, R - RIP,        O - OSPF, I - IS-IS, B - BGP, T - Table,        > - selected route, * - FIB route, q - queued, r - rejected, b - backup, ^ - HW install failed        t - trapped, o - offload failure S>*  0.0.0.0/0 [10/0] via 192.168.13.1, internal, weight 1, 05:19:57 C>*  192.168.4.0/24 is directly connected, GWInterf, 23:48:01 C>*  192.168.13.0/28 is directly connected, internal, 06:00:07 Let’s check the ARP table from FortiSwitch-2: S108DVAJT9-----D # get system arp Address           Age(min)   Hardware Addr      Interface 192.168.4.1       0          00:50:79:66:68:01  GWInterf 192.168.13.1      1          4a:ed:10:7f:00:01  internal A new Ethernet frame is created in the following way: Source 0C:AC:55:7C:00:02 Destination 00:50:79:66:68:01 Source 192.168.3.1 Destination 192.168.4.1 Now let´s validate the TTL value from FortiSwitch-1 under a packet capture which is 64: Now let´s validate the TTL value from FortiSwitch-2 which is 62, and the TTL always decreases till the destination: The reason why TTL decremented 2, Is because PC1 is doing 2 hops: PC1> trace 192.168.4.1 trace to 192.168.4.1, 8 hops max, press Ctrl+C to stop  1   192.168.3.254   0.648 ms  0.567 ms  0.439 ms  2   192.168.13.2   3.838 ms  3.328 ms  2.989 ms  3   *192.168.4.1   4.561 ms