Troubleshooting Tip: Connecting an ISP Router to a Managed FortiSwitch and terminating the Connection on FortiGate

[riteshpv]

Description	This article describes how multiple ISP connections terminate on multiple interfaces on FortiGate.
Scope	FortiGate v7.4, FortiSwitch v7.4
Solution	The requirement is to set up multiple ISP connections terminating on FortiGate over an L2 FortiLink setup. Topology: Network Setup: SW1, SW2, and SW3 are FortiSwitch devices. SW1 and SW2 are configured in an MCLAG-ICL pair and managed by FortiGate. Aggregate interface name: EXT (FortiGate ports a, b). FortiLink interface name: fortilink (FortiGate ports x1, x2). ISP Router Connections: ISP Router 1 is connected to SW3 with VLAN 30 tagged and terminated at VLAN interface VLAN30, created under the FortiLink interface. ISP Router 2 is connected to the MCLAG-ICL pair on port3, carrying different VLANs from the ISP: The ISP Router 2 to SW1 link carries VLAN 10 and terminates at VLAN interface VLAN10, created under the FortiLink interface. The ISP Router 2 to SW2 link carries VLAN 20 (PPPoE) and terminates at VLAN interface VLAN20, created under the external aggregate interface EXT. Since the FortiSwitch devices are managed by FortiGate, the configuration is pushed from FortiGate. Configuration on FortiGate: Configuring VLAN: FortiSwitch VLAN Configuration Guide Configuring Trunk: FortiSwitch Trunk Configuration Guide  Configuring Ports: FortiSwitch Port Configuration Guide VLAN 10 Configuration under FortiLink: config system interface     edit "VLAN10"         set vdom "root"         set mode dhcp         set allowaccess ping https ssh http         set role wan         set interface "fortilink"         set vlanid 10     next end Similarly, create VLAN 30 with vlanid 30. Create dummy vlan 20 under fortilink interface. config system interface     edit "VLAN20"         set vdom "root"         set interface "fortilink"         set vlanid 20     next end end VLAN 20 Configuration (PPPoE) under External Aggregate Interface (name EXT): config system interface     edit "EXT"         set vdom "root"         set type aggregate         set member "a" "b"         set device-identification enable         set lldp-transmission enable         set lacp-mode active     next     edit "VLAN20"         set vdom "root"         set mode pppoe         set allowaccess ping https ssh http         set role wan         set username "test@in"         set password <PASSWORD>         set interface "EXT"         set vlanid 20     next end FortiSwitch Configurations Pushed from FortiGate: SW1 Configuration: config switch-controller managed-switch     edit <SW1-SERAIL-NO>         config ports             edit port3                 set native-vlan VLAN10                 set dhcp-snooping trusted             next             edit "EXT-trunk"                 set allowed-vlans "VLAN20"                 set dhcp-snooping trusted                 set type trunk                 set mode lacp-active                 set mclag enable                 set members "port2"             next         end SW2 Configuration: config switch-controller managed-switch     edit <SW2-SERAIL-NO>         config ports             edit port3                 set native-vlan VLAN20                 set dhcp-snooping trusted             next             edit "EXT-trunk"                 set allowed-vlans "VLAN20"                 set dhcp-snooping trusted                 set type trunk                 set mode lacp-active                 set mclag enable                 set members "port2"             next         end SW3 Configuration: config switch-controller managed-switch     edit <SW3-SERAIL-NO>         config ports             edit port3                 set allowed-vlans "VLAN30"                 set dhcp-snooping trusted             next         end Note:  Since this is a FortiLink setup, trunks are automatically formed between FortiGate and FortiSwitch devices, as well as between FortiSwitch devices. VLANs are dynamically assigned on FortiSwitch FortiLink trunk ports. Troubleshooting: If ISP VLANs are not getting an IP from the ISP, use the following commands to diagnose the issue. On FortiSwitch (Example: SW3): Verify that port3 is up: diagnose switch physical-ports summary port3 Check if the ISP MAC address is learned on port3: diagnose switch mac-address list | grep port3 Verify data rate (RX Rate)  incrementing on port3: diagnose switch physical-port datarate port3 Capture and traffic received on FortiSwitch port connecting to ISP: port-mirror  Logs to Provide to TAC Before Opening a Case: Network Topology. FortiGate Logs: get system status execute switch-controller get-conn-status execute switch-controller get-sync-status all execute dhcp lease-list fnsysctl ifconfig FortiGate Backup Configuration. FortiSwitch Logs from the ISP-connected switch: show full-configuration diagnose debug crashlog read diag debug report