FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
cskuan
Staff
Staff

Description
This article esxplains the reason why interface status show as ‘down’ on all FPMs but show as ‘up’ on FIMs when the interface is connected.

Solution

Symptoms.
 

1) Interface shows up (green) on the Web Management GUI.

 
 
 
2) From debug commands ‘diagnose hardware deviceinfo nic’ on that interface shown show as ‘down’ on all FPMs but shown as ‘up’ on FIMs.
# diagnose hardware deviceinfo nic 2-C1
==========================================================================
Slot: 2 Module SN: FIM20Exxxxxxxx < --------
Description FGT-7000E Ethernet Driver
Driver Name FGT-7000E Ethernet Driver
System_Device_Name 2-C1
Current_HWaddr 70:4c:a5:xx:xx:xx
Permanent_HWaddr 70:4c:a5:xx:xx:xx
State up  < --------
Link up < --------
PHY Link up < --------
==========================================================================
Slot: 3 Module SN: FPM20E xxxxxxxx < --------
Description :FortiASIC NP6 Adapter
Driver Name :FortiASIC Unified NPU Driver
Name :np6_2

..........

========== Link Status ==========
Admin :up
netdev status :down < --------
autonego_setting:1
link_setting :0
link_speed :40000
link_duplex :1

==========================================================================
Slot: 4 Module SN: FPM20E3Exxxxxxxx < --------
Description :FortiASIC NP6 Adapter
Driver Name :FortiASIC Unified NPU Driver
Name :np6_2

............
========== Link Status ==========
Admin :up
netdev status :down
< --------
autonego_setting:1
link_setting :0

............
==========================================================================
Current slot: 1 Module SN: FIM01E xxxxxxxx < --------
Description FGT-7000E Ethernet Driver
Driver Name FGT-7000E Ethernet Driver
System_Device_Name 2-C1
Current_HWaddr 70:4c:a5:xx:xx:xx
Permanent_HWaddr 22:4c:a5:xx:xx:xx
State up < --------
Link up < --------
PHY Link up < --------

Explanation.

If that interface is part of the members of an Aggregate / LACP link.
If that interface failed to form the LACP.
It will show down on all FPMs.

Observed that interface 2-C1 has yet to form the LACP and still in negotiating state.
Besides that, on it shows 'down' in FPMs.
 
# diag netlink aggregate name LAG1

==========================================================================
Current slot: 1 Module SN: FIM01Exxxxxxxx < --------
LACP flags: (A|P)(S|F)(A|I)(I|O)(E|D)(E|D)
...........
status: up
…………

slave: 2-C1 < --------
  index: 0
  link status: up < --------
  link failure count: 7
  backup: 1, inactive: 1
  user_inactive: 0
  permanent MAC addr: 70:4c:a5:xx:xx:xx
  LACP state: negotiating < --------
  actor state: ASAODD
  actor port number/key/priority: 1 35 255
  partner state: ASIODD
  partner port number/key/priority: 1 1 255
  partner system: 0 00:00:00:00:00:00 < --------
  aggregator ID: 1
  speed/duplex: 40000 1
  RX state: DEFAULTED 5
  MUX state: WAITING 2 < --------

slave: 2-C2 < --------
  index: 1
  link status: up < --------
  link failure count: 4
  backup: 0, inactive: 0
  user_inactive: 0
  permanent MAC addr: 70:4c:a5:xx:xx:xx
  LACP state: established < --------
  actor state: ASAIEE
  actor port number/key/priority: 2 35 255
  partner state: ASAIEE
  partner port number/key/priority: 49 11681 32768
  partner system: 0 ac:75:1d::xx:xx:xx < --------
  aggregator ID: 2
  speed/duplex: 40000 1
  RX state: CURRENT 6
  MUX state: COLLECTING_DISTRIBUTING 4 < --------

==========================================================================
Slot: 3 Module SN: FPM20E3E17900470
status: up
……….
slave: 2-C1 < --------
  index: 0
  link status: down < --------
  link failure count: 2
  backup: 1, inactive: 1
  user_inactive: 0
  oid_vid: 129
  permanent MAC addr: 70:4c:a5:xx:xx:xx

slave: 2-C2 < --------
  index: 1
  link status: up < --------
  link failure count: 5
  backup: 0, inactive: 0
  user_inactive: 0
  oid_vid: 139
  permanent MAC addr: 70:4c:a5:xx:xx:xx

==========================================================================
Slot: 4 Module SN: FPM20E3E17900501
status: up
……….
slave: 2-C1 < --------
  index: 0
  link status: down < --------
  link failure count: 2
  backup: 1, inactive: 1
  user_inactive: 0
  oid_vid: 129
  permanent MAC addr: 70:4c:a5:xx:xx:xx

slave: 2-C2 < --------
  index: 1
  link status: up < --------
  link failure count: 5
  backup: 0, inactive: 0
  user_inactive: 0
  oid_vid: 139
  permanent MAC addr: 70:4c:a5:xx:xx:xx
Solution.

- Fix and form the LACP for the interface and the interface will be shown as ‘up’ on all FPMs and FIMs.
# diagnose netlink aggregate name LAG1
==========================================================================
Current slot: 1 Module SN: FIM01Exxxxxxxx
LACP flags: (A|P)(S|F)(A|I)(I|O)(E|D)(E|D)
.........
status: up < --------
.......
slave: 2-C1 < --------
  index: 0
  link status: up
  link failure count: 12
  backup: 0, inactive: 0
  user_inactive: 0
  permanent MAC addr: 70:4c:a5:xx:xx:xx
  LACP state: established < --------
  actor state: ASAIEE
  actor port number/key/priority: 1 35 255
  partner state: ASAIEE
  partner port number/key/priority: 49 11681 32768
  partner system: 0 ac:75:1d:xx:xx:xx < --------
  aggregator ID: 1
  speed/duplex: 40000 1
  RX state: CURRENT 6
  MUX state: COLLECTING_DISTRIBUTING 4

- Interface 2-C1 show up on all slots (both FIMs and FPMs) after bring up the LACP.
FW-ITDC-BSD-2 [FIM01] (global) # diagnose hardware deviceinfo nic 2-C1
==========================================================================
Current slot: 1 Module SN: FIM01E xxxxxxxx < --------
Description FGT-7000E Ethernet Driver
Driver Name FGT-7000E Ethernet Driver
System_Device_Name 2-C1
Current_HWaddr 70:4c:a5:xx:xx:xx
Permanent_HWaddr 70:4c:a5:xx:xx:xx
State up < --------
Link up < --------
PHY Link up < --------

.....................

==========================================================================
Slot: 3 Module SN: FPM20Exxxxxx
Description :FortiASIC NP6 Adapter
Driver Name :FortiASIC Unified NPU Driver
Name :np6_2
………….
========== Link Status ==========
Admin :up
netdev status :up < --------
autonego_setting:1

.................

==========================================================================
Slot: 4 Module SN: FPM20E3E17900515
Description :FortiASIC NP6 Adapter
Driver Name :FortiASIC Unified NPU Driver
Name :np6_2
.............
========== Link Status ==========
Admin :up
netdev status :up < --------
autonego_setting:1