|
The first step to troubleshoot CPU issues is always to determine the CPU module of usage. This issue is regarding the softirq module.
FGT-01 # get sys performance status
CPU states: 6% user 6% system 0% nice 65% idle 0% iowait 0% irq 23% softirq
CPU0 states: 3% user 2% system 0% nice 3% idle 0% iowait 0% irq 92% softirq
CPU1 states: 10% user 12% system 0% nice 78% idle 0% iowait 0% irq 0% softirq
CPU2 states: 4% user 6% system 0% nice 90% idle 0% iowait 0% irq 0% softirq
CPU3 states: 6% user 6% system 0% nice 88% idle 0% iowait 0% irq 0% softirq
Memory: 12332116k total, 2857160k used (23.2%), 9016844k free (73.1%), 458112k freeable (3.7%)
Average network usage: 428015 / 433566 kbps in 1 minute, 447142 / 452540 kbps in 10 minutes, 415709 / 421098 kbps in 30 minutes
Average sessions: 66776 sessions in 1 minute, 66971 sessions in 10 minutes, 68353 sessions in 30 minutes
Average session setup rate: 512 sessions per second in last 1 minute, 515 sessions per second in last 10 minutes, 524 sessions per second in last 30 minutes
When performing a Troubleshooting Tip: FortiGate CPU Profiling, several commands can be seen starting with the name e1000. These references a specific network driver. While this specific driver does work, it causes extra load on the FortiGate and should be avoided.
FGT-01 # diagnose sys profile show order
0xffffffff80392bcc: 254 __memcpy+0x6c/0x120
0xffffffff805e9918: 225 e1000_xmit_frame+0xa26/0xf24
0xffffffff805e8c14: 142 e1000_clean+0x31f/0x5fd
0xffffffff805ea03c: 99 e1000_clean_rx_irq+0x188/0x481
0xffffffffa0006c68: 50 ip_tuple_find_get+0xaf/0x18d
0xffffffff805e799c: 45 e1000_alloc_rx_buffers+0x22b/0x4b6
To confirm what drivers are used on the FortiGate, check any non-logical interface with get hardware nic <interface> or run diagnose hardware lspci -v which is also part of the Technical Tip: Download Debug Logs and 'execute tac report'.
get hardware nic port1
Name: port1
Driver: e1000
diagnose hardware lspci -v
02:00.0 Class 0200: Device 8086:100f (rev 01)
Subsystem: Device 15ad:0750
Flags: bus master, 66MHz, medium devsel, latency 0, IRQ 18
Memory at fd540000 (64-bit, non-prefetchable) [size=128K]
Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
I/O ports at 2000 [size=64]
[virtual] Expansion ROM at ebb00000 [disabled] [size=64K]
Capabilities: [dc] Power Management version 2
Capabilities: [e4] PCI-X non-bridge device
Kernel driver in use: e1000
Solution:
Instead of the e1000 driver the use the VMXNET 3 driver on the VMWare hypervisor.
Related document:
Choosing a network adapter for a virtual machine
|
