| 'IRQ' value can be seen as high sometimes in the 'get system performance status' result. A higher 'IRQ' value indicates an issue with the console output getting printed in the background or if collecting any debug, the chances are that the command output is getting printed in multiple consoles at the same time. get sys perf status
CPU states: 0% user 1% system 0% nice 87% idle 0% iowait 12% irq 0% softirq
CPU0 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq
CPU1 states: 0% user 1% system 0% nice 99% idle 0% iowait 0% irq 0% softirq
CPU2 states: 0% user 3% system 0% nice 97% idle 0% iowait 0% irq 0% softirq
CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU4 states: 0% user 1% system 0% nice 96% idle 0% iowait 0% irq 3% softirq
CPU5 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU6 states: 0% user 2% system 0% nice 98% idle 0% iowait 0% irq 0% softirq
CPU7 states: 0% user 0% system 0% nice 0% idle 0% iowait 100% irq 0% softirq In the 'get system performance status' command output, IRQ stands for Interrupt Requests. IRQ values in the output indicate the percentage of time the CPU is spending on handling hardware interrupts. These interrupts are signals sent by hardware devices to the CPU to request attention. Here are the key points regarding IRQ in the 'get system performance status' output: - IRQ allows devices to notify the kernel that they need attention.
- Hardware Interrupts directly affect the CPU.
- Software Interrupts can only be generated by the process currently running.
- Devices with high bandwidth may distribute interrupts to multiple cores using multiple IRQs (NP6 or NP4). To view interrupts for hardware, use the 'diagnose hard sys interrupts' command. For software interrupts, use the 'fnsysctl cat /proc/softirqs' command.
An abnormal increase in this part means there could be a loop in the network or the device is getting too many duplicate packets (mostly too many ARP packets). To check it, verify the traffic pattern on the interfaces that are connected to the L2 network: - Collect 'diagnose sys mpstat' to see the pattern of the 'irq' value.
- Collect 'diagnose debug info' to confirm if there are other debugs running in the background.
exe time
diagnose deb info
exe time
get sys perf
exe time
Example: graviton-kvm63 (global) # diagnose debug info
debug output: disable
console timestamp: disable
console no user log message: disable
zebos debug level: 306783954 (0x124926d2)
CLI debug level: 3
WAD console log: enable ===== IPS debug settings =====
IPS Engine <00235> Enabled debug actions: ===== IPS SSL debug settings =====
IPS Engine <00235> ssl debug level is "none" ===== IP router debug settings =====
BGP debugging status:
BGP debug level: ERROR
timestamp disabled ;; ;; RIPng debugging status:
RIPng debugging level is CRITICAL
timestamp disabled The current debug duration is 30 minutes; the timer has not started yet. - Use the command below to disable debug output to the console.
FGT-A # config global
FGT-A (global) # diagnose debug console no-user-log-msg enable
FGT-A (global) # end Logs to be collected. Putty Session 1: get sys status
get sys performance status
diagnose hardware sysinfo memory
diagnose sys vd stats
diagnose sys top-mem 20
diagnose sys top-fd 50
diagnose sys top 1 99 5
diagnose sys mpstat 2 5
fnsysctl ps aux
diagnose sys process dump <PID><----- Replace the PID with the highest IPS Engine using a high CPU.
diagnose sys process pstack <PID> <----- Replace the PID with the highest IPS Engine using a high CPU. diagnose sys process trace < PID > <----- Replace the PID with the highest IPS Engine using a high CPU. fnsysctl cat /proc/<< pid >>/status <----- Replace the PID with the highest IPS Engine using a high CPU.
diagnose ips session status diagnose sys profile report
diagnose debug report Putty Session 2: get sys stat
get sys perf stat
diagnose sys top 5 20 99999 Putty Session 3: If a specific CPU core is only going high, then CPU profiling can be done. diagnose sys profile cpumask <0>
diagnose sys profile start
- wait a few minutes diagnose sys profile stop
diagnose sys profile show
diagnose sys profile show order
diagnose sys profile show detail
diagnose sys profile sysmap | 
