FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
AlexC-FTNT
Staff
Staff
Article Id 392554

Description

 

This article describes the usage, limitations, and requirements of the 'fnsysctl' command on FortiGate devices.

 

Scope

 

FortiGate.

 

Solution

 

The fnsysctl command is frequently useful for advanced troubleshooting on FortiGate. Although several forum threads reference individual options, there is no single article summarizing all relevant information. This article consolidates key details and usage notes.

 

Important facts about the fnsysctl command:

  • Login must be performed using a user account with the super_admin profile.

  • On FortiGate Virtual Machines, a regular (paid) license is required. Free evaluation VM instances will return the following error:

    Unknown action 0.

  • This is a CLI-only command and has no graphical equivalent in the GUI.

  • The command executes locally on the FortiGate device where the session is initiated. To run it on a passive member of a high availability cluster, log in directly to the passive unit.

  • Tab completion does not work with this command.

  • The command can be used within automation stitches by configuring set action-type cli-script.

 

fnsysctl ifconfig

 

Shows detailed info on the physical interfaces, including drops/errors/MTU. Accepts optionally the name of the interface, for example fnsysctl ifconfig port1.

 

FGT-Perimeter # fnsysctl ifconfig
port1   Link encap:Ethernet  HWaddr 0A:7C:2A:D2:17:6F
        inet addr:10.100.100.227  Bcast:10.100.100.255  Mask:255.255.255.0
        link-local6: fe80::87c:2aff:fed2:176f prefixlen 64
        UP BROADCAST RUNNING MULTICAST  MTU:9001  Metric:1
        RX packets:3537 errors:0 dropped:0 overruns:0 frame:0
        TX packets:5436 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:1340257 (1.3 MB)  TX bytes:4360502 (4.2 MB)

port2   Link encap:Ethernet  HWaddr 0A:C2:8D:76:4D:8D
        inet addr:10.100.104.13  Bcast:10.100.104.255  Mask:255.255.255.0
        link-local6: fe80::8c2:8dff:fe76:4d8d prefixlen 64
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:23 errors:0 dropped:0 overruns:0 frame:0
        TX packets:129 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000
        RX bytes:644 (644  Bytes)  TX bytes:5888 (5.8 KB)

 

fnsysctl ls

 

Lists files/folders in the filesystem. Useful for post-incident investigation of FortiGate compromises, looking for a given CVE indicators of compromise (IOCs).

It accepts only 3 flags:

  • a - Show all files, including those starting with the dot in their name.
  • l - Show long output, i.e., not only names but timestamps, sizes.
  • A - almost all, do not show names starting with the dot (default so no need to specify).

 

Examples:

 

FGT-Perimeter #  fnsysctl ls -al  /tmp
drwxr-xr-x    2 0   0   Wed Oct 23 01:53:42 2024    40 $$auto-script$$
drwxrwxrwt   60 0   0   Wed Oct 23 02:03:46 2024  4780 .
drwxr-xr-x   18 0   0   Wed Oct 23 01:53:40 2024     0 ..
srwxr-xr-x    1 0   0   Wed Oct 23 01:53:42 2024     0 .auto_script_server
-rw-r--r--    1 0   0   Wed Oct 23 01:53:42 2024     0 .aws_addrs
srwxr-xr-x    1 0   0   Wed Oct 23 01:53:42 2024     0 .cloudapi_fconv.sock
srwxr-xr-x    1 0   0   Wed Oct 23 01:53:42 2024     0 .dhcpd.msg
srwxr-xr-x    1 0   0   Wed Oct 23 01:53:42 2024     0 .dns_local_server
FGT-Perimeter #  fnsysctl ls -a  /tmp
$$auto-script$$               .
.dns_local_server             .dns_local_server_for_proxy
.dnsproxy_unix_server  0      .fgfm_stream_clt_sock
.ipsengine001_0_0.url.socket  .ipsengine002_0_0.url.socket
.urlfilter0.sock              .wad512_0_0.url.socket
admin_server.crt              KEY-FILE
backtrace_log                 bwl_gui_to_url0_unix_sock

 

fnsysctl cat

 

Show contents of a file, not all files in the filesystem are accessible. Some examples.

  • Show Linux kernel version of the FortiGate (here FortiOS v7.4.3):

 

FGT-Perimeter #  fnsysctl cat /proc/version
Linux version 4.19.13 (root@build) (gcc version 10.3.0 (GCC)) #1 SMP Thu Feb 1 17:10:43 UTC 2024

 

When trying to access a prohibited file:

 

FGT-Perimeter # fnsysctl cat /tmp/cw_ac_key_bak.pem
cat: /tmp/cw_ac_key_bak.pem: Not allowed

 

  • Show open TCP connections to/from FortiGate itself:

 

FGT-Perimeter #  fnsysctl cat /proc/net/tcp
  sl  local_address rem_address   st tx_queue rx_queue
  tr tm->when retrnsmt   uid  timeout inode
   0: 00000000:28A0 00000000:0000 0A 00000000:00000000
   00:00000000 00000000     0        0 13871 1 ffff8880443a9200
   100 0 0 10 0 0:0/0:0/0:0 0
   1: 00000000:1E82 00000000:0000 0A 00000000:00000000
   00:00000000 00000000     0        0 17550 1 ffff88804a0ece00
   100 0 0 10 0 0:0/0:0/0:0 0
   2: 00000000:2904 00000000:0000 0A 00000000:00000000 00:00000000
   00000000     0        0 13877 1 ffff888042db2200 100 0 0 10 0
   0:0/0:0/0:0 0

 

The output is in hex, so it is much easier to use diagnose sys tcpsock | grep 0.0.0.0.

  • Show CPU info:

 

FGT-Perimeter # fnsysctl cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 85
model name      : Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
stepping        : 7
microcode       : 0x5003707
cpu MHz         : 2499.998
cache size      : 36608 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic

 

  • Get memory information:

 

FGT-Perimeter # fnsysctl cat /proc/meminfo
MemTotal:        1984244 kB
MemFree:          595988 kB
MemAvailable:     757016 kB
Buffers:           10140 kB
Cached:           597428 kB
SwapCached:            0 kB
Active:           591168 kB
Inactive:         141344 kB
Active(anon):     518884 kB
Inactive(anon):    47496 kB
Active(file):      72284 kB
...cut...

 

  • Show nturbo acceleration statistics fnsysctl cat /proc/nturbo/<n>/drv:

 

fnsysctl cat /proc/nturbo/<0>/drv

Turbo interface ID: 0

============================================================================
Driver RX/TX:      760818543/759413272
TX hang:        No

Free/Used buffers:   109675/2965
Alloc fail:     0, Bad qid:     0
queue ready: 0x0000007f, 0x00000000

RXQ_0(0,20806): IN 64201109 OUT 64201142 DROP 0 NRDY 0 Fullness 0, Peak 282
TXQ_0(0,20806): IN 64083848 OUT 64083848 DROP 0 SHAPER_DROP 0 USR_DROP 117056 BUFERR 0

RXQ_1(1,20808): IN 62241175 OUT 62241191 DROP 0 NRDY 0 Fullness 0, Peak 444
TXQ_1(1,20808): IN 62092654 OUT 62092654 DROP 0 SHAPER_DROP 0 USR_DROP 148288 BUFERR 0

RXQ_2(2,20807): IN 63028145 OUT 63028179 DROP 0 NRDY 0 Fullness 0, Peak 247
TXQ_2(2,20807): IN 62856041 OUT 62856041 DROP 0 SHAPER_DROP 0 USR_DROP 171904 BUFERR 0

RXQ_3(3,20809): IN 61829939 OUT 61830044 DROP 0 NRDY 0 Fullness 0, Peak 254
TXQ_3(3,20809): IN 61684861 OUT 61684861 DROP 0 SHAPER_DROP 0 USR_DROP 144928 BUFERR 0

RXQ_4(4,20810): IN 64154116 OUT 64154184 DROP 0 NRDY 0 Fullness 0, Peak 1408
TXQ_4(4,20810): IN 64009332 OUT 64009332 DROP 0 SHAPER_DROP 0 USR_DROP 144608 BUFERR 0

RXQ_5(5,20804): IN 63186535 OUT 63186600 DROP 0 NRDY 0 Fullness 0, Peak 221
TXQ_5(5,20804): IN 63097904 OUT 63097904 DROP 0 SHAPER_DROP 0 USR_DROP 88448 BUFERR 0

RXQ_6(6,20805): IN 63168351 OUT 63168429 DROP 0 NRDY 0 Fullness 0, Peak 432
TXQ_6(6,20805): IN 62918126 OUT 62918126 DROP 0 SHAPER_DROP 0 USR_DROP 250048 BUFERR 0

 

fnsysctl date

 

Show date in the Linux format, ignoring any options.

 

FGT-Perimeter #  fnsysctl date
Wed Oct 23 02:11:03 PDT 2024

 

fnsysctl df

 

Show filesystem usage, useful when hard disks) are attached to the FortiGate.

 

FGT-Perimeter # fnsysctl df -h
Filesystem                 Size    Used  Available Use% Mounted on
none                       1.3G   81.6M       1.2G   6% /tmp
none                       1.3G    4.7M       1.3G   0% /dev/shm
none                       1.3G   70.0M       1.2G   5% /dev/cmdb
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /data
/dev/nvme0n1p2             1.6G  141.7M       1.4G   9% /data2
/dev/nvme1n1p1            29.4G   54.8M      27.8G   0% /var/log
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /new_root/zebos/fortidev/etc/localtime
none                       1.3G   70.0M       1.2G   5% /new_root/eap_proxy/dev/cmdb
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /new_root/eap_proxy/etc/cert/ca
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /new_root/eap_proxy/fortidev/etc/localtime
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /new_root/eap_proxy_worker/etc/cert/ca
/dev/nvme0n1p1           231.9M  129.2M      89.9M  59% /new_root/eap_proxy_worker/fortidev/etc/localtime

 

fnsysctl du

 

Shows directory usage, accepts the following options:

  • -d n - Limit depth to n levels deep.

  • -a - Show/count files as well, not only directories.

  • -s - Show only the summary usage of all directories/files.

  • -L - Follow all symlinks.

 

Examples:

 

FGT-Perimeter #  fnsysctl du -s
715312  .
FGT-Perimeter #  fnsysctl du -L
4       ./new_root/eap_proxy_worker/fortidev/etc
4       ./new_root/eap_proxy_worker/fortidev
1256    ./new_root/eap_proxy_worker/etc/cert/ca
1256    ./new_root/eap_proxy_worker/etc/cert
1256    ./new_root/eap_proxy_worker/etc
0       ./new_root/eap_proxy_worker/dev/pts

...cut...

0       ./dev/shm/ips001
0       ./dev/shm/ips002
0       ./dev/shm/ips
3280    ./dev/shm
3280    ./dev
85811852        .
FGT-Perimeter #  fnsysctl du -d 1 -a
71960   ./new_root
20488   ./migadmin
5344    ./node-scripts
113596  ./bin
0       ./proc
0       ./fortidev
131464  ./data
142520  ./data2
0       ./boot
24      ./sbin
0       ./lib64
147440  ./tmp
11324   ./var
0       ./init
452     ./usr
0       ./etc
0       ./sys
67432   ./lib
0       ./root
3280    ./dev
715324  .

 

fnsysctl pwd

 

Show current working directory. Not very useful as cd is not accessible and the directory cannot be changed.

 

FGT-Perimeter # fnsysctl pwd
/

 

fnsysctl ps

 

List running processes. Useful together with the next command kill for restarting some stuck process on FortiGate. Most of the processes in FortiGate are run via Watch Dog which means killing them will shut the running process and will restart it immediately later.

 

FGT-Perimeter # fnsysctl ps
PID       UID     GID     STATE   CMD
1         0       0       S       /bin/initXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
2         0       0       S       [kthreadd]
3         0       0       I       [rcu_gp]
4         0       0       I       [rcu_par_gp]
6         0       0       I       [kworker/0:0H-kblockd]
8         0       0       I       [mm_percpu_wq]
9         0       0       S       [ksoftirqd/0]
10        0       0       I       [rcu_sched]
11        0       0       I       [rcu_bh]
12        0       0       S       [migration/0]
13        0       0       I       [kworker/0:1-events_power_efficient]
14        0       0       S       [cpuhp/0]
15        0       0       S       [cpuhp/1]
16        0       0       S       [migration/1]
17        0       0       S       [ksoftirqd/1]
19        0       0       I       [kworker/1:0H-kblockd]
20        0       0       S       [kdevtmpfs]
32        0       0       I       [kworker/1:1-events]
37        0       0       I       [kworker/1:2-mm_percpu_wq]
217       0       0       I       [kworker/u4:2-fortilink]
345       0       0       S       [khungtaskd]
346       0       0       S       [oom_reaper]

...cut...

2019      0       0       S       /bin/autod
2020      0       0       S       /bin/cloudapid
2021      65530   65530   S       /bin/eap_proxy
2026      0       0       S       /bin/dnsproxy
2045      0       0       S       /bin/wad 4
2046      0       0       S       /bin/wad 5
2047      0       0       S       /bin/wad 6
2048      0       0       S       /bin/wad 12
2049      0       0       S       /bin/wad 13
2050      0       0       S       /bin/wad 14
2051      0       0       S       /bin/wad 9
2052      0       0       S       /bin/wad 18 0
2053      0       0       S       /bin/miglogd 1
2095      0       0       S       /bin/ipsengine
2096      0       0       S       /bin/ipsengine
2119      0       0       S       /bin/urlfilter 0
2123      65531   65531   S       /bin/imi -L 2
2124      0       0       R       /bin/sshd
2125      0       0       S       /bin/newcli
2204      0       0       I       [kworker/u4:1-events_unbound]
2319      0       0       I       [kworker/u4:0-events_unbound]
2325      0       0       S       /bin/httpsd

 

fnsysctl kill

 

Kill a process by its ID (PID). The only option accepted is -s N, where N is the signal number to send as per Linux. Using the output of the fnsysctl ps above, httpsd (Admin GUI process) can be killed as follows:

 

fnsysctl kill 2325

 

There are usually multiple processes for the same function, so it is more practical to use the next command instead - fnsysctl killall.

 

fnsysctl killall

 

Kill/restart a process by name. The only option is the name of the process. The example above for killing all httpsd processes will be:

 

FGT-Perimeter # fnsysctl killall httpsd

 

  • When using killall it is not recorded in the crash log file (read with diagnose debug crashlog read).

  • Not all processes can be killed with it, for example, hasync.

 

fnsysctl mv

 

Move file in the filesystem. Most of the directories on the FortiGate are read-only, but some, like tmp are not. This command will ask for the username/password explicitly.

 

FGT-Perimeter # fnsysctl mv  /tmp/ipsshm.urldb-whitelist /tmp/ipsshm.urldb-whitelist.orig
Admin:admin
Password:
FGT-Perimeter # fnsysctl ls -al  /tmp/ipsshm.urldb-whitelist.orig
-rw-r--r--    1 0        0       Wed Oct 23 02:15:02 2024           810912 /tmp/ipsshm.urldb-whitelist.orig
 
 
Warning: With file moves as FortiGate may stop functioning if a crucial file is deleted.
 

The obvious use for this command is for attackers who have broken into FortiGate to hide their traces.

 

fnsysctl printenv

 

The only environment variable I was able to catch with this was the type of Terminal used.

 

FGT-Perimeter # fnsysctl printenv
TERM=vt220

 

fnsysctl grep

 

Search the contents of a file/files. The usual grep options are available:

 

        -i      Ignore case distinctions
        -l      List names of files that match
        -H      Prefix output lines with filename where match was found
        -h      Suppress the prefixing filename on output
        -n      Print line number with output lines
        -q      Quiet
        -v      Select non-matching lines
        -s      Suppress file open/read error messages
        -c      Only print count of matching lines
        -A      Print NUM lines of trailing context
        -B      Print NUM lines of leading context
        -C      Print NUM lines of output context