Technical Tip: Short list of processes on the FortiGate

Description

This article describes the various processes running on the FortiGate, including an explanation of the processes and how to list the running processes on the FortiGate.

Scope

FortiGate.

Solution

Listing running processes on the FortiGate:

To list the processes that are running in memory, run the following command:

diagnose sys top <refresh interval> <max number of lines> <iterations>

The above command can be run as-is ('diagnose sys top'), or it can be run with additional parameters to adjust the refresh rate of the data (default is 5 seconds), how many lines are displayed (default is 20), and the number of iterations that should be run (default is unlimited). For example, the following version of the command displays up to 200 processes every 10 seconds for 3 iterations:

diagnose sys top 10 200 3

Additionally, the output of the top command can be sorted in certain ways:

• Hit the 'p' key to sort processes by CPU usage.

• Hit the 'm' key to sort by memory usage.

• Hit the 'n' key to sort by process ID value (very useful when gathering a sorted list of all processes running on the FortiGate).

Finally, the column output of top can be interpreted as follows:

Normal States: S, R, and D (for a short time).

Abnormal States: Z and D (if not for a short time).

List of Processes/Daemons on the FortiGate and their responsibilities:

A to D.

acd: Aggregate Controller process; handles LACP link-aggregation on the FortiGate.

alertmail: Alert mail daemon; handles email alerts sent out by the FortiGate for functions such as Automation Stitches. Can be debugged to show the process by which the FortiGate sends outgoing emails (including SMTP commands and server return codes).

authd: General authentication daemon; notably handles managing user authentication, including Fortinet Single Sign-On (FSSO. It helps in monitoring and managing user sessions, ensuring that authentication states are maintained correctly.

autod: Automation daemon; handles Automation Stitch and auto-script functionality and debugging.

bcm.user: Userspace process that communicates and manages the internal hardware switch (for FortiGates with Integrated Switch Fabrics based on the BCM SDK). Do not kill this process manually, as it will cause an outage for FortiGate interfaces connected to the internal ISF until a system reboot is conducted.

bgpd: Handles the Border Gateway Protocol (BGP) dynamic routing protocol; part of the ZebOS Routing Daemons. Notably, bgpd handles IPv4 and IPv6 within a single daemon (as opposed to RIP and OSPF having separate IPv4 and IPv6 daemons).

cfmd: Connectivity Fault Management daemon; useful for diagnosing/resolving issues within Ethernet networks (when configured). Introduced in FortiOS v7.4.1 as part of change #765007. See also: Connectivity Fault Management.

cid: Client Identification daemon. Handles Device Identification/Detection functionality on the FortiGate in FortiOS v7.2.0 and later (i.e., creating MAC address entries and snooping traffic to identify a given device on the network). Before v7.2.0, this daemon was named src-vis (it has since been renamed/updated as of change #740250).

clearpass: ClearPass daemon. Handles REST API integration with ClearPass Policy Manager to allow for dynamic address objects. See also: ClearPass integration for dynamic address objects.

cloudapid: Cloud service communication daemon. Added in FortiOS v7.4.0 (as per change #754765) to provide support for integrating with cloud APIs in general (e.g. standardized polling of cloud services from the FortiGate).
Notably, its first function was to support integration with Fortinet's FortiConverter service (e.g., allowing FortiConverter ticket creation and pulling/pushing of converted config files from source FortiGate to destination FortiGate).

cmdbsvr: Handles the internal database used to store the running state and configuration of the FortiGate. Also handles the saving of configuration changes to the filesystem/persistent boot storage.
It is important to keep in mind that cmdbsvr has child processes, which can be seen when the parent is at work:

• cmdbsvr_iprope.

• cmdbsvr_prodcsu.

• cmdbsvr_writeco.

• cmdbsvr_cfgsave.

• cmdbsvr_checksu.

csfd: Cooperative Security Fabric daemon. Handles Security Fabric functionality on the FortiGate.
It was previously named nstd and was renamed/rebranded as of FortiOS v5.4.1 (as per change #308676).

cw_acd / cw_acd_helper / cu_acd: Handles management of FortiAP/FortiSwitch via CAPWAP traffic.

dhcpd / dhcpcd / dhcprd / dhcpsd: Daemons for handling DHCP functions on the FortiGate (c = client, s = server, r = relay).

dnsproxy: DNS helper process that performs several tasks, including handling DNS caching on the FortiGate, DNS Filtering and inspection, DNS server functionality, etc.

In FortiOS v6.4 and earlier, dnsproxy was always handling DNS traffic in a proxy-like fashion (even when performing flow-based inspection). In FortiOS v7.0 and later, DNS Filter duty is now handled by the IPS Engine. See also: DNS filter handled by IPS engine in flow mode.

E to H.

eap_proxy: EAP Proxy daemon. Handles WiFi authentication for EAPoL protocols, particularly when the FortiGate is performing Local Authentication for WPA2-/WPA3-Enterprise.

fas: Handles FortiToken Cloud API communication (FTC, originally named FAS, hence the name of the daemon).

fcnacd: FortiClient NAC daemon; handles communication between FortiGate and FortiClient EMS.

fgfmd: FortiGate-FortiManager daemon; handles the establishment of fgfm secure management tunnels to FortiManager/FortiGate Cloud. 

fgtlogd: Handles FortiAnalyzer and FortiGate-Cloud-based logging functionality on the FortiGate.

Previously, FortiAnalyzer and FortiGate-Cloud logging functionality was handled within miglogd. As of FortiOS v7.2.0 and later, fgtlogd is a new daemon that subscribes to miglogd's log publishing functionality and handles logging functionality to Fortinet-proprietary logging destinations (as per change #687202).

fltund: Refers to FortiLink tunnel daemon and is responsible for managing the FortiLink tunnel operations within the FortiGate system. This daemon is part of the FortiLink feature, which is used to connect and manage FortiSwitch devices through a FortiGate.

fnbamd: Handles remote user authentication (LDAP, RADIUS, FortiToken, etc.) as well as x.509 certificate verification. Designed to handle multiple authentication requests simultaneously without blocking other processes.

foauthd: Handles authentication when Web Filter Override is configured.

forticldd: FortiCloud daemon; handles communication between the FortiGate and FortiCloud services (such as FortiGate Cloud).

forticron: Performs internal job scheduling, similar to Linux crontab.

fortilinkd: Handles management controller functionality for FortiSwitches connected to the FortiGate via FortiLink.

fsso_ldap: Handles the task of polling LDAP servers to obtain groups for FSSO usage on the FortiGate.

fsvrd: FortiService daemon. Listens to and forwards connection requests to fcnacd and nstd (now called csfd). Introduced in FortiOS v5.4.1 (as per change #188261).

ftm2: FortiToken daemon. Handles FortiToken Mobile authentication on the FortiGate, including processing token push notifications.

getty: Linux daemon that handles serial console logins on the FortiGate.

hasync: Performs object/session/configuration synchronization between members of a FortiGate High-Availability (HA) cluster. Can be restarted safely without disrupting traffic.

hatalk: Handles heartbeat/cluster communication between members of a FortiGate HA cluster. It is not recommended to restart this process or it can lead to cluster instability and/or split-brain behavior.

httpclid: Handles the Web GUI-based CLI console for FortiOS v7.0 and later.

httpsd: Provides HTTPS and REST API services on the FortiGate, including the administrative web GUI.

http_authd: Handles administrator web authentication for FortiOS v7.6.4 and later. See also Technical Tip: In FortiOS v7.6.4 and later, administrator web authentication is handled by the http_authd process.

I to L.

iked: Handles the negotiation and operation of all IPsec VPN tunnels on the FortiGate.

ikecryptd: New multi-process daemon in FortiOS v7.0 and later (added as part of Change #675373). Supports the IKED daemon by offloading IPsec Diffie-Hellman calculations to multiple worker processes (allowing for the usage of multiple CPUs and CPx co-processors, which improves performance).

• ikecryptd spawns a main manager process along with multiple child worker processes. These can be seen in the output of diagnose sys top-fd 100 | grep ikecryptd, where the child processes will be named 'ikecryptd_dhwX'.

• Debug information for this process can be printed using diagnose vpn ikecrypt info.

• This process can also be further configured under the config system ike in the CLI. Refer to the following for more information: config system ike.

• The process may be disabled by default when upgrading from FortiOS v6.4 and earlier to FortiOS v7.0+. The option 'set dh-multiprocess' will enable/disable the usage of ikecryptd.

imi: Management daemon for centralized configuration/monitoring of ZebOS daemons; part of the ZebOS Routing Daemons.

init (aka initXXXXXXXXXXX): Linux init process. The first process to start that initializes the system and all other processes.

insmod: Linux utility for inserting modules into the Linux Kernel.

iotd: Internet of Things (IoT) Detection daemon. Works with the CID daemon to send queries to FortiGuard servers regarding detected MAC addresses and receive back confidence scores and device info corresponding to the MAC address.
Introduced in FortiOS v6.4.0 (as per change #607855) and ties into the IoT Detection subscription service available on FortiGates (see also: IoT detection service).

ipmc_sensord: Daemon responsible for monitoring onboard hardware sensors (e.g., temperature, PSU, fan speed, etc.).

ipsengine: Performs the actual security inspection of traffic in flow-based Firewall Policies using pattern-matching and signature-/heuristic-matching.

• One of the ipsengine workers will have a 'master:1' flag signifying it as the master engine. The master ipsengine handles additional tasks, such as cleaning up SSL cache entries and updating signature databases on the Content Processor.

• For more information on the IPS Engine series of processes (ipsengine, ipshelper, ipsmonitor), refer to the following article: Technical Tip: Introduction of IPS process.

ipshelper: Handles configuration management tasks for IPS Engine as a whole, including monitoring CMDB config changes related to IPS and pushing them to the ipsengine workers.

• Also handles the compilation of the IPS rule database and generates DFA entries for upload to the Content Processor (CP8/CP9, if present). Note that this behavior only occurs if IPS Acceleration (IPSA) is in use (see also: IPSA offloads flow-based pattern matching)

ipsmonitor: Oversees ipsengine workers and serves as a watchdog, starting/stopping/restarting engines as required.

Restarting ipsmonitor with 'diagnose test application ipsmonitor 99' gracefully restarts all ipsengine processes with no disruption to flow-based traffic sessions.

isisd: Handles the Intermediate System-to-Intermediate System (IS-IS) dynamic routing protocol; part of the ZebOS Routing Daemons.

l2tpd: Handles legacy L2TP dial-up VPN functionality on the FortiGate. Frequently pairs with the IKE process for L2TP over IPsec remote-access VPN tunnels (see also: L2TP over IPsec).

lldprx: Handles receiving (rx) of Link-Layer Discovery Protocol traffic on the FortiGate.

lldptx: Handles transmitting (tx) of Link-Layer Discovery Protocol traffic on the FortiGate.

First introduced in FortiOS v5.2 (as per change #224654). Before this, FortiOS could only receive LLDP frames and not transmit/advertise them.

locallogd: Handles local memory- and disk-based logging functionality on the FortiGate. Previously, local logging functionality was handled within miglogd. As of FortiOS v7.2.4 and later, locallogd is a new daemon that subscribes to miglogd's log publishing functionality and handles local logging functionality (as per changes #687201 and #892541).

M to P.

merged_daemons: Legacy process that handles ipsufd (IPS URL Filter Daemon) as of Change #458157.

miglogd: Handles event logging on the FortiGate.

kmiglogd: the Kernel Log Daemon. In FortiOS v6.4.2 and later, miglogd's complex functionality was divided among multiple separate daemons. The main miglogd process now solely handles the building and publishing of log messages, and other daemons subscribe to miglogd to utilize these published logs (as per change #584449).

mvl.user: The process is the Marvell switch daemon. It manages switch and interface configuration, monitors link status, and collects interface accounting data.

newcli: Manages the creation or termination of CLI connections (management/telnet/GUI).

node: Handles several tasks related to the FortiOS Web GUI, including report management, WebSockets, Web CLI in the GUI, and proxying traffic to/from the administrative web GUI. For more information on the function of the node daemon and some causes for high memory usage by the process, see Technical Tip: High memory usage of node process.

nsm: Network Services Module; component of the ZebOS routing software that manages the routing table by adding/removing routes.

nstd: Network Segmentation Tree daemon. Has since been renamed to csfd (Cooperative Security Fabric daemon) as of FortiOS v5.4.1 (as per change #308676).

ntpd: Handles Network Time Protocol (NTP) functionality on the FortiGate (i.e., acting as an NTP client to external NTP servers, as well as listening as a local NTP server, if configured).

ospfd: Handles the IPv4 Open Shortest Path First (OSPF) dynamic routing protocol; part of the ZebOS Routing Daemons.

ospf6d: Handles the IPv6 Open Shortest Path First v3 (OSPF6) dynamic routing protocol; part of the ZebOS Routing Daemons.

pdmd: Handles the Protocol-Independent Multicast Dense-Mode (PIM-DM) multicast routing protocol; part of the ZebOS Routing Daemons.

pimd: Handles the IPv4 Protocol-Independent Multicast Sparse-Mode (PIM-SM) multicast routing protocol; part of the ZebOS Routing Daemons.

pim6d: Handles the Protocol-Independent Multicast for IPv6 (PIMv6-SM) multicast routing protocol; part of the ZebOS Routing Daemons.

pyfcgid: Python Configuration Daemon; provided Web GUI services alongside httpsd.

• Was removed from FortiOS as of versions 6.2.4 and 6.4.0 (see Resolved Issue #599284).

• See also: Technical Tip: pyfcgid (python config daemon) entry in FortiGate crash log.

Q to T.

quard: Quarantine daemon. Handles FortiSandbox communication/file-submission for the FortiGate, as well as communication with FortiAnalyzer for quarantined file archive handling. Notably, this process is debugged using a slightly different name (i.e. 'diagnose debug application quarantine <debug level>' and 'diagnose test application quarantined <test level>').

reportd: Handles local report generation based on existing logs, as well as FortiView functionality. As of FortiOS v6.4.2 and later, reportd is a subscriber daemon to logs published by miglogd (as per change #584449).

ripd: Handles the IPv4-based Routing Information Protocol (RIP) dynamic routing protocol; part of ZebOS Routing Daemons.

ripngd: Handles the IPv6-based Routing Information Protocol next generation (RIPng); part of the ZebOS Routing Daemons.

samld: Handles SAML authentication on the FortiGate (both when the FortiGate is acting as an SP or as an IdP). First introduced in FortiOS v6.2 as per change #570207.

scanunitd: Scanning unit; handles Antivirus scanning of certain traffic types.

See Inspection mode differences for antivirus.

sdnvpnd: Handles the configuration done by the Cloud SDN Orchestration VPN wizard for AWS introduced in FortiOS v8.0.

sflowd: Handles sFlow and NetFlow functionality on the FortiGate (e.g. packet sampling and session statistic tracking).

smbcd: SMB Client daemon. Handles SMB connections made from the FortiGate to SMB file servers (i.e. for SMB bookmarks used with SSL-VPN Web Mode).

snmpd: Handles SNMP connections as well as processing traps/queries.

src-vis: Source Visibility daemon. Handles Device Identification/Detection functionality on the FortiGate (e.g. creating MAC address entries and snooping traffic to identify a given device on the network). Renamed/replaced by the cid daemon as of FortiOS v7.2.0 and later (as per change #740250).

sshd: Handles management connections to the FortiGate via SSH.

sslvpnd: Handles all SSL-VPN related tasks, including authentication/authorization and VPN tunneling on the FortiGate.

stpd: Handles Spanning Tree on the FortiGate, specifically supporting Rapid Spanning Tree (RSTP); first introduced as part of change #152528.

syslogd: Handles syslog functionality on the FortiGate (i.e., packaging and transmitting log events to remote log servers using the syslog protocol).

• Previously, syslog functionality was handled within miglogd. As of FortiOS v7.0.0 and later, syslogd is a new daemon that subscribes to miglogd's log publishing functionality and handles syslog-based logging functionality (as per change #665697).

• This process also fully replaces the older rlogd process.

tvc: Tunnel Virtual Connection daemon. Added in FortiOS v7.0.0 (per Change #653386) as part of a new feature that allows the FortiGate to act as an SSL VPN client to another FortiGate.

Troubleshooting TVC assists in identifying problems during tunnel setup, emphasizing the interaction and negotiation between the FortiGate and the client.

U to X.

updated: Handles communication with FortiGuard for on-device database and licensing updates (such as IPS/AV databases, Internet Service DB, etc.).

uploadd: Fortinet upload daemon; handles the uploading of log files to FortiGate Cloud.

urlfilter: Handles Web Filtering functionality on the FortiGate, including verifying client URL requests against the Web Filter profile, monitoring/maintaining lists of FortiGuard rating servers, and updating the urlfilter cache.

• As of FortiOS v6.2.1 and later, multiple URLfilter daemons can be spawned as worker processes managed by the wf_monitor daemon (per Change #526481).

• Before FortiOS v6.2.1, urlfilter was a single-process daemon. Enabling multiple URLfilter worker processes has resulted in substantial increases in total performance.

voipd: Performs proxying/inspection of SIP traffic using the SIP Application Layer Gateway (ALG).

In FortiOS v6.4 and earlier, voipd handled SIP traffic in a proxy-based manner, regardless of the inspection-mode. In FortiOS v7.0 and later, flow-based SIP inspection is handled by the IPS Engine (see also: Flow-based SIP inspection).

vwl: Virtual WAN Link daemon (aka SD-WAN). Handles packet routing along the SD-WAN configuration.

wad: Handles traffic proxying on the FortiGate, including Explicit Web Proxy, proxy-based security-inspection, and proxy-based Firewall Policy traffic.

• In FortiOS v6.0 and earlier, inspection modes were set on a per-VDOM or global basis. In FortiOS v6.2.1 and later, it became possible to set Firewall Policies to be either flow-based or proxy-based, which results in traffic largely being inspected by the IPS Engine or WAD, respectively (see: Flow versus proxy policy improvement v6.2.1).

• Refer to the following article for more in-depth information regarding the various sub-processes of WAD: Technical Tip: Overview of WAD process structure.

 
wf_monitor: Web Filter monitor daemon (parent to urlfilter daemon). A new daemon that was introduced as part of the new multi-process URL filter project aimed to improve performance. Introduced in FortiOS v6.2.1 and later (per change #526481).

• The wf_monitor daemon is responsible for starting, restarting, and killing urlfilter worker processes; monitoring worker statuses and gathering debugging statistics; and managing shared memory caches for urlfilter workers.

• This daemon enables multiple URL filter worker processes, which improves Web Filtering performance by scaling across multiple CPU cores.

 
Y to Z.

zebos_launcher - Launcher daemon for the ZebOS routing software (handles routing tables on the FortiGate).

The following applications/daemons can be further diagnosed (this list is available in FortiOS v6.2; older firmware may have fewer options, or different names). 

This list is available by entering the command 'diagnose debug application ?'. Example output:

http                  HTTP proxy.
smtp                  SMTP proxy.
ftpd                  FTP proxy.
pop3                  POP3 proxy.
imap                  IMAP proxy.
nntp                  NNTP proxy.
proxy                 Proxy.
radvd                 Router adv daemon.
miglogd               Log daemon.
kmiglogd              Kernel Log daemon.
forticldd             FortiCloud daemon.
alertmail             Alert mail daemon.
ppp                   PPP daemon.
l2tp                  L2TP daemon.
pptp                  PPTP daemon.
pptpc                 PPTP client.
authd                 Auth daemon.
foauthd               FortiguardOverride auth daemon.
fcnacd                FortiClient NAC daemon.
fcld                  Fclicense daemon.
fssod                 FSSO daemon.
dhcps                 DHCP server.
dhcp6s                DHCPv6 server.
update                Update daemon.
vpd                   VPN policy daemon.
fnbamd                Fortigate non-blocking auth daemon.
eap_proxy             EAP proxy daemon.
ipsmonitor            ips monitor
ipsengine             ips sensor
urlfilter             Urlfilter daemon.
wf_monitor            WF monitor, parent of urlfilter daemon.
ddnscd                DDNS client daemon.
dhcprelay             DHCP relay daemon.
dhcp6r                DHCPv6 relay.
snmpd                 SNMP daemon.
chassis               Chassis daemon.
scanunit              Scanunit daemon (File scanning).
emailfilter           Emailfilter module.
wpad                  Port access entity daemon.
wpad-crash-hexdump    Dump wpad crash in hexedecimal format.
wpa-show-keys         Dump keys in wpad or wpas log.
wpa-timestamp         Dump timestamp in wpad or wpas log.
wifi                  WiFi setting.
dnsproxy              DNS proxy module.
sflowd                sFlow protocol module.
hatalk                HA protocol module.
hasync                HA synchronization module.
harelay               HA relay module. Relays the slave daemons' local-out tcp connection to the public network
hamonitord            HA monitor module.
quarantine            Quarantine daemon.
dhcpc                 DHCP client module.
zebos-launcher        ZebOS launcher daemon.
zebos                 ZebOS
modemd                MODEM daemon.
radiusd               RADIUS daemon.
sshd                  Sshd daemon.
sslvpn                SSL VPN proxy daemon
guacd                 Guacamole proxy daemon
info-sslvpn           SSL-VPN info daemon for Fortinet top bar.
sessionsync           Session sync daemon.
ipldbd                Ipldbd daemon.
crl-update            CRL update daemon.
alarmd                Alarmd daemon.
forticron             Forticron daemon.
uploadd               Upload daemon.
smbcd                 SMB client daemon.
samld                 SAML SSO daemon.
acd                   Aggregate Controller
alicloud-sdn          AliCloud SDN controller
alicloud-ha           AliCloud HA controller
sip                   SIP ALG.
sccp                  SCCP ALG.
ike                   IKE daemon.
ocvpn                 Overlay Controller VPN.
fgfmd                 FortiGate/FortiManager communication daemon.
wccpd                 WCCP daemon.
waocs                 WAN acceleration object cache storage.
wabcs                 WAN acceleration byte cache storage.
garpd                 VIP gratuitous ARP daemon.
scep                  SCEP
ipsufd                IPS URL filter resolver daemon.
cw_acd                Capwap AC daemon.
cw_acd_helper         Capwap AC helper daemon.
cw_acd_wpad           CAPWAP AC and WPA daemon (wpad).
cw_acd_wlev           CAPWAP AC daemon wireless event notification.
cu_acd                caputp AC daemon
fortilinkd            fortilink daemon
flcfgd                fortilink configuration daemon
fltund                FortiLink tunnel daemon
rsyslogd              Rsyslogd daemon.
reportd               report daemon
dlp                   DLP
vrrpd                 VRRP daemon.
fgd_alert             FortiGuard alert message.
ntpd                  NTPd daemon.
fsd                   Forti-start daemon.
dlpfingerprint        DLP fingerprint daemon.
httpsd                HTTPS daemon.
stp                   Spanning Tree Protocol daemon.
spareblock            Set debug spare block count.
lted                  USB LTE daemon.
lldprx                Link Layer Discovery Protocol (LLDP) Receiver
lldptx                Link Layer Discovery Protocol (LLDP) Transmitter
src-vis               Source Visibility daemon.
wiredap               Wired AP (802.1X port-based auth) daemon.
dhcp6c                DHCPv6 client.
server-probe          Server probe daemon.
link-monitor          Link monitor daemon.
pppoed                PPPoE client Daemon.
ovrd                  Override daemon.
extenderd             Extender Wan daemon.
awsd                  Amazon Web Services (AWS) daemon.
netxd                 NetX REST API daemon.
gcpd                  Google Cloud Platform daemon.
azd                   Microsoft Azure daemon.
ocid                  Oracle Cloud Infrastructure (OCI) daemon
openstackd            OpenStack SDN connector daemon.
kubed                 Kubernetes daemon.
vmwd                  VMware vSphere daemon
init                  System init process.
mrd                   Mobile router daemon.
dssccd                PCI DSS Compliance Check daemon.
radius-das            RADIUS DAS daemon.
csfd                  Security Fabric daemon.
fsvrd                 FortiService daemon.
virtual-wan-link      Virtual-Wan-Link daemon.
ftm-push              FTM-Push daemon.
cmp                   CMPv2.
sdncd                SDN Connector daemon.
ptpd                  Precision Time Protocol daemon.
autod                Automation daemon.
bfdd                  BFD daemon.
fsso_ldap          FSSO LDAP daemon.
cmdbrsv            Applies configuration changes
updated            FortiGuard updates
wad                  WAN optimization, explicit proxy, proxy based inspection for HTTP and HTTPS, and FTP
proxyworker     Proxy-based inspection for IMAP, POP, SMTP

Some applications can be seen in the list of top processes and cannot be debugged or investigated in-depth, because the information may not be useful for troubleshooting.

Related articles:

• Technical Tip: How to list processes in FortiOS

• Technical Tip: Diagnose sys top CLI command

• Technical Tip: Restarting internal processes/daemons