- Run the following command to identify all running httpsd daemons:
diagnose system process pidof httpsd
143
1075
1080
1081
1092
1098
1100
1104
1105
1106
1108
1110
1111
1112
1113
1116
- Use the following command to display the CPU utilization and status of each daemon.
diagnose system top 10 50 | grep httpsd
httpsd 1106 R 96.0 0.7 0
httpsd 1081 R 87.6 0.7 1
httpsd 1112 R 87.6 0.7 3
httpsd 1105 R 77.8 0.7 2
httpsd 1098 S 1.9 0.7 1
.
.
Restart the httpsd daemon and verify if the following are present:
- The ID of the daemons changes rapidly, sometimes every few seconds.
- CPU utilization of the httpsd rises quickly.
If all of these symptoms occur, check the number of active admin sessions to the FortiGate:
get system info admin status
If there are multiple admin sessions, disconnect them from the GUI and only let one admin via SSH if the high CPU continues.
It will be prudent to check the httpsd real-time debug. The following is a real example that reports the same high CPU symptoms from httpsd daemons. In the output below, there are multiple HTTP POST requests targeting the login page coming from different IPs.
diagnose debug app httpsd -1
diagnose debug console timestamp enable
diagnose debug enable
[httpsd 1168 - 1764537551 info] fweb_debug_init[451] -- New POST request for "/logincheck" from "X.X.X.233:56302"
[httpsd 1168 - 1764537551 info] fweb_debug_init[453] -- User-Agent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36"
[httpsd 1168 - 1764537551 info] fweb_debug_init[455] -- Handler "logincheck-handler" assigned to request
[httpsd 1168 - 1764537551 info] logincheck_handler[454] -- entering vdom for login_attempt (vdom='root')
[httpsd 1171 - 1764537551 info] fweb_debug_init[451] -- New POST request for "/logincheck" from "X.X.X.221:48580"
[httpsd 1171 - 1764537551 info] fweb_debug_init[453] -- User-Agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.6 Safari/605.1.15"
[httpsd 1170 - 1764537553 info] fweb_debug_init[451] -- New POST request for "/logincheck" from "X.X.X.212:57384"
[httpsd 1170 - 1764537553 info] fweb_debug_init[453] -- User-Agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.6
[httpsd 1167 - 1764537553 info] fweb_debug_init[451] -- New POST request for "/logincheck" from "X.X.X.235:51474"
[httpsd 1167 - 1764537553 info] fweb_debug_init[453] -- User-Agent: "Mozilla/5.0 (Macintosh; Intel Mac OS X 15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/18.6 Safari/605.1.15"
During HTTPS daemon debugging on a FortiGate, log entries such as`[httpsd ... info] logincheck_handler[565] -- login attempt completed with code -100`indicate repeated failed or incomplete administrative login attempts.
The code '-100' reflects unsuccessful authentication, which may result from automated scripts, multiple browser sessions, or unauthorised probing. These repeated attempts increase the workload on existing httpsd worker processes. This activity elevates httpsd CPU usage and can cause it to appear at the top of 'diagnose sys top' output.
The source of such attempts can be restricted by configuring a local-in-policy to block access to the administrative HTTPS service from untrusted addresses.
To disable debugging:
diagnose debug reset
diagnose debug disable
Go to Log & Report -> System Events, and analyse the event logs for multiple HTTPS login attempts and note the interface that is receiving these login attempts.
Alternatively sniffer command can also be executed to check incoming requests via CLI:
diagnose sniffer packet <wan-interface> 'host <wan-interface-ip> and port <admin-port>' 4 0 l
Workaround:
- Disable the HTTPS service from the 'Administrative access' of the internet-facing port, then restart the HTTPS daemon. This setting can be found under Network -> Interface -> [select interface] -> Deselect HTTPS from 'Administrative Access'.
In the CLI:
config system interface
edit "wan"
set allowaccess ping # (remove https, ssh unless required)
next
end
The httpsd processes can be restarted by executing the command 'fnsysctl killall httpsd' or 'diagnose system kill 11 [pid_of_httpsd]'.
If GUI Administrative Access needs to be enabled on the internet-facing ports, consider using a Local‑In Policy to restrict GUI access from only specific Public IP Addresses.
config firewall local-in-policy
edit 1
set intf "wan"
set srcaddr "MY_ADMIN_IP"
set dstaddr "all"
set action accept
set service "HTTPS"
next
edit 2
set intf "wan"
set srcaddr "all"
set dstaddr "all"
set action deny
set service "HTTPS"
next
end
As best practice, a 'local-in policy' can be configured with a GEO address object as follows:
First: Create an address object with 'geography' type:
config firewall address
edit "COL-GEO-object"
set type geography
set country "CO"
next
end
Second: Use it as 'srcaddr' in the 'local-in policy':
config firewall local-in-policy
edit 1
set intf "wan"
set srcaddr "COL-GEO-object"
set dstaddr "all"
set action accept
set service "HTTPS"
next
edit 2
set intf "wan"
set srcaddr "all"
set dstaddr "all"
set action deny
set service "HTTPS"
next
end
Note:
If the GUI access service does not use port 443, ensure the service is created with the correct port and then added to the 'local-in policy'. To review the administrative port, input this command:
show full-configuration | grep 'set admin-sport'
Related articles:
Technical Tip: Debugs for troubleshooting high CPU Issues
Technical Tip: How to configure a local-in policy to only allow specific subnet using srcaddr-negate...
Technical Tip: Creating a Local-In policy (IPv4 and IPv6) on GUI
|
