Technical Tip: FortiAuthenticator swap memory in use, symptoms and solutions

[kiri]

Description	This article describes symptoms and solutions for swap memory being in use on a FortiAuthenticator. FortiAuthenticators may have the following symptoms: Admins being unable to log in to the GUI without a clear reason. Logs or debug failing to download, with timeouts or other web server errors. Web server errors like 500 or internal server errors after simple admin actions. Users being unable to log in via FortiAuthenticator services, push notifications not working. Tokens failing to provision. Before the condition sets in, the operations above are usually possible.
Scope	FortiAuthenticator 6.5.x, 6.6.x
Solution	If these symptoms are experienced, run the following CLI command to check memory usage: get hard mem Look for this section in the output: ... ### Memory info MemTotal: 2029704 kB MemFree: 246556 kB MemAvailable: 618012 kB ... Mlocked: 0 kB SwapTotal: 1048572 kB SwapFree: 948572 kB Dirty: 16 kB ... If SwapFree ≠ SwapTotal, swap memory is in use. If SwapFree = SwapTotal, the issue may not be memory-related. The following steps help identify the cause and resolve the issue. FortiAuthenticator does not have enough memory. Version 6.4 requires at least 2 GB RAM for 1–500 users. Versions 6.5 and 6.6 require at least 4 GB RAM for 1–500 users. Higher license tiers require more memory. Check the FortiAuthenticator-VM sizing guide. Allocating the correct resources should resolve the issue. Memory Leak. If the system meets the required specs but still experiences issues, check the kernel log: https://<FortiAuthenticatorIp>/debug/kernlog Look for messages indicating memory-related crashes, such as: Example 1: Segmentation Fault (Segfault): 2025-03-24T21:34:15.249194+01:00 fortiauth kernel: [28995.800096] wmid[28977]: segfault at 0 ip 00007f7eec3c2618 sp 00007ffd57fe6988 error 4 in libc.so.6[7f7eec281000+155000] Example 2: Out of Memory ( oom-kill or oom_kill ) : 2024-10-17T15:05:59.739590+02:00 FortiAuthenticator kernel: [15576.788064] Out of memory: Killed process 6063 (httpd) total-vm:2042176kB, anon-rss:945536kB, file-rss:0kB, shmem-rss:0kB, UID:33 tables:3608kB oom_score_adj:0 If a memory leak is detected: Rebooting will temporarily restore functionality, but the issue will return. Check FortiAuthenticator OS release notes for known/fixed memory issues. Upgrade the FortiAuthenticator OS if a fix is available. If the issue persists after upgrading, open a TAC case with: The output of 'get hard mem'. Kernlog output. Debug reports: GUI -> Logging -> Log Access -> Logs -> Downloads -> Summary or System. High Load from Unusual Activity or Brute Force Attacks. If there are no indications of a memory leak, check raw logs for unusual authentication attempts: go to GUI -> Log Access -> Logs. Run a quick search for: 'Authentication' -> Lists successful and failed attempts. 'Invalid' -> Shows failed login attempts with invalid usernames. If login attempts are unexpectedly high, a brute force attack might exhaust memory. Some quick-to-employ measures are: FortiGate automation stitches. Login-block policies. Geofencing. Related article: Technical Tip: Best practices on hardening FortiAuthenticator environments