FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Ehanssen
Staff
Staff
Article Id 370354
Description This article describes high memory usage in the wadding process due to the ssl.fts.str.fstr_buffer_bytes buffer.
Scope All FortiOS versions.
Solution

The buffer ssl.fts.str.fstr_buffer_bytes is part of the wadding process and is a generic buffer object for processing SSL traffic. Depending on the proxy usage this buffer can quickly rise and memory and can cause the device to be pushed into memory conserve mode. It is symptomatic to see the memory drop eventually as the buffer clears, so it is not to be confused with Troubleshooting Tip: Wad memory leak in object ssl.fts.str.fstr_buffer_bytes on firmware 7.0.8 and 7...

 

Since in this instance it is genuine traffic causing the issue, restarting the wad process with the diagnose test application wad 99 will not be helpful. The memory consumption would quickly rise to the previous level.

 

Down below is an example of the symptom where the wad using up 1.5GB of memory. In this example, the wad process id 18721 refers to a wad worker. For a more detailed description of the WAD process, refer to Technical Tip: Overview of WAD process structure.

 

Example:

 

get hardware memory

MemTotal: 8170500 kB
MemFree: 2134140 kB
Cached: 864804 kB
Active: 3571352 kB
Active(anon): 3478064 kB
Shmem: 545320 kB
Slab: 316624 kB


diagnose sys top-mem 99

wad (18721): 1581347kB     
ipshelper (291): 263091kB
wad (347): 106108kB
wad (17922): 93174kB
node (277): 88499kB
wad (346): 82280kB
wad (17936): 78335kB

 

diagnose wad stats worker

ssl.fts.str.fstr_buffer now 90310 max 741110 total 55899610
ssl.fts.str.fstr_buffer_bytes 1178626093

 

Linking the buffer usage to throughput requires multiple points of data. For that, refer to the article Technical Tip: Tracking WAD throughput in the CLI.

This article can be used to track the number of bytes sent through the wad process. Since the ssl.fts.str.fstr_buffer_bytes buffer is part of the diagnose wad stats worker command, the proposed teraterm monitoring script can be used here as is.

 

There is no uniform workaround for such instances. The following is an outline of how to circumvent the issue:

  1. Disable UTM features on the matching proxy policies. Since this buffer is generic, it is affected by UTM feature usage.
  2. In the case of virtual servers, use a flow-based VIP instead. This will prevent the usage of advanced settings the virtual server offers like SSL offloading.
  3. Switch from proxy-based inspection in the firewall rules to flow-based rules. This can be implemented in a more fine-grained fashion if the issue can be linked to specific connections.