Hello FortiSOAR SMEs. I am trying to implement enrichment of indicators in a time bound manner, below is the usecase details. FortiSOAR VM has below configuration. FSR Version: v7.4.2 ENT License without any SME and Agents. vCPU# : 16 RAM: 64GB HA: Active / Active Use-case - On-Create event triggered enrichment of 1000 numbers indicators such as IPv4, URL, FQDN, SHA1, SHA256 and MD5 hashes within 30 minutes, where each indicator takes 2 minutes to get enriched by 3rd party threat intelligence platforms (VirusTotal, InfoBlox,..etc) Below tunables are configured. 1. Workflow workers: 16/etc/celery/celeryd.conf: CELERYD_OPTS="--concurrency=16" Postgres shared buffer: 2GB/var/lib/pgsql/12/data/postgresql.conf: shared_buffers = 2048MB ElasticSearch Xms and Xmx 8GB:/etc/elasticsearch/jvm.options.d/fsr.options:-Xms8g-Xmx8g "max connection" defaults to 200 in Postgresql. 2. Playbook where enrichment happens, runs parallelly and asynchronously for each Indicator through "On-Create" event trigger. Observing error: too many database connections Batch processing is not an option as enrichment is triggered with "On-Create" event in Indicator module. Kindly suggest this usecase solution.

A

Anonymous_User

Contributor III

Question

FortiSOAR vertical scaling requirement

Forum|Forum|2 years ago
November 20, 2023
8 replies
3280 views

Hello FortiSOAR SMEs.

I am trying to implement enrichment of indicators in a time bound manner, below is the usecase details.

FortiSOAR VM has below configuration.

FSR Version: v7.4.2 ENT License without any SME and Agents.

vCPU# : 16

RAM: 64GB

HA: Active / Active

Use-case - On-Create event triggered enrichment of 1000 numbers indicators such as IPv4, URL, FQDN, SHA1, SHA256 and MD5 hashes within 30 minutes, where each indicator takes 2 minutes to get enriched by 3rd party threat intelligence platforms (VirusTotal, InfoBlox,..etc)

Below tunables are configured.

1.

Workflow workers: 16
/etc/celery/celeryd.conf: CELERYD_OPTS="--concurrency=16"
Postgres shared buffer: 2GB
/var/lib/pgsql/12/data/postgresql.conf: shared_buffers = 2048MB
ElasticSearch Xms and Xmx 8GB:
/etc/elasticsearch/jvm.options.d/fsr.options:
-Xms8g
-Xmx8g
"max connection" defaults to 200 in Postgresql.

2. Playbook where enrichment happens, runs parallelly and asynchronously for each Indicator through "On-Create" event trigger.

Observing error:

too many database connections

Batch processing is not an option as enrichment is triggered with "On-Create" event in Indicator module.

Kindly suggest this usecase solution.

A

Anonymous_UserAuthor

Contributor III

If there is enough memory and CPU available on the machine during the max workload, you could increase the PG max connections to 300

A

Anonymous_UserAuthor

Contributor III

Thanks.

But what is the number that defines "enough memory and CPU" ?

Is there specification available from benchmarking within a timelimit and numbers of indicators for enrichment ?

Use-case - On-Create event triggered enrichment of 1000 numbers indicators such as IPv4, URL, FQDN, SHA1, SHA256 and MD5 hashes within 30 minutes, where each indicator takes 2 minutes to get enriched by 3rd party threat intelligence platforms (VirusTotal, InfoBlox,..etc)

A

Anonymous_UserAuthor

Contributor III

Please check the output of top command for memory in use and the CPU load average. Should be less than max memory and the number of cores respectively. Then you can increase the max connections and monitor the CPU and memory usage again

A

Anonymous_UserAuthor

Contributor III

With above bruteforce approach of benchmarking at customer end, it is extremely difficult to pursuade customer do run a benchmarking test. Customer asks for benchmark and we do not have specification. Your answer is not helpful.

A

Anonymous_UserAuthor

Contributor III

For benchmarking, please refer to test run results at https://docs.fortinet.com/document/fortisoar/7.4.2/performance-benchmarking/989364/fortisoar-performance-benchmarking-for-v7-4-2