FortiSOAR vertical scaling requirement

[malayamanas_FTNT]

Hello FortiSOAR SMEs.

I am trying to implement enrichment of indicators in a time bound manner, below is the usecase details.

FortiSOAR VM has below configuration.

FSR Version: v7.4.2 ENT License without any SME and Agents.

vCPU# : 16

RAM: 64GB

HA: Active / Active

Use-case - On-Create event triggered enrichment of 1000 numbers indicators such as IPv4, URL, FQDN, SHA1, SHA256 and MD5 hashes within 30 minutes, where each indicator takes 2 minutes to get enriched by 3rd party threat intelligence platforms (VirusTotal, InfoBlox,..etc)

Below tunables are configured.

1.

• Workflow workers: 16
  /etc/celery/celeryd.conf: CELERYD_OPTS="--concurrency=16"
• Postgres shared buffer: 2GB
  /var/lib/pgsql/12/data/postgresql.conf: shared_buffers = 2048MB
• ElasticSearch Xms and Xmx 8GB:
  /etc/elasticsearch/jvm.options.d/fsr.options:
  -Xms8g
-Xmx8g
• "max connection" defaults to 200 in Postgresql.

2. Playbook where enrichment happens, runs parallelly and asynchronously for each Indicator through "On-Create" event trigger.

Observing error: 

too many database connections

Batch processing is not an option as enrichment is triggered with "On-Create" event in Indicator module. 

Kindly suggest this usecase solution.

[spooja]

If there is enough memory and CPU available on the machine during the max workload, you could increase the PG max connections to 300

[malayamanas_FTNT]

Thanks.

But what is the number that defines "enough memory and CPU" ?

Is there specification available from benchmarking within a timelimit and numbers of indicators for enrichment ?

Use-case - On-Create event triggered enrichment of 1000 numbers indicators such as IPv4, URL, FQDN, SHA1, SHA256 and MD5 hashes within 30 minutes, where each indicator takes 2 minutes to get enriched by 3rd party threat intelligence platforms (VirusTotal, InfoBlox,..etc)

[spooja]

Please check the output of top command for memory in use and the CPU load average.  Should be less than max memory and the number of cores respectively. Then you can increase the max connections and monitor the CPU and memory usage again

[malayamanas_FTNT]

With above bruteforce approach of benchmarking at customer end, it is extremely difficult to pursuade customer do run a benchmarking test. Customer asks for benchmark and we do not have specification. Your answer is not helpful.

[spooja]

For benchmarking, please refer to test run results at https://docs.fortinet.com/document/fortisoar/7.4.2/performance-benchmarking/989364/fortisoar-perform...

[malayamanas_FTNT]

How to import below sample playbook soultion ?

zip file is not supported as an import while inside Automation >> Playbook >> Import.

Then, I tried to unzip and then import info.json which was failed.

Then tried to import through configuration import wizard, that was also failed.

Kindly reply - How to import this test playbook to further understand it.

[malayamanas_FTNT]

Hello FortiSOAR SMEs, 

Anyone out there to help me !

[aghutke]

Hi, 

We have updated the attachments, please check them now

Thank You