FortiSIEM ClickHouse Deployment Architecture: Supervisor and Worker Node Configuration

[gauravpawar]

We are planning a FortiSIEM ClickHouse deployment with an expected EPS of 15,000, using the following architecture:

1 Supervisor Node (without a dedicated data disk — i.e., no Disk 5)

1 Worker Node (with a data disk, intended to store all event data)

We have a few queries regarding this setup:

Is it possible to install the Supervisor without a data disk, considering that all data will reside on the Worker and the Supervisor will function solely as a Keeper node?

Can we configure the Worker with both “Data” and “Query” roles enabled, and create a ClickHouse cluster with a single shard and one replica without supervisor ?

Could you please recommend the most suitable and supported architecture for this 1 Supervisor + 1 Worker node setup?

@Secusaurus @Anthony_E could you please help here 

[Secusaurus · Accepted solution]

Hi @gauravpawar,

For official statements, please follow the official sizing guide: https://docs.fortinet.com/document/fortisiem/7.4.0/sizing-guide-clickhouse/965243/fortisiem-sizing-g...

In my experience, setting up the supervisor without data disk does not work, since you need to have a data disk for initial deployment and lateron for the keeper storage. You cannot connect workers before the initial deployment, therefore the initial ClickHouse setup will use the Supervisor as first node. After going through the full setup, you might probably be able to reduce the disk size - but as far as I understand, still, the defined ClickHouse disk must be available for Keeper activities.

But leaving the fact aside that you will need a (small) disk, you can configure the system to store the data entirely on the Worker(s) and let the Supervisor only be Keeper. This is a very common setup.

One of the main benefits of using Workers is redundancy and data backups as the same data exists on multiple Workers. So, in my opinion, using a single Worker does not really improve the setup compared to a All-In-One deployment. Yes, if you use separate hardware, you can reduce load on the Supervisor. But for 15,000 EPS, the load is not too high that splitting is vital.

Best,

Christian

[gauravpawar]

Thanks Christian