Multi-Site HA Deployment with FortiSIEM 7.3.0 on KVM – Is This Possible?

[beingarif]

I’m planning to deploy FortiSIEM 7.3.0 on KVM in a Multi-Site High Availability (HA) setup with ClickHouse, and I’d like to confirm if this is feasible. The planned architecture is as follows:

For a Two-Site Deployment:

• Supervisors with DB: 3 (1 Primary Leader, 2 Followers) [2 in one site, 1(Follower in the other].
• Keeper Nodes: 3 Keepers (2 in one site, 1 in the other).
• Shards: 1 shard:
  • 2 Workers per shard.(2 Replicas)
  • 1 Worker in each site.
  • Workers having both Data (Ingest) and Query flags set, ensuring local storage with replication within the shard.
• Latency: Should be <100ms for efficient replication.

Would this setup be supported on KVM with the latest FortiSIEM 7.3.0? Has anyone successfully implemented a similar architecture? Will this also support Automatic HA..??

Looking forward to your insights!

This is supported in Version 7.2.4 Reference: https://docs.fortinet.com/index.php/document/fortisiem/7.0.0/high-availability-and-disaster-recovery... 

[Secusaurus]

Hi @beingarif,

Have a look at this discussion, it covers almost the same question:

https://community.fortinet.com/t5/FortiSIEM-Discussions/Multi-Site-HA-Deployment-Implementation-in-F...

And the docs:

https://docs.fortinet.com/document/fortisiem/7.3.0/high-availability-and-disaster-recovery-procedure...

Note, that the deployment of 7.3.0 works a little bit different to the one on previous versions. It's a real HA deployment now, before, it was just a sync process and required the DR license.

For clickhouse deployment, keep in mind, the cluster is only able to store to the db, if a majority of the keeper nodes is available. So, when using your cluster design above, if site 1 fails, you only have read-only-access and cannot store new events.

In the same manner (majority), the only deployment of HA in 7.3.0 I would recommend, would be to use three supervisors. I am not quite sure, if the HA-failover is designed to work with only two supervisors.

KVM is not an issue, I know about very large setups entirely based on KVM.

For the workers, you should have a direct high-performance connection. In our experience, the internet between two workers is adding too much latency and issues on top, so they constantly get out of sync. But looking at your design with a load balancer in front, I would assume you plan to have them physically close to each other.

Don't forget that you should load-balance the traffic to the workers, but should not load-balance it to the supervisor(s), as there can only be one primary.

Have you had a look as well at the sizing recommendations? As keeper-nodes can also be attached to the supervisors directly, you could reduce some complexity in your design. Also, in our experience, you can live with one shard for quite a high amount of EPS. As new shards only make sense with new hardware (multiple workers on the same hardware don't get more efficient), depending on your expected EPS, you can save a lot of money in the first step.

Best,

Christian

[FSM_FTNT]

Just to clarify what @Secusaurus mentioned, currently in 7.3.0 we do not support HA with two Super nodes, even with manual failover. So you need to run with 3 Super nodes and that will provide the auto failover.

Make sure you have low latency, stable network and high bandwidth. When adding either a ClickHouse replica or Super node there can be a significant surge in traffic as data is replicated.

If you lack the networking requirements, you will have trouble with cluster replication and possibly stability.