Seeking Multitenant VDOM Recommendation

[Nauticus]

I am trying to determine the best design for VDOMs in a FortiGate that will be used to connect to multiple different customer sites via IPsec tunnels.  Customers must be isolated from each other and I have more customers than the max 10 vdom limit, so a dedicated vdom for each customer is not an option.  My plan is to have:

root vdom (Global shell)
mgmt-vdom (Fortigate mgmt functions and administration - type "Admin" vdom - has internet access via traffic-vdom)
traffic-vdom (has wan interface, function is to provide other vdoms wan/internet access)
customers-vdom (customers shared vdom, vpn tunnels for each customer)
vendor-vdom (third-party vendor for monitoring customer sites and providing access to vendors cloud applications - vpn tunnel to vendors cloud)

No LAN resources for customers, I am only handling traffic from customers to vendor via the IPsec tunnels.

What is the best and most secure way to configure vdoms and IPsec tunnels.

Inter vdom links from other vdoms to traffic vdom?
No vdom links, but using subinterfaces of the wan port for each customer.  Wan located in traffic vdom, subinterfaces of wan located in customers-vdom.
Where to terminate vpns - all in traffic vdom?  Or customer vpns in Customers vdom, Vendor vpn in vdom? etc.

Segregate by only using IPsec interfaces for each customer?  Or both IPsec and Vlan interfaces?

How to best leverage hardware acceleration in the design

The more I research the more confused I become as to how to approach this.

Thanks

[Toshi_Esumi]

I can not disclose the detail for our set up in public, which is serving to our customers, for security reason. It would violate some security audits/standards.
But at least I can say, if you terminate customer VPNs at a different shared VDOM, all customers would share the same routing-table unless you separate them with VRF in the VDOM. Which is redundant while you're using VDOMs. If you terminate the VPNs at the customer's VDOM by assigning public IPs to each customer VDOM you can separate them automatically.
To do that, you have to use either vdom-link or npu-vlink (the latter is recommended for performance) unless you dedicate some physical ports for loopback cable from a shared vdom port to a customer port. That would quickly run out of available ports when the number of customers grow.

Toshi   

[Nauticus]

Hi Toshi,

Since I have more customers than allowed vdoms, I have to have customers share a vdom.  I think you are saying to have dedicated vdoms for each customer with a unique public IP for each as well?

[Toshi_Esumi]

Sharing local routing table between customers wouldn't satisfy most of, if not all of, security audits/standards. If 10 VDOMs are not enough you should add more FGTs. If customers figure it out, they would likely leave. At least I would never sign up your service.
Sharing a public IP isn't impossible to but again it would require internal network to be on the same routing table or requires VRF separation in the VDOM. I never used VRF, because I never needed, so I can't tell you exactly how you can do and how it would work.

Toshi

[Nauticus]

Hi Toshi,

Understood.  If I have dedicated vdoms for each customer and a traffic vdom with wan interface, would I create subinterfaces under WAN with public IPs for each customer?  Would I then assign the subinterfaces toeach respective customer vdom, or leave them in traffic vdom?  Thanks

[Toshi_Esumi]

That's one option. Another is just routing customer public IPs/subnets. That part (VDOM) is just a part of internet that can be accessed from anywhere in the world.

Toshi