Inter VDOM dynamic routing : OSPF or BGP ?

[FlorianFlux]

Hi guys,

I have the following scenario implemented, all Vdoms are connected among themselves through a Transparent vdom (only one allow all policy applied so transparent vdom acts as a L2 switch).

Now I would like to implement dynamic routing protocol so i can redistribute BGP routes from WAN vdom to others, and others can redistribute connected networks to other vdoms etc.

I did not find nothing about doing such thing on internet. I would create an OSPF area0 and put all vdoms associated to it.

Some colleagues advice I should do iBGP. Not sure about this.

Any thought about the setup ?

thank you very much for your feedback.

Running 6.2.4 on 600E

[emnoc]

I guess that would work,  but I would not waste a vdom between outer edge and inside vdom. I would just stacked them and run a interior routing protocol.

stacked vdom  https://socpuppet.blogspot.com/2014/09/a-stacked-vdom-concept-with-fortigate.html

You can now filter routes to the inside. Please do not do a full-bgp view and redistribute these routes into IGP.

Ken Felix

[FlorianFlux]

Hi Ken,

thank you very much for your explanations, i understand the fact that wasting a vdom is not ideal, but in our case all vdoms need to talk with each others, so making a full mech topology would be a pain (we have 5 vdoms so far).

Not sure to understand "Please do not do a full-bgp view and redistribute these routes into IGP."

I will go for OSPF and obviously filter redistribution between the OSPF and BGP.

Thank you for your advice !

[emnoc]

Not sure to understand "Please do not do a full-bgp view and redistribute these routes into IGP." I will go for OSPF and obviously filter redistribution between the OSPF and BGP.

Just keep in mind that routes take memory. So If your redistributing a 0.0.0.0/0 into ospf  & sharing it down stream into the 2other vdom and advertising a few routes into bgp you would be fine. That's all  that I'm implying here. I still think a simpler mesh stack would be better.

edge  ASNxxxx ( what ever your asn# ) 

         ebgp peer to vdom1 AS65001

         ebgp peer to vdom2 AS65002

What would be even more better is to not use a inter-vdom-link so if you have  the need for HA you can run  1 or 2 vdom on the 2nd FGT ha member , and would not be bound by any  virtual interface link speeds limits. A 600E has quite a few interfaces . You could cable down a lacp-bundle 2 members to your core-switch and if your in a cisco or juniper virtual switch chassis you could easily build redundancy.

FWIW;  You gain absolutely nothing with the additional "transparent vdom" just one more area that could cause issues, another area for management of  policy , and if you have issues, you have very little options to eliminate it from the equation.

And the next obvious reasons ;  "  vdom1/2 and vdomMain you have the same sessions now in triplets  regardless if you just have an "any" policy in the transparent vdom or specific "policy-id"  rules.

think about this more and it would be clear ;

  A host in vdom1 connecting  to www.example.com

     vdom1 = 1 session 

     vdom-transparent = 1 session

     vdom-main   = 1 session 

     total session  = 3  for one client connection.

 A host in vdom1 connecting  to a server in vdom2

     vdom1 = 1 session 

     vdom-transparent = 1 session

     vdom2   = 1 session 

     total session  = 3  for one client connection.

 A host in vdom1 connecting  to a server in vdomMain

     vdom1 = 1 session 

     vdom-transparent = 1 session

     vdommain   = 1 session 

     total session  = 3  for one client connection.

and so on.....No matter what you do,  unless the traffic stays  internal to that vdom, you now have 3 sessions before you even get to your own edge-router. A bad and poor design imho.

Thinking deeper and if your edge space is big enough, emac-vlan would be better if the 3 vdoms ( vdom1/vdom2/vdomMain ) need connectivity to the same edge-router.

Another option that is doable and we just did this 5 years ago is to run the main vdom down to a internal core-router.  So vdomMain, vdom1,vdom2 all connects to this router on 802.1q tags, So traffic between vdom1<>vdom2 goes to the router and do not touch the mainvdom. This saves you  a vdom, this saves having to have a 3rd session, reduce the resources use by firewall overall. Etc.......

This reduce the complexity of the design and make it easier over all to diagnose if and when you have issues. One of my colleague design that (d.bohls) with internal router running in a vrf-lite. As you add more customers/tenant  ( i.e a vdom ) you only have to rip off a new 802.1q tag assign a /31 and move on.

No need for a "ménage à trois" of 3 firewalls just  to send traffic to you edge ;)

And finally in regards to this comment

"only one allow all policy applied so transparent vdom acts as a L2 switch" 

If it's transparent vdom it's still managing traffic at layer3 and upwards. IT IS  NOT ACTING AS A L2 SWITCH in the  tru sense of switching. 

just my 2ct observation and based on my experience. But you have better designs that are simpler

Ken Felix

[FlorianFlux]

Hi Ken,

thank you for your clear and detailed answer, the performance aspect and poor deign are some point i definitely take into consideration.

Colleagues told that the transparent VDOM solution was the only way found, back in the days, to propagate a vlan across several vdoms. There are actually Enhanced-MAC vlans (which i was not aware of until today), that seem to work just fine, i'm testing this in lab right now.

If this works the way i want, this plus your advice will make us save some resources and complexity.

If i get trouble or questions related to EMAC vlan i will post another thread but this really seems like the way to go for what we want to do.

https://docs.fortinet.com/document/fortigate/6.2.0/cookbook/212317/enhanced-mac-vlans

Example 1 is what we achieve using transparent VDOM, i guest we've been doing it the wrong way.

I really appreciate the time you took to answer and your valuable feedback.

Florian