Skip to main content
Zenith
New Member
November 11, 2013
Question

Three interfaces in different VDOMs but same subnet?

  • November 11, 2013
  • 12 replies
  • 29511 views
Hi guys, I' m setting up a 100D with three VDOMs; a root and two customer VDOMs. On the WAN side the 100D is connected to the ISP switch. The ISP typically issues you a subnet to be used for your firewall WAN interface(s) then you can register additional subnets of public IPs and have them routed to the WAN interface IP of a particular firewall. As we wanted the two customer VDOMs to be separate and have their own ranges of public IPs we took three physical interfaces on the FG to act as WAN interfaces, so one physical WAN interface per VDOM. I then asked the VSP for a range of public IPs to use for these interfaces. I added the first IP (1.2.3.4/30) to the root VDOM WAN interface no problem, but when I then try to add the second IP (1.2.3.5/30) to one of the customer VDOM WAN interfaces the FG gives an error saying the IP subnet is in-use on another interface. Is there any way around this or do I need to request separate interface IP subnet ranges from the ISP? They have to setup HSRP IPs and all sorts of stuff on each interface subnet they have to setup, so I' d prefer not to have to do this! Thanks for any thoughts!

    12 replies

    emnoc
    New Member
    November 11, 2013
    A topology drawing would be nice. As for the interfaces can you post the cfgs and have you ensured that the WAN uplinks are in 3 unique vdoms ?
    Zenith
    ZenithAuthor
    New Member
    November 11, 2013
    Thanks for replying! I' ve just anonymised a network diagram I had done and attached it, if you need more details just shout! Just to explain the IP situation a bit better - - The ISP have given us a range like 1.2.3.4/30 with a default gateway of say 1.2.3.5. - If we had three independent firewalls connected to the ISP switch we could set the WAN IP of one to say 1.2.3.6/30, the next 1.2.3.7/30 and finally the next 1.2.3.8/30. (In fact this is the way we do have a couple of physically separate firewalls in there.) - Assuming we want to use more ranges of IPs for servers behind these firewalls we would request a range and ask the ISP to forward it to one of the three WAN IPs above depending on which customer is going to use the IPs. I' ll post a screenshot of the Interfaces page in a moment, but yes definitely separate VDOMs setup. You' ll see in my screenshot I' m trying to set an IP on the port15/port16 interfaces, the error you get is " IP address is in same subnet as the others" .
    Zenith
    ZenithAuthor
    New Member
    November 11, 2013
    Interfaces screenshot.
    Zenith
    ZenithAuthor
    New Member
    November 11, 2013

    Actually if you take a look at this thread - https://forum.fortinet.com/FindPost/99727 there' s a drawing I did half way down which shows the IP addresses and WAN interfaces much better!

    Zenith
    ZenithAuthor
    New Member
    November 11, 2013
    I see this in the FortiOS5 manual which is presumably the root of my problem - " FortiGate unit interfaces cannot have overlapping IP addresses, the IP addresses of all interfaces must be on different subnets. This rule applies to both physical interfaces and to virtual interfaces such as VLAN subinterfaces. Each VLAN subinterface must be configured with its own IP address and netmask. This rule helps prevent a broadcast storm or other similar network problems." But under it this - " If you are unable to change your existing configurations to prevent IP overlap, enter the CLI command config system global and set ip-overlap enable to allow IP address overlap. If you enter this command, multiple VLAN interfaces can have an IP address that is part of a subnet used by another interface. This command is recommended for advanced users only." I guess I just want to understand what the implications are if I do enable ip-overlap, what sorts of topology issues should I be watching out for to avoid broadcast storms etc? I assume it is disabled by default for a fairly good reason but don' t want to discover that reason at 5pm some Friday when it is live :). Thanks again for the replies!
    rwpatterson
    New Member
    November 11, 2013
    Personally, I would back up and see why I would really need to have the same subnet on 3 interfaces... Sounds like a weak network design to me.
    ORIGINAL: Zenith Hi guys, I' m setting up a 100D with three VDOMs; a root and two customer VDOMs. On the WAN side the 100D is connected to the ISP switch. The ISP typically issues you a subnet to be used for your firewall WAN interface(s) then you can register additional subnets of public IPs and have them routed to the WAN interface IP of a particular firewall. As we wanted the two customer VDOMs to be separate and have their own ranges of public IPs we took three physical interfaces on the FG to act as WAN interfaces, so one physical WAN interface per VDOM. I then asked the VSP for a range of public IPs to use for these interfaces. I added the first IP (1.2.3.4/30) to the root VDOM WAN interface no problem, but when I then try to add the second IP (1.2.3.5/30) to one of the customer VDOM WAN interfaces the FG gives an error saying the IP subnet is in-use on another interface. Thanks for any thoughts!
    If you use one VDOM as the gateway and use inter-VDOM links, then you do not need a WAN interface in each VDOM.
    Zenith
    ZenithAuthor
    New Member
    November 11, 2013
    It wouldn' t be a big deal to change it as it is not live yet, but from reading the documentation (admittedly a few months back) it seemed like routing the traffic through the root-vdom wouldn' t really be suitable in this case, correct me if I' m wrong though! The two clients are entirely separate businesses with their own public IP address ranges, various different VPNs, different administration teams etc. So it wouldn' t be a runner for an admin in one company being able to log into the root-vdom and potentially make changes that could cause issues with the other company and visa-versa, which is my understanding of how this would work. Essentially we want them to have two separate firewalls altogether, but the benefit of putting them in the same firewall in separate VDOMs is they can share the HA infrastructure of the two FGs and the cost of this.
    rwpatterson
    New Member
    November 11, 2013
    They wouldn' t need to touch the root VDOM. They would have an Internet handoff from that VDOM in their own that they would use. They would only have access to their side of the inter-VDOM link.
    Zenith
    ZenithAuthor
    New Member
    November 12, 2013
    OK I must have misunderstood that, can you give me a bit more info about how this would work? Will they be able to setup VPNs to their own VDOM, have ranges of public IPs NAT' d in their VDOM etc.? Thanks!
    emnoc
    New Member
    November 12, 2013
    Bob' s suggestion, is correct, you might be better off with one inter-vdom. Do you really need 3 vdom is now the real question? Also how do you plan on any hardware redundancy (2nd 3rd 4th FGT down the line ? ) I would really look hard at why you think you need 3 vdoms. If it to provide management authority to 3 different orgs, than I would agreed with your scope and logic. But if not that, than you need to really look at the extra hassle for vdom' ing your firewall. Traffic is only going to go where your fwpolicies allow it to go :) If this is a true multi-tentant housting firewall, than vdom away :) On the drawing, you had in the other thread ( btw, good job it shed a lot of light and reduce any confusion ) you mention the 3 context don' t necessary have to communicate today but the requirement might come up later, could you just apply fwpolicies between the various lans( vlans ) between vdom context 1 2 3 . FWIW, I would rather leverage the ISP handoffs in a redundant WAN1 + WAN2 , than to see 3 separate handoffs with no redundancy. Now with that said, most people enable multi-vdom & intervdom-routing to actually reduce the number of uplink interface or where the model has a limited number of ports to begin with. A 100D, would not be hamper by the pure number of uplinks. On the Address overlap, try to avoid that if possible, but I don' t think that should be a case in your setup & should not be a issue ( i think ) in a multi-vdom firewall. I' m doing that today with a pair of cisco 5558X and the 2 contexts are in the same uplink subnet sharing a port-channel interface. Neither context knows that the other guy is in the same physical firewall. The ports 15 and 16 in yoru case, should be 2 unique interfaces and in a unique vdom. And can be in the same subnet if I had to guess. Can you share the config system interface and on what your trying todo? And when/where do you get any error messages?
    Zenith
    ZenithAuthor
    New Member
    November 13, 2013
    Hi emnoc, Yeah we absolutely need at least two VDOMs as this is a multi-tenanted setup. I' ve left it out of the discussion to keep things simple, but there are actually two 100Ds in a cluster with redundant switches on the LAN side, and on the ISP side we actually connect to two separate ISP switches (so three connections in each). As I say I don' t mind going for the two VDOMs routing through the root-vdom but my reading of the documentation was that this would require setting routes/fwpolicies in the root-vdom which isn' t really a runner if either tenant needs to log into root to do this regularly. If we can just set it up once and leave it then that would be fine. The other complication is that both tenants have fairly complex needs with probably 10 VPNs each, 64+ public IP addresses each etc. so my question was how this would be possible using just the one WAN interface into a root VDOM? Will each tenant be able to manage the NATing of their IPs and will they be able to terminate VPNs in their VDOM or will they need to terminate in the root-vdom and be routed into each VDOM? I hadn' t actually turned on the overlap-subnet setting so as you think that should be OK (no broadcast storms :) ) I' ll do that and then see if I get any more error messages, pretty sure it will solve my problem of the three interfaces but wanted to check any longterm consequences before doing it, and also interested in the idea of routing through the root-vdom
    ede_pfau
    SuperUser
    SuperUser
    November 13, 2013
    I would definetely recommend to avoid subnet overlap. What my first thought was, do your /30 subnets really NOT overlap? Say, I' ve got a 10.11.12.1/28 from the ISP (as I am planning to subdivide the range into three /30s I will need an address space of at least 4x4 addresses): .0 is broadcast .1 to .14 is hosts .15 is network Now to subdivide, I use a /30 mask which gives me the subnets .0 (usable: .1, .2) .4 (usable: .5, .6) ... .12 (usable: .13, .14) My point is: if you are not very precise with your starting IP address you may get an overlap within one /30 subnet. Your example of using 1.2.3.4 will not allow a .5 and a .6 to be in different subnets, but .4/30 and .8/30 and .12/30 will.