Unable to Correctly Set WAN Port on Fortigate 600D

[bellis1]

We are implementing a Fortinet infrastructure into one of our main buildings. We are essentially building a new network for these users and we are looking for this system to mesh well with our existing devices. I am currently setting up a Fortigate 600D running v5.4.4.

I am trying to set up the WAN port on the Fortigate but every IP I assign to the port is erroring out, claiming that it belongs to our main firewall. The goal is to have a PAT so one external IP address (that belongs to our organization) should do it. At first we were trying to chase down ARP tables from different devices to figure out why that specific IP might be claimed, but as I started to toy around I noticed that even private IPs, externals that didn't belong to us, and IPs I was making up on the spot were claiming to be taken by our main firewall. Is this a bug with the Fortigate Software? 

As a work around, I was able to set the WAN IP to 0.0.0.0/0.0.0.0, disable the port, set the IP correctly to one of our globals, then enable the port but it's triggering all sorts of ARP flaps on multiple devices now, so I'd like to avoid that route. Any suggestions? 

[ede_pfau · Accepted solution]

I know what Bob is suggesting. My feeling is that it won't help, and no, this is not common practice.

There are 3 ways to associate an address with an interface:

- define it as primary address (GUI/CLI)

- define it as secondary (tertiary,...) address (secondary in GUI, others in CLI)

- define a VIP (virtual IP) associated with this interface

All of which would work even as a single measure (ie., a VIP would even work if there is no interface address at all).

But the common way is to define a primary IP on the WAN interface. Throwing an error message could be a bug but could as well be rightful.

One thing to keep in mind is that the FGT is a router. No 2 interfaces may use addresses from the same collision domain. You can check address overlap for instance by looking at the routing table (Routing > Monitor).

Lately, I've had serious trouble with the GUI blocking an address setting because "address overlap with management port setting". Management ports have the special quality that they explicitely allow an address from a otherwise configured subnet, for instance the LAN port.

It turned out the the new "role" parameter in the GUI/CLI caused this. Setting the address in CLI always worked, and setting the role to "undefined" let me set the address in the GUI as well. You might check this.

If you still have problems then please provide more info on the address spaces used: which addresses are assigned to the ports, which netmasks are used (check them!). I guess you use a transfer net between the HQ FGT (WAN port) and the branch FGT.

[tanr · Accepted solution]

I think the key point from ede_pfau was that in general no two interfaces on the FortiGate can use addresses that are in the same subnet.  If they did, since the FortiGate is a router and (mostly) not a switch, you would have some messy routing problems.

*However*, there are some ways to have multiple IPs on the same interface (such as defining secondary IPs on a single interface) or to have multiple physical interfaces (with the same IP) that are all on the same subnet or part of the same vlan (defining them as members of a hardware or software switch, or as an aggregate depending on what you're doing).  If you want to get complicated you can even define multiple VDOMs on the FortiGate and since each VDOM functions like a separate router each could have their own interface with its own IP for each of your subnets.

I think the confusion is that we're not really clear on what you're doing.  If you could provide a simple network diagram of what you want to do, and maybe what the motivation is, we could probably give better answers.

[tanr]

Please note I'm NOT a FortiSwitch expert, so I might not be describing this the best way.  Hopefully someone else will jump in if I give you incorrect info.

If you are going to segregate your VLANs you'll probably want to change their subnets to match CIDR. A calculator that lets you plug in CIDRs or ranges is https://www.ipaddressguide.com/cidr. Remember not to use a VLAN ID of 1 as that is considered the default vlan.

Regarding DHCP and the LAN interface, if you're using FortiLink to the FortiSwitches you don't need to have separate cables for the lan interface. Everything should be able to go over the FortiLink, which acts as a vlan trunk and manages the switches as a stack. With the FortiLink controlling the switches, when you use the GUI to create FortiSwitch vlan interfaces those interfaces are actually getting created on top of the FortiLink interface. Your DHCP servers get created for those vlan interfaces. This might be made a little bit clearer by looking at http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-managing-fortiswitch-330-54/VLANconfi...  which shows the actual CLI objects that are getting created for this. You could make the FortiLink an aggregate interface if you want more bandwidth.

Regarding FortiAPs, if you're controlling them from the FortiGate and creating tunnelling SSIDs you'll want the FAPs on a vlan interface (probably untagged at the port they connect to) for which you've enabled CAPWAP. That will be separate from the SSID interface you create.

I'd suggest trying to test out simple cases before you do all this. Add a single FortiSwitch connected to a single FortiLink port on the FortiGate. Create a single vlan on top of it and set up security policies to let a user connected to the switch out to the wan (only outbound security policies). Add a second FortiSwitch connected to the first FortiSwitch by ISL. Get this working for some simple wired cases before you add in the FortiAPs. Try out the FortiAPs first with bridged SSIDs (so the FortiSwitch port they are on has the CAPWAP vlan untagged and whatever vlan the bridged SSID is tagged), then with tunnelled SSIDs, etc.

One gotcha if you do these simple test cases first is that the FortiGate won't let you change interfaces if they are referenced elsewhere and won't let you change vlan ids either. So converting from a simple config to a more complex one can mean almost starting over from scratch. One way you can partially work around this is to work with interface zones. Any new interface you create goes into an appropriate zone, then your security policies and most of your other work is with the zone. That way you can always just remove the interface from one zone and put it in another.

BTW, a lot of this is in the documentation and cookbook, but it can be difficult to sift out.  I keep hoping an updated version of UTM Security with Fortinet (https://www.amazon.com/UTM-Security-Fortinet-Mastering-FortiOS/dp/1597497479) will come out and include some of these details.

[bellis1]

Been a day or so since I've last responded. My co-worker has been helping me figure this setup out. Just an update of where I'm at...

The FortiGate has basic internet connectivity. We defined multiple VLANs on the FortiLink port. For each VLAN we created, we also created an IPv4 policy which allows access internet connectivity. Each VLAN was tested successfully for pushing out the correct DHCP ranges (and all had internet connectivity). We extended the test to Access Points and defined a range inside of the SSID that the FortiAPs will broadcast. We were able to successfully connect to the SSID and again, get internet access.

This setup was tested using 1 FortiGate, 1 FortiSwitch and multiple FortiAPs. When we tried to introduce a second FortiSwitch, we were unable to manage both switches from the FortiGate.

Port 4 on the FortiGate is defined as a FortiLink and is connected to FortiSwitch1 on port 27. Port 28 on FortiSwitch1 is connected to port 27 on FortiSwitch2. I am unable to see FortiSwitch2 from the FortiGate.

We were able to configure FortiSwitch2 manually and successfully tested both our Access Point VLAN and our pre-defined SSID. We had internet connectivity, but none of it would be possible without connectivity from the MGMT interface. 

Is there a way centrally manage multiple FortiSwitches from the FortiGate? I have seen something about stacking so I want to be clear- each switch will have a separate IP Address and be racked in separate IDF's across the building. These will not be part of a traditional stack. 

[tanr]

Inter-switch links should be automatic.  That is, if your FortiGate has a FortiLink connection to FortSwitch #1, then adding a connection between FortiSwitch #1 and FortiSwitch #2 should automatically register it with the FortiGate to be authorized.  Unless your FortiGate's FortiLink interface has fortilink-stacking disable?

I'm offsite today, so can't easily check my configs.

[bellis1]

Thanks for the response. 

The FortiGate definitely has a FortiLink connection to FortiSwitch1 via Port 4 to Port 27. FortiSwitch1 is displayed from the FortiGate in the "Managed FortiSwitch" section with the a black circle and a FortiLink logo on/around the port itself. Port 28, which is the uplink for FortiSwitch2 is green, but shows neither a circle or a FortiLink logo. 

The FortiAPs connected to FortiSwitch2 showed up for authorization but the switch has yet to. I ran the following commands from the FortiGate via the CLI but nothing has appeared to change. 

config system interface

edit port4

set fortilink-stacking enable 

Based on independent research, I'm starting to think I need to define the FortiLink as a Logical Interface. Does this sound right to you? From the documentation, it appears that this allows both ports 27 and ports 28 on FortiSwitch1 to be dedicated FortiLinks (FortiSwitch1 and FortiSwitch2's uplinks, respectively).

I can only assume this is a trickle down technology which would allow me to define FortiLinks down the line (on switches 2, 3 and 4)? 

[tanr]

Ah, sorry, missed your comment that this wouldn't actually be a stack.  In that case, you won't have any inter-switch links because those would be what handled the stacking (and trunking).

First off, I should ask: Is that what you want?  You don't want to manage the switches as a stack?  With a stack you still control which ports on which switches allow specific vlans, and you can still see the separate switches.  It just makes the management and trunking easier.  If you want to do stacking you'll need to have "set fortilink-stacking enable" on whatever your FortiGate's FortiLink interface is.  With my own stack, the switch ports I have connected by ISL were both set as auto-discovery-fortilink enable.

If you do want to control each switch separately from the FortiGate, then you're looking at example "Single FortiGate managing multiple FortiSwitches (using hardware or software switch interface)" example in http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-managing-fortiswitch-330-54/Stacking.....  In that case, you need your FortiLink interface needs to actually be a hardware or software switch interface, to which you add multiple physical port members.  This is still seen as a single interface, but you can then directly connect to each switch.  You'll also need to have "set fortilink-stacking disable" on your FortiLink interface.  I don't have this configured myself, so haven't done much testing with that setup.

[bellis1]

I come from a Brocade environment where a traditional "stack" refers to 2 or more switches connected together via QSFP+ ports. All of our stacks in other buildings are in the same IDF. You can control each switch and its ports individually from the "active controller", but each stack has 1 single IP address and are treated as a single unit. 

Our five switches will be spanned across five different IDFs in a 100,000+ square foot facility. I would prefer each switch have its own IP address to make for easier troubleshooting but that is not necessarily required. My biggest concern is centralized management from the Fortigate. I still want to be able to control which ports on which switches allow specific vlans. 

I ran a "set fortilink-stacking enable" on the FortiLink interface last week but the ISL between SW2 and SW3 did not auto-configure as it should. With that being said, I have had to reload the FortiGate a few times to make changes to the FortiLink interface so I'm wondering if that would be required. 

Looking at that diagram you provided, I see that all FortiSwitches are connected directly to the FortiGate. This is not necessarily the infrastructure I am looking to implement. A single switch will be directly connected to the FortiGate but that is all. The second switch will be uplinked through the first switch, the third switch will be uplinked through the second switch, the fourth switch will be uplinked through the third switch and the fifth switch will be uplinked through the fourth switch. So technically there will be only one FortiLink configured on the FortiGate itself. The rest of the links will be from FortiSwitches. Is a SW/HW switch still the best option for this setup? 

Thank you so much for the help. You are very much appreciated! 

[tanr]

If you want to chain your switches together as you describe, then that is a stack, just without switches in the same rack.   That is more like the "Single FortiGate managing a stack of several FortiSwitches" example, just without a standby FortiLink and without the redundant ISL from the last FortiSwitch in the stack to the first.  In this config you don't need to use the SW/HW switch on the FortiGate, though your FortiLink and ISLs can still be aggregates to give you more bandwidth.

I'm basically doing the same sort of stack, with just two small FortiSwitches and no standby FortiLink or redundant ISL return loop.  For me, this is really just a proof of concept test to see if this will work for us at our main office.  I'll probably add a third switch to the setup to test both the stack config and the two-tier config.

If you have a stacked (or second-tier) FortiSwitch that isn't getting automatically recognized when you connect it to the first managed FortiSwitch, have you confirmed that the ports being connected have auto-discovery-fortilink enabled?

If you need to set properties for a FortiSwitch port on a switch that is already being managed by the FortiGate you will need to create custom switch commands as in http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-managing-fortiswitch-330-54/Additiona....  Note that the 5.4.x documentation doesn't tell you that to put in a return character you use the special sequence "%0a".

I haven't been able to directly control the FortiSwitches by their IPs once they are managed by the FortiGate, but have used the custom commands as above.  I'm not sure I like the trade off of control vs. visibility.

[Prab]

Hi Bellis1,

Just a comment, you can have 2 or more ports with the same IPv4 address on the FG. ;)

You could use the Software, hardware or an aggregate interface. Basically here you shall group different ports together & they all will have same IP address. Please make sure you understand the limitation of these port types before implementing them.

Thanks & regards,

Prab