VLAN Switch Mode with CLI on FGT 100D 5.4.1

[tanr]

Hi All,

TLDR;

  Anybody familiar with the "VLAN Switch Mode" that is supposedly accessible through CLI only for the FGT 100D?   A usable example or set of CLI commands would be great.

More Details:

I'm spending my weekend doing initial setup of a FortiGate 100D and 300D, to replace older (non-FortiGate) hardware at two locations, both of which have multiple managed switches with a number of vlans.  This is all with 5.4.1.  The two locations have an always-on vpn connection.  Everything is already up and running with the old hardware.

The 100D is going to the remote site, with only two small managed switches and a smaller number of vlans.

My initial plan for the 100D was to remove most of its physical ports from membership in the "lan" hard-switch interface, create appropriate vlan interfaces as children of the ports (multiple in some cases so it can be used as a trunk), and connect to the switches in exactly the same way.  However, it seemed a waste to use all those separate switch ports when the 100D had plenty itself...

I've scanned through the forums and found plenty of references telling me that a FortiGate's vlan interfaces can only send and received tagged packets, but I also ran into a few documents that specifically referred to the 100D and 200D and described a "VLAN Switch Mode", that seemed to imply that a hardware switch on the 100D or 200D could be set to have a particular vlan, but with an untagged trunk port.  This supposedly is doable only from CLI.

I've searched the following documents and posts, among others, but haven't found any method that works in 5.4.1 to change an existing switch with type hard-switch to type switch-vlan.  Similarly, attempting to create a new switch object with type switch-vlan also fails.  (I can post the attempts and failures if needed.)

Tech Note that describes VLAN Switch Mode for 5.4

  http://kb.fortinet.com/kb/documentLink.do?externalID=FD37588 

Ken Felix blog post the describes this, but seems to only be controlling a FortiSwitch

  http://socpuppet.blogspot.com/2015/01/fortigate-switch-controller.html

FortiOS 5.2 Forum Post regarding VLAN switch mode, with mention of a trunk

  https://forum.fortinet.com/tm.aspx?m=127058

Before I get to the point of exhaustively trying combinations and posting the many errors they generate, has anybody successfully set up a 100D or 200D with a switch of type switch-vlan?  Did it still force all switch ports to be vlan tagged, or did it allow untagged?  If it allowed untagged, please let me know the CLI commands you used.

Probably just chasing ghosts, but thought I'd check.

[nswetland]

I would like to know more about this, too..   I look at this link: http://kb.fortinet.com/kb/documentLink.do?externalID=FD37588 but it doesn't really tell you what they are used for exactly.

I also read through what others have suggested here: http://cookbook.fortinet.com/redundant-architecture/ about the vlan-switch configuration, but again, it didn't really seem to explain what its for.

The primary things I am trying to figure out is creating trunk ports that I can send over the POE ports to my Access Points, since I have SSIDs based on VLAN tags...    But I also want the same VLANs to have a trunk to the physical ports.   In an ASA it was as simple as "switchport trunk allowed VLAN X,Y,Z" and then "switchport mode trunk" on the ethernet interfaces...    Is this what Switch-Vlan is for?

[tanr]

I think that cookbook article is only valid for 5.2.x.  It refers to system>global>internal-switch-mode and config>system>global>virtual-switch-vlan, neither of which exist in 5.4.x.

With the FGT you can create multiple tagged VLAN interfaces on top of a single physical interface, creating a trunk port that is restricted to just those VLANs.  Your switch and AP need to be able to handle tagged packets, though.

From the docs: http://help.fortinet.com/fos50hlp/54/Content/FortiOS/fortigate-system-administration-54/VLANs/VLANs%...

If you're generating VLAN ID's dynamically at the AP, that's a whole different problem, though.

[nswetland]

Thanks for the reply.

So, I am basically going to create, say VLAN 10,20,30 underneath port 16?

Say VLAN10 is my native VLAN, Do I need to assign IPs to the VLANs, and leave port16 at 0.0.0.0 and then all the devices behind that use their specific VLAN ip as their gateway?  

e.g.

PORT16:  IP = 0.0.0.0/24

VLAN 10 = 10.0.10.254/21

VLAN 20  = 10.0.20.254/24

VLAN 30 = 10.0.20.254/24

And, then if I want to also have wired clients on, say port2 that goes into my workstation switch...  I want to trunk VLAN 10,20,30 on that as well...  But say I apply VLAN 10 to port2, I try and apply 10.0.10.253/21 for the IP on VLAN10, it conflicts with Port16-VLAN10...  Do I leave the IP address blank for each iteration of the VLAN past the first one, and then just still point all the client GWs to the original definition (in my example, where I defined it on Port16)?

I am used to being able to just Create a VLAN interface with an ASA, and apply it to any number of physical interfaces I want, and the VLAN just references the original VLAN interface definition...  

I try to look for documentation on this, but it seems they always want to refer back to a FortiSwitch (not going to happen), or use different terminology or something...

It is very frustrating that we cant define VLANs and then attach them easily to the interfaces, and once we define a VLAN, if you need to make a change to it, you basically have to delete the one you made, and re-create it with your changes...  Those seem like pretty common features someone would want.

Have you made any headway with your original question?  What purpose is this VLAN Switch Mode option?  I ran into this thread when I was trying to figure out my problem, thinking it was a feature that I could b use...

[tanr]

Note that this is all on 5.4.1.  And yup, the non-movable, non-changeable after creation aspect of FGT vlans is pretty painful.  As is the way things get locked as soon as they are used, making moving an in-use interface into a zone almost impossible.

I believe you can leave the physical port you're using as a trunk at 0.0.0.0 0.0.0.0 (not /24). 

The vlan interfaces under the port (type=VLAN, interface=port16) each get their own IP and subnet.  They can each function as a DHCP server.

On the managed switch side (not the FGT), the switch port you connect to should be set to only accept tagged frames for those ports.  Generally, then, you'd set most other switch ports to only allow one specific vlan, and to only accept UNTAGGED packets from their connected hosts.  That makes it difficult for a compromised host to spoof a different vlan.

The exception would be the switch port connected to your AP (assuming your AP is vlan aware), which should again only accept tagged packets for the specific vlans.

The above is all without using the 100D-200D hardware switch interface (with 5.4.1 you get a default hardware switch called "lan" that holds port1-port16).  Changing such an internal switch to "VLAN Switch Mode" was what I had hoped would give the possibility of having the equivalent of a simple managed switch (like the description above).  Looks like not, though.

Setting up multiple ports on the FGT with the same vlan ID and same subnets is going to cause problems (like loops).

Even without matching vlans and subnets, if you've got multiple ports inside a FGT hardware switch object that connect to the same switch, you could still get loops. You can turn on STP for the switch object if needed - I think that gives MSTP, not RSTP.  I'm not too familiar with STP on the FGT's -- I usually try to avoid needing it!

If you need more bandwidth for a particular VLAN, you might try 802.3ad link aggregation with vlans.  Note something I've tried, though.

Definitely not expert on any of this, so anybody with more know-how feel free to correct me!

[nswetland]

Thanks for the info.  One of the reasons I want to assign a VLAN to multiple ports is for my Access Points.  They are POE and VLAN aware, so I trunk out 3 or 4 VLANs to each device from a POE switch to them, and I was hoping I could reclaim those switch ports on my main switches and just use the POE ports on my 200D.  I dont believe this should cause a loop, as only one physical port per VLAN would be plugged into my actual switch.  It was one of the reasons I bought this device, and one of the things I asked the salesperson about before I bought it, so it would be nice to have it working as I was told it would.

Ideally, I would set up the FGT ports identical to my old Cisco ASA and just swap them out, save for the reconfiguration on the WiFi APs.  I might have to reconsider some of my network design as a result, which is very unfortunate.  I just got this unit, and it is my first FortiNet device, and I am already second guessing the whole ecosystem.

[tanr]

I played with this a little more on the 100D.

You can create a hardware switch interface that contains multiple physical ports, then create one or more vlans on that hardware switch.  All ports within the hardware switch will then match the vlans, without the conflict of having multiple separate ports defined with the same subnet.  All the ports would be tagged, not untagged, so this doesn't solve my issue.

I think you could create a hardware switch like that, turn on STP for it, then plug its ports into each of your APs, and into your switch.

My guess is that would work connecting trunk ports that include vlans 10, 20, and 30 into each AP and the switch.

However, if you want to use the FGT more like a fully managed switch, with one port trunking vlan 10 and 20, another trunking 20 and 30, etc., I don't know how to do that with the FGT.  Anybody else know?

If you don't find a solution, or get a better one from the forums, I'd open a ticket with TAC and see what they have to say.

[vinch100]

For the record and because I had the request from one of our customer, it is possible to achieve that on 100D series (hopefully 140D-PoE as well because that 's the model I just ordered) and 200D series.

I confirm that the following configuration is working on our own 100D :

- Create new VLAN interface on port 11, tag 110, name it "VLAN110_port11" or any other name

- Create new Software switch interface including "VLAN110_port11" and "port12"

- Plug access point (+power injector) with VLAN 110 tagged on port 11

- Plug laptop, untagged, on port 12

=> Switching works.

I'll confirm that it also works on 140D-PoE asap

[tanr]

@vinch100, it looks like you're running 5.2.x instead of 5.4.x?

So untagged packets from the laptop on untagged non-vlan port 12 are going out VLAN 110 tagged port 11 with vlan tags added without any special security policy allowing this?  And it works the other direction as well, with tags getting removed?

At best that sounds like adding the non-vlan port 12 to the switch made it function like a untagged vlan port that is forcing packets to VLAN 110.  At worst that is breaking the separation you would get on a normal switch with different vlans, where it would not allow that communication.

Or do you have some specific security policy rules allowing this?  Or maybe I'm just missing something about the way the Software Switch works in 5.2.x.

[vinch100]

Yes running 5.2.10, we will upgrade to 5.4 in the next few weeks.

To clarify : when you build a software switch you can see it as a VLAN interface on a standard switch : every interface included in the software switch will be part of the switch, regardless of their tagging configuration (at least on 100D and 200D series)

In my config that means that the interface "port11" can be in any other subnet (or even another software switch) and will be treated a a separate security interface.

When I ping from the port 12 to interface "VLAN110_port11" while looking ar "port11" I only see 802.1Q packets :

MyFW # diag sniffer packet port11 '' 4
interfaces=[port11]
filters=[]
pcap_lookupnet: port11: no IPv4 address assigned
4.306491 port11 -- 802.1Q vlan#110 P0
4.306775 port11 -- 802.1Q vlan#110 P0
4.307486 port11 -- 802.1Q vlan#110 P0
4.308241 port11 -- 802.1Q vlan#110 P0
6.493099 port11 -- 802.1Q vlan#110 P0
7.499202 port11 -- 802.1Q vlan#110 P0

The VLAN interface "VLAN110_port11" will be switched with "port12" without going through the security check process (as a proof I just ran a diag debug and nothing shows up) :

MyFW # diag debug flow filter clear
MyFW # diag debug flow filter daddr 10.110.1.2
MyFW # diag debug flow trace start 10
MyFW #

=> Only packet going from the software switch interface (called "HybridSwitch" in my conf) will be processed against security rules

Diag sniffer during a ping shows the packet going out on port 11 (but doesn't display the tag even if it's present, which is somehow confusing) :

MyFW # diag sniffer packet any 'host 10.110.1.2' 4
interfaces=[any]
filters=[host 10.110.1.2]
3.097889 port12 in 10.110.1.100 -> 10.110.1.2: icmp: echo request
3.097898 VLAN110_Port11 out 10.110.1.100 -> 10.110.1.2: icmp: echo request
3.097902 port11 out 10.110.1.100 -> 10.110.1.2: icmp: echo request
3.097904 eth0 out 10.110.1.100 -> 10.110.1.2: icmp: echo request

I hope that it's more clear ?