No reliable connection with hardware lan switch and bridged ssid

[sanderl]

Ok hold on, this is going to be hard to believe and to describe. I have troubleshooted a lot and cannot find out where the problem lies. Suddenly I had this vague problems, of sites not loading, DNS not resolving, etc. Maybe related to upgrading to 7.0.10 or 7.0.11. Maybe not.

What does not work (but had always worked like this - for years):

• I have a (existing) hardware switch "lan" with an SSID bridged to that (no VLANs).
• The IP address is on the lan switch and the SSID is bridged.
• Created a new test policy, top placed any/any allow, no filtering, NAT to internet.
• When I connect a mobile to this SSID and start roblox (don't ask - this is a prio 1 for days now) it does not load any game.

What does work:

• I have created a (new) test vlan (99) with an IP address on it, and a test SSID bridged to that vlan (99), connected to the lan switch as my FortiAPs reside there.
• Created a new test policy, under the top placed any/any allow, no filtering, NAT to internet.
• When I connect a mobile to this SSID and start roblox it does load games.

EDIT: Roblox is "the" way of proving/testing above. As described a lot more is not working smoothly, but a refresh of the page will do. Roblox seems to be a lot more  "picky" in the coneection stability.

Both "networks" are giving out the same DNS servers.

I have 6 VLANS connected via the lan hardware switch which all work(ed) well for years. Of which 3 have an IP adress on the VLAN interface and 3 are connected in a software switch with a port.

--> this can also be a separate topic because since this week I discovered the Fortigate does not allow me to select a VLAN anymore as a member of a softwareswitch (!), but this used to work and still works. Nothing to find in any release notes...

I cannot find any mentioning of any change in behavior. Also I have no active subscription on this device (81E) and thus cannot call support.

Is there anything I can do to (more) narrow down this issue?

[gfleming]

So now you're certain it has to do with the firmware upgrade. Before it was  maybe/maybe not. But one thing that is certain is you have an 'old config' that works on 7.0.10 when you reverted. And a 'new config' that you've built/changed since upgrading to 7.0.11. 

So again, I will ask: what changes did you make after upgrading to 7.0.11?

[sanderl]

Before (up to 7.0.10) it worked for years with that config. Then I upgraded (to 7.0.10), it failed. I was not sure at that moment it was the OS. Then I had to do something and started to get rid of software switches (which connected ports with tunneled SSISs, also worked for years). Because thát was suddenly not possible anymore (bridged ssids still are).

I started to change to use VLANs and combine them with all (but the ssid bridge to lan) FortiAPs.

So test99 was bridge to vlan 99, worked both on the lan switch and new hardwareswitch.

Then vlan98 with ssid 98, bridge directly to the lan switch (worked not!). Bridged to the new hardware switch, worked!

Then you advised me to revert 1 firmware. And then it all worked and I got my older software switches back.

So to answer your question. All but the bridge ssid to the old lan (hardware) switch I moved from a software swith to a hardware switch with a vlan.

I am 100% sure the firmware is the issue.

[gfleming]

There's just not that many changes from 7.0.10 to 7.0.11. Review the release notes. Do you see anything that applies to your config? I don't... If you were going from say 6.2 to 7.0.11 sure I would say most likely it's a firmware/config thing or you didn't follow the upgrade path. But you've gone up one tiny point release with no major changes to the code or features or behaviour of the OS.

I don't even see how you could have software switches working in 7.0.10 and not in 7.0.11. Can you show the config from 7.0.10 so I can validate it?

[sanderl]

I can share that, but not publicly... suggestion?

[gfleming]

Up to you. Or just show the relevant stuff in the interface configurations that doesn't have anything sensitive in it?

While you're at it can you post a diagram of your network? It's getting a bit muddled talking of lan switch, hwswitch, test etc etc. Losing track of what is what.. thanks.

[sanderl]

Sorry, I am not sure what config you are now asking for... In my opinion it does not really matter. Its just the existing harewareswitch (lan) that introduced the problems after upgrading to 7.0.11. But please let me know.

BTW: This was not exactly the setup at the start of the topic. but the bridging of the main ssid to the lan switch is not changed (and does still not work correctly in 7.0.11 while is does in 7.0.10 (directly).

[gfleming]

I"m just asking for the software switch configuration that you were talking about that worked on FOS 7.0.10 and not on 7.0.11.

Also just curious why do you have two hardware switches? Why not just have all VLANs under one switch and connect your switch uplink(s) to it?

Also just so you know the config you have in place will result in all L2 traffic transmitting through the FGT—it might cause some resource contention. You might be better off doing a STP ring topology with one of the uplinks blocked. That's for another day, though....

Does the FAP work in the managed switch with all SSID and bridging scenarios? Is it only a problem on the unmanaged switch?

When you say you are testing the SSID by bridging directly to the hw switch, how are you accomplishing that?

[sanderl]

Here you go sir:

<I"m just asking for the software switch configuration that you were talking about that worked on FOS 7.0.10 and not on 7.0.11.>

This is a snap of the config where tunneled ssids were connected to a softwareswitch together with a port.

config system switch-interface
    edit "IP CAM Segment"
        set vdom "root"
        set member "IPSecMon" "IPSecMon2" "port3"
    next
    edit "IOTSOL"
        set vdom "root"
        set member "IOTSOLar" "VLAN20-IOTSOL"
    next
    edit "Isolated-LN"
        set vdom "root"
        set member "Isolated-Labour" "VLAN27-IsoLN"
    next
end
config system interface
    edit "wan1"
        set vdom "root"
        set mode dhcp
        set allowaccess ping
        set type physical
        set alias "Internet"
        set monitor-bandwidth enable
        set role wan
        set snmp-index 1
        set dns-server-override disable
    next
    edit "wan2"
        set vdom "root"
        set mode dhcp
        set allowaccess ping fgfm
        set type physical
        set role wan
        set snmp-index 2
    next
    edit "dmz"
        set vdom "root"
        set type physical
        set role lan
        set snmp-index 3
    next
    edit "ha"
        set vdom "root"
        set type physical
        set snmp-index 4
    next
    edit "port1"
        set vdom "root"
        set type physical
        set snmp-index 41
    next
    edit "port2"
        set vdom "root"
        set type physical
        set snmp-index 32
    next
    edit "port3"
        set vdom "root"
        set type physical
        set snmp-index 17
    next
    edit "port4"
        set vdom "root"
        set type physical
        set snmp-index 42
    next
    edit "port5"
        set vdom "root"
        set type physical
        set snmp-index 43
    next
    edit "port6"
        set vdom "root"
        set type physical
        set snmp-index 18
    next
    edit "port7"
        set vdom "root"
        set type physical
        set snmp-index 36
    next
    edit "port8"
        set vdom "root"
        set type physical
        set snmp-index 12
    next
    edit "port9"
        set vdom "root"
        set type physical
        set snmp-index 11
    next
    edit "port10"
        set vdom "root"
        set type physical
        set snmp-index 10
    next
    edit "port11"
        set vdom "root"
        set type physical
        set snmp-index 31
    next
    edit "port12"
        set vdom "root"
        set type physical
        set snmp-index 33
    next
    edit "modem"
        set vdom "root"
        set mode pppoe
        set status down
        set type physical
        set snmp-index 5
    next
    edit "naf.root"
        set vdom "root"
        set type tunnel
        set src-check disable
        set snmp-index 30
    next
    edit "l2t.root"
        set vdom "root"
        set type tunnel
        set snmp-index 44
    next
    edit "ssl.root"
        set vdom "root"
        set type tunnel
        set alias "SSL VPN interface"
        set snmp-index 6
    next
    edit "IP CAM Segment"
        set vdom "root"
        set ip 192.168.3.254 255.255.255.0
        set allowaccess ping
        set type switch
        set device-identification enable
        set role lan
        set snmp-index 8
    next
    edit "IOTSOL"
        set vdom "root"
        set ip 192.168.4.254 255.255.255.0
        set allowaccess ping
        set type switch
        set device-identification enable
        set lldp-transmission enable
        set role lan
        set snmp-index 21
    next
    edit "Isolated-LN"
        set vdom "root"
        set ip 192.168.27.1 255.255.255.0
        set allowaccess ping https http
        set type switch
        set device-identification enable
        set lldp-transmission enable
        set role lan
        set snmp-index 15
    next
    edit "lan"
        set vdom "root"
        set ip 192.168.1.254 255.255.255.0
        set allowaccess ping https ssh http fgfm fabric ftm
        set type hard-switch
        set stp enable
        set device-identification enable
        set role lan
        set snmp-index 7
        set auto-auth-extension-device enable
    next
    edit "2.4G"
        set vdom "root"
        set type vap-switch
        set alias "b"
        set role lan
        set snmp-index 16
    next
    edit "5G"
        set vdom "root"
        set type vap-switch
        set alias "b"
        set role lan
        set snmp-index 19
    next
    edit "IOTSOLar"
        set vdom "root"
        set type vap-switch
        set role lan
        set snmp-index 20
    next
    edit "LoopbackSSLVPN"
        set vdom "root"
        set ip 192.168.40.254 255.255.255.0
        set allowaccess ping
        set type loopback
        set role lan
        set snmp-index 35
    next
    edit "Isolated-Labour"
        set vdom "root"
        set type vap-switch
        set role lan
        set snmp-index 39
    next
    edit "VLAN27-IsoLN"
        set vdom "root"
        set role lan
        set snmp-index 40
        set interface "lan"
        set vlanid 27
    next
    edit "VLAN20-IOTSOL"
        set vdom "root"
        set role lan
        set snmp-index 13
        set interface "lan"
        set vlanid 20
    next
end

<Also just curious why do you have two hardware switches? Why not just have all VLANs under one switch and connect your switch uplink(s) to it?>

Because I am now in the process where I am getting rid of the above. And as described many times before :) I have created a new hardwareswitch to where I have moved the softwareswitch IP interfaces. the "leftovers" are still connected to the old hardwareswitch and yes they need to move to the new HW-switch. But this needs proper timing and then I need to move all APs and other wired connections from "lan" to "HW-Switch".

<Also just so you know the config you have in place will result in all L2 traffic transmitting through the FGT—it might cause some resource contention. You might be better off doing a STP ring topology with one of the uplinks blocked. That's for another day, though....>

I know, 1st things first.

<Does the FAP work in the managed switch with all SSID and bridging scenarios? Is it only a problem on the unmanaged switch?>

FAPs worked for 5 years via unmanaged switch, with the "main SSID" bridged to vlan 0 (the hardware "lan" switch. Up until v 7.0.11 there it broke.

Another FAP connected to the managed switch also works well, but obviously via another management vlan and via trunks etc. --> not the issue now.

<When you say you are testing the SSID by bridging directly to the hw switch, how are you accomplishing that?>

So:

Don't ask too complex questions... the problem is very simple and it only is there in 7.0.11. In 7.0.10 it was gone directly. Nothing fancy here...

BTW, this forum software is dragon.

[gfleming]

Can you show the configs for the VAPs too?

I'm asking complex questions because to be honest it's not clear when you describe things. For example, you say refer to things like 'the hardware "lan" switch'. Are you talking about the Netgear physical switch or the hardware switch on the FortiGate named "lan"? If it's the FortiGate lan hardware switch then "bridging the AP directly to the hw switch"  IMO makes more sense to just say you are bridging to VLAN 1 (NOT 0!!) or in other words, the native VLAN.

And please confirm prior to this you were running on 7.0.10 for a long time and things were working? Or did you come from an even earlier OS release?

[sanderl]

So long: in all versions above worked. it only stopped in 7.0.11. and when reverting to 7.0.10 it works.

Again, I am talking here about plain simple bridging ssid (vlan 0!) default to the hardwareswitch (lan).

This firewall only goes back to 6.4.5. But the config was inherited from another Fortigate (60E) and goes back working up to even 5.4 I believe so... 6.0, 6.2, 6.4 then 7.0

I understand its not too clear. Its not easy to describe this.

I Only talk about fortigate when I talk about lan switch. My lan switch is called "lan" and it's a hardware switch.

See the screenshot. By default the is no vlan (0) when you create a bridge vlan. It just puts the taffic untagged on the wire. In the new situation indeed I have vlan 1 in my managed netgear, as a native vlan and there the ap is in the native vlan 1 to be managed. There I am going to create vlan 10 as my new "lan" segment (what is now the lan switch - on the fortigate)

BTW, I really appreciate you taking the time to help me. I hope al is now clear.

Please see the screen shot for the SSIDs... there you see vlan0 for the main ssid (bridged to the lan-switch, on the fortigate), this worked for years, but stopped (not totally but partially! read the roblox - and others problems) in 7.0.11

The VAPs:

config wireless-controller vap
    edit "2.4G"
        set ssid "24G"
        set passphrase ENC nononsM1bSAVaNJKNfHw52IYfM1pJh4/u1oNRTiXQXVIJIxUq9KEPpt1clrIHhNKhnh7ZLuYBeYLm4tFfE+6etO/aojWnc4X6RwZoCLyNkWfpjw2CJ3LUOMVeLxOHQYj99u33yLy+5FgyBLfy7sVvrpOU/1DmugyxImEnUMbpMHuut7d7bo2QF2dBUx+9ovrlYEKw==
        set local-bridging enable
        set local-authentication enable
        set schedule "always"
        set alias "b"
    next
    edit "5G"
        set ssid "5G"
        set passphrase ENC nonoiscg43VFwt8JctrS9BJHccbXPb9JD38HsREFWK5cT+tzE9gxxT1j8FDB/AKltqZu3UAwExg9uR6F610wj2jEf8COryS1Iot7J58zquZtPRMEikA3dlymyMB9BDgxb2Q4j8AlXda4pHfvYOrID5YIdaC254aY+46wStayHAspaiDuJPOmxjTVpaYfOnFE9eA==
        set local-bridging enable
        set local-authentication enable
        set schedule "always"
        set alias "b"
    next
    edit "IOTSOLar"
        set ssid "IOTSOL"
        set passphrase ENC nononENOL4cTNikBamuySHDyGyfc55v0JNRyl92uD1NrsLP5hywsnthcIjqwd0hma8biipIE+jtMpf3EwkH/RwfLldFzFSuN7ZSqjv87yEpaN2CDOIOUn8fWjfH1ggv4/sGjVUerc9OesjZgVAYnEcxGko3rpjY6HOgJtwHLhE4lSSbjDD1dbKM/9rJIvN80N+FXg==
        set schedule "always"
        set quarantine disable
    next
    edit "IPSecMon"
        set ssid "IPSecMon"
        set passphrase ENC nonoZ5BKGsomx0olfkBUtkyYI/hH0sILJJC8RuhJqAulL/3sD/d88Qif10842CsFZXP142Z3MNEsBm/QdsbfozrcmGJj/fgZvrZl84yUQ4+EUAST/kQm8m6J6PJ43gWUoZRWukce/U9Ul9lK98nrIPeId9L/XGltA3vnqanIs5KyJ1adulUm/wZvsTBn3bXphjH1Q==
        set schedule "always"
        set quarantine disable
    next
    edit "IPSecMon2"
        set ssid "IPSecMon2"
        set passphrase ENC nonoFuUQ90TvmAP2GUJRbebR8tCmMGxJYXwpIeP1m1vbfFfVcsZ71zEgGTmA+qUKOOSuYH9xkLw1PjH+dXYbBGCA84uOCByt1F8HjqQqGyUHPe4OVV7LstKpD85twmyrivlLfl0gN7M2Rl+I6gguiBKc4bR1rtG4KdTP92LVte1fJbialQYBTFyEIahSOllkQgwdHw==
        set schedule "always"
        set quarantine disable
    next
    edit "Isolated-Labour"
        set ssid "Isolated-LN"
        set passphrase ENC nonoQSKgf/QQzrxiB75TPQsHG7UyicR+lSE3kwXILUvT9ZlNRPlvff7wNoiyBa2bfdYMfgk7983SlhmFI+5IHkJ00Pj5Z2M1bkrloLA2weXZJf3zHc9kVmuoX9RWMZGp+47esd1tFIUGClj8EL/C+TUiSVW/V/aLEYr/3ErrBbpdKo1kqBi6mISB6ZwEZo9gekLcKw==
        set schedule "always"
        set quarantine disable
    next
end