Unreliable traffic between segments

[sanderl]

I start a new topic while this problem is still not resolved and since the other topic about this same issue is not accurate anymore. This topic starts with a clear description of the issue including an overview of the situation.

This is the old topic (for reference please do not refer anymore to it to avoid confusion).

The network schematic:

the issue:

If connected wireless to ssid "test99" all is fine. No issues and superfast.

If connected to "ssid" clients experience weird issues.

2 examples:

1. Roblox games do not (never) load, and give an error message.

2. Downloading apps/update from google play store is terribly slow ... 1%,2%,3%...

I deliberately do not (yet include) and config/capture (see also previous topic).

extra information:

I never had any of these issue, for over ~7 years. It started after upgrading to 7.0.11. And if you read the old topic, 2 things constantly cross: software switch vs hardware switch, that issue is now gone. All is connected to 1 hardware switch and VLANs with bridged ssids.

The netgear is configured to tag VLANs on the uplink and to the FAP. furthermore there are no (known) issues in the network other than this...

Please help out in finding if this could be a configuration issue or bug. Thank you!

[sanderl]

Hi Graham,

connected a fresh FortiAP to port 5 on HW-Switch, and ssid (vlan 10) on it. Exactly same problem.

Regarding the captures. Indeed no "strange" behavior.

For the "faulty" one look at the timestamps, it takes a long time, very much packets an NO connection.

For the successful one, very short time, less packets and good connection.

Tell me how to send the config to you. Can I mail it?

[gfleming]

The differnce in timing is 20 secs vs 30 secs... not that much IMO.

In the faulty I do see some ICMP TTL messages. Which leads me back to another experiement I wanted you to try.

Can you change the IP of VLAN 10 on the FortiGate to something random like 172.29.241.1/24 and then manually change one of your VLAN 10 PC's to have an IP in that subnet like 172.29.241.5 and default gateway 172.29.241.1. Please see if that changes behaviour.

As for your config file can you upload it to cloud storage somewhere and then DM me the link?

[sanderl]

Changed the IP address on VLAN10, also used static on the phone and used google 8.8.8.8 and 1.1.1.1 as DNS.

EDIT: to be clear, the phone could browse internet normally but the behavior as with the original IP adress on the fortigate was the same.

No change in behavior...

I will DM you.

[sanderl]

You said:

nothing really standing out.... except a couple things:

1. is there a reason you are using a virtual switch and not just a regular physical switch?

2. can you disable your DDOS policies and see if that makes a difference

For 1:

Yes, I now see this in my config:

end
config system physical-switch
    edit "sw0"
        set age-val 0
    next
end
config system virtual-switch
    edit "HW-Switch"
        set physical-switch "sw0"
        config port
            edit "port5"
            next
            edit "port7"
            next
            edit "port9"
            next
            edit "port10"
            next
            edit "port11"
            next
            edit "port12"
            next
        end
    next
end

But that is weird as I never created that as a softwareswitch. In fact in the GUI it even shows as a hardwareswitch:

For 2, YES! Disabled DoS policies and that instantly solves all issues.

But now... How on earth is that possible to be effective on one VLAN but not the other!?

And what about the difference (introduction) of the issue in 7.0.11. What changed?

What was resolved in this BUG ID in 7.0.11? Did that introduce new errors/problems?

807629

NP7 dos-offload triggers an established TCP session to have synproxy process issues.

(You have my config)

[gfleming]

Excellent! We have progress!!

That bug does not apply to you.

However, for now I would check your DDOS logs and see what is triggering...

[sanderl]

Why would it differ between 2 identical VLANs? How can I see those logs?

[gfleming]

Logs are seen at Log&Report->Security Events->Filter on "Anomaly"

However your DoS policy #2 does not have logging enabled. And IMO it is the one blocking. The thresholds for each anomaly are all set super low at "1" and you are including US GEO IP in your "Geo-Block" source. So yeah. That's going to cause problems.

So either enable logging on that policy and confirm in your logs or re-enable policy 1 which has sane thresholds set and logging enabled and see how things behave.

[sanderl]

I still don't get why it should trigger on vlan 10 and not on 99. The dos policies have no relation with a vlan what so ever? A dos policy is active on the wan interface incoming !?

The "1" is active only on specific geo-ip countries, of which are also used in the policy to block all traffic from... why would that "trigger" anything...?

[gfleming]

You are only blocking "Geo_Block" towards a VIP group.

If a device on your network initiates traffic to anything within that Geo-Block range it will still work no problem (because return traffic is allowed based on the outbound policy)

At the end of the day you have a VERY restricitive DOS policy on your FortiGate. I would suggest following some of the steps I've outlined above.

DoS is being applied and looking at individual sessions to individual endpoints. It's not JUST looking at traffic hitting the WAN port. tcp_dst_session anomaly for example will look at the number of TCP sessions hitting a specific destination. In this case could be a phone or laptop on your network.

And just again to remind you, your "Geo-Block" address group includes US. There are A LOT of servers within the US that could be used as legitimate from within your network.

And without seeing your DoS logs I'm not going to make any further guesses as to why it's working on VLAN 10 and not on VLAN 99. But I've already given you some ideas around this. Up to you if you want to take the time to investigate it on your end.

[gfleming]

my only theory right now as to why VLAN 99 is OK is that the DOS policy is being triggered by VLAN 10 because tehre are a number of devices with sessions already coming from roblox servers.

When you switch to VLAN 99 nothing is triggered yet so initial connection works. I'd be curious to see what happens if you keep a device on VLAN 99 long term with a number of other devices on that VLAN.