Hi all -
Had an incident on my network that I'm having trouble getting to the bottom of.
We have 7 FortiAPs (320c's) connecting to our Fortigate firewall as controller. They're on a private vLAN with no other devices. This morning all 7 disconnected with this pair of errors:
Jan 17 10:05:56 <local7.notice> date=2020-01-17 time=10:05:56 devname=SanJose-HQ devid=FG600B3911600212 logid=0104043553 type=event subtype=wireless level=notice vd="root" logdesc="Physical AP fail" sn="FP320C3X14007549" ap="S-WAP-03" profile="resv-dflt-FP320C3X14007549" ip=10.1.132.106 meshmode="mesh root ap" snmeshparent="N/A" action="ap-fail" reason="Control message maximal retransmission limit reached" msg="AP S-WAP-03 failed." Jan 17 10:05:56 <local7.notice> date=2020-01-17 time=10:05:56 devname=SanJose-HQ devid=FG600B3911600212 logid=0104043552 type=event subtype=wireless level=notice vd="root" logdesc="Physical AP leave" sn="FP320C3X14007549" ap="S-WAP-03" profile="resv-dflt-FP320C3X14007549" ip=10.1.132.106 meshmode="mesh root ap" snmeshparent="N/A" action="ap-leave" reason="Control message maximal retransmission limit reached" msg="AP S-WAP-03 left."
After about 10 minutes they reconnected and started function fine again. None of the APs had rebooted or lost power. What's interesting here (maybe) is that it doesn't appear to be a broadcast flood or anything causing packets to be dropped; instead for several seconds before the outage they all got ping deny responses from the controller, e.g.:
Jan 17 10:05:50 <local7.notice> date=2020-01-17 time=10:05:50 devname=SanJose-HQ devid=FG600B3911600212 logid=0001000014 type=traffic subtype=local level=notice vd=root srcip=10.1.132.106 srcintf="port3" dstip=10.1.132.1 dstintf="root" sessionid=1533065762 proto=1 action=deny policyid=0 dstcountry="Reserved" srccountry="Reserved" trandisp=noop service="PING" app="PING" duration=0 sentbyte=0 rcvdbyte=0 sentpkt=0 appcat="unscanned" crscore=30 craction=131072 crlevel=high
So, their heartbeat pings were getting denied by the default implicit deny rule (policyid=0) but there's no reason for that that I can see. And it's also very odd to me that the problem then fixed itself without intervention. Any idea why the controller would suddenly stop accepting pings from its APs? We had no other network outage at the time, and the regular Fortigate firewall rules all continued functioning normally in the meantime.
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1698 | |
1092 | |
752 | |
446 | |
228 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.