| Description |
This article describes the procedure to troubleshoot configuration sync issues and the possible remedies, for FortiAP Access points that are managed by a FortiGate (with the integrated Wireless-Controller functionality). There could be certain scenarios in which the configuration changes made on FortiGate might not reach the FortiAPs, due to config sync issues, capwap issues, etc. |
| Scope | FortiGate, FortiAP. |
| Solution |
Table of Contents: Wireless Event logs in FortiGate and FortiAP Common Event log errors related to config sync issues Review crashlogs in FortiGate and FortiAP Review AP configs in FortiGate Review errors in FortiGate Wireless-Controller CLI Review system resources utilization FortiGate Wireless-Controller debugs
This article describes scenarios where FortiAPs are showing online and connected to FortiGate, but AP-related configurations on FortiGate are not in sync with FortiAPs, and new config changes, like updates to SSID config from FortiGate, are failing to get installed/take effect on the FortiAPs. The first step is to ensure the FortiAP version is compatible with the FortiGate/FortiSwitch firmware versions and review the compatibility guide, especially during upgrades of any device in the network. If there was a recent upgrade of FortiGates/FortiSwitches, it might have become incompatible if FortiAP is running an older, incompatible version.
FortiAP and FortiGate 7.x compatibility matrix FortiSwitch and FortiGate 7.x compatibility matrix
Wireless Event logs in FortiGate and FortiAP. The WiFi event logs in the FortiGate provide useful information on any CAPWAP errors while the FortiGate's wireless controller is trying to manage the FortiAPs, like config sync after new config changes. In the FortiGate GUI, go to 'Log & Report' -> System Logs -> Wi-Fi Events as shown in the example below, and review any critical/warning logs indicating possible issues.
Common Event log errors related to config sync issues. Here are some common event log errors related to FortiAP config sync issues and the possible reasons:
05-04-2025 21:02 0 ap ap-status ap-fail 80211 WLAN DEL error AP <FortiAP1> failed 05-04-2025 21:02 0 ap ap-status ap-fail 80211 WLAN ADD error AP <FortiAP1> failed
Indicates possible with capwap tunnel being unstable between FortiGate & FortiAP, review remaining logs around the time of these errors, and additionally enable debugs discussed later in the section to identify the cause of the issue.
05-04-2025 02:43 0 ap ap-status ap-fail Received unexpected DISCOVERY REQ, session tear down AP <FortiAP1> failed
Indicates that discovery requests are being received from the FortiAP even though there is already an active CAPWAP session with that AP. These discovery requests might cause the existing session to be torn down and re-created.
05-04-2025 02:43 0 ap ap-status ap-fail Control message maximal retransmission limit reached AP <FortiAP1> failed
Indicates that the CAPWAP packet exchanges are not stable and FortiGate is unable to maintain connectivity to FortiAP.
logdesc="Physical AP fail" action="ap-fail" reason="ECHO REQ is missing" msg="Failure happened on FortiAP1."
This error indicates that FortiGate is unable to maintain keepalives from the FortiAP. The ECHO REQ and RESPONSE message exchanges are not complete, and are likely caused by unstable connectivity. A packet sniffer for capwap packets can be done to analyze further.
FortiAP diagnostic logs from within FortiGate GUI under 'Managed FortiAPs' section. Logs directly from FortiAP can be viewed from within the FortiGate by navigating to WiFi & Switch Controller ->Managed FortiAPs -> Select the FortiAP -> Diagnostics and Tools.
Review crashlogs in FortiGate and FortiAP. In the FortiGate CLI, using the 'diagnose debug crashlog read', check for any crash log entries for 'application cw_acd' and entries like 'cw_acd previously crashed 1 times'. cw_acd is the wireless controller daemon in FortiGate, and crashes with this daemon could indicate FortiAP connectivity/config-sync issues. Open a ticket with Fortinet TAC with the crashlog output to investigate further.
Similarly, check for crashlog entries in FortiAP CLI using the command "diag_debug_crashlog read", and share with Fortinet TAC for further review.
Review AP configs in FortiGate. If the event logs do not indicate an issue, as a next step, review the current SSID configs on FortiAP and check what is missing. Log in to FortiAP via SSH from FortiGate and use the command 'vcfg' or 'cw_diag -c vap-cfg' to review the SSID and FortiAP profile configs, compare it with the corresponding configs in FortiGate, and compare to see if there is any difference between the two.
FortiAP# vcfg
Or:
FortiAP-431F # cw_diag -c vap-cfg -------------------------------VAP Configuration 1---------------------------- Radio Id 0 WLAN Id 2 Corp-SSID ADMIN_UP(INTF_UP) init_done 0.0.0.0/0.0.0.0 unknown (-1) vlanid=0, intf=wlan02, vap=0x1bdb0a8, bssid=d4:76:a0:db:ea:0a 11ac_ss12: mcs0/1 mcs1/1 mcs2/1 mcs3/1 mcs4/1 mcs5/1 mcs6/1 mcs7/1 mcs8/1 mcs9/1 mcs10/1 mcs11/1 mcs0/2 mcs1/2 mcs2/2 mcs3/2 mcs4/2 mcs5/2 mcs6/2 mcs7/2 mcs8/2 mcs9/2 mcs10/2 mcs11/2 . . .
Verify that there are as many VAP config entries as there are [Radios X SSID] entries configured in the FortiGate. If any entry is missing, it indicates some part of the config has not been synced with the FortiAP. Also, check to see that the corresponding WLAN IDs have "ADMIN_UP" status as shown below.
FortiAP# cw_diag -c vap-cfg | grep WLAN
Review errors in Fortigate Wireless-Controller CLI. There are several diagnostic wireless controller CLIs available in FortiGate that can be used to quickly check errors that could be affecting communication with FortiAPs. Review the configs in CLI first to ensure it is as intended using the CLI command 'show wireless-controller vap'.
Use the below CLI command to check for any errors:
FortiGate# diagnose wireless-controller wlac -c wtp FortiGate-81F #
Here are some common errors seen with this diagnostic command:
FortiGate# diagnose wireless-controller wlac -c wtp | grep failure last failure : 34 -- 80211 WLAN DEL error
The possible reasons for this error are the same as explained in section 2, FortiAP L2/L3 connectivity issues.
FortiGate# diagnose wireless-controller wlac -c wtpprof 231G | grep country Issue: country name : --
If the country name/code field is empty or incorrect, it could cause connectivity issues or config-sync issues. Ensure these fields are accurate and not empty.
The following is what an output should look like (good example):
cfg country : 250
FortiGate# diagnose wireless-controller wlac -c wtp | grep failure
Control message max retransmission errors indicate possible task completion failures or even connectivity issues with FortiAP., review the event logs and apply packet sniffers on the FORTIGATE to analyze further on possible causes for the issue.
FortiGate# cw_diag -c radio-cfg Radio 0: Failed
Ensure the Radios have not gone into a failed state, restart the FortiAP or factory reset to try to recover from the issue, and then attempt to push the configs from FortiGate again by authorizing and adding the FortiAP.
Review system resources utilization. Check for any overutilization of system resources like CPU/memory/storage on both FortiGate and FortiAP using the below CLI commands. This could cause the cw_acd daemon to not get enough cycle cycles (or the cw_acd daemon itself is the top CPU/memory consuming process)and cause config-sync and other issues with FortiAPs.
Look for the cw_acd daemon to see if it is busy and hogging system resources.
FortiGate# get system performance top Run Time: 210 days, 0 hours and 45 minutes cw_acd 275 S 2.3 2.9 3 node 235 S 1.3 1.8 3 FortiGate# get sys performance status
Note: Restarting cw_acd daemon in the FortiGate might be a possible remedy, but upon restarting this process, all the FortiAPs will get disconnected and reconnect.
Use the CLI command below to check the status of CPU and memory utilization on the FortiAP. Use the FortiGate GUI option to SSH to FortiAP or directly console/SSH to FortiAP to run the commands below.
FortiAP# cw_diag sys-performance CPU Load : 6% Memory Usage: 34%
FortiAP# otop -n 3 -d 3 Mem: 352820K used, 538468K free, 1252K shrd, 15208K buff, 59912K cached
FortiAP# top
Example output of the 'top' CLI command on a FortiAP:
Monitor the resource utilizations over several minutes with the above CLI, top processes in the list, and any anomalies in the above output.
FortiGate uses the CAPWAP protocol (Control and Provisioning of Wireless Access Points) to manage FortiAPs over two channels:
Use the packet sniffer on FortiGate with filters as shown below and verify the bidirectional CAPWAP control and data traffic flows. In the example below, FortiGate wireless-controller (internal3) IP address is 172.16.218.1, and the FortiAP IP address (DHCP assigned) is 172.16.218.10.
FortiGate-81F # diagnose sniffer packet any "udp port 5246 or udp port 5247" 4 . . .
Additionally, packet captures in PCAP format can be collected from the FortiGate in the GUI for more detailed analysis of the packets by going to Network -> Diagnostics -> Packet capture and enabling the appropriate filters as shown in the example below:
Packet captures in the FortiGate GUI:
Wireshark view of the pcap downloaded from FortiGate GUI, showing CAPWAP and DLTS traffic between FortiAP and FortiGate.
Note: For more detailed analysis of CAPWAP traffic between FortiGate and FortiAP, use the packet sniffer configuration discussed in the documentation here: CAPWAP and wireless traffic packet sniffer configurations.
The following are some additional diagnostic CLI commands that can be used while troubleshooting configuration push issues.
FortiGate:
get sys status get hardware status diagnose debug crashlog read show wireless-controller wtp show wireless-controller vap diagnose wireless-controller wlac -c ws show diagnose debug report
FortiAP:
fap-get-status <----- Basic info about the FortiAP unit. rcfg <----- Radio configurations, compare with FortiGate. top <----- Shows the top processes using CPU/memory/swap. Kp or cw_diag kernel-panic <----- Check for any kernel crash entries. fap-tech <----- FortiAP tech report.
Additional FortiAP diagnostic CLIs are listed here: FortiAP CLI diagnostic commands.
FortiGate Wireless-Controller debugs. Following cw_acd wireless-controller related debugs can be enabled on FortiGate, which will print useful debugs. Ensure debugs are enabled with filters where available (like wlac sta_filter), as it can be verbose and could cause CPU spikes.
cw_acd daemon debugs: FortiGate# diagnose debug console timetamp enable FortiGate# diagnose debug enable FortiGate# diagnose debug disable
wlac sta debugs: FortiGate#diagnose wireless-controller wlac sta_filter <MAC> <verbose> Debug messages will be on for 30 minutes. FortiGate# diagnose debug disable
Note: More details on wireless-controller WLAC debug processes are available in Troubleshooting Tip: Debugging a wireless client connection issue using client MAC address.
Debug commands can be run on the FortiAP as well, for the CAPWAP WTP daemon called cwWtpd, by running the following CLI commands. Use these commands with caution, as the output can be verbose.
Syntax : cw_debug app <app_name> [debug_var] FortiAP# cw_debug app cwWtpd 255 FortiAP# cw_debug on <debugs will print> FortiAP# cw_debug off
If the issue with config sync is still not remediated after following the steps above, the SSID profile may have been corrupted. This may occur after power outages and unexpected reboots of FortiGate. Any subsequent configuration changes from FortiGate might not get pushed to the FortiAP.
To troubleshoot this, start by making a small number of incremental changes to the SSID config in FortiGate, then check to see if the changes were reflected on the FortiAP VAP config. If not, this indicates a possible corruption of one of the SSID configurations. The next step is to unlink all the attached SSIDs from the specific FortiAP under WiFi & Switch Controller -> Managed FortiAPs -> Edit the specific FortiAP -> SSIDs -> Manual, and re-add one at a time (for both Radios) to see which specific SSID profile is causing the issue, as shown in the screenshot below.
After locating the SSID profile with the issue, create a clone of that SSID profile and replace it with the original one. Retry again by assigning this new SSID and confirm FortiAP is receiving this VAP config.
Manual selection of SSID profiles to incrementally check which SSID profile might be corrupted:
Note:
FortiGate# config system npu
set capwap-offload : Enable/disable offloading managed FortiAP and FortiLink CAPWAP sessions.
The following are some common terms used in the context of FortiGate as wireless-controller and FortiAPs.
Troubleshooting Tip: The SSID is not broadcasting Issue on FortiAP/FortiGate setup Technical Tip: High memory due to the cw_acd process and potential causes FortiAP CLI diagnostic commands Technical Tip: Changing country setting on a wireless controller |
