FortiNAC-F 7.2.5, New core switch, Contact Lost issues

[nkuhl30]

We replaced our core switch (HP 8212zl) with an Aruba CX 6405 back in June. We hired a consultant to reconfigure the config to be compatible with the new CX OS. Ever since then, FortiNAC is intermittently reporting Contact Lost events for our Aruba Controllers and Aruba APs. The controllers and the APs are on the same VLAN, and there are no ACLs or firewalls in-between VLAN 1 (new core) and VLAN 40 (wireless network).

There are zero port errors on the core and it's not an STP issue. The contact lost events only occur with the wireless controllers and APs, and not with our other edge switches or servers.

If we check the Aruba wireless side of things, none of the controllers or APs lose network connectivity. There's just a brief/random comms issue when FortiNAC reaches out to poll via ICMP.

Does anyone have any ideas? Fortinet support is claiming it's a network issue and not on their end. I could really use some help.

Thank you.

[salemneaz]

Hi, would you please try the article reference given below and troubleshoot the snmp issues if any.

https://community.fortinet.com/t5/FortiNAC/Technical-Note-Troubleshooting-Poll-failures/ta-p/195480

[nkuhl30]

Thanks, I've already looked at this.

[ebilcari]

What is the event message, Contact lost via ICMP or SNMP?

FNAC should try to ping the network device 4 times and than generate the event if they timed out. If the ping is successful, than it tries an SNMP GET to some OIDs. This seems like a known behavior with Aruba AP but shouldn't affect the services.

[nkuhl30]

It's contact lost via SNMP but there aren't any SNMP errors. We enabled the contact polling debug and confirm that FortiNAC is reaching out to the controllers and APs via ICMP and intermittently failing. It's only happening with the controllers and APs. I can't reproduce it and it sometimes happens 4 times a day, sometimes once a day, and sometimes it skips a day. Very frustrating.

You mentioned a known behavior with Aruba AP. What is the known behavior?

[ebilcari]

It seems like the Aruba APs will not always respond to pings, maybe the AP treats it as low priority traffic and during peak time refuses to respond. If the ping checks fails, than the SNMP check is not done by FNAC and the contact lost via SNMP event is triggered. This is more like an Aruba limitation, but currently this doesn't affect the performance or cause any undesired output apart from the event creation.

FNAC uses a common procedure to check all the devices via ICMP and it seems that this behavior appear only for this vendor. From FNAC side this procedure can not be changed, maybe check the vendor documentation if it allows any change to give more priority to the ICMP traffic.

If for some reason this events need to be hidden (triggers some alert in a 3rd party software), the event may be limited to a group of device as shown below:

[garibachi]

Ticket was opened last Thursday. Updated firewall on 6/14. I updated the ticket referencing the update to 7.2.5 and they admitted that its possibly the cause though no one else has reported this. From other conversations with FortiNAC folks, most are not on the newest versions for firewalls, etc.

[nkuhl30]

Are you having a similar issue with your FortiNAC instance as we are? You mentioned your firewall but I'm confused about what your firewall has to do with FortiNAC.

[AEK]

Since you said there are some ping loss then it is normal that FNAC may losses contact.

Try perform ping statistics to the WLC from different sources on your network, that will probably reveal where the issue is.

[nkuhl30]

What's interesting is that if I CLI-in to FortiNAC, and perform a ping to both of our wireless controllers, it will never drop a ping and the issue won't happen. As soon as I stop the pings from the CLI, the issue will occur within a few hours. FortiNAC CLI has a max timeout period of 8 hours so I can perform pings for 8 hours until it stops automatically.