CAPWAP with fortigate 60D is not working stable

[earthcroeser]

Hi

I have fortigate 60D running 5.4.1

2 fortiswitches 124D with S124DN-v3.4-build192 running

2 forti aps 321 with FP321C-v5.4-build0339. the fortiaps are connectect through the fortiswitches with the fortigate.

The reason why I bought fortinet solutions because of the good security and the central management.

Problem is that the capwap tunnels are instable. Once they are up they stay up, but dont reboot any of the systems or you risk that switch or AP is not able to establish capwap again.

date=2016-07-28 time=19:16:29 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000064 is connected date=2016-07-28 time=19:16:19 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:68:BA ip=169.254.254.2 lease=604800 hostname="S124DN3W15000064" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:16:10 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:16:10 logid=0100038409 type=event subtype=system level=information vd=root logdesc="SSL connection closed" dstip=208.91.113.205 dstport=514 action=disconnect status=success msg="SSL connection to 208.91.113.205 is successfully closed." date=2016-07-28 time=19:16:09 logid=0100038408 type=event subtype=system level=information vd=root logdesc="SSL connection established" dstip=208.91.113.205 dstport=514 action=connect status=success msg="SSL connection to 208.91.113.205 is successfully established." date=2016-07-28 time=19:15:53 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000133 is connected date=2016-07-28 time=19:15:49 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:70:46 ip=169.254.254.3 lease=604800 hostname="S124DN3W15000133" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:15:34 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session dtls terminated (ev 21): S124DN3W15000133 is disconnected" action=session-leave srcip=169.254.254.3 date=2016-07-28 time=19:15:25 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:15:15 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:15:06 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:68:BA ip=169.254.254.2 lease=604800 hostname="S124DN3W15000064" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:14:55 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:14:37 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000133 is connected date=2016-07-28 time=19:14:33 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:70:46 ip=169.254.254.3 lease=604800 hostname="S124DN3W15000133" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:14:29 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session dtls terminated (ev 21): S124DN3W15000133 is disconnected" action=session-leave srcip=169.254.254.3 date=2016-07-28 time=19:14:26 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:14:15 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:14:03 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session dtls terminated (ev 21): S124DN3W15000064 is disconnected" action=session-leave srcip=169.254.254.2 date=2016-07-28 time=19:14:00 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:13:58 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:68:BA ip=169.254.254.2 lease=604800 hostname="S124DN3W15000064" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:13:45 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000064:-login:169.254.254.2 failed:-7624" date=2016-07-28 time=19:13:32 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000133 is connected date=2016-07-28 time=19:13:28 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:70:46 ip=169.254.254.3 lease=604800 hostname="S124DN3W15000133" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:13:25 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000064:-login:169.254.254.2 failed:-7624" date=2016-07-28 time=19:13:12 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session dtls terminated (ev 21): S124DN3W15000133 is disconnected" action=session-leave srcip=169.254.254.3 date=2016-07-28 time=19:13:09 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:13:06 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000064 is connected date=2016-07-28 time=19:12:56 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:68:BA ip=169.254.254.2 lease=604800 hostname="S124DN3W15000064" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:12:49 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:12:32 logid=0100022891 type=event subtype=system level=error vd=root logdesc="Fortilink configuration daemon log" msg="UpdSwConf:S124DN3W15000133:-login:169.254.254.3 failed:-7624" date=2016-07-28 time=19:12:15 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session join: S124DN3W15000133 is connected date=2016-07-28 time=19:12:11 logid=0100026001 type=event subtype=system level=information vd=root logdesc="DHCP Ack log" interface="root-sw" dhcp_msg="Ack" mac=08:5B:0E:E4:70:46 ip=169.254.254.3 lease=604800 hostname="S124DN3W15000133" msg="DHCP server sends a DHCPACK" date=2016-07-28 time=19:12:04 logid=0100032546 type=event subtype=system level=warning vd=root logdesc="Application crashed" action=crash msg="Pid: 00245, application: cu_acd, Firmware: FortiGate-60D v5.4.1,build1064b1064,160608 (GA) (Release), Signal 11 received, Backtrace: [0x01590f50] [0x300c0080]" date=2016-07-28 time=19:12:04 logid=0100022900 type=event subtype=system level=notice vd=root logdesc="CAPUTP session status" msg="session dtls terminated (ev 21): S124DN3W15000133 is disconnected" action=session-leave srcip=169.254.254.3

when I reboot my fortiswitches they start to flap. when one switch has established capwap the other one goes down,etc, strangely my fortiaps stay connected (event viewer:ap-fail - Reason Control message maximal retransmission limit reached)

somebody can help I am getting crazy? multiple support cases loggebut no progress so far.

Kind regards

[Toshi_Esumi · Accepted solution]

One idea I can think of without much experience&knowledge is trying static IPs for the APs without using DHCP. That would probably avoid the problem whaterver the cause is. Also if possible, hook up one of APs directly to the controller (I assume the FG60D) by skipping the FortiSwitch to see if it change the situation.

[Toshi_Esumi · Accepted solution]

One idea I can think of without much experience&knowledge is trying static IPs for the APs without using DHCP. That would probably avoid the problem whaterver the cause is. Also if possible, hook up one of APs directly to the controller (I assume the FG60D) by skipping the FortiSwitch to see if it change the situation.

[earthcroeser]

thx for your reply. I am not in the ability to connect the AP directly to controller, because of limited cabling foreseen. but this should anyway work. no?

How can I get a static ip in the AP's?

[Toshi_Esumi · Accepted solution]

Everything you're doing is probably "supposed to work". But we avoid many "supposed to work" things due to bugs and constraints and future concerns.

If you didn't change any FortiAPs setting before they should still have 192.168.1.2/24 configured even though it requests a DHCP IP. Set your PC with one of IPs in the subnet and directly hook up to the AP, then you can get to GUI admin page with http://192.168.1.2. The default user/pass is either admin/(no password) or admin/admin depending on the firmware version. The GUI is quite intuitive and you can figure out how to set up a static IP and the controller's IP manually. By the way TAC must have check the version of your APs if it's compatible with FG60D's firmware version. It's specified with a release note of the AP version.

[MikePruett]

I know the 60D supports only 5 tunneled AP's normally. Perhaps something is screwy with the switches and everything in place as it is.

This solution definitely should work. I have it deployed in many environments just as you (at least from my understanding of your environment) and it works fine. 

Definitely try static IPing if you must. It shouldn't be necessary though. Any chance you have possible cable issues? or maybe the FortiAPs are trying to pull an IP before the switch they are connected to are fully booted causing them to hang?

Try static IPing the devices and let us know how it goes!

[PC88]

Hi there,

If you're using DHCP, can I suggest using option 138 on your DHCP server for the CAPWAP server IP.

Have a read through this...  http://kb.fortinet.com/kb/documentLink.do?externalID=FD33978

Thanks,

Paul

[romanr]

Hi,

DHCP is fine for the Fortilink Interface - You don't need any options additionaly!

How does you FortiLink interface look like?

How does your cabling look like from the FGT to the switches? And in between?

Can you show us the output of "exec switch-controller get-physical-conn root FortiSwitch-Stack-"YourFortilinkInterfacename" "

What are your Switch Controller Global settings from the CLI?

Have you had you Fortiswitches upgraded to Build 192 before connecting them the first time to the switch controller? (if not do a "exec factoryreset" on the switch)

Br,

Roman