Please note that normally when your are doing failover, the FortiAP will try to discover new controller only when the retrasnmit count (default is 3) configured on the Fortigate fails. It is normally it will take 2 -3 min for the AP to join the new controller.
Kindly let us known how much time it is taking to FortiAP joins to new controller. Does both the Fortigate are in sync, when you do the HA failover.
You can fine tune to below keep alive settings on the Fortigate.
#config wireless-controller global
#set max-retransmit 3
# config wireless-controller timers
(timers) # set echo-interval 60 (sec)
(timers) # end