Reliably using IKEV2 (Forticlient)

With IPSEC being removed from the new Forticlient and SSL-VPN being removed from the Fortigates themselves, I've been migrating everyone to IKEV2 using EMS.

For around 100 users I would say 80 of them are connecting fine using IKEV2, LDAP and 2FA (Fortitokens) however around 20% are consistently having issues and end up reverting back to SSL-VPN.

I've created both an UDP and TCP (443) IKEV2 profile for people to try. The TCP did solve some issues but a lot of people just cannot use IKEV2. I'm pretty sure it's likely their ISP/Router blocking it but I'm just wondering if there are any other tips I could check for when setting up the client on the Fortigate?

I've forced NAT Traversal and setup IKE fragmention. Any one else had issues which changing any settings helped at all?

Thanks!