SAML Entra ID Connection - server IP address could not be found

We have been going through the process of setting up our Firewall to access Forticlient connections. All went well with connecting via just a Pre-Shared Key but we wanted to implement Saml using Entra ID. We went through multiple documents and have everything setup correctly (At least we think so). When we try to connect we get the popup to provide our credentials. After the credentials are submitted we are shown a <FQDN> server IP address could not be found.

I made another connection using an external browser and when I make the connection I can see it sending the request to our Public facing IP address and then resolves to <FQDN>/remote/saml/login. The message in this window is 'can't reach this page'. 

There are Firewall policies setup to allow all traffic from the VPN Tunnel so we can verify that it works before scaling back but still does not work. The only thing that comes to mind is I read that the <FQDN> needs to be reachable from the internet. Ours is not so wondering if that is what ultimately is causing our problem. If that is true do most organizations allow access to the Firewall interface or is something else created to allow this connection. Sorry if this is a basic question but I inherited the firewall so wasn't around when it was initially setup.