EMS with Azure and auto SSL VPN on user login, failing at graph API connection.

[aguerriero]

I setup EMS and fortigate both with SAML configurations and both systems work. A user can be SAML SSO verified through EMS and a user can access SSL VPN with SAML SSO as well.

I tried to enable azure AD auto ssl vpn login and I get an error when the fortigate attempts to connect to the microsoft graph API to verify the users session token. I believe these are the steps that need to happen for a successful auto login. Step number 5 fails with the below debug errors. I verified the CLI can resolve DNS and ping the microsoft graph API.

1. user attempts saml

2. ems/forticlient talks to azure

3. azure provides a token to the forticlient

4. forticlient gives token to fortigate

5. fortigate uses graph api to validate token

6. fortigate authorizes forticlient

[saneeshpv_FTNT]

Hi,

Why don't you collect a packet sniffer from FortiGate while you attempt the authentication process. As per debug logs DNS is successful but the connection is failing to the received IP address with socket error. Capture will help to check further. 

# #diagnose sniffer packet <interface> none 4 10 a

Here interface would be your exit interface of FortiGate. 

Best Regards,

[aguerriero]

Attached is the result of the packet capture and the debugs running at the same time. I thought sessions initiated by the fortigate would not show up in a packet capture. If they are supposed to then it looks like the fortigate is not even trying to connect to the graph API.

02-0C-01-FGT-10E-A1 # config vdom

02-0C-01-FGT-10E-A1 (vdom) # edit ZTNA-GW
current vf=ZTNA-GW:12

02-0C-01-FGT-10E-A1 (ZTNA-GW) # diag debug app saml -1
Debug messages will be on for 9 minutes.

02-0C-01-FGT-10E-A1 (ZTNA-GW) # diag debug app sslvpn -1
Debug messages will be on for 9 minutes.

02-0C-01-FGT-10E-A1 (ZTNA-GW) # diag debug enable

02-0C-01-FGT-10E-A1 (ZTNA-GW) # diagnose sniffer packet ZTNA-EMAC-0051 "tcp and (port 443 or port 11443)" 4 1000 a
interfaces=[ZTNA-EMAC-0051]
[380:ZTNA-GW:4766]allocSSLConn:310 sconn 0x7f912ccba000 (12:ZTNA-GW)
[380:ZTNA-GW:4766]SSL state:before SSL initialization (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:before SSL initialization (X.X.X.34)
[380:ZTNA-GW:4766]got SNI server name: HOST.DOMAIN.COM realm (null)
[380:ZTNA-GW:4766]client cert requirement: no
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS read client hello (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write server hello (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write change cipher spec (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data:(null)(X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data (X.X.X.34)
[380:ZTNA-GW:4766]got SNI server name: HOST.DOMAIN.COM realm (null)
[380:ZTNA-GW:4766]client cert requirement: no
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS read client hello (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write server hello (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 write encrypted extensions (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write certificate (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 write server certificate verify (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write finished (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data:(null)(X.X.X.34)
[380:ZTNA-GW:4766]SSL state:TLSv1.3 early data (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS read finished (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write session ticket (X.X.X.34)
[380:ZTNA-GW:4766]SSL state:SSLv3/TLS write session ticket (X.X.X.34)
[380:ZTNA-GW:4766]SSL established: TLSv1.3 TLS_AES_256_GCM_SHA384
[380:ZTNA-GW:4766]req: /remote/info
[380:ZTNA-GW:4766]capability flags: 0x1cdf
2023-08-01 16:35:49.554531 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: syn 2281064911
2023-08-01 16:35:49.554549 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: syn 1096045289 ack 2281064912
2023-08-01 16:35:49.641975 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096045290
2023-08-01 16:35:49.647279 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: psh 2281064912 ack 1096045290
2023-08-01 16:35:49.647289 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: ack 2281065429
2023-08-01 16:35:49.647843 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096045290 ack 2281065429
2023-08-01 16:35:49.740325 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: psh 2281065429 ack 1096045389
2023-08-01 16:35:49.744101 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: 1096045389 ack 2281065952
2023-08-01 16:35:49.744108 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: 1096046849 ack 2281065952
2023-08-01 16:35:49.744113 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096048309 ack 2281065952
2023-08-01 16:35:49.745060 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096049485 ack 2281065952
2023-08-01 16:35:49.833981 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096048309
2023-08-01 16:35:49.838759 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096050696
2023-08-01 16:35:49.877242 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: psh 2281065952 ack 1096050696
2023-08-01 16:35:49.877561 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096050696 ack 2281066026
2023-08-01 16:35:49.877662 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096050983 ack 2281066026
2023-08-01 16:35:49.882214 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: psh 2281066026 ack 1096050696
2023-08-01 16:35:49.882450 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096051270 ack 2281066458
2023-08-01 16:35:49.969210 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096051270
2023-08-01 16:35:50.016640 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096051953
[380:ZTNA-GW:4766]Destroy sconn 0x7f912ccba000, connSize=1. (ZTNA-GW)
[380:ZTNA-GW:4766]SSL state:warning close notify (X.X.X.34)
2023-08-01 16:36:14.993014 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: psh 1096051953 ack 2281066458
2023-08-01 16:36:14.993134 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51780: fin 1096051977 ack 2281066458
2023-08-01 16:36:15.084662 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: ack 1096051978
[374:ZTNA-GW:4761]allocSSLConn:310 sconn 0x7f912c523800 (12:ZTNA-GW)
[374:ZTNA-GW:4761]SSL state:before SSL initialization (X.X.X.34)
[374:ZTNA-GW:4761]SSL state:fatal decode error (X.X.X.34)
[374:ZTNA-GW:4761]SSL state:error:(null)(X.X.X.34)
[374:ZTNA-GW:4761]SSL_accept failed, 1:unexpected eof while reading
[374:ZTNA-GW:4761]Destroy sconn 0x7f912c523800, connSize=5. (ZTNA-GW)
[376:ZTNA-GW:4764]allocSSLConn:310 sconn 0x7f912c4d7800 (12:ZTNA-GW)
[376:ZTNA-GW:4764]SSL state:before SSL initialization (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:before SSL initialization (X.X.X.34)
[376:ZTNA-GW:4764]got SNI server name: HOST.DOMAIN.COM realm (null)
[376:ZTNA-GW:4764]client cert requirement: no
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS read client hello (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write server hello (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write certificate (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write key exchange (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write server done (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write server done:(null)(X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write server done (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS read client key exchange (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS read change cipher spec (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS read finished (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write session ticket (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write change cipher spec (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSLv3/TLS write finished (X.X.X.34)
[376:ZTNA-GW:4764]SSL state:SSL negotiation finished successfully (X.X.X.34)
[376:ZTNA-GW:4764]SSL established: TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384
[376:ZTNA-GW:4764]req: /remote/saml/autoauth?type=azure
[376:ZTNA-GW:4764]User Agent: FortiSSLVPN (Windows NT; SV1 [SV{v=02.01; f=07;}])
[376:ZTNA-GW:4764]sslvpn_auth_check_usrgroup:3000 forming user/group list from policy.
[376:ZTNA-GW:4764]sslvpn_auth_check_usrgroup:3046 got user (4) group (1:0).
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:1908 validating with SSL VPN authentication rules (1), realm ((null)).
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:1994 checking rule 1 cipher.
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2002 checking rule 1 realm.
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2013 checking rule 1 source intf.
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2052 checking rule 1 vd source intf.
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2543 rule 1 done, got user (0:0) group (1:0) peer group (0).
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2551 got user (0:0) group (1:0) peer group (0).
[376:ZTNA-GW:4764]sslvpn_validate_user_group_list:2898 got user (4:0), group (1:0) peer group (0).
[376:ZTNA-GW:4764]sslvpn_update_user_group_list:1807 got user (4:0), group (1:0), peer group (0) after update.
[376:ZTNA-GW:4764][fsv_found_saml_server_name_from_auth_lst:122] Found SAML server [AZURE-SAML] in group [azure-saml-users]
[376:ZTNA-GW:4764]dns_query():296 tried IPv4 0 graph.microsoft.com
[376:ZTNA-GW:4764]dns_on_read():178 got result
[376:ZTNA-GW:4764]proxy: connect() error: Network is unreachable
[376:ZTNA-GW:4764]connect_to_server:209 fail to create socket.
[376:ZTNA-GW:4764]fsv_saml_autoauth_state_cleanup with user (null)/grp size 0
[376:ZTNA-GW:4764]Destroy sconn 0x7f912c4d7800, connSize=3. (ZTNA-GW)
[376:ZTNA-GW:4764]SSL state:warning close notify (X.X.X.34)
2023-08-01 16:36:48.924529 ZTNA-EMAC-0051 -- X.X.X.34.51803 -> X.X.X.16.11443: syn 1242389074
2023-08-01 16:36:48.924549 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51803: syn 2225783893 ack 1242389075
2023-08-01 16:36:49.017046 ZTNA-EMAC-0051 -- X.X.X.34.51803 -> X.X.X.16.11443: ack 2225783894
2023-08-01 16:36:49.035499 ZTNA-EMAC-0051 -- X.X.X.34.51803 -> X.X.X.16.11443: fin 1242389075 ack 2225783894
2023-08-01 16:36:49.035758 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51803: psh 2225783894 ack 1242389076
2023-08-01 16:36:49.036040 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51803: fin 2225783901 ack 1242389076
2023-08-01 16:36:49.131418 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: syn 272956864
2023-08-01 16:36:49.131434 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: syn 3624586122 ack 272956865
2023-08-01 16:36:49.131467 ZTNA-EMAC-0051 -- X.X.X.34.51803 -> X.X.X.16.11443: rst 1242389076 ack 2225783901
2023-08-01 16:36:49.229009 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624586123
2023-08-01 16:36:49.233965 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: psh 272956865 ack 3624586123
2023-08-01 16:36:49.233976 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: ack 272957274
2023-08-01 16:36:49.234650 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: 3624586123 ack 272957274
2023-08-01 16:36:49.234657 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: 3624587583 ack 272957274
2023-08-01 16:36:49.234661 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: psh 3624589043 ack 272957274
2023-08-01 16:36:49.236852 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: psh 3624590219 ack 272957274
2023-08-01 16:36:49.238578 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: fin 2281066458 ack 1096051978
2023-08-01 16:36:49.238583 ZTNA-EMAC-0051 -- X.X.X.34.51780 -> X.X.X.16.11443: rst 2281066459 ack 1096051978
2023-08-01 16:36:49.329945 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624587583
2023-08-01 16:36:49.329952 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624589043
2023-08-01 16:36:49.329955 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624590219
2023-08-01 16:36:49.334952 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624591275
2023-08-01 16:36:49.341344 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: psh 272957274 ack 3624591275
2023-08-01 16:36:49.343991 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: psh 3624591275 ack 272957432
2023-08-01 16:36:49.436461 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624591549
2023-08-01 16:36:49.436543 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: 272957432 ack 3624591549
2023-08-01 16:36:49.436547 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: psh 272958892 ack 3624591549
2023-08-01 16:36:49.436553 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: ack 272959920
2023-08-01 16:36:49.463286 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: psh 3624591549 ack 272959920
2023-08-01 16:36:49.463393 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: fin 3624591580 ack 272959920
2023-08-01 16:36:49.554915 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624591580
2023-08-01 16:36:49.559988 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: fin 272959920 ack 3624591581
2023-08-01 16:36:49.559997 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: ack 272959921
2023-08-01 16:36:49.560005 ZTNA-EMAC-0051 -- X.X.X.34.51804 -> X.X.X.16.11443: ack 3624591581
2023-08-01 16:36:49.560011 ZTNA-EMAC-0051 -- X.X.X.16.11443 -> X.X.X.34.51804: ack 272959921

[aguerriero]

I tried posting debug and packet capture output twice with embedded code sample and it looks like it was posted successfully but looks like it doesn't actually post.

This is the packet capture with debugs running at the same time. 

[aguerriero]

I submitted a TAC case.

[saneeshpv_FTNT]

HI @aguerriero,

Thats great as the capture doesn't show any connection to Graph API URL and all it shows is your VPN traffic on port 10443. Looks like a detailed investigation is necessary with the configuration and log here. Share the feedback in forum if you find a solution. 

Hope you have seen this document for the setup. If not, please review your setup against the article as well.

https://fortinetweb.s3.amazonaws.com/docs.fortinet.com/v2/attachments/967cd9f0-70ff-11ed-8e6d-fa163e... 

Best Regards,

[aguerriero]

Support from Fortinet called me. The graph API connection is broken in VDOM mode and also possibly with VRFs. He said that 7.2.6 and 7.4.1 should address this issue.

[UQTR_Beaudet]

Any success with 7.2.6 or 7.4.1 since then? 

I was trying to test Auto-Connect Azure AD and ended up with the same error as you connecting to Graph on our Fortigate 7.2.5. 

Since there seems to be a big DNS bug on 7.2.6 I am holding off for now. Just curious to know if it is now working for you if you upgraded.

thanks!

[aguerriero]

I am waiting for ZTNA tags to be fixed before I upgrade from 7.2.5.

7.2.6 seems a little risky at this point. 

[UQTR_Beaudet]

Yep same here. The "cosmetic" DNS bug doesn't appeal to us :rolling_on_the_floor_laughing:

Will wait patiently for 7.2.7 and no way are we jumping on the 7.4 train. 

I tried finding info on whether this connection mechanism would work well with hybrid joined PCs... still unclear! Will know more when that bug is squashed.

hybrid joined meaning that the PC is joined to our on-premises AD Domain but fully synced to Azure AD. It seems hopeful since I am at the same point as you so far.