Created on
07-08-2025
07:22 AM
Edited on
08-11-2025
05:09 AM
By
Anthony_E
This article describes an overview of Dial-up IPSec authentication and policy matching on the FortiGate.
FortiGate.
FortiGate offers Dial-Up IPSec VPN tunnels as one method to securely connect an endpoint to a protected network. IPSec provides methods to authenticate a connection and ensure access is only granted to appropriate users.
Common authentication methods in an IPSec VPN context include local users on the FortiGate, or authentication via LDAP, RADIUS, or SAML.
Both IKEv1 and IKEv2 have ways to support these methods.
IKEv1 uses the XAUTH framework to this end. IKEv2 utilizes EAP instead and may encounter some limitations due to how EAP works.
In principle, users can be authenticated in one of two ways:
Authentication via IPSec Settings.
This option means user groups are specified directly in the IPSec tunnel configuration.
In IKEv1 setups, a group may be set in the Xauth section:
In IKEv2 setups, a group may be set via CLI:
config vpn ipsec phase1-interface
edit "IKEv2-test"
set eap enable
set eap-identity send-request
set authusrgrp "vpn-group"
next
end
With either configuration, FortiGate will expect the VPN client to submit user credentials, or in case of SAML authentication, will redirect the client to the SAML IdP to authenticate.
Authentication via Policy Configuration.
FortiGate can also require user credentials if no user group is specified outright in the VPN configuration. In this case, IPSec authentication behaves somewhat similarly to SSL VPN authentication, in that FortiGate checks for any user groups attached to policies that use the IPSec tunnel interface as source, and attempts to authenticate the user to any of the groups it finds.
In IKEv1, this behavior is achieved by setting the XAUTH configuration to 'Inherit from policy'.
In IKEv2, this behavior is achieved via CLI, by enabling EAP and setting 'eap-identity' to 'send-request', but leaving the 'authusrgrp' value blank.
config vpn ipsec phase1-interface
edit "IKEv2-test"
set eap enable
set eap-identity send-request
next
end
As the FortiGate looks to authenticate the user to any possible matching group, this will result in similar behavior as with SSL VPN authentication:
Note:
Further details on IPSec and SAML authentication can be found for example in FortiGate documentation.
Notes on IKEv2 and LDAP authentication: Technical Tip: IKEv2 dial up VPN with LDAP authentication.
An example of IKEv2 and RADIUS authentication: Technical Tip: IKEv2 Dialup IPsec tunnel with RADIUS and FortiToken MFA.
If an IPSec VPN user is authenticated through a group defined in the IPsec configuration itself, this does not lead to a cached authentication session: The user authentication cannot be found in Firewall User Monitor or via the command 'diagnose firewall auth list'.
This means that while the user had to provide credentials to connect to IPSec VPN, FortiGate cannot apply the user's group memberships to match policies. If users authenticate via a group set in the IPSec configuration directly, any policies they should match must not have groups set.
If users have authenticated via groups defined in policies, and no group is specified in IPSec configuration directly, then users will match policies with groups based on what groups they authenticated with, as expected.
Essentially, either configuration is valid:
If groups are set in both IPSec configuration AND policies, the FortiGate will essentially try to authenticate the user twice.
The first authentication happens during IPsec tunnel setup, and does not result in a cached authentication session FortiGate can re-use for policy matching.
As the user is not authenticated for purposes of policy matching, the FortiGate will attempt to authenticate the user again, should no matching, group-less policy be available. This may either result in a login screen for the user to enter their credentials again (if the traffic is of a nature that FortiGate can intercept and trigger authentication, usually only HTTP/HTTPS), or the traffic being denied if FortiGate cannot trigger an authentication prompt.
Debug flow output may look something like this in a case where FortiGate is not able to trigger authentication:
trace_id=1 func=print_pkt_detail line=5932 msg="vd-root:0 received a packet(proto=6, x.x.x.78:48874->y.y.y.11:3389) tun_id=a.a.a.150 from tunnel1. flag [S], seq 2098054241, ack 0, win 64240"
[...]
trace_id=1 func=__vf_ip_route_input_rcu line=1989 msg="find a route: flag=00000000 gw-b.b.b.49 via port2"
trace_id=1 func=__iprope_fwd_check line=810 msg="in-[tunnel1], out-[port2], skb_flags-02000008, vid-20, app_id: 0, url_cat_id: 0"
trace_id=1 func=__iprope_tree_check line=535 msg="gnum-100004, use addr/intf hash, len=9" <--- gnum-100004 means checking against forward traffic policies.
trace_id=1 func=__iprope_check_one_policy line=2140 msg="checked gnum-100004 policy-1, ret-no-match, act-accept"
trace_id=1 func=__iprope_check_one_policy line=2140 msg="checked gnum-100004 policy-2, ret-matched, act-accept" <--- matches policy 2 based on source/destination
trace_id=1 func=__iprope_user_identity_check line=1903 msg="ret-no-match" <--- policy 2 has a user group set and does NOT match based on authentication.
[...]
trace_id=1 func=__iprope_check_one_policy line=2140 msg="checked gnum-100004 policy-0, ret-matched, act-accept" <---- implicit deny is matched; there are no group-less policies that match at all
[...]
trace_id=1 func=__iprope_check line=2404 msg="gnum-3, check-ffffffffa002cac7" <---- gnum-3 means FortiGate is checking what protocols it can trigger authentication on, to see if it can prompt authentication and learn user and group information that way.
trace_id=1 func=__iprope_check_one_policy line=2140 msg="checked gnum-3 policy-4294967295, ret-no-match, act-drop"
[...]
trace_id=1 func=__iprope_check_one_policy line=2140 msg="checked gnum-3 policy-4294967295, ret-matched, act-drop"
trace_id=1 func=__iprope_check_one_policy line=2374 msg="policy-4294967295 is matched, act-drop"
trace_id=1 func=__iprope_check line=2421 msg="gnum-3 check result: ret-matched, act-drop, flag-00000020, flag2-00000000"
[...]
trace_id=1 func=fw_forward_handler line=840 msg="Denied by forward policy check (policy 0)" <--- FortiGate cannot force authentication through its implicit captive portal because the traffic is RDP. The traffic is denied because it is not considered authenticated, and there are no matching policies.
Troubleshooting Tip: Dialup IPsec VPN user-info not displaying in 'Assets & Identities'
Technical Tip: Using group based firewall policy for Dial-Up VPN to restrict network access
Technical Tip: A quick guide to FortiGate SSL VPN authentication and common issues and misunderstand...
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2025 Fortinet, Inc. All Rights Reserved.