Is there a guide to make virtual IPs work on SSL VPN interfaces?

[ariel-anieli]

Hello,

We have two questions:

1. Is there a guide to make virtual IPs work on SSL VPN interfaces?
2. May hairpin NAT work using secondary IP interfaces?

We are trying to make hairpin NAT work on 3000F. Any help will be much appreciated.

We followed this KB on hairpin NAT (https://community.fortinet.com/t5/FortiGate/Technical-Tip-Configuring-Hairpin-NAT-VIP/ta-p/195448#M2...). Our use case is similar to Example 1.

Campus1 (VPN / LAN IPs) --- FW (WAN IPs) -- internalServer (VIP).

At the FortiGate side, the VPN tunnels land on an SSL interface (ssl.root); ssl.root is defined on a WAN interface (port35).

Two firewall policies are defined:

1. a NAT ingress rule, from ssl.root to port35, allowing internal users (VPN admin pool) to reach the virtual IP (defined as destination address object) balancing on an internal server.
2. an egress rule, from port35 to a DMZ, allowing the NAT'ed requests to reach the load balancer virtual IP.

This translates into these policies:

# show firewall policy 1 
config firewall policy
    edit 96
        set uuid 40edc776-13ac-51f0-4ffe-76f817f3a8fb
        set srcintf "ssl.root"
        set dstintf "port35"
        set action accept
        set srcaddr "VPN admin pool"
        set dstaddr "internal-vip"
        set schedule "always"
        set service "SVC_HTTP"
        set logtraffic all
        set nat enable
        set ippool enable
        set poolname "PRIMARY"
    next
end

# show firewall address internal-vip 
config firewall address
    edit "aichor-dataplane-vip"
        set uuid b89f3c46-13ac-51f0-a148-d5b310b9adee
        set associated-interface "port35"
        set color 1
        set subnet 5.23.18.38 255.255.255.255
    next
end

# show firewall policy 2 
config firewall policy
    edit 93
        set uuid bde1c42c-9556-51f0-1920-baa5f4d44b9f
        set srcintf "port35"
        set dstintf "DMZ"
        set action accept
        set srcaddr "RFC1918" "WAN_primary_ip"
        set dstaddr "external-vip-https"
        set schedule "always"
        set service "SVC_NGINX_EXT_HTTP_TCP_32443"
        set inspection-mode proxy
        set logtraffic all
    next
end

# show firewall vip external-vip-https 
config firewall vip
    edit "external-vip-https"
        set uuid 5508607c-1f64-51f0-b372-bc95c2b4275d
        set type server-load-balance
        set server-type tcp
        set extip 5.23.18.39
        set extintf "any"
        set monitor "nginx-internal-ingress-hc-tcp-32444" "ping"
        set extport 443
        config realservers
            edit 1
                set ip 10.120.0.16
                set port 32444
                set max-connections 1000
            next
            edit 2
                set ip 10.120.0.151
                set port 32444
                set status standby
                set max-connections 1000
            next
            edit 3
                set ip 10.120.1.149
                set port 32444
                set status standby
                set max-connections 1000
            next
        end
    next
end

We were expecting policy ID 1 to match first, then policy ID 2. We observed policy ID 2 got directly matched.

# diagnose firewall iprope lookup 10.120.37.1 12345 5.23.18.39 443 tcp ssl.root 
<src [10.120.37.1-12345] dst [85.233.198.42-443] proto tcp dev ssl.root> matches policy id: 0

# diagnose firewall iprope lookup 10.120.37.2 12355 5.23.18.39 443 tcp port35 
<src [10.120.37.2-12355] dst [85.233.198.43-443] proto tcp dev port35> matches policy id: 2

To work around the mismatch, we added the internal IPs in srcaddr of policy ID 2 (the address object RFC1918), and see this message in diag debug flow

id=65308 trace_id=144 func=fw_pre_route_handler line=191 msg="VIP-10.120.0.123:32443, outdev-unknown"

A complete extract,

# diagnose debug flow trace start 5
# id=65308 trace_id=144 func=print_pkt_detail line=6005 msg="vd-root:0 received a packet(proto=6, 10.120.37.1:50042->5.23.18.39:443) tun_id=0.0.0.0 from ssl.root. flag [S], seq 856634200, ack 0, win 64240"
id=65308 trace_id=144 func=init_ip_session_common line=6206 msg="allocate a new session-3dcd1101"
id=65308 trace_id=144 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=69, len=13"
id=65308 trace_id=144 func=get_new_addr line=1280 msg="find SNAT: IP-5.23.18.36(from IPPOOL), port-50042"
id=65308 trace_id=144 func=fw_pre_route_handler line=191 msg="VIP-10.120.0.123:32443, outdev-unknown"
id=65308 trace_id=144 func=__ip_session_run_tuple line=3525 msg="DNAT 5.23.18.38:443->10.120.0.123:32443"
id=65308 trace_id=144 func=__vf_ip_route_input_rcu line=1989 msg="find a route: flag=00000000 gw-0.0.0.0 via DATA"
id=65308 trace_id=144 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=36, len=9"
id=65308 trace_id=144 func=fw_forward_handler line=1002 msg="Allowed by Policy-2:"
id=65308 trace_id=144 func=ip_session_confirm_final line=3179 msg="npu_state=0x100, hook=4"

As mentioned above, the addresses used in the VIPs are secondary IPs of port35; we are also wondering if the problem might come from that.

Your feedback is much appreciated,

Ariel

[funkylicious]

hi,

from my experience, hairpin nat works great with VIPs but i dont think they work with VS ( virtual server / load-balance ) and the extintf i usually leave it to any so I can use the VIP as destination ( even for hairpin NAT ) on the real destination interface where the traffic needs to go, in your case DMZ/DATA.

also, i personally prefer not to assign more than 1 IP on the WAN, if I have/want to use SNAT ( oubound to internet ) for certain traffic to use a specific IP other the one on WAN i just create a ip pool for that IP.

what should be the benefit of policy 96 ? it just uses the WAN IP as a destination for service SVC_HTTP ( port 80? ) but it shouldnt really do anything, except if you want to access the GUI for mgmt if it listens on port 80 for the secondary ip .

for WAN to DMZ access you would need a policy like , port35 > DATA ( servers 10.120.* are there i assume ? ) , src all , dst external-vip-https, svc https ( no NAT )

for SSLVPN to DMZ i would do something similar, ssl.root > DATA , src vpn subnet + usergroup , dst VS ( external-vip-https ) , service https ( no NAT assuming there's a default route back to FGT )

[ariel-anieli]

Hi @funkylicious,

Thanks for having answered, it's much appreciated.

From your answer I understood that you would:

1. use `any` as extintf, so the VIP may be used as dstaddr.
2. assign one IP to the WAN, and use an IP pool if SNAT is needed.
3. define a policy from WAN to DMZ, with external-vip-https as dstaddr for all IPs  and no NAT (which should work, thank to Point 1).
4. define a policy from SSLVPN to DMZ, with internal IPs as srcaddr and VS as dstaddr, without NAT (and assuming there is a default route back to FGT).

This implies that, for doing an hairpin NAT with internal IP ingressing from ssl.root (over WAN interface), it should be enough to set one policy and a VIP (with extintf as any).

Is that correct?

We will have a try on this, and let you know about our findings.

For your question on policy  1 (there is a typo; instead of IDs 96 and 93, it is 1 and 2), we followed the hairpin KB as said before. Our use case matching Example 1, we defined two policies.

[ariel-anieli]

Hi again, @funkylicious;

This follow from my previous response. We have tried Point 1 to 4; and there is no hairpin NAT happening between SSLVPN and DMZ.

We have defined policy ID 3,

# show firewall policy 3 
config firewall policy
    edit 3
        set uuid c8bce004-d519-51f0-ecb5-41b49aee0b1a
        set srcintf "ssl.root"
        set dstintf "DMZ"
        set action accept
        set srcaddr "VPN admin pool"
        set dstaddr "test-vip"
        set schedule "always"
        set service "SVC_HTTP"
        set logtraffic all
        set nat enable
        set ippool enable
        set poolname "PRIMARY"
    next
end

PA10-R104U34-F3501F-FW1 # show firewall vip test-vip
config firewall vip
    edit "eqx-test-vip"
        set uuid 7234d330-d515-51f0-c42e-f2549a316cbc
        set comment "For ID 3"
        set type server-load-balance
        set server-type tcp
        set extip 5.23.18.42
        set extintf "any"
        set monitor "nginx-external-ingress-hc-tcp-32443"
        set extport 443
        config realservers
            edit 1
                set ip 10.120.0.123
                set port 32443
            next
        end
    next
end

# diagnose firewall iprope lookup 10.120.37.1 12345 5.23.18.42 443 tcp ssl.root 
<src [10.120.37.1-12345] dst [5.23.18.42-443] proto tcp dev ssl.root> matches policy id: 0

The IPs used in the VS comes are secondary IPs of port35. May hairpin NAT work with secondary IPs?

That is our second question. Thanks for your hints on the first question.

Below a debug flow and the interface configuration,

# diagnose debug flow filter
        vf: any
        proto: any
        Host addr: any
        host saddr: 10.120.37.1-10.120.37.1
        host daddr: 5.23.18.42-5.23.18.42
        port: any
        sport: any
        dport: any

# diagnose debug flow trace start 5 

# id=65308 trace_id=198 func=print_pkt_detail line=6005 msg="vd-root:0 received a packet(proto=6, 10.120.37.1:36974->5.23.18.42:443) tun_id=0.0.0.0 from ssl.root. flag [S], seq 2996138729, ack 0, win 64240"
id=65308 trace_id=198 func=init_ip_session_common line=6206 msg="allocate a new session-3f429e7f"
id=65308 trace_id=198 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=69, len=13"
id=65308 trace_id=198 func=get_new_addr line=1280 msg="find SNAT: IP-5.23.18.36(from IPPOOL), port-36974"
id=65308 trace_id=198 func=fw_pre_route_handler line=191 msg="VIP-10.120.0.123:32443, outdev-unknown"
id=65308 trace_id=198 func=__ip_session_run_tuple line=3525 msg="DNAT 5.23.18.42:443->10.120.0.123:32443"
id=65308 trace_id=198 func=__vf_ip_route_input_rcu line=1989 msg="find a route: flag=00000000 gw-0.0.0.0 via DATA"
id=65308 trace_id=198 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=36, len=9"
id=65308 trace_id=198 func=fw_forward_handler line=839 msg="Denied by forward policy check (policy 0)"

# show sys interface port35 
config system interface
    edit "port35"
        set vdom "root"
        set ip 5.23.18.36 255.255.255.248
        set allowaccess ping https ssh http
        set type physical
        set mediatype sr4
        set alias "WAN"
        set lldp-reception enable
        set estimated-upstream-bandwidth 25000000
        set estimated-downstream-bandwidth 25000000
        set monitor-bandwidth enable
        set role wan
        set snmp-index 35
        set secondary-IP enable
        set forward-error-correction disable
        set speed 100Gfull
        config secondaryip
            edit 1
                set ip 5.23.18.37 255.255.255.248
                set allowaccess ping
            next
            edit 2
                set ip 5.23.18.38 255.255.255.248
                set allowaccess ping
            next
            edit 3
                set ip 5.23.18.41 255.255.255.248
                set allowaccess ping
            next
            edit 4
                set ip 5.23.18.42 255.255.255.248
                set allowaccess ping
            next
            edit 5
                set ip 85.233.198.43 255.255.255.248
                set allowaccess ping
            next
        end
    next
end

 Have a good day,

Ariel

[funkylicious]

ok, based on the ones from above i would suggest the following:

1. secondary IPs are not necessary/mandatory, I would delete them from port35 or at least for .42 which you are testing VIP/VS with.

you want to use a specific Public IP for NAT ( SNAT ) just create/enable a IP Pool in the firewall policy in question and that's it. you can still use them like this and it removes the administrative overhead;

2. firewall policy lookup from CLI/GUI i dont think it will match the rule that activates hairpin nat ( rule 3 ) .

in your example i can see a VIP called test-vip but in the output you are showing a VS eqx-test-vip or is there a typo ?

for this firewall policy i would disable NAT

a, get router info routing-table details 10.120.0.123 would help, since I am seeing via DATA in the flow trace and the rules show dstintf DMZ , instead of DATA. which is the correct outgoing interface to reach destination 10.120.0.123 is it DMZ or DATA ? the command will help clarify.

[funkylicious]

this is a rather similar setup that I had to do for a client that wanted to access a hairpin nat VIP from sslvpn and for some reason it works, although i dont understand it's logic ( https://community.fortinet.com/t5/Support-Forum/VIP-Hairpin-NAT-issue/td-p/381414 ) 

try it out and see if it helps your case.

config firewall policy
    edit 0
        set srcintf "ssl.root"
        set dstintf "WAN"
        set action accept
        set srcaddr "SSLVPN_TUNNEL_ADDR1"
        set dstaddr "VIP-EXT_ADDRESS"
        set schedule "always"
        set service "ALL"
        set groups "SSL VPN"
    next
    edit 0
        set srcintf "WAN"
        set dstintf "LAN"
        set action accept
        set srcaddr "SSLVPN_TUNNEL_ADDR1"
        set dstaddr "viP_10.0.0.2-3050" "viP_10.0.0.2-3389"
        set schedule "always"
        set service "ALL"
    next
end

config firewall vip
    edit "viP_10.0.0.2-3050"
        set extip PUB-IP
        set mappedip "10.0.0.2"
        set extintf "any"
        set portforward enable
        set extport 3050
        set mappedport 3050
    next
    edit "viP_10.0.0.2-3389"
        set extip PUB-IP
        set mappedip "10.0.0.2"
        set extintf "any"
        set portforward enable
        set extport 53389
        set mappedport 3389
    next
end

show firewall address VIP-EXT_ADDRESS
config firewall address
    edit "VIP-EXT_ADDRESS"
        set subnet PUB-IP 255.255.255.255
    next
end

[ariel-anieli]

Thank you very much for giving us your time, @funkylicious; it's much appreciated.

Indeed, the interface is DATA, not DMZ. My bad. We disabled NAT on policy 3, and it still doesn't match traffic.

# get router info routing-table details 10.120.0.123

Routing table for VRF=0
Routing entry for 10.120.0.0/21
  Known via "connected", distance 0, metric 0, best
  * is directly connected, DATA



# diagnose debug flow trace start 2

# id=65308 trace_id=210 func=print_pkt_detail line=6005 msg="vd-root:0 received a packet(proto=6, 10.120.37.3:57648->5.23.18.42:443) tun_id=0.0.0.0 from ssl.root. flag [S], seq 2416725053, ack 0, win 64240"
id=65308 trace_id=210 func=init_ip_session_common line=6206 msg="allocate a new session-3f57ee7e"
id=65308 trace_id=210 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=69, len=13"
id=65308 trace_id=210 func=get_new_addr line=1280 msg="find SNAT: IP-5.23.18.36(from IPPOOL), port-57648"
id=65308 trace_id=210 func=fw_pre_route_handler line=191 msg="VIP-10.120.0.123:32443, outdev-unknown"
id=65308 trace_id=210 func=__ip_session_run_tuple line=3525 msg="DNAT 5.23.18.42:443->10.120.0.123:32443"
id=65308 trace_id=210 func=__vf_ip_route_input_rcu line=1989 msg="find a route: flag=00000000 gw-0.0.0.0 via DATA"
id=65308 trace_id=210 func=__iprope_tree_check line=524 msg="gnum-100004, use int hash, slot=36, len=9"
id=65308 trace_id=210 func=fw_forward_handler line=839 msg="Denied by forward policy check (policy 0)"
id=65308 trace_id=211 func=print_pkt_detail line=6005 msg="vd-root:0 received a packet(proto=6, 10.120.37.3:57648->5.23.18.42:443) tun_id=0.0.0.0 from ssl.root. flag [S], seq 2416725053, ack 0, win 64240"

  From what we see, network-wise, the configurations looks correct. But policy-wise, no traffic got matched.

Thanks for VPN configuration snippets, we are adapting them to our use case.

[funkylicious]

since you are NAT-ing the traffic from SSLVPN ( SNAT: IP-5.23.18.36(from IPPOOL) )  i would also create/add a object with this IP to the rule in question since the traffic is no longer 10.127.30.X as source.