How to Enable Local Traffic over IPsec Tunnel

[qskwood]

I route all Internet traffic through an IPsec tunnel except the connection to the remote tunnel peer.

I have been able to get SSL VPN connections, FTM acknowledgements, SNMP, etc. working in this way however since upgrading from 6.0.14 to 6.0.15 I have not been able to get FTM acknowledgements and SNMP working, although the SSL VPN connection itself does, and I cannot identify why. I do not see anything that has changed other than upgrading.

I have validated the local traffic is hitting the implicit deny.

fg90d $ di de flow filter addr 172.16.0.100

fg90d $ di de flow filter dport 161

fg90d $ di de flow trace start 1000

fg90d $ di de en

fg90d $ id=20085 trace_id=103 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:56936->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=103 func=init_ip_session_common line=5693 msg="allocate a new session-00000612"
id=20085 trace_id=103 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"

I have validated there is in fact a (newly created after seeing the issue, as a troubleshooting step) local in policy that matches this traffic which should be allowing it.

fg90d $ show firewall local-in-policy
config firewall local-in-policy
    edit 1
        set intf "Reverse-Proxy"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set service "ALL"
        set schedule "always"
    next
    edit 2
        set intf "wan1"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set service "ALL"
        set schedule "always"
    next
end

I have validated the objects in use are not configured in a way other than what I expect, and that SNMP is enabled on wan1.

fg90d $ show system interface Reverse-Proxy
config system interface
    edit "Reverse-Proxy"
        set vdom "root"
        set type tunnel
        set estimated-upstream-bandwidth 100000
        set estimated-downstream-bandwidth 100000
        set role wan
        set snmp-index 6
        set interface "wan1"
    next
end

fg90d $ show system interface wan1
config system interface
    edit "wan1"
        set vdom "root"
        set ip 172.16.0.100 255.255.255.0
        set allowaccess ping https ssh snmp ftm
        set type physical
        set alias "WAN"
        set estimated-upstream-bandwidth 100000
        set estimated-downstream-bandwidth 100000
        set role wan
        set snmp-index 1
    next
end

fg90d $ show firewall address all
config firewall address
    edit "all"
        set uuid caf1e28a-1dc9-51ed-2370-7ff1519e9b30
    next
end

fg90d $ show firewall service custom ALL
config firewall service custom
    edit "ALL"
        set category "General"
        set protocol IP
    next
end

I have validated that a non-local in policy (which was sufficient before) is configured to allow this traffic.

fg90d $ show firewall policy 7
config firewall policy
    edit 7
        set name "LibreNMS_to_FG90D_SNMP"
        set uuid ca072b8a-4ac6-51ed-121d-068c43fd588a
        set srcintf "Reverse-Proxy"
        set dstintf "wan1"
        set srcaddr "Web-Server"
        set dstaddr "FG90D_WAN"
        set action accept
        set schedule "always"
        set service "SNMP"
        set logtraffic all
        set fsso disable
    next
end

I have validated these objects are configured in the way I expect.

fg90d $ show firewall address FG90D_WAN
config firewall address
    edit "FG90D_WAN"
        set uuid ee3c3d46-3bb8-51ed-7daf-b3e57d78f113
        set subnet 172.16.0.100 255.255.255.255
    next
end

fg90d $ show firewall address Web-Server
config firewall address
    edit "Web-Server"
        set uuid 94770698-480a-51ed-c20a-2350d2177167
        set subnet 1.2.3.4 255.255.255.255
    next
end

fg90d $ show firewall service custom SNMP
config firewall service custom
    edit "SNMP"
        set category "Network Services"
        set tcp-portrange 161-162
        set udp-portrange 161-162
    next
end

I have validated this traffic was being allowed before.

Am I missing something here?

[Stephen_G]

Hello qskwood,

Thanks for getting in touch with us about this issue. Your post was automatically marked as spam in error: we're sorry for that. I have moved it out of spam quarantine and I am looking into getting a solution for you.

Kind regards,

Stephen

[distillednetwork]

You initial debug was cut off and can not see the deny, could you run it again and post the full output?  Also, try to run it with just the filter of:

di de flow filter addr <snmp server ip>

This will allow us to see the bi-directional traffic.  The request may be coming in but the reply could be having an issue as well.

[qskwood]

You are correct. Here is the updated debug with a more expanded filter as well as the full context.

fg90d $ di de dis

fg90d $ di de flow filter clear

fg90d $ di de flow filter addr 1.2.3.4

fg90d $ di de flow trace start 1000

fg90d $ di de en

fg90d $ id=20085 trace_id=1019 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1019 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdac"
id=20085 trace_id=1019 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1019 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1020 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1020 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdad"
id=20085 trace_id=1020 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1020 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1021 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1021 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdae"
id=20085 trace_id=1021 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1021 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1022 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1022 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdaf"
id=20085 trace_id=1022 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1022 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1023 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1023 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb0"
id=20085 trace_id=1023 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1023 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1024 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:38850->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1024 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb1"
id=20085 trace_id=1024 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1024 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1025 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1025 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb2"
id=20085 trace_id=1025 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1025 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1026 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1026 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb4"
id=20085 trace_id=1026 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1026 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1027 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1027 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb5"
id=20085 trace_id=1027 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1027 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1028 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1028 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb6"
id=20085 trace_id=1028 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1028 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1029 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1029 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb8"
id=20085 trace_id=1029 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1029 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1030 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=17, 1.2.3.4:59797->172.16.0.100:161) from Reverse-Proxy. "
id=20085 trace_id=1030 func=init_ip_session_common line=5693 msg="allocate a new session-0002bdb9"
id=20085 trace_id=1030 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1030 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"

[qskwood]

Adding the SNMP monitor to my admin login's trusted hosts list allowed this traffic.

How would I allow this monitoring traffic without also allowing admin logins from this IP address? Likewise with the FTM push notification server at port 4433. I can, I suppose, lock down SNMP monitoring to a relatively trustworthy IP address and not worry about admin logins. I cannot realistically lock down FTM push notifications this way; any IP can connect using the SSL VPN.

fg90d $ di de dis

fg90d $ di de flow filter clear

fg90d $ di de flow filter dport 4433

fg90d $ di de flow trace start 1000

fg90d $ di de en

fg90d $ id=20085 trace_id=1335 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1335 func=init_ip_session_common line=5693 msg="allocate a new session-0002c10f"
id=20085 trace_id=1335 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1335 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1336 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1336 func=init_ip_session_common line=5693 msg="allocate a new session-0002c112"
id=20085 trace_id=1336 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1336 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1337 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1337 func=init_ip_session_common line=5693 msg="allocate a new session-0002c123"
id=20085 trace_id=1337 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1337 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1338 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1338 func=init_ip_session_common line=5693 msg="allocate a new session-0002c12c"
id=20085 trace_id=1338 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1338 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1339 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1339 func=init_ip_session_common line=5693 msg="allocate a new session-0002c12d"
id=20085 trace_id=1339 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1339 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1340 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1340 func=init_ip_session_common line=5693 msg="allocate a new session-0002c134"
id=20085 trace_id=1340 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1340 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"
id=20085 trace_id=1341 func=print_pkt_detail line=5529 msg="vd-root received a packet(proto=6, 1.2.3.4:55028->172.16.0.100:4433) from Reverse-Proxy. flag [S], seq 4219844010, ack 0, win 65535"
id=20085 trace_id=1341 func=init_ip_session_common line=5693 msg="allocate a new session-0002c145"
id=20085 trace_id=1341 func=vf_ip4_route_input line=1604 msg="find a route: flags=80000000 gw-172.16.0.100 via root"
id=20085 trace_id=1341 func=fw_local_in_handler line=403 msg="iprope_in_check() check failed on policy 0, drop"

[qskwood · Accepted solution]

The resolution to the underlying problem is to stop using trusted hosts and instead use a combination of custom local in policies (for traffic within broadcast domains that needs to be denied and allowed) and normal firewall policies (for traffic from one broadcast domain to another.) In one relatively specific use case I needed both.

I now use a normal firewall policy to allow SNMP only from the needed Internet source IP for and implicitly deny it from elsewhere. I then use local in policies to reject HTTPS, SSH, and SNMP from the immediate upstream network, while retaining access from the needed hosts.

When I say custom local in policies, I mean ones that override the automatic ones created when enabling admin access on an interface. When administrative service access was enabled on the wan1 interface, local in policies were created behind the scenes that allowed all traffic directly entering the wan1 interface to access the firewall's HTTPS server, SSH server, and SNMP agent.

The custom policies shown below can be used to

1. restrict HTTPS, SSH, and SNMP access (policy ID 1 implicitly denies all sources)
2. allow SNMP access from only the Web-Server address object (policy ID 2)
3. allow HTTPS and SSH access from only the Trusted-Host address group object (policy ID 3)

fg90d $ show firewall local-in-policy
config firewall local-in-policy
    edit 3
        set intf "wan1"
        set srcaddr "Trusted-Host"
        set dstaddr "FG90D_WAN"
        set action accept
        set service "HTTPS" "SSH"
        set schedule "always"
    next
    edit 2
        set intf "wan1"
        set srcaddr "Web-Server"
        set dstaddr "FG90D_WAN"
        set action accept
        set service "SNMP"
        set schedule "always"
    next
    edit 1
        set intf "wan1"
        set srcaddr "all"
        set dstaddr "FG90D_WAN"
        set service "HTTPS" "SSH" "SNMP"
        set schedule "always"
    next
end

In this example, although the incoming SNMP traffic was from the Reverse-Proxy interface, local in policy ID 1 affected it despite being specific to wan1. This IPsec tunnel used wan1 so maybe this is why. Both a normal firewall policy and local in policy were needed for this specific use case where all WAN traffic routes through an IPsec tunnel. I suppose the firewall treats the IPsec traffic under the hood as directly entering the wan1 interface since that is where the tunnel terminates. In more modern firmware this behavior may work differently (in my opinion, it probably should treat these as separate but that may non-trivial to implement.)