Traffic going to odd address instead of tunnel?

[rhardiman]

We expanded a subnet at a remote site and traffic from our main site to addresses in the new part of the remote subnet does not work. I have the correct subnet mask on the routes and on the IPSec VPN tunnel. I see the traffic in Forward Traffic being accepted and destined for the VPN interface, but if I do a traceroute the next hop after our firewall is 10.10.10.1 which is not on any network, route, or interface that we have at any site.

Traceroutes from a workstation show the firewall as the first hop and 10.10.10.1 as the second. Traceroutes from the firewall show that address as the first hop.

I'm sure more info is needed, please let me know what I can provide.

[rhardiman]

These all look healthy, except the diag debug flow, which gives the following:

id=65308 trace_id=1861 func=print_pkt_detail line=5879 msg="vd-root:0 received a packet(proto=1, MY IP:1->192.168.111.38:2048) tun_id=TUNNEL IP from VPN TUNNEL NAME. type=8, code=0, id=1, seq=2183."
id=65308 trace_id=1861 func=resolve_ip_tuple_fast line=5967 msg="Find an existing session, id-14df90ee, original direction"
id=65308 trace_id=1861 func=npu_handle_session44 line=1224 msg="Trying to offloading session from VPN TUNNEL NAME to lan, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x02040000"
id=65308 trace_id=1861 func=ip_session_install_npu_session line=364 msg="npu session installation succeeded"
id=65308 trace_id=1861 func=fw_forward_dirty_handler line=442 msg="state=00010204, state2=00000001, npu_state=02000400"

I can't find any info about that dirty handler message and am not sure what it means. 

Additionally, I ran get router info routing-table details 192.168.111.38 on both ends of the tunnel and both show correct routes, which adds to my confusion.

[rhardiman]

Bit of a breakthrough, I found that 10.10.10.1 DOES exist on the remote site router as the dmz interface. I changed that address and the traffic tried to go to the new address, so that IS where it's getting that from. Our dmz is tied to a physical port that is disconnected, so why is it trying to route traffic there? Even if I check the route to the dmz interface ip, it tells me it would use the default route out rather than the dmz interface.

[pminarik]

If the other side is FortiGate

+ if the tunnel does not have an IP assigned on the other side

=> the other side will not have an IP to use for the traceroute reply (it wants to use the egress interface IP towards the destination = the traceroute initiator), so it will use something "random". (typically the lowest-id interface's IP addr)

In order to see a meaningful IP in the traceroutes, add IPs to your tunnel interfaces (any sane /32s should be fine). If you want them to pass through tunnels and be router properly, make sure these new IPs are covered in phase2 selectors, routing tables, and firewall policies (if needed).

Nothing is being routed thorugh that DMZ IP/interface, it's just an artifact of the FGT not knowing which IP to use for tracert reply.

[Toshi_Esumi]

I think because the DMZ IP is the smallest IP on the remote FGT. If you change it to like 254.254.254.254/32, probably you would see another interface IP in your traceroute.

But, the problem you want to solve is not the IP in the traceroute, but reachability when you expanded the remote subnet from 192.168.111(or 110).0/24 to 192.168.110.0/23. Right? Can you actually ping the destination 111.38 from the remote FGT? And, are you sure the destination device has the /23 in the subnet mask? I'm assuming the device's default GW is the remote FGT's interface IP.

Toshi