Help with WAN Additional IPs and Testing

[imei-ianv]

Hey team,

My setup is:

FG100E on 7.0.10

wan1 - 203.1.1.1/30 with additional IPs routed 203.2.2.1/27

wan2 - 165.1.1.1/30 with additional IPs routed 165.2.2.1/28

static route - 0.0.0.0/0 via 203.1.1.1 gateway - priority 10

static route - 0.0.0.0/0 via 165.1.1.1 gateway - priority 20

services setup as firewall policies using Virtual IPs on 203.2.2.1/27 just fine

Ever since this FG100E was setup, I have been suspicious of the additional IPs on 165.2.2.1/28 actually working.

The plan would be, if wan1 goes down for an extended period of time, outbound internet is fine as the link monitor disables the route to wan1, but we would then manually change DNS to point to IPs on 165.2.2.1/28, matching preconfigured firewall policies.

I wanted to setup a test, to prove I can use them.

Picked one test device. Configured a Policy Route to point its traffic out of wan2. Then a Firewall Policy to NAT its outbound traffic using IP 165.2.2.2. Public what is my IP pages show the correct IP.

Then I configured a Firewall Policy and Virtual IP pointing 165.2.2.2 to this device for port 3389 (something that I know works right now).

I can't get through.

Looking at diag sniffer and diag debug flow shows that the traffic is hitting the FGT, but gives me:

2023-03-20 16:46:22 id=20085 trace_id=1 func=init_ip_session_common line=6024 msg="allocate a new session-009ab520, tun_id=0.0.0.0"
2023-03-20 16:46:22 id=20085 trace_id=1 func=get_new_addr line=1225 msg="find DNAT: IP-10.1.1.107, port-0(fixed port)"
2023-03-20 16:46:22 id=20085 trace_id=1 func=fw_pre_route_handler line=181 msg="VIP-10.1.1.107:3389, outdev-wan2"
2023-03-20 16:46:22 id=20085 trace_id=1 func=__ip_session_run_tuple line=3489 msg="DNAT 165.2.2.2:3389->10.1.1.107:3389"
2023-03-20 16:46:22 id=20085 trace_id=1 func=ip_route_input_slow line=2267 msg="reverse path check fail, drop"
2023-03-20 16:46:22 id=20085 trace_id=1 func=ip_session_handle_no_dst line=6110 msg="trace"

And my online research tells me, it can be due to FGT not having a route back to the source.

And I don't, when checking get router info routing-table all. But I also don't for any IP in 203.2.2.1/27, and their traffic is working ok.

So, what should I do? Configure a route for the additional IPs to ... where?

This can't be routing to the destination 10.1.1.107, because it has one.

Or maybe this should be a question of, what is the proper/correct way to configure additional routed IPs into a FGT?

[funkylicious]

I would suggest creating an SD-WAN zone w/ those 2, wan1 and wan2 and that might solve your issue.

[imei-ianv]

How does an SDWAN help?

[imei-ianv]

Found a solution? No. I know where SDWAN is, but am not prepared to put it into production without understanding it further.

One thing I have found but haven't tested yet, is that on a WAN interface you can specify 'Secondary IP address'. Couldn't find much online about it and it's interaction with routing and the error I am getting.

[Toshi_Esumi]

It generally means your link-monitor is somehow not removing the default route to wan1. Did you confirm it's not there any more when wan1 was down?
Also make sure you have below configured:

config system global
  set snat-route-change enable  (by default it's disabled)
end

Toshi

[imei-ianv]

For my tests, I intended to design it so that no wan link needs to go down. So, no, wan1 was never down.

[Toshi_Esumi]

I think if you shut down wan1 it would work because the FGT currently seems to be seeing the wan1 should be the returning path. So once wan1 is gone, it would be forced to use wan2.

But even when wan1 is up, I don't see a particular reason your test setting doesn't work.

I would try below then test again.

# diag sys session filter src <your_test_source_IP>

# diag sys session clear

then make sure:

# diag sys session list
total session 0

If still doesn't work after this, I have no other idea so I would open a ticket at TAC.

Toshi