Snat and dnat with 2 wans

[nicolasross]

Hi ! This will be rather complex, so bear with me.

In our setup, we have 2 wans with 2 isp and sd-wan enabled to load-balance/failover outbout trafic.

When a "regular" computer, phone whatever go out to the internet, SD-Wan will chose a link and that works fine. For most servers where we don't mind where the outbound trafic comes from, in the VIP, we enabled port forwarding and setup only incoming ports like 80, 443, and the like. Doing so outbound connections comes from the main IP of either of the wan link. But, for some servers, it's very important that outgoing trafic comes from the VIP of that server. And for those cases, we do have to VIP, one for each of the ISP external IP for that server. So for exemple we have : config firewall vip     edit "ext-4d-as-cyb-ws-test1"         set extip 1.1.1.1         set extintf "any"         set arp-reply disable         set mappedip "192.168.127.109"     next     edit "ext-4d-te-cyb-ws-test1"         set extip 2.2.2.2         set extintf "any"         set arp-reply disable         set mappedip "192.168.127.109"     next end

Policy looks like :

config firewall policy     edit 204         set srcintf "any"         set dstintf "any"         set srcaddr "all"         set dstaddr "ext-4d-te-cyb-ws-test1" "ext-4d-as-cyb-ws-test1" <---         set action accept         set schedule "always"         set service "HTTP" "HTTPS"     next end

Where 1.1.1.1 is the external IP for isp1, and guess that 2.2.2.2 is for isp2. Incoming traif is perfectly fine, comes in trough one wan link, goes out the same. Whenever some other computer on the network tries to talk to either of the externel IP, it just works.

When that server makes an outbound connection, it will pick up a link according to sd-wan policy, BUT the SNAT IP will be 1.1.1.1, regardless of the link choosen. While our ISP don't have an egress filter in place to prevent spoofing, it works more or less reliably. So outbound goes to wan2, and inbound replies comes to wan1.

That is far from ideal. So, back in the time where we setup the dual-isp sdwan and all, I added :

set srcintf-filter "wan1" to the first VIP, and wan2 to the second. That works perfecly. When the outbound connection would go to wan1, it'll pick 1.1.1.1 as SNAT ip, and 2.2.2.2 when going out on wan2. Now enter the problem part of this. With srfinf-filter enabled, when a someone on the network tries to access those exernal IPs, a SYN is sent out on the wan link choosen for that source, and the trafic goes out the wan link, but nothing comes back to the server. Followed me so far ?

Any one have an idea ?

[emnoc]

I don't follow you last part in your controlling your traffic flow, you should be able to do some this  with the SD-WAN rules and  your policy. The SYN are probably sent to the address of the VIP and is not replied. You probably need a hairpin and two policies

The "set srcintf-filter" is truly required here and is a must since you have a vip with ANY and  the same mapped-ip.

As far as spoof and thinking that is good, that is bad and leads to bad practices and future  possible issues if ISP and it's peer have src-verification enabled in the path.

just my 2cts observation

Ken Felix

[nicolasross]

What I was saying is that without srcintf-filter, I can reach the VIP from anywhere, even inside the network, as the FG will catch the traffic end sends it to the internal IP. In that case trafic goes from lan to dmz for exemple, dnated to the internal IP.

But with srcintf-filter, the trafic seems to begin to egress the wan interface, and into to void, without reaching the internal ip. And that is the problem I have. Trafic log show it's snated to the ip of the wan, as if I'd try to reach google.

[sw2090]

You could create an sd-wan rule that forces your server to outbound via a specific wan.

Alas in case of this wan failing the traffic will get load balanced over the other wan(s) until it gets back online again.

This is because health check sees that wan is down and takes it out of service so the sd-wan rule that does the outbount forcing cannot match anymore.

Then always the one last sd-wan rule will match and this is the Loadbalancing one.

[nicolasross]

sw2090 wrote:

You could create an sd-wan rule that forces your server to outbound via a specific wan.

Alas in case of this wan failing the traffic will get load balanced over the other wan(s) until it gets back online again.

This is because health check sees that wan is down and takes it out of service so the sd-wan rule that does the outbount forcing cannot match anymore.

Then always the one last sd-wan rule will match and this is the Loadbalancing one.

Yes, but still with the srcintf-filter, other IPs on the network won't be able to reach the external ip.