Dual WANs - Separate subnets

[ctzchris]

I have 2 separate networks, one using WAN1 and the other using WAN2 for internet (separate switches on LAN and WAN sides).  On the LAN side, port1 is for the first network and port 4 is for the second network.

I thought I had everything working well until we ran into a situation where one of the networks had to access internet facing services on the other network via their public IP. 

Policy routes are configured so all traffic from port1 goes outbound on WAN1 and all from port4 goes outbound to WAN2.

I also have a stop policy routing configured where if the destination IP of the other WAN is matched then it should drop to static routes, which I thought would send out to the ISP and route back in on the other WAN.  This does not appear to be happening.  No matter what I do with policy routes (stop routing, or force it out to the next hop) it seems to be ignored.

When accessing the services from a different network, ie not behind the Fortigate, then everything works which tells me the VIP is configured properly.

Any advice would be greatly appreciated.

[Toshi_Esumi]

It's been a while since we used policy routes last time, which wouldn't fit well in any failover situations. So I don't remember well but I thought it worked when you have the same/similar routes and you can specify one way by the policy route or the other depending on some parameters, mainly by the source. If the destination is "connected" it might now work. That's my guess.

But based on what you described, it seems to be the perfect situation to split your FG into two vdoms so if one side needs to access the outside interface of the other vdom, it would go out to the ISP and comeback to the other side. Just like you have two separate FWs with two separate internet circuits.

Only in case you have to connect them internally for whatever the reason is, you can use vdom-link to "leak" packets to the other side. The best part is they have separate routing-tables without contamination by the other's.

[ctzchris]

The problem was related to policy routes and WAN priority.  After giving wan2 a lower priority the policy routes started responding, originally it seemed to ignore whatever I put in there short of all/wan1, all/wan2 policy routes.  Setting a policy route for LAN-to-LAN with the destination as the LAN/inside IP of the VIP got the IP's responding as expected.  I also had to change the VIPs to 'any' interface.  

After all that the decision was made to move the WAN2 VIPs (only 2) over to WAN1 and use WAN2 for failover instead.  It was a good learning experience though.  Next time I need to keep things completely separate, I will definitely use 2 VDOMs.  I'm pretty sure that is what they designed it for.  Thank you Toshi for the suggestion.