Facing an issue with NAT

[siil-itman]

I'm new to fortinet hardware and am currently having fun and games with a fortigate 401F running v7.4.8
We have two internet connections, a static connection with a /29 subnet of public IP's and a broadband connection. Most traffic is routed out of the broadband and we plan to pin only a few systems to the static connection.
For our static WAN,
.89 is the gateway
.90 is the firewall
.91-.94 we will use for services
The .90 is set as the address on the WAN and the 91-94 are added as secondary addresses.

I'm trying to set up a DMZ with one system in it at the moment. The DMZ network is running on a separate switch plugged directly into the firewall with the gateway on the firewall.

I've set policy routes for the internal traffic followed by a policy route for the external traffic plus an outbound rule using an ip pool with the address I want the server pinned on (.93)
If I run a "what's my IP" from the server it reports the .93 address and if I run curl commands or an apt update, it can pull information from the internet through the correct public IP as seen in the logs

What I can't get working is the incoming NAT from .93 into the server!

Everything I've read so far tells me that all I need to do is setup a VIP (1 to 1 or port to port - tried both) with the public IP and internal IP then setup a firewall rule with the Static WAN as the incoming interface, the DMZ interface as the outgoing and the virtual IP as the destination.
It's not working! I've got a web service on the DMZ system that I can access from all parts of the internal network but can't access from outside. The internal rules are getting triggered when I access the web service but I'm seeing nothing triggering on the outside to inside rule!
I've tried setting an inbound policy rule too but that breaks the working outbound connection

I'm probably missing something obvious here but I don't know what the hell it is!! If anyone can steer me in the right direction, it would be a great help before I take a hammer to the firewall!!

[Toshi_Esumi]

The first thing I would do is to sniff the out-to-in packets if they're really hitting the wan interface and the FGT is sending those out to DMZ interface with the real/local server IP. If hitting wan but not going out DMZ, obviously something is misconfigured in the policy and/or VIP.

Toshi

[siil-itman]

@Toshi_Esumi I'm seeing 'syn' attempts on the WAN interface but nothing beyond that so it's probably the routing policy or VIP setup that's at fault!

Below is the outbound route which is working fine for traffic requests from the server

Outbound Route

The inbound policy route looks to be the one causing the issue and the most confusion.
Incoming interface needs to be the WAN port
Source address, should that be 0.0.0.0 or the public IP
I'm assuming that the destination address needs to be the server in the DMZ
I've got the protocol set to TCP and destination port 80
I've set it to forward traffic to the DMZ port and gateway. 
The traffic is not getting there so there's something wrong with the rule as i've set the firewall pretty loosely at the moment until we get this working!

[Toshi_Esumi]

Just show us the inbound policy and the vip config in question. And if the out-to-in access to the server is HTTP or HTTPS, make sure you disabled HTTP/HTTPS admin access at port2.

It's unlikely affecting it though because your interface IP is .90 and the packet's dst is .93.

Toshi

[siil-itman]

We got it fixed after a lot of swearing! It sprang into life after we turned on asymmetric routing!
Thank you for your assistance @Toshi_Esumi  

[Toshi_Esumi]

It would not be a solution in most situation since if you enable asymm routing, the FGT wouldn't work as FW any more. It would work as just a router.
Instead, you should resolve why incoming and outgoing are through different interfaces.

Toshi

[siil-itman]

Fair enough @Toshi_Esumi 
We went through the fault finding yesterday
The diagnostic trace showed

func=ip_route_input_slow line=1696 msg="reverse path check fail, drop"

We then when through this
which is where we came up with the asymmetric routing

What we have set is laid out below but we need your advice on the inbound bit!
The default route for most users and systems is set to use
1.WAN (port17)
Admin Distance 10
Priority 1
This is a broadband connection

A small number of systems are going to use
2.STATIC (port2)
Admin Distance 8
Priority1
This is a static internet connection with a /29

The systems using the static connection are in a DMZ
DMZ-SW (port15)
This is set as the gateway for NET-DMZ a /24 subnet

We have set policy routes for the outgoing traffic as per below

Incoming IF	Outgoing IF	Source	Destination
DMZ-SW	Port-LAN	SRV-Web1	NET-LAN
DMZ-SW	2.Static	SRV-Web1	0.0.0.0/0

VIP's are set for each of the public IPs as type overload with just a single IP for each IP pool

Firewall we have setup a rule

From	To	Source	Destination	IP Pool	NAT
DMZ-WS	2.Static	SRV-Web1	All	public-94	Yes

With this setup, we can reach the internet and a "what's my IP" query showed the correct public IP.

For the incoming traffic, we setup Virtual IP's

Name	Interface	Mapped From	Mapped To
Web1-80	2.Static	***.***.***.94 (TCP:80)	***.***.***.10 (TCP:80)
Web1-443	2.Static	***.***.***.94 (TCP:443)	***.***.***.10 (TCP:443)

On the firewall we've setup 

From	To	Source	Destination	Service
2.Static	DMZ-SW	All	Web1-80 Web1-443	HTTP HTTPS

The bit we are struggling with is what to set for the inbound routing.
We can't do 1:1 routing on the public IP's as we are going to be running more than one service through them

Are we supposed to set a policy route?
If so, should the source address be 0.0.0.0/0 or the public IP for that service
We have tried both of the below and it's not been working for either

Incoming IF	Source	Destination	Protocol	Port	Forwarding	Outgoing IF
2.Static	0.0.0.0/0	SRV-Web1	TCP	80	Y	DMZ-SW
2.Static	***.***.***.94	SRV-Web1	TCP	80	Y	DMZ-SW

Please advise us on what we need to set to get port 80 traffic to the specific public ip on 2.Static into SRV-Web1 via the DMZ-SW interface as non of the guides and threads i've found online so far have been any help!

[Toshi_Esumi]

Whichever the /29 subnet is routed by the ISP, one default route needs to be pointing back to the incoming interface. Otherwise ends with "reverse path check fail, drop". Do you have default routes to both interfaces in your routing table? Check in below command:
   get router info routing-t all

Toshi

[siil-itman]

Routing table for VRF=0
S* 0.0.0.0/0 [5/0] via 192.168.18.1, port17, [1/0]
C ***.***.***.88/29 is directly connected, port2
C ***.***.9.0/24 is directly connected, port15

The last one is the DMZ subnet

[Toshi_Esumi · Accepted solution]

The routing table shows the port17 default route has admin distance 5. Since the admin distance of the default route toward port2 is higher than that, it wouldn't show up in the routing table. You need to match it with 5.

Toshi