Fortigate not routing request from client

[netserv]

Good evening, 

Experiencing an issue with FG Routing can anyone assist, 

I have the following setup on a Proxmox node: 

1. Ubuntu (Client) 

2. Foritgate Firewall 

The Client has two interfaces attached with the following networks: 

1. 192.168.61.X for management 

2. 192.168.10.x (shared network to reach the Fortigate) LAN

The Fortigate has three interfaces attached with the following networks: 

1. 192.168.61.X for management 

2. 192.168.10.x (shared network to reach the client ) LAN

3. 192.168.100.x (WAN Interface to reach wan router)

I cannot get internet on the client through the FG.  I can ping between both the client and the FG on the  192.168.10.0 network but if i try to ping an external network from the client with the FG as the gateway there is no reply. 

After running sniffers and debug flow i can see the packet reaching the Fortigate with the following output: 

FortiOS-VM64-KVM # diag deb en
FortiOS-VM64-KVM # diag sniffer packet any 'host 192.168.10.2'
Using Original Sniffing Mode interfaces=[any]
filters=[host 192.168.10.2] 0.262386 192.168.10.2 -> 8.8.8.8: icmp: echo request
1.286294 192.168.10.2 -> 8.8.8.8: icmp: echo request
2.310243 192.168.10.2 -> 8.8.8.8: icmp: echo request

FortiOS-VM64-KVM # diag deb en
FortiOS-VM64-KVM # diag deb flow filter saddr 192.168.10.2
FortiOS-VM64-KVM # diag debug flow trace start 100
FortiOS-VM64-KVM # id=20085 trace_id=1 func=print_pkt_detail line=5742 msg="vd-root:0 received a packet(proto=1, 192.168.10.2:7->8.8.8.8:2048) from port2. type=8, code=0, id=7, seq=89."
id=20085 trace_id=1 func=init_ip_session_common line=5913 msg="allocate a new session-0000047c"
id=20085 trace_id=1 func=vf_ip_route_input_common line=2621 msg="find a route: flag=04000000 gw-192.168.100.1 via port1"
id=20085 trace_id=2 func=print_pkt_detail line=5742 msg="vd-root:0 received a packet(proto=1, 192.168.10.2:7->8.8.8.8:2048) from port2. type=8, code=0, id=7, seq=90."
id=20085 trace_id=2 func=init_ip_session_common line=5913 msg="allocate a new session-0000047d"
id=20085 trace_id=2 func=vf_ip_route_input_common line=2621 msg="find a route: flag=04000000 gw-192.168.100.1 via port1"
id=20085 trace_id=3 func=print_pkt_detail line=5742 msg="vd-root:0 received a packet(proto=1, 192.168.10.2:7->8.8.8.8:2048) from port2. type=8, code=0, id=7, seq=91."
id=20085 trace_id=3 func=init_ip_session_common line=5913 msg="allocate a new session-0000047e"
id=20085 trace_id=3 func=vf_ip_route_input_common line=2621 msg="find a route: flag=04000000 gw-192.168.100.1 via port1"

there is a statiic route on the Fortigate using the WAN interface and from the FG i can ping anything on the internet so there is internet access on the firewall:

FortiOS-VM64-KVM # exe ping www.google.com
PING www.google.com (142.251.35.228): 56 data bytes 64 bytes from 142.251.35.228:
icmp_seq=0 ttl=113 time=46.6 ms 64 bytes from 142.251.35.228:
 
FortiOS-VM64-KVM # get router info routing-table all C
candidate default Routing table for VRF=0 S*
C 0.0.0.0/0 [5/0] via 192.168.100.1, port1
C 192.168.10.0/24 is directly connected, port2
C 192.168.61.0/24 is directly connected, mgmt
192.168.100.0/24 is directly connected, port1
 

Based on the debugs it seems fortigate cannot find a route once the intial packet comes from the LAN network but has no trouble if the traffic initiates from itself. 

can anyone assist in troubleshooting issue.

thanks for your assistance. 

regards, 

#fortigate, #proxmox

[amouawad]

My guess is that the firewall policy on the FortiGate does not have NAT enabled. This means when the packets are sent to the router the source IP will be from 192.168.10.2 which I'm guessing the router doesn't have a route to (so will use it's default route to the internet).

So the quick fix is to enable NAT on the FortiGate firewall policy.

[Toshi_Esumi]

I agree with amouawad. Becaue the FGT never gets reply from the router in the flow debug and tries to create a new session every time. If you sniff packets on port1, you would see nothing is coming back when the client pings.
But my quick fix would be putting in a static route for 192.168.10.0/24 toward the FGT(192.168.100.x) on port1 at the wan router.

Toshi

[netserv]

Thanks for your reply Toshi_Esumi, 

The thing is that because NAT is enabled the WAN Router shouldn't see any 10.0/24 addresses only addressed natted based on the FG interface connected to it and reply to that. 

I do see what youre saying though, however since my wan router is a Huawei unit provided by my ISP it doesnt seem to have adjustable routing like the home linksys series etc. 

if you think of anything else let me know though, seems FG NAT is not working to be honest 

[netserv]

Hi amouawad, 

thanks for your reply, the firewall policy does have NAT enabled. 

[amouawad]

Just for a quick sanity check, are you able to change the source address to 'all' in the policy and test (just to make sure it's not the address configuration).

Also can you try the following sniffer and ping 9.9.9.9 from the client PC (take note the number 4 at the end)? 

diag sniffer packet any 'host 9.9.9.9' 4

If we do the sniffer on the destination IP and specify debug level 4, we should see the interface names in the capture, and hopefully see a ping come in port2 and another exit from port1.

Just trying to isolate where the issue is here for now.

[netserv]

"Just for a quick sanity check, are you able to change the source address to 'all' in the policy and test (just to make sure it's not the address configuration)."

Yes I had it on ALL ALL previously for the sake of testing it still didn't work. That's when I specified the subnet. 

I will do the 

diag sniffer packet any 'host 9.9.9.9' 4

and post the results as soon as i get back home. on work at the moment. 

Thanks so much for your help. 

[netserv]

Hi amouawad, 

             So i attempted the steps you outlined: 

Quick note, i changed the LAN IP's to 192.168.2.0/24 

and the fortigate interface to 192.168.2.2 

here are the results: 

FortiOS-VM64-KVM # diag deb enable

FortiOS-VM64-KVM # diag sniffer packet any 'host 192.168.2.3' 4
Using Original Sniffing Mode
interfaces=[any]
filters=[host 192.168.2.3]
4.549108 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
5.567788 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
6.592218 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
7.616188 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
8.640176 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
9.664197 port2 in arp who-has 192.168.2.2 tell 192.168.2.3
9.664237 port2 out arp reply 192.168.2.2 is-at fa:a7:58:a1:db:c5
9.664263 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
10.687812 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
11.711693 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
12.736394 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
13.760405 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
14.784011 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
15.807769 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
16.831612 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
17.855592 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
18.880027 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
^C
17 packets received by filter
0 packets dropped by kernel

FortiOS-VM64-KVM # diag deb enable

FortiOS-VM64-KVM # diag sniffer packet any 'host 8.8.8.8' 4
Using Original Sniffing Mode
interfaces=[any]
filters=[host 8.8.8.8]
4.055568 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
5.078804 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
6.103220 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
7.127214 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
8.150861 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
9.174722 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
10.199210 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
11.222843 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
12.247123 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
13.270727 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
14.294718 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
15.318632 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
16.343061 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
17.367040 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
18.391047 port2 in 192.168.2.3 -> 8.8.8.8: icmp: echo request
^C
16 packets received by filter
0 packets dropped by kernel

Both filters (source and destination) dont seem to give any additional details even with the debug level set to 4. 

do you think there is anything else we can look at ? 

[Toshi_Esumi]

Then, somehow the NAT is not working. Or based on the flow debug, it's not matching any policy for whatever the reason is. Something is quite odd. You should open a ticket at TAC.

Normal NAT flow should show like below. As in the 4th line, it's matching policy#1 and SNATed.

id=20085 trace_id=24 func=print_pkt_detail line=5727 msg="vd-root:0 received a packet(proto=1, 172.16.2.104:1->1.1.1.1:2048) from down_link. type=8, code=0, id=1, seq=78."
id=20085 trace_id=24 func=init_ip_session_common line=5898 msg="allocate a new session-004584b2"
id=20085 trace_id=24 func=vf_ip_route_input_common line=2621 msg="find a route: flag=00000000 gw-<MY_GW> via ppp1"
id=20085 trace_id=24 func=fw_forward_handler line=799 msg="Allowed by Policy-1: SNAT"
id=20085 trace_id=24 func=ids_receive line=298 msg="send to ips"
id=20085 trace_id=24 func=__ip_session_run_tuple line=3470 msg="SNAT 172.16.2.104-><MY_IP>:60417"
id=20085 trace_id=24 func=ipd_post_route_handler line=490 msg="out ppp1 vwl_zone_id 0, state2 0x0, quality 0.

Toshi

[netserv]

Yeah i believe that you are right NAT is not working, based on the debug output im not seeing anything nat related. 

"Something is quite odd"  Exactly my thoughts. 

Your output is what id expect..