FG100F not routing between subnets

[jdw314]

Basic functions not working and I cant figure out why.    This is my first Fortigate firewall.  Experienced with Palo Altos.   

I have a remote industrial facility with 2 WANs and multiple LAN subnets all on a single LAN interface.   

I have configured my subnets under Port 1 of the firewall with their respective VLAN IDs and IP/mask. 

From each subnet, I can ping and access the Fortinet firewall on the respective IP. 

However no traffic can make it from one subnet to another.  The FG100F is not routing between the subnets.   

I have traffic rules in place for the intra LAN traffic that should be allowed, with NAT disabled.    And for diagnostic purposes I created an allow all rule from one subnet to another, and still nothing.

I was expecting the FG100F to automatically route between subnets as long as policy allows the traffic, but it appears these devices do not do that?

So I added static routes from one subnet to another.   Still does not work.  

The traffic logs are completely blank - no dropped traffic, no allowed traffic, nothing.   I have all the log options turned on to log everything.  

One thing I do like about this FG100F is it has a nice packet capture tool.  I ran packet capture on one of the subnet interfaces while maintaining a standing ping from one subnet to another, but the packets for the ping do not show up.  

What is the secret to get this thing to route from one subnet to another where policy allows?

Thanks

[Debbie_FTNT]

Hi @jdw314

and welcome to the Fortinet family :).

To make sure I have a correct grasp of your FortiGate - you have a 100F, with multiple VLAN interfaces all on the same port, and are looking to have the FortiGate route traffic between the VLANs, correct?
The basic configuration you will need:
- routing information
-> FortiGate will know the connected routes of the VLAN interfaces, based on its own IP in there, but will need static or dynamic routing for any subnets beyond the connected ones
- policies from vlan interface to vlan interface (not the physical interface!), with action allow and optional security profiles, NAT, etc.

From your description, it sounds as if you already have these two things in place.

For troubleshooting, there are two very useful commands:

- the diag sniffer command, which is basically tcpdump and gives you the same information as the packet capture tool

dia sniff packet <interface|any> 'host x.x.x.x and/or host a.a.a.a and port (c or d)' 4 <number of packets> <a/l, for absolute (GMT) or local timestamp>

end the output with Ctrl+C
-> this will show what interface a packet comes in on, and what interface it leaves the FortiGate on. In the case of VLAN interfaces, you should see the same packet twice - once on the physical interface, once on the VLAN interface (if you don't see it on the VLAN interface, this could mean a missing VLAN tag, for example)

- the dia de flow set of commands; this displays packet handling and routing lookup

dia de reset
dia de flow filter addr <x.x.x.x>
dia de flow filter proto <IP protocol, for example 1, meaning icmp, or 6, meaning tcp>
dia de flow filter port <a>
dia de flow filter [...]
dia de flow trace start <number of packets>
dia de en

This will start dumping packets in CLI and show what the FortiGate does with them, if this is for a new session setup.
If there is an existing traffic session, this set of commands might not show anything, as traffic would be offloaded to an ASIC and thus not visible to debug.

In that case, you might want to clear sessions first:

dia sys session filter src <x.x.x.x>
dia sys session list ## to see the sessions
dia sys session clear ## to clear the filtered sessions; if no filter is applied, this clears ALL sessions

#dia sys session filter clear ## to clear the session filter in place

To end the debug:

dia de dis
dia de reset

These two commands should give you a starting point as to what FortiGate is doing with the traffic.

Regarding logging - your 100F does not have a disk, if I remember, meaning it logs to memory be default, and memory logging is restricted to severity warning or higher.

This can be changed via CLI:

config log memory filter
set severity info
end

Also, if for some reason the traffic is hitting implicit deny, you might need to enable logging in the implicit deny as well; this can be done in the policy table by clicking on the log setting in implicit deny and selecting 'All'.

I hope this helps you figure out what's going on :)

[jdw314]

@Debbie_FTNT  Thank you for your reply with good suggestions.  I have tried them and unfortunately I am still unable to solve this problem.   

I set up a continuous ping between two end point devices, one on VLAN 120 and the other on VLAN 180 which are both sub interfaces of Port 1.  And I have a rule that allows all ICMP traffic from VLAN 120 to VLAN 180 and another rule that allows all ICMP VLAN 180 to VLAN 120.

With that test setup in place, I tried to use the tools you suggested.    I did Set Severity INFO, but that made no difference - no traffic was logged in the traffic logs.  I thought maybe I needed to save the config after setting severity info, but when I tried to save it it said no changes.   

I tried the sniff packet tool, and as you stated it is basically the same as the packet capture.  One possibly vital clue is that the sniff packet/packet capture report does not show the continuous ping on any interface - its not on VLAN 120, its not on Port 1, its not on VLAN 180, etc.   The FG doesnt seem to believe that it has received those ping packets.   Which likely explains why it is not routing them, but the real question is where did they go?    From the end point devices, I can ping the FG100F on their respective subnets for VLAN 120 and VLAN 180 without any difficulty.  

I was not able to make the debug flow tool work.   I copied what you pasted above, except for the line 

dia de flow filter [...]

which I believe was placeholder for additional filter criteria.   But I got no output at all.  Nothing displayed when I did dia de en.   I'm clearly doing something wrong when I use that tool but I'm not sure what.   Although there should be no sessions offloaded to the ASIC right now since the only traffic right now is my continuous ICMP ping, nevertheless I tried clearing sessions anyway but that resets my connection to the CLI.   So if it causes any output to be generated, I don't get to see it.  

From the CLI, if I do execute ping with source in AUTO I can successfully ping both of the end points on VLAN 120 and VLAN 180.   But if I set the source to the IP of the FG on one of the subnets, then I can only ping the endpoint on that same layer 2 subnet.  It wont route its own pings to the other subnet.   

Any further suggestions?  Is there any way to get live help on something like this through Fortinet?   One difficulty we have as contract engineers is that we don't purchase the devices directly, nor do we hold the licenses to them, and we have no input on what support packages the client purchases.   Our role is only to integrate the device into the overall network and configure it.   For this reason we try to steer our clients to OEMs who provide at least basic support to anyone operating the device.  

Thanks

[Debbie_FTNT]

Hey @jdw314,

live support is only possible through a support ticket.

If the device has even a basic support package, that should include online support, which is all that is required to get a call or remote session with a support engineer.

The FortiGate's serial number is usually sufficient to open a ticket; if you do not have access to the support account the FortiGate is in, that could complicate matters a little, though. In that case, you might want to speak to whoever registered the device in their support account to have your own email added to their account to get access to the support page for things like downloading firmware to upgrade, and opening support tickets.

As for other things to troubleshoot:

If traffic is not hitting the FortiGate, try to determine where it is going instead.

Perhaps run a traceroute command on your VLAN clients to see what hops it goes through?

@ESCHAN_FTNTsuggestion is also a good idea - check the routing settings on your hosts.

Let us know how it goes :)

[ESCHAN_FTNT]

Hi jdw314

When device in VLAN120 trying to ping device in VLAN180, normally the traffic would go to default gateway (unless you have specified a static route configured on the device). In this case, I am thinking the device you trying to ping from VLAN120 default gateway is set to somewhere else (not FGT). This explains that when you try to ping to VLAN180, the ping do not even reach the FGT (but sending to the configured default gateway), but on the same device you can ping FGT IP in VLAN180 because they are in the same subnet. Please do check the default gateway setting (or static route) for both the device in VLAN120 and also VLAN180 (You need to check on VLAN180 also for return traffic).

[Toshi_Esumi]

If both VLAN subnets are configured on the same port1, and the IP on the both VLANs are configured (or pulled via DHCP) as GW on the ping source and destination devices, those are connected networks and no need for any extra routing.

As long as the ping packets are coming in, and a policy exists between two VLAN interfaces, nothing should block the packets to come though the FGT.

When you sniff, you might need to diable asic offloading ("set auto-asic-offload disable") on the policy to see all. Otherwise, you would see only first a couple of ping packets before the session is offloaded.

My guess is it's not even hitting the FGT. Just run sniffing after disabling offload on the incoming vlan interface, or use "any" then add "4" option in Debbie_FTNT's port.

[Toshi_Esumi]

Just to make sure, one policy can cover only one direction. You need a set of policies to allow both directions of pinging.