Routing issue - FortiGate VM02 - intra-interface routing

[farroar]

I have a virtual VM02 FortiGate. Two interfaces, inside and transit. Behind the transit interface is multiple subnets. All subnets can ping to the interface of the the transit interface. What I cannot get to work is connectivity between subnets VIA the fortigate.

The policies are in place to allow traffic and I do see inbound traffic hitting the counters. An example of a policy is:

source int: transit

dest int: transit

source IP: 10.0.0.0/24

dest IP: 10.0.2.0/24

The only way I can get this communication to work is to enable NAT on the policy. This works fine for ping but will break directed IP traffic.

I have all of the static routes in place and have no issues again with FortiGate to subnet traffic, it's just when the traffic traverses the fortigate between the subnets. I can only imagine it has something to do with the interface being the same ingress and egress. I don't think this could trigger RPF rules since the routes are there (flow logs do not show any issues that I can see)

I cannot use VLANs to separate the traffic and I cannot utilize additional interfaces. 

I also tried to set the allowed-traffic-redirect option to enabled and it didn't seem to help

[orani]

Your policy does not seem correct. Your source and destination interfaces are same.

[ede_pfau]

Your policy is 100% correct (if not a bit unusual), this is what is needed in your situation.

The hint with NAT points to the hosts: if they don't know where to send reply traffic to 'unknown' subnets they would just drop it. You could sniff for reply traffic on the target interface - if correct, you'll see the request coming in, and the reply going out.

So, please check the setting of the default route on your hosts. If DHCP provided, this is not much work.

And RPF doesn't apply here as all subnets are directly connected (are they?). If you host multiple subnets on one interface you need to create static routes for them.

[farroar]

Here's a bit more context, I didn't want to bring it up since it might muddy the waters a bit.

This is in Azure and is running a multi-subscription model. Each VNet consists of multiple subnets. Each subnet (and each host) has a default route pointing to the transit interface IP of the FortiGate. There are multiple VNets 'peered' back to the VNet where the FortiGate is placed. SO, from the FortiGate's perspective, all of these other networks are reachable via the same interface... AND all communication in both directions will use the single interface. The details of why this happens in all based on the architecture of Azure. It is kinda like a router on a stick but without the VLANs. It is a bit different but makes sense in the context of Azure.

When I turn on NAT on the policies, the FortiGate is doing source NAT to the IP address of the interface. The hosts just respond to that IP and it works.

I have done packet captures on all points. I see the data get to the firewall but it does not then leave the firewall (when not using NAT). SO, I am fairly sure that the firewall is preventing this traffic but I'm not sure why.

This is like a hairpin, but without the NAT.

[farroar]

This is kind of what I would think is happening and what would fix it, but it doesn't seem to help:

https://kb.fortinet.com/kb/documentLink.do?externalID=FD43937