AWS Fortigate Setup - possible routing issue.

[jreynolds]

Hi,

Note: This post was attempted earlier but seems not to have reach the forum. Am reposting with more detail. We've been evaluating the Fortigate appliance in AWS and followed setup guides (http://docs.fortinet.com/...e-aws-deployment.pdf). It's gone ok so far but we've hit an odd issue and were hoping for some pointers. We're not too experienced with with Fortigate so perhaps have overlooked something. Background [ul]

AWS Routing:Two route tables created[ul]

Public[ul]

has 10.0.5.0 subnet associated

default route is AWS Internet Gateway (IGW)[/ul]

Internal[ul]

has 10.0.217.0 subnet associated

default route is Internal interface of Fortigate Appliance[/ul][/ul][/ul][ul]

Fortigate Appliance [ul]

Two interfaces: Public (10.0.5.93) & Internal (10.0.217.79)

IPv4 Policies[ul]

From: Internal, To: External, Source: All, Dest: All, Service: All_ICMP, NAT: Enabled

From: Internal, To: External, Source: All, Dest: All, Service: HTTP/HTTPS, NAT: Enabled

Explicit Deny in place[/ul]

Can SSH into both Public & Internal interfaces and successfully ping external IP/FQDN[/ul][/ul][ul]

Test servers: [ul]

Test server 1: 10.0.217.215

Test server 2: 10.0.210.77[/ul][/ul]Problem Description [ul]

Both Test Servers are unable:

ping external IP/FQDN

reach via HTTP/HTTPS any external IP/FQDN

Both ICMP and HTTP/HTTP IPv4 Policies show Statistic Counters increases when each Test Server attempts outbound connection. 

The Explicit Deny Policy show's zero statistical counters[/ul]   Troubleshooting [ul]

Test 1: Explicit Proxy[ul]

We enabled the Explicit Proxy Feature

Enabled Explicit Proxy on the Private interface

Network->Explicit Policy: Default Settings.

Created an Explicit Proxy Policy -  Source: All, Destination: All

Configured Test Server 1 & 2 browser to use Fortigate Private IP as Proxy server

Both Test Servers were able to successfully browse to external FQDNs[/ul][/ul][ul]

Test 2: Different Route Table for Private Interface[ul]

Associated Private interface subnet with an existing route table

Existing route table default route = an existing NAT server with public IP

Both Test Servers were able to successfully browse FQDNs via HTTP and HTTPS

Note that the public IP seen as the source of the HTTP/HTTPS request is the NAT public IP, not the Fortigate public IP[/ul][/ul] Summary [ul]

Can reach only reach external IP/FQDN from Test servers when Explicit Proxy or NAT server is used as internal interface route 

Seems to be an issue with routing??

The only time we've seen something similar to this in AWS is when an internal server has no public IP or is not using a NAT instance. Effectively no route back from the internet. This seems to be confirmed by the fact that the IPv4 Policies work ok when routing via the existing NAT instance. It's as if the fortigate private interface isn't routing to/from the public interface, yet seems to be doing just then when Explicit Proxy is enabled.[/ul]  Any help appreciated.

[ede_pfau]

As noone else is responding...how about the routing on the FGT? At least it'll need a default route as it has to handle network addresses other than those directly on the interfaces.

Could you please post a 'get router info routing all' from the FGT?

[jreynolds]

Thanks for your reply Ede.

Here's the output:

S* 0.0.0.0/0 [5/0] via 10.0.5.1, port1 [5/0] via 10.0.217.1, port2 C 10.0.5.0/24 is directly connected, port1 C 10.0.217.0/24 is directly connected, port2

Given that traffic can traverse without issue when Explicit Proxy is enabled, I'd have thought that routing would be ok. Might be showing my ignorance here.

Thanks again

[MikePruett]

I noticed that you don't mention a policy that allows DNS traffic out. Obviously, this would only cause problems with your FQDN tests and not the IPs but just something that caught my eye at a quick glance. This is for test1. 

[jreynolds]

Good point, Mike. Thanks. We're using an internal DNS server so lookups are directed to that. The FQDN resolution is working fine (can be seen to resolve okay in the cli for pings, telnet etc). Thanks again though. Any other tips/insights welcomed. Cheers.

[MikePruett]

make a machine do a continuous ping to one of the boxes. Run a trace flow on the FortiGateto see if it is properly entering and leaving the FortiGate. That will provide some insight into what could be happening (if the FortiGate is wigging or not).

The more info you have on that the better IMO. Will keep you from pulling your hair out and punching babies haha

[jreynolds]

Thanks Mike. Will give it a try and let you know.

btw, babies are safe but hair was entirley pulled out last week due to this one...sigh..

[ede_pfau]

I think the second default route is the culprit. What do you need it for (0.0.0.0/0 via port2 to 10.0.217.1)? If all your servers are in the internal 10.0.217.0 LAN then the 'connected' route will do fine. If you use IPs different from that you would create one route per subnet for these and not use a default route.

So, I'd delete the internal default route and retest.

As mentioned, you could run diagnostics on the FGT, 'diag debug flow' will clearly show you where the traffic is going. But this takes a bit of effort if you don't run that everyday. Besides, planning always tops debugging.

Regarding DNS, no wonder the FGT can resolve FQDNs as it's using it's system DNS. I would let all hosts use the FGT as DNS (per DHCP), and let the FGT relay DNS to the DNS you wish to use. This way, you have only one spot where to configure an external DNS, and you can prevent rogue DNS requests completely (just deny any DNS from internal to public).

@Mike: the baby joke is just rude and tasteless. It happens in real life, and each time is one time too many.