aws internet traffic and aws

[toan2552]

Hi,

I have an IPsec connection set up from AWS to Fortigate.

In AWS, there's a private subnet containing various services, including EC2 instances.

Within my Fortinet, there are two networks - a DMZ network and an internal network, and they can communicate with the EC2 instances without any issues.

However, I'm currently facing a challenge: I want to enable internet access for the EC2 instances through the IPsec connection, following this path:

EC2 ===> Fortigate 1 ===> Internet

To achieve this, I've configured the AWS route table to have a route with destination 0.0.0.0/0 pointing to the virtual private gateway (VGW) to handle internet-bound traffic.

On the Fortigate side, I've implemented two policies. The first policy is to allow traffic from the WAN to AWS IPsec, and the second policy is to allow traffic from AWS IPsec to the WAN.

Unfortunately, despite these configurations, the setup isn't functioning as expected. When capturing traffic on the Fortigate, the results show:

1 0.000000 192.168.16.44 8.8.8.8 ICMP 60 Echo (ping) request id=0x0001, seq=51820/27850, ttl=128 (no response found!)

This suggests that the ping request from 192.168.16.44 (presumably one of the EC2 instances) to 8.8.8.8 (Google's DNS server) did not receive a response.

I'd appreciate any guidance or suggestions to troubleshoot and resolve this issue.

Thank you.

[saneeshpv_FTNT]

Hi,

Is your FortiGate Located in AWS or are your referring to a FortiGate appliance on-premise and a VPN connection between AWS VPG and an On-premise FortiGate device?

If the FortiGate is on-premise and you are receiving traffic from the AWS VPC EC2 instance on the FortiGate, please check you have the required policy from AWS-IPsec(Fortigate VPN interface) to WAN and  have enabled Source NAT on this policy to use outgoing interface IP  (ie WAN interface IP). 

You can refer to traffic logs to confirm if this traffic is matching the right policy on the FortiGate or not and NAT is applied successfully. 

If my understanding of your setup is not correct, please share the diagram for better understanding.

Best Regards,

[toan2552]

Hi Saneeshpv_FTNT

My proposal is to set up the AWS network as a private link, acting as a DMZ or internal network. The reason for this choice is that the Fortigate AWS SaaS incurs additional costs, and I am already paying for the Fortigate appliance.

The network

                            { interna..-----{--\ 

fortigate -----------{  dmz----------{ --------- internet

                           {  aws----------{--/

internal network (192.168.3.0/24) 

dmz network (192.168.1.0/24)

aws network (192.168.16.0/24)

Of course, the internal and DMZ networks have access to the internet in both directions.

internal and dmz  has access to aws network

but aws has not access to internet, dmz and internal

on aws route 0.0.0.0/0 is on aws ipsec vpn

I do a diagnostic by ping from e2c 192.168.16.44

diagnose sniffer packet any 'host 192.168.16.44 and icmp' 4 0 l
interfaces=[any]
filters=[host 192.168.16.44 and icmp]
2023-07-20 09:58:30.882948 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:58:35.887153 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:58:40.908303 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:58:45.908813 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:58:50.909318 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:58:55.910043 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:59:00.910546 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:59:05.911074 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:59:10.911382 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:59:15.911851 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-20 09:59:20.912448 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request

Thank you

[saneeshpv_FTNT]

Hi,

Your AWS network (VPC) is connected to your FortiGate via IPsec VPN tunnel I am assuming, and you are receiving the traffic on the FortiGate Firewall through this VPN from EC2 instance. If so, please perform a quick sniffer with only "Destination IP and ICMP" as filter to understand if this traffic is exiting your FGT firewall over the internet WAN interface or not and below is the sniffer command.

diagnose sniffer packet any 'host 8.8.8.8 and icmp' 4 0 l

At this moment make sure you stop any other ping traffic to 8.8.8.8 apart from the one from the EC2 instance. 

Also did you make sure NAT is enabled in on the internet bound policy from AWS to WAN?

Best Regards,

[toan2552]

Nat from aws to wan is enable

this is the command result

# diagnose sniffer packet any 'host 8.8.8.8 and icmp' 4 0 l
interfaces=[any]
filters=[host 8.8.8.8 and icmp]
2023-07-21 10:14:35.712754 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:14:40.713158 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:14:45.713753 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:14:50.714274 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:14:55.714779 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:15:00.715286 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
2023-07-21 10:15:05.715707 FG-AWS-2 in 192.168.16.44 -> 8.8.8.8: icmp: echo request
^C
7 packets received by filter
0 packets dropped by kernel

kind regards

[saneeshpv_FTNT]

Hi,

It looks like traffic is not going out of FGT Firewall as we could only see In packets only.

Please enable Debug flow on the firewall and share the output with me.

Enable Debug flow from Putty (SSH session) with session logging enabled.

========================================================

diag debug reset
diag debug flow filter addr 8.8.8.8
diag debug flow filter protocol 1
diag debug flow show function-name enable
diag debug flow show iprope enable
diag debug flow trace start 1000
diag debug console timestamp enable
diag debug enable

Generate Traffic and After Capturing logs disable debug and Reset Debug flow

============================================================

diag debug disable
diag debug reset

Also share the FGT relevant configuration for verification.

Best Regards,

[toan2552]

Hi

thank you

2023-07-21 10:39:01 id=20085 trace_id=1 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55777."
2023-07-21 10:39:01 id=20085 trace_id=1 func=init_ip_session_common line=5995 msg="allocate a new session-00372a04"
2023-07-21 10:39:01 id=20085 trace_id=1 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:01 id=20085 trace_id=1 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:01 id=20085 trace_id=1 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:01 id=20085 trace_id=1 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:01 id=20085 trace_id=1 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:06 id=20085 trace_id=2 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55778."
2023-07-21 10:39:06 id=20085 trace_id=2 func=init_ip_session_common line=5995 msg="allocate a new session-00372a64"
2023-07-21 10:39:06 id=20085 trace_id=2 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:06 id=20085 trace_id=2 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:06 id=20085 trace_id=2 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:06 id=20085 trace_id=2 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:06 id=20085 trace_id=2 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:11 id=20085 trace_id=3 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55779."
2023-07-21 10:39:11 id=20085 trace_id=3 func=init_ip_session_common line=5995 msg="allocate a new session-00372aa0"
2023-07-21 10:39:11 id=20085 trace_id=3 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:11 id=20085 trace_id=3 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:11 id=20085 trace_id=3 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:11 id=20085 trace_id=3 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:11 id=20085 trace_id=3 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:16 id=20085 trace_id=4 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55780."
2023-07-21 10:39:16 id=20085 trace_id=4 func=init_ip_session_common line=5995 msg="allocate a new session-00372b66"
2023-07-21 10:39:16 id=20085 trace_id=4 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:16 id=20085 trace_id=4 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:16 id=20085 trace_id=4 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:16 id=20085 trace_id=4 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:16 id=20085 trace_id=4 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:21 id=20085 trace_id=5 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55781."
2023-07-21 10:39:21 id=20085 trace_id=5 func=init_ip_session_common line=5995 msg="allocate a new session-00372e10"
2023-07-21 10:39:21 id=20085 trace_id=5 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:21 id=20085 trace_id=5 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:21 id=20085 trace_id=5 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:21 id=20085 trace_id=5 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:21 id=20085 trace_id=5 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:26 id=20085 trace_id=6 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55782."
2023-07-21 10:39:26 id=20085 trace_id=6 func=init_ip_session_common line=5995 msg="allocate a new session-00372e95"
2023-07-21 10:39:26 id=20085 trace_id=6 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:26 id=20085 trace_id=6 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:26 id=20085 trace_id=6 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:26 id=20085 trace_id=6 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:26 id=20085 trace_id=6 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:31 id=20085 trace_id=7 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55783."
2023-07-21 10:39:31 id=20085 trace_id=7 func=init_ip_session_common line=5995 msg="allocate a new session-00372f15"
2023-07-21 10:39:31 id=20085 trace_id=7 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:31 id=20085 trace_id=7 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:31 id=20085 trace_id=7 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:31 id=20085 trace_id=7 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:31 id=20085 trace_id=7 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:36 id=20085 trace_id=8 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55784."
2023-07-21 10:39:36 id=20085 trace_id=8 func=init_ip_session_common line=5995 msg="allocate a new session-00372f79"
2023-07-21 10:39:36 id=20085 trace_id=8 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:36 id=20085 trace_id=8 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:36 id=20085 trace_id=8 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:36 id=20085 trace_id=8 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:36 id=20085 trace_id=8 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:41 id=20085 trace_id=9 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55785."
2023-07-21 10:39:41 id=20085 trace_id=9 func=init_ip_session_common line=5995 msg="allocate a new session-00372fbf"
2023-07-21 10:39:41 id=20085 trace_id=9 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:41 id=20085 trace_id=9 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:41 id=20085 trace_id=9 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:41 id=20085 trace_id=9 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:41 id=20085 trace_id=9 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:46 id=20085 trace_id=10 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55786."
2023-07-21 10:39:46 id=20085 trace_id=10 func=init_ip_session_common line=5995 msg="allocate a new session-00373044"
2023-07-21 10:39:46 id=20085 trace_id=10 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:46 id=20085 trace_id=10 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:46 id=20085 trace_id=10 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:46 id=20085 trace_id=10 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:46 id=20085 trace_id=10 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:51 id=20085 trace_id=11 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55787."
2023-07-21 10:39:51 id=20085 trace_id=11 func=init_ip_session_common line=5995 msg="allocate a new session-003730cf"
2023-07-21 10:39:51 id=20085 trace_id=11 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:51 id=20085 trace_id=11 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:51 id=20085 trace_id=11 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:51 id=20085 trace_id=11 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:51 id=20085 trace_id=11 func=ip_session_handle_no_dst line=6079 msg="trace"
2023-07-21 10:39:56 id=20085 trace_id=12 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55788."
2023-07-21 10:39:56 id=20085 trace_id=12 func=init_ip_session_common line=5995 msg="allocate a new session-0037316a"
2023-07-21 10:39:56 id=20085 trace_id=12 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:39:56 id=20085 trace_id=12 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:39:56 id=20085 trace_id=12 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:39:56 id=20085 trace_id=12 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:39:56 id=20085 trace_id=12 func=ip_session_handle_no_dst line=6079 msg="trace"
diag debug 2023-07-21 10:40:01 id=20085 trace_id=13 func=print_pkt_detail line=5824 msg="vd-root:0 received a packet(proto=1, 192.168.16.44:1->8.8.8.8:2048) from FG-AWS-2. type=8, code=0, id=1, seq=55789."
2023-07-21 10:40:01 id=20085 trace_id=13 func=init_ip_session_common line=5995 msg="allocate a new session-003731c3"
2023-07-21 10:40:01 id=20085 trace_id=13 func=iprope_dnat_check line=5121 msg="in-[FG-AWS-2], out-[]"
2023-07-21 10:40:01 id=20085 trace_id=13 func=iprope_dnat_tree_check line=823 msg="len=0"
2023-07-21 10:40:01 id=20085 trace_id=13 func=iprope_dnat_check line=5134 msg="result: skb_flags-02000008, vid-20, ret-no-match, act-accept, flag-00000000"
2023-07-21 10:40:01 id=20085 trace_id=13 func=ip_route_input_slow line=2264 msg="reverse path check fail, drop"
2023-07-21 10:40:01 id=20085 trace_id=13 func=ip_session_handle_no_dst line=6079 msg="trace"

[saneeshpv_FTNT]

HI,

The packet is getting dropped with below error "reverse path check fail, drop", which means Firewall is dropping the traffic as it doesn't have a route back to the source (here the EC2 instance IP) during RPF check. You need to look into your configuration to confirm this.

Best Regards,

[toan2552]

I don't know what exactly happen, on theory is not difficult to do this.

I have two static router to find ec2 machine

Yo can see on these pictures

[saneeshpv_FTNT · Accepted solution]

Hi,

I can see two routes here. Which one is Active in FIB?

Your traffic enters Firewall on FGT-AWS2, so it should have an active route to AWS EC2 instance via FGT-AWS2 only.

Best Regards,

Regards,