AWS tunnel issue

[Anatoli]

Hi, I would want to ask a question on the issue that was raised earlier.

I have two wan lines, I have set up both tunnels on WAN1 I'm not sure if his setup is accurate. Should one tunnel be installed in Wan 1 and the other tunnel in Wan 2?

Still, I have one of them tunnelled in shut down. The problem is that the client from the SAP server does not reach an AWS server, but the person from the AWS server reaches the SAP server for my client. Any thoughts?

show system interface "TO AWS 2"

config system interface

edit "TO AWS 2"

set vdom "root"

set ip 169.254.85.6 255.255.255.255

set allowaccess ping

set type tunnel

set tcp-mss 1379

set remote-ip 169.254.85.5 255.255.255.252

set snmp-index 69

set interface "wan1"

set mtu-override enable

set mtu 1427

diagnose debug enable

FortiGate1 # diagnose debug flow filter daddr 169.254.85.5

FortiGate1 # diagnose debug flow trace start 1

FortiGate1 # id=65308 trace_id=1 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 169.254.85.6:83->169.254.85.5:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=83, seq=888."

id=65308 trace_id=1 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-03871a2d, original direction"

id=65308 trace_id=1 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"

id=65308 trace_id=1 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"

id=65308 trace_id=1 func=esp_output4 line=896 msg="IPsec encrypt/auth"

id=65308 trace_id=1 func=ipsec_output_finish line=629 msg="send to 91.1X.X.X via intf-wan1"

execute ping 169.254.85.5

PING 169.254.85.5 (169.254.85.5): 56 data bytes

64 bytes from 169.254.85.5: icmp_seq=0 ttl=254 time=35.4 ms

64 bytes from 169.254.85.5: icmp_seq=1 ttl=254 time=35.7 ms

[DPadula · Accepted solution]

Hi Anatoli,

You can have both tunnel on the same WAN1 or have each tunnel on each WAN interface.
The question is, what you are trying to achieve?
Is WAN1 and WAN2 both active or WAN2 is just a backup of WAN1 and take over only when wan1 is down?
I

[DPadula]

If you run the command 'diag sniffer packet any "host 10.0.0.5 and icmp" 4 0' while you are pinging 10.40.6.119 it will show the ping request being sent via tunnel interface. 

Then you stop the ping, run the command again and send the ping from the AWS to the SAP server. You will see the ICMP requests and replies. 

If you have troubles proving to AWS that FGT is properly configured raise a case with our TAC and we will help collecting the logs/outputs. 

[Anatoli]

I neglected to mention that the client uses Fortiswitch.

for your consideration This time, the aws guy  had to ping  from  this server10.40.6.58  

FortiGate1 # diagnose  sniffer packet  any 'host 10.0.0.5 and   icmp  ' 4 0
interfaces=[any]
filters=[host 10.0.0.5 and   icmp  ]
5.815189 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo request
5.815262 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo request
6.843189 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo request
6.843217 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo request
7.863180 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo request
7.863244 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo request
8.887175 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo request
8.887212 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo request

////////////////////////////////////////////////////////////////////////////////////////////////////////

FortiGate1 # diagnose sniffer packet any ' host 10.40.6.58 and icmp ' 4 0 1
interfaces=[any]
filters=[ host 10.40.6.58 and icmp ]
217.439922 TO AWS 2 in 10.40.6.58 -> 10.0.0.5: icmp: echo request
217.440009 SAP out 10.40.6.58 -> 10.0.0.5: icmp: echo request
217.440012 fortilink out 10.40.6.58 -> 10.0.0.5: icmp: echo request
217.440016 x1 out 10.40.6.58 -> 10.0.0.5: icmp: echo request
217.440062 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo reply
217.440090 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo reply
218.440458 TO AWS 2 in 10.40.6.58 -> 10.0.0.5: icmp: echo request
218.440498 SAP out 10.40.6.58 -> 10.0.0.5: icmp: echo request
218.440501 fortilink out 10.40.6.58 -> 10.0.0.5: icmp: echo request
218.440505 x1 out 10.40.6.58 -> 10.0.0.5: icmp: echo request
218.440547 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo reply
218.440569 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo reply

[DPadula · Accepted solution]

The first diagnose command is what you need to prove to the AWS team that FGT is sending the traffic but no reply is seen. You can ask them to do a packet capture on AWS side and confirm they can see the ICMP request and if the server is sending a ICMP reply. 

[amrit]

you can use these additional steps to verify the traffic 

1. Verify the routing details on fortigate for the IP 10.40.6.119

"get router info routing-table details 10.40.6.119" -- in the output of this command I expect  the outgoing interface as your IPSec tunnel to AWS

If the route is present and is the best route to 10.40.6.119 then

Verify the policy from the incoming interface to the outgoing tunnel interface. 

To verify the incoming interface you can again use "get router info routing-table details 10.0.0.5". Also make sure the policy should have correct source and destination subnets

[Anatoli]

Hi @amrit  so thanks for your reply 

I attach  to you the tester

[amrit]

The following sniffer shows that traffic is getting forwarded to the AWS tunnel, so I don't think the problem is with the Fortigate. You may need to check with the AWS 

FortiGate1 # diagnose  sniffer packet  any 'host 10.0.0.5 and   icmp  ' 4 0
interfaces=[any]
filters=[host 10.0.0.5 and   icmp  ]
5.815189 SAP in 10.0.0.5 -> 10.40.6.58: icmp: echo request
5.815262 TO AWS 2 out 10.0.0.5 -> 10.40.6.58: icmp: echo request

Check the output of the following commands- this is to verify if the traffic is being sent

In SAP to AWS policy disbale the asic

config firewall policy 

edit <id>

set auto-asic-offload disable

end

end

di de flow filter addr 10.40.6.58

di de flow filter prot 1

di de flow show function-name en

di de flow trace start 100

di de en

[Anatoli]

I would want to express my gratitude for your assistance. @amrit @DPadula 

The problem was resolved. The AWS route database needed to be cleaned, and then the ping succeeded.