AWS tunnel issue

[Anatoli]

Hi, I would want to ask a question on the issue that was raised earlier.

I have two wan lines, I have set up both tunnels on WAN1 I'm not sure if his setup is accurate. Should one tunnel be installed in Wan 1 and the other tunnel in Wan 2?

Still, I have one of them tunnelled in shut down. The problem is that the client from the SAP server does not reach an AWS server, but the person from the AWS server reaches the SAP server for my client. Any thoughts?

show system interface "TO AWS 2"

config system interface

edit "TO AWS 2"

set vdom "root"

set ip 169.254.85.6 255.255.255.255

set allowaccess ping

set type tunnel

set tcp-mss 1379

set remote-ip 169.254.85.5 255.255.255.252

set snmp-index 69

set interface "wan1"

set mtu-override enable

set mtu 1427

diagnose debug enable

FortiGate1 # diagnose debug flow filter daddr 169.254.85.5

FortiGate1 # diagnose debug flow trace start 1

FortiGate1 # id=65308 trace_id=1 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 169.254.85.6:83->169.254.85.5:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=83, seq=888."

id=65308 trace_id=1 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-03871a2d, original direction"

id=65308 trace_id=1 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"

id=65308 trace_id=1 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"

id=65308 trace_id=1 func=esp_output4 line=896 msg="IPsec encrypt/auth"

id=65308 trace_id=1 func=ipsec_output_finish line=629 msg="send to 91.1X.X.X via intf-wan1"

execute ping 169.254.85.5

PING 169.254.85.5 (169.254.85.5): 56 data bytes

64 bytes from 169.254.85.5: icmp_seq=0 ttl=254 time=35.4 ms

64 bytes from 169.254.85.5: icmp_seq=1 ttl=254 time=35.7 ms

[DPadula · Accepted solution]

Hi Anatoli,

You can have both tunnel on the same WAN1 or have each tunnel on each WAN interface.
The question is, what you are trying to achieve?
Is WAN1 and WAN2 both active or WAN2 is just a backup of WAN1 and take over only when wan1 is down?
I

[Anatoli]

Hi

anyone can i help me ?

[DPadula · Accepted solution]

Hi Anatoli,

You can have both tunnel on the same WAN1 or have each tunnel on each WAN interface.
The question is, what you are trying to achieve?
Is WAN1 and WAN2 both active or WAN2 is just a backup of WAN1 and take over only when wan1 is down?
I

[Anatoli]

Hi @DPadula  thanks for your reply .

• Wan 2 is  just a backup take over when wan1 is down .

• The goal is to connect my customer's SAP system to an AWS servers.

AWS servers 10.40.6.0 must be reached using the customer's fortigate IP address of 10.0.0.0.

For instance, the customer cannot reach from SAP server 10.0.0.5 to AWS server 10.40.6.119.

From AWS 10.40.6.119 to SAP server 10..0.0.5 it is Pinging

[DPadula]

Because you said that traffic from the server 10.40.6.119 can reach 10.0.0.5 we show that tunnels is up and rule from AWS to SAP is correct. You should check and confirm if you do have a rule from SAP to AWS. The sniffer show the traffic arriving from the SAP server via SAP interface, but we don't see a ICMP echo request out on the tunnel interface. Sounds like missing rule or wrong rule configuration. 
You can use diag debug flow trace to confirm that. 

[Anatoli]

Yes it is correct from 10.0.0.5 server lan citrix no ping to 10.40.6.119 server aws  . From 10.40.6.119 to 10.0.0.5 ping is ok 

I attach you the test

FortiGate1 # diagnose debug flow filter saddr 10.0.0.5

FortiGate1 # diagnose debug flow filter daddr 10.40.6.119

FortiGate1 # diagnose debug flow trace start 100

FortiGate1 # diagnose debug enable

FortiGate1 # execute ping-options source 10.0.0.5

FortiGate1 # execute ping 10.40.6.119
id=65308 trace_id=102 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 10.0.0.5:156->10.40.6.119:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=156, seq=0."
id=65308 trace_id=102 func=init_ip_session_common line=5980 msg="allocate a new session-049ff447, tun_id=0.0.0.0"
id=65308 trace_id=102 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"
id=65308 trace_id=102 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"
id=65308 trace_id=102 func=esp_output4 line=896 msg="IPsec encrypt/auth"
id=65308 trace_id=102 func=ipsec_output_finish line=629 msg="send to X.X.76.41 via intf-wan1"
id=65308 trace_id=103 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 10.0.0.5:156->10.40.6.119:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=156, seq=1."
id=65308 trace_id=103 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-049ff447, original direction"
id=65308 trace_id=103 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"
id=65308 trace_id=103 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"
id=65308 trace_id=103 func=esp_output4 line=896 msg="IPsec encrypt/auth"
id=65308 trace_id=103 func=ipsec_output_finish line=629 msg="send to X.X.76.41 via intf-wan1"
id=65308 trace_id=104 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 10.0.0.5:156->10.40.6.119:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=156, seq=2."
id=65308 trace_id=104 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-049ff447, original direction"
id=65308 trace_id=104 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"
id=65308 trace_id=104 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"
id=65308 trace_id=104 func=esp_output4 line=896 msg="IPsec encrypt/auth"
id=65308 trace_id=104 func=ipsec_output_finish line=629 msg="send to X.X.76.41 via intf-wan1"
id=65308 trace_id=105 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 10.0.0.5:156->10.40.6.119:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=156, seq=3."
id=65308 trace_id=105 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-049ff447, original direction"
id=65308 trace_id=105 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"
id=65308 trace_id=105 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"
id=65308 trace_id=105 func=esp_output4 line=896 msg="IPsec encrypt/auth"
id=65308 trace_id=105 func=ipsec_output_finish line=629 msg="send to X.X.76.41 via intf-wan1"
id=65308 trace_id=106 func=print_pkt_detail line=5795 msg="vd-root:0 received a packet(proto=1, 10.0.0.5:156->10.40.6.119:2048) tun_id=0.0.0.0 from local. type=8, code=0, id=156, seq=4."
id=65308 trace_id=106 func=resolve_ip_tuple_fast line=5883 msg="Find an existing session, id-049ff447, original direction"
id=65308 trace_id=106 func=ipsecdev_hard_start_xmit line=669 msg="enter IPSec interface TO AWS 2, tun_id=0.0.0.0"
id=65308 trace_id=106 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"
id=65308 trace_id=106 func=esp_output4 line=896 msg="IPsec encrypt/auth"
id=65308 trace_id=106 func=ipsec_output_finish line=629 msg="send to X.X.76.41 via intf-wan1"
PING 10.40.6.119 (10.40.6.119): 56 data bytes

--- 10.40.6.119 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss

[DPadula]

Hi Anatoli,

Just a tip, every time that you use the 'diag debug flow' we suggest that you also enable the 'diagnose debug flow show function-name enable'.

Looking into the output collected (id=65308 trace_id=106 func=_do_ipsecdev_hard_start_xmit line=229 msg="output to IPSec tunnel TO AWS 2 vrf 0"), clearly the traffic is been sent via TO AWS tunnel, proving that your FGT is configured correctly (firewall policies and routing). The issue seems to be on the other side of the tunnel.

The next step is to check the AWS side when the traffic is originating from the SAP client. 

[Anatoli]

Hi @DPadula how ro check the aws side ? Any idea?

[DPadula]

Is it a tunnel directly to AWS correct?

If yes, you need to check with AWS support. You can share the 'debug flow trace' collected showing that you are sending them some traffic but you cannot see the traffic back. 

[Anatoli]

I guess  it is directly  I was working with an AWS person on a tester. He assured me that his perspective is accurate, but I'm not sure how he can present it clearly.