Hi folks,
This is my setup:
VPC with one (or two) public subnets and two private subnets, which are distributed between two AZ.
My fortigate vm has an interface in 10.11.0.0/24 (external) and 10.11.10.0/24 (internal). In Availability Zone A everything is working as expected. But when I try to get the private subnet 10.11.11.0/24 working with the fortigate vm, it doesn't work as expected.
Ping between instances in different AZs is working.
First attempt
Trying to ping from fortigate to 10.11.11.20 (instance-b) without setting a static route.
FGTAWS # execute ping 10.11.11.20
PING 10.11.11.20 (10.11.11.20): 56 data bytes
64 bytes from 10.11.11.20: icmp_seq=0 ttl=64 time=6.3 ms
64 bytes from 10.11.11.20: icmp_seq=1 ttl=64 time=5.7 ms
64 bytes from 10.11.11.20: icmp_seq=2 ttl=64 time=5.3 ms
64 bytes from 10.11.11.20: icmp_seq=3 ttl=64 time=7.2 ms
64 bytes from 10.11.11.20: icmp_seq=4 ttl=64 time=7.0 ms
debug flow output mixed with sniffer
# diagnose debug flow filter proto 1
# diagnose sniffer packet any "proto 1" 4
18.147088 port1 out 10.11.0.4 -> 10.11.11.20: icmp: echo request
18.153952 port1 in 10.11.11.20 -> 10.11.0.4: icmp: echo reply
2017-08-11 15:38:25 id=20085 trace_id=178 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.0.4:11776->10.11.11.20:2048) from local. type=8, code=0, id=11776, seq=0."
2017-08-11 15:38:25 id=20085 trace_id=178 func=init_ip_session_common line=5047 msg="allocate a new session-00001263"
2017-08-11 15:38:25 id=20085 trace_id=179 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.20:11776->10.11.0.4:0) from port1. type=0, code=0, id=11776, seq=0."
2017-08-11 15:38:25 id=20085 trace_id=179 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, reply direction"
2017-08-11 15:38:25 id=20085 trace_id=179 func=vf_ip_route_input_common line=2583 msg="find a route: flag=80000000 gw-10.11.0.4 via root"
19.160461 port1 out 10.11.0.4 -> 10.11.11.20: icmp: echo request
19.167117 port1 in 10.11.11.20 -> 10.11.0.4: icmp: echo reply
2017-08-11 15:38:26 id=20085 trace_id=180 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.0.4:11776->10.11.11.20:2048) from local. type=8, code=0, id=11776, seq=1."
2017-08-11 15:38:26 id=20085 trace_id=180 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, original direction"
2017-08-11 15:38:26 id=20085 trace_id=181 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.20:11776->10.11.0.4:0) from port1. type=0, code=0, id=11776, seq=1."
2017-08-11 15:38:26 id=20085 trace_id=181 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, reply direction"
20.170464 port1 out 10.11.0.4 -> 10.11.11.20: icmp: echo request
20.177254 port1 in 10.11.11.20 -> 10.11.0.4: icmp: echo reply
2017-08-11 15:38:27 id=20085 trace_id=182 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.0.4:11776->10.11.11.20:2048) from local. type=8, code=0, id=11776, seq=2."
2017-08-11 15:38:27 id=20085 trace_id=182 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, original direction"
2017-08-11 15:38:27 id=20085 trace_id=183 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.20:11776->10.11.0.4:0) from port1. type=0, code=0, id=11776, seq=2."
2017-08-11 15:38:27 id=20085 trace_id=183 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, reply direction"
21.180472 port1 out 10.11.0.4 -> 10.11.11.20: icmp: echo request
21.187344 port1 in 10.11.11.20 -> 10.11.0.4: icmp: echo reply
2017-08-11 15:38:28 id=20085 trace_id=184 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.0.4:11776->10.11.11.20:2048) from local. type=8, code=0, id=11776, seq=3."
2017-08-11 15:38:28 id=20085 trace_id=184 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, original direction"
2017-08-11 15:38:28 id=20085 trace_id=185 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.20:11776->10.11.0.4:0) from port1. type=0, code=0, id=11776, seq=3."
2017-08-11 15:38:28 id=20085 trace_id=185 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, reply direction"
22.190455 port1 out 10.11.0.4 -> 10.11.11.20: icmp: echo request
22.197581 port1 in 10.11.11.20 -> 10.11.0.4: icmp: echo reply
2017-08-11 15:38:29 id=20085 trace_id=186 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.0.4:11776->10.11.11.20:2048) from local. type=8, code=0, id=11776, seq=4."
2017-08-11 15:38:29 id=20085 trace_id=186 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, original direction"
2017-08-11 15:38:29 id=20085 trace_id=187 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.20:11776->10.11.0.4:0) from port1. type=0, code=0, id=11776, seq=4."
2017-08-11 15:38:29 id=20085 trace_id=187 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-00001263, reply direction"
routing-table
# get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
S* 0.0.0.0/0 [5/0] via 10.11.0.1, port1
C 10.11.0.0/24 is directly connected, port1
C 10.11.10.0/24 is directly connected, port2
S 10.55.0.0/24 [10/0] is directly connected, port2
it goes through the external interface. But when i configure a static route (10.11.11.0/24) to the external interface, i have the same result as in second and third attempt.
Second attempt
Add Secondary IP Address 10.11.11.5/255.255.255.0 to internal interface
FGTAWS # execute ping 10.11.11.20
PING 10.11.11.20 (10.11.11.20): 56 data bytes
--- 10.11.11.20 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
debug flow output mixed with sniffer
# diagnose debug flow filter proto 1
# diagnose sniffer packet any "proto 1" 4
2017-08-11 15:33:52 id=20085 trace_id=173 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.5:11520->10.11.11.20:2048) from local. type=8, code=0, id=11520, seq=0."
2017-08-11 15:33:52 id=20085 trace_id=173 func=init_ip_session_common line=5047 msg="allocate a new session-000011a2"
2017-08-11 15:33:53 id=20085 trace_id=174 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.5:11520->10.11.11.20:2048) from local. type=8, code=0, id=11520, seq=1."
2017-08-11 15:33:53 id=20085 trace_id=174 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-000011a2, original direction"
2017-08-11 15:33:54 id=20085 trace_id=175 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.5:11520->10.11.11.20:2048) from local. type=8, code=0, id=11520, seq=2."
2017-08-11 15:33:54 id=20085 trace_id=175 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-000011a2, original direction"
3805.210453 root out 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3805.210457 root out 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3805.210459 root out 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3805.210461 root in 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3805.210472 root in 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3805.210473 root in 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
2017-08-11 15:33:55 id=20085 trace_id=176 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.5:11520->10.11.11.20:2048) from local. type=8, code=0, id=11520, seq=3."
2017-08-11 15:33:55 id=20085 trace_id=176 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-000011a2, original direction"
2017-08-11 15:33:56 id=20085 trace_id=177 func=print_pkt_detail line=4903 msg="vd-root received a packet(proto=1, 10.11.11.5:11520->10.11.11.20:2048) from local. type=8, code=0, id=11520, seq=4."
2017-08-11 15:33:56 id=20085 trace_id=177 func=resolve_ip_tuple_fast line=4967 msg="Find an existing session, id-000011a2, original direction"
3808.250454 root out 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3808.250458 root out 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3808.250460 root in 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
3808.250471 root in 127.0.0.1 -> 10.11.11.5: icmp: host 10.11.11.20 unreachable
routing-table
# get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
S* 0.0.0.0/0 [5/0] via 10.11.0.1, port1
C 10.11.0.0/24 is directly connected, port1
C 10.11.10.0/24 is directly connected, port2
C 10.11.11.0/24 is directly connected, port2
S 10.55.0.0/24 [10/0] is directly connected, port2
Third attempt
Added static route 10.11.11.0/24 to internal interface. Same result as Second attempt
Solution?
Does any of You have a tip or a solution for this problem?
Best regards
