Fortigate VM64-AZURE VPN IPSEC connection to other Azure vNETs

[JakeBlues]

Hi all,

I'm trying out Fortigate VM64-AZURE on Azure.

My task is to have it as IPsec VPN terminator which allows authorized clients to connect to some Azure vNets that are in peering with VM64 Vnet.

Fortigate is in its own Vnet (call it FwvNet), and clients connect to this vnet without problems, i.e. I've put a test vm in the internal vnet, and clients can ping it, ssh and whatever.

I've put in peering FwvNet with another vNet, say AppvNet.

From the test machine I can ping VMs on Appvnet, but from the IPsec clients I can't. 

What am I missing?

Thanks for your help

[ntaneja]

Hi JakeBlues

Connect your VPN client and then open putty to FGT device and run below commands:

Putty 1: 
diag debug reset 
diag debug en 
diag debug console timestamp enable 
diag debug flow filter clear 
diag debug flow filter addr X.X.X.X <<------[Replace X.X.X.X with Destination IP] 
diag debug flow filter proto 1 
diag debug flow trace start 999

Before executing above commands, ensure that there is no existing sessions from the IP . After executing above commands, ping to destination IP . Once you see the output generated, enter following commands to turn off debugging- 

diag debug disable 
diag debug reset 


Putty 2: 

diag vpn ike gateway list 
diag vpn tunnel list 
get router info routing-table details <src ip> 
get router info routing-table details <dst ip> 

Thanks

[JakeBlues]

Hi,

here are the results:

FW01-FGT-A # diag debug reset
diag debug en
diag debug console timestamp enable
diag debug flow filter clear
diag debug flow filter addr 10.129.1.6
diag debug flow filter proto 1
diag debug flow trace start 999

FW01-FGT-A # diag debug en

FW01-FGT-A # diag debug console timestamp enable

FW01-FGT-A # diag debug flow filter clear

FW01-FGT-A # diag debug flow filter addr 10.129.1.6

FW01-FGT-A # diag debug flow filter proto 1

FW01-FGT-A # diag debug flow trace start 999

FW01-FGT-A # diag vpn ike gateway list

vd: root/0
name: FW01-RA_0
version: 1
interface: port1 3
addr: 10.254.2.4:4500 -> <MY_PUBLIC_IP>:45786
tun_id: 10.254.0.224/::10.0.0.52
remote_location: 0.0.0.0
virtual-interface-addr: 169.254.1.1 -> 0.0.0.0
created: 360s ago
xauth-user: XXXXXXXXXXXXXXXXXXXX
2FA: no
FortiClient UID: XXXXXXXXXXXXXXXXXXXXXXXXXX
assigned IPv4 address: 10.254.0.224/255.255.255.255
nat: me peer
IKE SA: created 1/1 established 1/1 time 140/140/140 ms
IPsec SA: created 1/1 established 1/1 time 30/30/30 ms

id/spi: 242 655ed11b02f96ec1/7c06ad49929bda73
direction: responder
status: established 360-360s ago = 140ms
proposal: aes256-sha256
key: a2c502d822ace80e-559df85dad2a45e7-5ec974981d3bae11-17341b3f019e4d61
lifetime/rekey: 86400/85769
DPD sent/recv: 00000000/00000f06

FW01-FGT-A # diag vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=FW01-RA ver=1 serial=1 10.254.2.4:0->0.0.0.0:0 tun_id=10.0.0.1 tun_id6=::10.0.0.1 dst_mtu=0 dpd-link=on weight=1
bound_if=3 lgwy=static/1 tun=tunnel/1 mode=dialup/2 encap=none/520 options[0208]=npu frag-rfc accept_traffic=1 overlay_id=0
proxyid_num=0 child_num=1 refcnt=3 ilast=47557091 olast=47557091 ad=/0
stat: rxp=14935 txp=8572 rxb=1004786 txb=552988
dpd: mode=on-demand on=0 idle=20000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
fec: egress=0 ingress=0
run_tally=0
------------------------------------------------------
name=FW01-RA_0 ver=1 serial=34 10.254.2.4:4500-><MY_PUBLIC_IP>:45786 tun_id=10.254.0.224 tun_id6=::10.0.0.52 dst_mtu=1500 dpd-link=on weight=1
bound_if=3 lgwy=static/1 tun=intf/0 mode=dial_inst/3 encap=none/9096 options[2388]=npu rgwy-chg rport-chg frag-rfc run_state=0 accept_traffic=1 overlay_id=0
parent=FW01-RA index=0
proxyid_num=1 child_num=0 refcnt=6 ilast=1 olast=1 ad=/0
stat: rxp=351 txp=88 rxb=25575 txb=15193
dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
natt: mode=keepalive draft=32 interval=10 remote_port=45786
fec: egress=0 ingress=0
proxyid=FW01-RA proto=0 sa=1 ref=3 serial=1 add-route
src: 0:0.0.0.0-255.255.255.255:0
dst: 0:10.254.0.224-10.254.0.224:0
SA: ref=3 options=82 type=00 soft=0 mtu=1422 expire=42829/0B replaywin=2048
seqno=59 esn=0 replaywin_lastseq=0000015f itn=0 qat=0 hash_search_len=1
life: type=01 bytes=0/0 timeout=43188/43200
dec: spi=3d238656 esp=aes key=16 4f8556fb9fc3c5b316b8bbd8fc607a51
ah=sha1 key=20 4b3b078b8f1a7e7fa80aa04e2a770c3cf3f6ec6e
enc: spi=ce9efab6 esp=aes key=16 bf3d3d3918f5349ed4496cc696dd3e6d
ah=sha1 key=20 70d627b08d9dac3eed12675f3259b5cc9f5f2cac
dec:pkts/bytes=702/51150, enc:pkts/bytes=176/36489
npu_flag=00 npu_rgwy=<MY_PUBLIC_IP> npu_lgwy=10.254.2.4 npu_selid=17 dec_npuid=0 enc_npuid=0

FW01-FGT-A # get router info routing-table details 10.254.0.224 (the IP I get using forticlient)

Routing table for VRF=0
Routing entry for 10.254.0.224/32
Known via "static", distance 15, metric 0, best
* via FW01-RA tunnel 10.254.0.224 vrf 0

FW01-FGT-A # get router info routing-table details 10.129.1.6 (the IP I try to reach)

Routing table for VRF=0
Routing entry for 10.129.1.0/24
Known via "static", distance 10, metric 0, best
* vrf 0 10.254.0.1, via port2

FW01-FGT-A #

Thanks

[ntaneja]

Hi JakeBlues

We didnt get the output for the commands in putty 1 session.
Putty 1: 
diag debug reset 
diag debug en 
diag debug console timestamp enable 
diag debug flow filter clear 
diag debug flow filter addr X.X.X.X <<------[Replace X.X.X.X with Destination IP] 
diag debug flow filter proto 1 
diag debug flow trace start 999

Before executing above commands, ensure that there is no existing sessions from the IP . After executing above commands, ping to destination IP . Once you see the output generated, enter following commands to turn off debugging- 

diag debug disable 
diag debug reset 


After running 1st set of commands, you need to connect client and try pinging the dst IP.
We should see some packets on FGT device if packet is reaching it.

Thanks

[JakeBlues]

Hi.

First part isn't missing.

Simply I didn't catch any output, while trying out a different IP on Fortigate internal LAN gave me a result (and this VM was pingable).

Nevertheless, a route print command on my windows PC shows:

IPv4 Route table
===========================================================================
Active route:
Network address Mask Gateway Interface Metric
0.0.0.0 0.0.0.0 10.0.125.1 10.0.125.17 35
0.0.0.0 0.0.0.0 10.0.126.252 10.0.126.35 25
10.0.0.0 255.0.0.0 10.254.0.4 10.254.0.224 2

i.e. the Forticlient gives me route to 10.0.0.0/8 via the firewall as it should be.

Ciao

[JakeBlues]

Any ideas?

[pa_iva]

Hi JakeBlues,

Can you ping the destination IP 10.129.1.6, directly from the Firewall?

Also, do you have a Firewall policy that matches that traffic flow?

With source interface "Your IPSEC Tunnel" -> Destination Interface Port2 and destination address is permited.

Try to do a debug flow as already requested, if it shows nothing it looks like the traffic isn't hitting the tunnel.

Have you tried to do a traceroute from your windows PC to 10.129.1.6?

Regards,

[JakeBlues]

By the way, I've replaced the 10.129.1.6 with 10.129.1.5 for other reasons (independent from this issue).

Anyway, from web cli and from ssh I can ping 10.129.1.5.

From my client connected with Forticlient I can't.

Traceroute to this IP gives:

Traccia instradamento verso 10.129.1.5 su un massimo di 30 punti di passaggio

1 * * * Richiesta scaduta.
[...]
30 * * * Richiesta scaduta.

(Richiesta scaduta = Request time out)

There's another odd behaviour: If I ping a VM on the same vNet of Fortigate's internal port, I can do it.

If I do a traceroute on it I get

Traccia instradamento verso 10.254.0.5 su un massimo di 30 punti di passaggio

1 23 ms 23 ms 23 ms 169.254.1.1
2 33 ms 25 ms 24 ms 10.254.0.5

I.e. I have an APIPA address as first hop. Is it ok?

Ciao

[pa_iva · Accepted solution]

Hi Jakeblues,

The APIPA address as first hop is ok, it's just an IP automatically assigned to the virtual tunnel interface.

One thing I noticed, the client is getting the IP 10.254.0.224, but on the last test you say that:

---

@JakeBlues wrote:

There's another odd behaviour: If I ping a VM on the same vNet of Fortigate's internal port, I can do it.

If I do a traceroute on it I get

Traccia instradamento verso 10.254.0.5 su un massimo di 30 punti di passaggio

1 23 ms 23 ms 23 ms 169.254.1.1
2 33 ms 25 ms 24 ms 10.254.0.5

---

So, it seems that for the Client Address Range, you're using the same subnet that is also a direcly connected network to the Fortigate at Azure (internal port). This will lead to some routing issues, the client address pool should be in a range that is not currently in use. Can you please verify this and post the output of your full routing table?

get router info routing table all

get router info routing table database

[JakeBlues]

Hi,

FW01-FGT-A # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
* - candidate default

Routing table for VRF=0
S* 0.0.0.0/0 [10/0] via 10.254.2.1, port1, [1/0]
S 10.0.10.0/24 [10/0] via 10.254.0.1, port2, [1/0]
S 10.254.0.0/22 [10/0] via 10.254.0.1, port2, [1/0]
C 10.254.0.0/24 is directly connected, port2
S 10.254.0.224/32 [15/0] via Forticlient-RA tunnel 10.254.0.224 vrf 0, [1/0]
C 10.254.2.0/24 is directly connected, port1
C 169.254.1.1/32 is directly connected, Forticlient-RA

FW01-FGT-A # get router info routing-table database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
V - BGP VPNv4
> - selected route, * - FIB route, p - stale info

Routing table for VRF=0
S *> 0.0.0.0/0 [10/0] via 10.254.2.1, port1, [1/0]
S *> 10.0.10.0/24 [10/0] via 10.254.0.1, port2, [1/0]
S *> 10.254.0.0/22 [10/0] via 10.254.0.1, port2, [1/0]
C *> 10.254.0.0/24 is directly connected, port2
S *> 10.254.0.224/32 [15/0] via Forticlient-RA tunnel 10.254.0.224 vrf 0, [1/0]
C *> 10.254.2.0/24 is directly connected, port1
C *> 169.254.1.1/32 is directly connected, Forticlient-RA

Ciao