Technical Note: Configuring and verifying a GRE over IPsec tunnel using 'encapsulation gre'

[sha-1_FTNT]

Description

This article describes how to configure and troubleshoot a GRE over IPsec tunnel using “encapsulation gre” between a FortiGate and a Cisco router

Scope

Support for IPsec “encapsulation gre” is available as of FortiOS 5.4
Support for IPsec transport-mode, traffic selector restriction and dynamic routing with IPsec “encapsulation gre” is available as of FortiOS 5.6 and 5.4.6

Solution

Diagram

 
The following topology is used:

PC1(.1) - 10.1.1.0/24-port2-[ FGT ]-port1----(198.51.100.1) =======
                                                                  I
                                                 (Internet)       I GRE over IPsec tunnel
                                                                  I
PC2(.2) - 10.2.2.0/24-gi0/0-[ Cisco_RTR ]-gi1/0-(192.0.2.2) =======  
 
 

Design

• Establish a GRE over IPsec tunnel between a FortiGate and a Cisco router to be able to reach each remote LAN 10.x.x.x
• IPsec in transport mode is used since data packets are already tunneled in GRE
• OSPF is used as dynamic routing protocol (multicast traffic, hence the need for GRE-IPsec with some vendors)

Why a GRE over IPsec tunnel instead of a plain IPsec tunnel ?

 
Common reasons are:
 
1-    Multicast over IPsec
 

Some vendors do not support multicast traffic (OSPF, streaming,…) directly inside an IPsec tunnel.

The multicast traffic is therefore tunneled in GRE which itself is protected by IPsec. Only the unicast GRE traffic between the GRE endpoints is exposed to IPsec.

FortiOS supports multicast traffic directly inside IPsec.

There is therefore no requirement to use GRE-IPsec to carry multicast traffic between two FortiGates.

 
 
2-    Traffic selector simplification
 

Some vendors do not support negotiating wildcard traffic selectors (i.e., any-any selectors: src-subnet=0.0.0.0/0 and dst-subnet=0.0.0.0/0)

Either they require the exhaustive list of all local-subnets and all remote-subnets,

Or they require specifying all the possible combination of (local <-> remote) subnets.

FortiOS supports any-any selectors (src-subnet=0.0.0.0/0 and dst-subnet=0.0.0.0/0).

There is therefore no requirement to use GRE-IPsec to simplify the traffic selector configuration between two FortiGates.

 
 
GRE over IPsec configuration with FortiOS
 
Tight integration between GRE and IPsec (“encapsulation gre”) is available as of FortiOS 5.4
 
“encapsulation gre” implementation in FortiOS from 5.4.0 to 5.4.5 however suffers these limitations:
   -       only IPsec in tunnel-mode is supported (no support for IPsec in transport-mode),
   -       the traffic selectors cannot be restricted to the GRE endpoints,
   -       dynamic routing is not supported
 
All the above “encapsulation gre” limitations are removed as of FortiOS 5.4.6 and 5.6.0:
   -       IPsec is supported in both transport-mode and tunnel-mode,
   -       traffic selectors can be restricted to the GRE endpoints addresses and GRE protocol (ip/47)
   -       dynamic routing is supported
 
 
The GRE over IPsec configuration in this article relies on the above “encapsulation gre” enhancements available as of FortiOS 5.6 and 5.4.6.
 
The scenario covered in this article is also available with independent configuration of GRE settings and IPsec settings. See the Related Articles section.
 
 

Limitations

 

• The inner GRE traffic cannot be hardware offloaded to NPU (NP6, NP4)
• IPsec in transport-mode cannot be offloaded to NPU (NP6, NP4)

  

Configuration

CLI configuration of the FGT
 
#
# Port1 is the Internet-facing interface
# Port2 is the LAN interface
#
 
config system interface
    edit "port1"
        set ip 198.51.100.1 255.255.255.0
        set alias "Internet"
    next
    edit "port2"
        set ip 10.1.1.254 255.255.255.0
        set alias "LAN"
    next
end
 
#
# IPsec with GRE encapsulation (GRE over IPsec)
#
 
config vpn ipsec phase1-interface
    edit "toCisco"
        set interface "port1"
        set encapsulation gre
        set proposal aes128-sha1
        set dhgrp 14
        set remote-gw 192.0.2.2
        set psksecret fortinet
    next
end
 
config vpn ipsec phase2-interface
    edit "toCisco"
        set phase1name "toCisco"
        set proposal aes128-sha1
        set dhgrp 14
        set protocol 47                     // restrict traffic selectors to GRE protocol (ip/47)
        set auto-negotiate enable
        set encapsulation transport-mode    // transport-mode for IPsec (tunneling already done by GRE)
    next
end
 
 
#
# Local and remote overlay IP addresses over the GRE-IPsec tunnel
#
 
config system interface
    edit "toCisco"
        set ip 10.255.255.1 255.255.255.255
        set remote-ip 10.255.255.2
    next
end
 
#
# Firewall Policies
#
 
config firewall address
    edit "10.1.1.0/24"
        set comment "Local LAN"
        set subnet 10.1.1.0 255.255.255.0
    next
    edit "10.2.2.0/24"
        set comment "Remote LAN"
        set subnet 10.2.2.0 255.255.255.0
    next
end
 
config firewall policy
 
    Allow traffic between the local LAN (port2) and the remote LAN (GRE-IPsec)
    edit 1
        set name "to remote LAN"
        set srcintf "port2"
        set dstintf "toCisco"
        set srcaddr "10.1.1.0/24"
        set dstaddr "10.2.2.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "local LAN to remote LAN"
    next
    edit 2
        set name "from remote LAN"
        set srcintf "toCisco"
        set dstintf "port2"
        set srcaddr "10.2.2.0/24"
        set dstaddr "10.1.1.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "remote LAN to local LAN"
    next
 
    Should the remote LAN subnet (10.2.2.0/24) be missing in the routing table (e.g., OSPF adjacency is down)
    packets destined to 10.2.2.0/24 would match the default-route and the ‘Internet Access’ policy  
    This ‘Deny Internet’ policy ensures that packets destined to the remote LAN never match the ‘Internet Access’  
    policy  
    edit 3
        set name "Deny Internet"
        set srcintf "port2"
        set dstintf "port1"
        set srcaddr "all"
        set dstaddr "10.2.2.0/24"
        set schedule "always"
        set service "ALL"
        set logtraffic all
        set comments "Prevent remote LAN access to leak over the Internet"
    next
 
    Internet Access
    edit 4
        set name "Internet Access"
        set srcintf "port2"
        set dstintf "port1"
        set srcaddr "10.1.1.0/24"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "Internet Access"
        set nat enable
    next
 
 
#
# OSPF routing
#
 
config router ospf
    set router-id 10.1.1.254
    config area
        edit 0.0.0.0
        next
    end
    config ospf-interface
        edit "toCisco"
            set interface "toCisco"
            set mtu-ignore enable    // ignore the MTU discrepancy between the FortiOS GRE-IPsec interface
                                     // and the IOS GRE interface
            set network-type point-to-point
        next
    end
    config network
        edit 1
            set prefix 10.1.1.254 255.255.255.255     // enable OSPF on the LAN interface
        next
        edit 2
            set prefix 10.255.255.1 255.255.255.255   // enable OSPF on the GRE-IPsec tunnel
        next
    end
end
 
 
#
# Static routes
#
 
config router static
    edit 1
        set gateway 198.51.100.254
        set device "port1"
        set comment "default-route to Internet ISP"
    next
end

CLI configuration of the Cisco Router
 
!
! IPsec configuration
!
 
crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 14
 
crypto isakmp key fortinet address 198.51.100.1
 
crypto ipsec transform-set aes128-sha1-transport esp-aes esp-sha-hmac
 mode transport
 
ip access-list extended encryptionDomain
 permit gre host 192.0.2.2 host 198.51.100.1
 
crypto map gre_over_ipsec 10 ipsec-isakmp
 set peer 198.51.100.1
 set transform-set aes128-sha1-transport
 set pfs group14
 match address encryptionDomain
 
 
!
! GRE tunnel interface
!
 
interface Tunnel0
 ip address 10.255.255.2 255.255.255.252    ! overlay subnet over the GRE tunnel
 ip ospf mtu-ignore
 tunnel source GigabitEthernet1/0
 tunnel destination 198.51.100.1
 
 
! LAN
 
interface GigabitEthernet0/0
 ip address 10.2.2.254 255.255.255.0
 ip nat inside
 
! Internet
 
interface GigabitEthernet1/0
 ip address 192.0.2.2 255.255.255.0
 ip nat outside
 crypto map gre_over_ipsec    ! apply IPsec to the traffic matching the crypto map
 
 
! OSPF
 
router ospf 1
 router-id 10.2.2.254
 network 10.2.2.254 0.0.0.0 area 0
 network 10.255.255.2 0.0.0.0 area 0
 
 
! SNAT for Internet Access
 
ip nat inside source list natAcl interface GigabitEthernet1/0 overload
 
ip access-list extended natAcl
 permit ip 10.2.2.0 0.0.0.255 any
 
! Static route
 
! default-route to Internet ISP
ip route 0.0.0.0 0.0.0.0 192.0.2.253
 

Verification

 
 
Verify the routing table (RIB)

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
O       10.2.2.0/24 [110/101] via 10.255.255.2, toCisco, 00:32:59
O       10.255.255.0/30 [110/1100] via 10.255.255.2, toCisco, 00:32:59
C       10.255.255.1/32 is directly connected, toCisco
C       10.255.255.2/32 is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
C       198.51.100.0/24 is directly connected, port1

 
 
Verify that PC1 and PC2 can ping each other
 

root@PC1:~# ping -c 5 10.2.2.2
PING 10.2.2.2 (10.2.2.2) 56(84) bytes of data.
64 bytes from 10.2.2.2: icmp_seq=1 ttl=62 time=44.4 ms
64 bytes from 10.2.2.2: icmp_seq=2 ttl=62 time=41.1 ms
64 bytes from 10.2.2.2: icmp_seq=3 ttl=62 time=53.5 ms
64 bytes from 10.2.2.2: icmp_seq=4 ttl=62 time=50.4 ms
64 bytes from 10.2.2.2: icmp_seq=5 ttl=62 time=47.8 ms
 
--- 10.2.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 41.148/47.487/53.538/4.368 ms
 
 
PC2> ping 10.1.1.1
84 bytes from 10.1.1.1 icmp_seq=1 ttl=62 time=47.694 ms
84 bytes from 10.1.1.1 icmp_seq=2 ttl=62 time=46.940 ms
84 bytes from 10.1.1.1 icmp_seq=3 ttl=62 time=47.815 ms
84 bytes from 10.1.1.1 icmp_seq=4 ttl=62 time=46.889 ms
84 bytes from 10.1.1.1 icmp_seq=5 ttl=62 time=46.941 ms
 
5 packets transmitted, 5 received, 0% packet loss

Troubleshooting

 
 
Verify the GRE-IPsec tunnel interface status
 

FGT # diag netlink interface list | grep -A1 "toCisco"
if=toCisco family=00 type=768 index=19 mtu=1438 link=0 master=0
ref=20 state=off start fw_flags=0 flags=up p2p run noarp multicast
 
 
FGT # get sys interface | grep -A1 "toCisco"
== [ toCisco ]
name: toCisco   ip: 10.255.255.1 255.255.255.255   status: up    netbios-forward: disable    type: tunnel   netflow-sampler: disable    sflow-sampler: disable    scan-botnet-connections: disable    src-check: enable    wccp: disable
 

 
 
Verify OSPF status
 

FGT # get router info ospf status
Routing Process "ospf 0" with ID 10.1.1.254
 Process uptime is 49 minutes
 Process bound to VRF default
 Conforms to RFC2328, and RFC1583Compatibility flag is disabled
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Do not support Restarting
 SPF schedule delay 5 secs, Hold time between two SPFs 10 secs
 Refresh timer 10 secs
 Number of incomming current DD exchange neighbors 0/5
 Number of outgoing current DD exchange neighbors 0/5
 Number of external LSA 0. Checksum 0x000000
 Number of opaque AS LSA 0. Checksum 0x000000
 Number of non-default external LSA 0
 External LSA database is unlimited.
 Number of LSA originated 4
 Number of LSA received 2
 Number of areas attached to this router: 1
    Area 0.0.0.0 (BACKBONE)
        Number of interfaces in this area is 2(2)
        Number of fully adjacent neighbors in this area is 1
        Area has no authentication
        SPF algorithm last executed 00:27:06.140 ago
        SPF algorithm executed 5 times
        Number of LSA 2. Checksum 0x007c7f
 
 
FGT # get router info ospf interface
port2 is up, line protocol is up
  Internet Address 10.1.1.254/24, Area 0.0.0.0, MTU 1500
  Process ID 0, Router ID 10.1.1.254, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 10.1.1.254, Interface Address 10.1.1.254
  No backup designated router on this network
  Timer intervals configured, Hello 10.000, Dead 40, Wait 40, Retransmit 5
    Hello due in 00:00:01
  Neighbor Count is 0, Adjacent neighbor count is 0
  Crypt Sequence Number is 3700
  Hello received 0 sent 165, DD received 0 sent 0
  LS-Req received 0 sent 0, LS-Upd received 0 sent 0
  LS-Ack received 0 sent 0, Discarded 0
toCisco is up, line protocol is up
  Internet Address 10.255.255.1/32, Area 0.0.0.0, MTU 1438
  Process ID 0, Router ID 10.1.1.254, Network Type POINTOPOINT, Cost: 100
  Transmit Delay is 1 sec, State Point-To-Point
  Timer intervals configured, Hello 10.000, Dead 40, Wait 40, Retransmit 5
    Hello due in 00:00:08
  Neighbor Count is 1, Adjacent neighbor count is 1
  Crypt Sequence Number is 2977
  Hello received 244 sent 303, DD received 2 sent 113
  LS-Req received 1 sent 1, LS-Upd received 3 sent 4
  LS-Ack received 4 sent 2, Discarded 92
 
 
FGT # get router info ospf neighbor
 
OSPF process 0:
Neighbor ID     Pri   State           Dead Time   Address         Interface
10.2.2.254        1   Full/ -         00:00:36    10.255.255.2    toCisco
 
 
FGT # get router info ospf database brief
 
                Router Link States (Area 0.0.0.0)
 
Link ID         ADV Router      Age  Seq#     CkSum Flag Link count
10.1.1.254      10.1.1.254      1689 80000004 60a6  0031 4
10.2.2.254      10.2.2.254      2451 80000002 1bd9  0002 3
 
 
FGT # get router info ospf route
C  10.1.1.0/24 [1] is directly connected, port2, Area 0.0.0.0
O  10.2.2.0/24 [101] via 10.255.255.2, toCisco, Area 0.0.0.0
O  10.255.255.0/30 [1100] via 10.255.255.2, toCisco, Area 0.0.0.0
C  10.255.255.1/32 [100] is directly connected, toCisco, Area 0.0.0.0
 

 
 
Verify the routing table (RIB)
 

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
O       10.2.2.0/24 [110/101] via 10.255.255.2, toCisco, 00:41:46
O       10.255.255.0/30 [110/1100] via 10.255.255.2, toCisco, 00:41:46
C       10.255.255.1/32 is directly connected, toCisco
C       10.255.255.2/32 is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
C       198.51.100.0/24 is directly connected, port1
 

Verify the kernel routes (FIB)
 

FGT # get router info kernel
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.254/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.255/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.1/32 pref=10.255.255.1 gwy=0.0.0.0 dev=19(toCisco)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.1/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.255/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.1/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.255/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->0.0.0.0/0 pref=0.0.0.0 gwy=198.51.100.254 dev=3(port1)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/24 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.2.2.0/24 pref=0.0.0.0 gwy=10.255.255.2 dev=19(toCisco)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.0/30 pref=0.0.0.0 gwy=10.255.255.2 dev=19(toCisco)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.2/32 pref=10.255.255.1 gwy=0.0.0.0 dev=19(toCisco)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/24 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/24 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
 

 
 
Verify the IPsec tunnel status
 

## phase1 IKE SA

 
FGT # diagnose vpn ike gateway list
 
vd: root/0
name: toCisco
version: 1
interface: port1 3
addr: 198.51.100.1:500 -> 192.0.2.2:500
virtual-interface-addr: 10.255.255.1 -> 10.255.255.2
created: 3710s ago
auto-discovery: 0
IKE SA: created 1/1  established 1/1  time 230/255/280 ms
IPsec SA: created 1/5  established 1/5  time 130/276/490 ms
 
  id/spi: 5 dc8687e453780573/ab4f308821fa8ec5
  direction: initiator
  status: established 3710-3710s ago = 230ms
  proposal: aes128-sha1
  key: e2450f2fc786cb09-56dfeebd3af8373f
  lifetime/rekey: 86400/82389
  DPD sent/recv: 00000000/00000000 
 

## phase2 IPsec SA

 
FGT # diagnose vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=toCisco ver=1 serial=2 198.51.100.1:0->192.0.2.2:0
bound_if=3 lgwy=static/1 tun=intf/0 mode=auto/1 encap=GRE/5 options[0005]=edg
encap-addr: 198.51.100.1->192.0.2.2
proxyid_num=1 child_num=0 refcnt=20 ilast=3 olast=3 auto-discovery=0, itn-status=0
stat: rxp=596 txp=663 rxb=305600 txb=266138
dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=toCisco proto=47 sa=1 ref=2 serial=1 auto-negotiate transport-mode
  src: 47:0.0.0.0/0.0.0.0:0
  dst: 47:0.0.0.0/0.0.0.0:0
  SA: ref=3 options=8327 type=00 soft=0 mtu=1438 expire=2825/0B replaywin=2048 seqno=30 esn=0 replaywin_lastseq=00000024
  life: type=01 bytes=0/0 timeout=3300/3600
  dec: spi=b0e2b4d7 esp=aes key=16 449524748c5e1f249680d4f982078e15
       ah=sha1 key=20 714bf3e5f5df9f25794727424b03ef5e4db7f009
  enc: spi=34740cc7 esp=aes key=16 81114b9a3ec521fd5901576dc156edad
       ah=sha1 key=20 190871a618de28ee7672404f3c5b6b31066b1391
  dec:pkts/bytes=36/3024, enc:pkts/bytes=47/6392
 

 
 
Verify the sniffer trace when PC1 attempts to ping PC2
 

## ICMP traffic between PC1 and PC2

 
FGT # diag sniffer packet any 'host 10.2.2.2 and icmp' 4
interfaces=[any]
filters=[host 10.2.2.2 and icmp]
 
3.578106 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.578250 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.609041 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
3.609113 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
4.578467 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.578491 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.607866 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
4.607899 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
5.579690 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.579739 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.597982 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
5.598007 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
6.581236 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.581266 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.610108 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
6.610131 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
7.583133 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
7.583155 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
7.611372 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
7.611387 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
20 packets received by filter
0 packets dropped by kernel
 
 

## IPsec traffic (ESP) sent and received by the FGT

 
FGT # diagnose sniffer packet any 'esp' 4
interfaces=[any]
filters=[esp]
 
3.145196 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
3.165217 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
4.146018 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
4.182590 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
5.147144 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
5.179591 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
6.148544 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
6.169862 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
7.150249 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
7.172710 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
10 packets received by filter
0 packets dropped by kernel

  
Verify the debug flow when PC1 attempts to ping PC2
 

FG1 # diag debug flow filter clear
FG1 # diag debug flow show function-name enable
show function name
 
FG1 # diag debug flow show iprope enable
show trace messages about iprope
 
FG1 # diag debug flow filter proto 1
FG1 # diag debug flow filter addr 10.2.2.2
 
FG1 # diag debug flow trace start 1000
FG1 # diag debug enable 
 

## ICMP echo-request from PC1 to PC2

 
id=20085 trace_id=3 func=print_pkt_detail line=5204 msg="vd-root received a packet(proto=1, 10.1.1.1:202->10.2.2.2:2048) from port2. type=8, code=0, id=202, seq=1."
id=20085 trace_id=3 func=init_ip_session_common line=5367 msg="allocate a new session-0000015f"
id=20085 trace_id=3 func=iprope_dnat_check line=4773 msg="in-[port2], out-[]"
id=20085 trace_id=3 func=iprope_dnat_check line=4786 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=3 func=vf_ip_route_input_common line=2578 msg="find a route: flag=04000000 gw-10.255.255.2 via toCisco"
id=20085 trace_id=3 func=iprope_fwd_check line=670 msg="in-[port2], out-[toCisco], skb_flags-02000000, vid-0, app_id: 0, url_cat_id: 0"
id=20085 trace_id=3 func=__iprope_check line=2102 msg="gnum-100004, check-ffffffffa0020979"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1873 msg="checked gnum-100004 policy-1, ret-matched, act-accept"
id=20085 trace_id=3 func=__iprope_user_identity_check line=1698 msg="ret-matched"
id=20085 trace_id=3 func=__iprope_check line=2102 msg="gnum-4e20, check-ffffffffa0020979"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1873 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1873 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1873 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check line=2121 msg="gnum-4e20 check result: ret-no-match, act-accept, flag-00000000, flag2-00000000"
id=20085 trace_id=3 func=__iprope_check_one_policy line=2073 msg="policy-1 is matched, act-accept"
id=20085 trace_id=3 func=__iprope_check line=2121 msg="gnum-100004 check result: ret-matched, act-accept, flag-08010000, flag2-00004000"
id=20085 trace_id=3 func=iprope_fwd_auth_check line=726 msg="after iprope_captive_check(): is_captive-0, ret-matched, act-accept, idx-1"
id=20085 trace_id=3 func=fw_forward_handler line=706 msg="Allowed by Policy-1:"
 
id=20085 trace_id=3 func=ipsecdev_hard_start_xmit line=178 msg="enter IPsec interface-toCisco"
id=20085 trace_id=3 func=esp_output4 line=888 msg="IPsec encrypt/auth"
id=20085 trace_id=3 func=ipsec_output_finish line=522 msg="send to 198.51.100.254 via intf-port1" 
 

## ICMP echo-reply from PC2 to PC1

 
id=20085 trace_id=4 func=print_pkt_detail line=5204 msg="vd-root received a packet(proto=1, 10.2.2.2:202->10.1.1.1:0) from toCisco. type=0, code=0, id=202, seq=1."
id=20085 trace_id=4 func=resolve_ip_tuple_fast line=5279 msg="Find an existing session, id-0000015f, reply direction"
id=20085 trace_id=4 func=vf_ip_route_input_common line=2578 msg="find a route: flag=00000000 gw-10.1.1.1 via port2"
 

 
 
Verify the session
 

FG1 # diag sys session filter clear
 
FG1 # diag sys session filter dst 10.2.2.2
 
FG1 # diag sys session filter proto 1
 
FG1 # diag sys session list
 
session info: proto=1 proto_state=00 duration=10 expire=49 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=toCisco/ vlan_cos=0/255
state=may_dirty
statistic(bytes/packets/allow_err): org=84/1/1 reply=84/1/1 tuples=2
tx speed(Bps/kbps): 7/0 rx speed(Bps/kbps): 7/0
orgin->sink: org pre->post, reply pre->post dev=4->19/19->4 gwy=10.255.255.2/10.1.1.1
hook=pre dir=org act=noop 10.1.1.1:202->10.2.2.2:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.2.2.2:202->10.1.1.1:0(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=0000015f tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
total session 1
 

Example of a decrypted GRE over IPsec packet containing PC1’s Echo-Request
 

## The ESP (IPsec) packet

 
Ethernet II, Src: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02), Dst: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Destination: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Source: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02)
    Type: IPv4 (0x0800)
 
Internet Protocol Version 4, Src: 198.51.100.1, Dst: 192.0.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 152
    Identification: 0xcaba (51898)
    Flags: 0x00
    Fragment offset: 0
    Time to live: 63
    Protocol: Encap Security Payload (50)
    Header checksum: 0xc442 [correct]
    Source: 198.51.100.1
    Destination: 192.0.2.2
 
Encapsulating Security Payload
    ESP SPI: 0x34740cc7 (880020679)
    ESP Sequence: 117
    ESP IV: 778b201ea8b76cd873667da2b3655545
    Pad: 010203040506
    ESP Pad Length: 6
    Next header: Generic Routing Encapsulation (0x2f)
    Authentication Data [correct] 
 

## The original IP packet carried inside the GRE packet

 
Generic Routing Encapsulation (IP)
    Flags and Version: 0x0000
    Protocol Type: IP (0x0800)
 
Internet Protocol Version 4, Src: 10.1.1.1, Dst: 10.2.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 84
    Identification: 0x2e18 (11800)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 63
    Protocol: ICMP (1)
    Header checksum: 0xf68b [correct]
    Source: 10.1.1.1
    Destination: 10.2.2.2
 
Internet Control Message Protocol
    Type: 8 (Echo (ping) request)
    Code: 0
    Checksum: 0x97cc [correct]
    Identifier (BE): 202 (0x00ca)
    Identifier (LE): 51712 (0xca00)
    Sequence number (BE): 1 (0x0001)
    Sequence number (LE): 256 (0x0100)
    Data (56 bytes)

Related Articles

Technical Note: Configuring and verifying a GRE over IPsec tunnel