Technical Note: Configuring and verifying a GRE over IPsec tunnel

[sha-1_FTNT]

Description

This article describes how to configure and troubleshoot a GRE over IPsec tunnel between a FortiGate and a Cisco router

Scope

Support for GRE tunneling and GRE over IPsec in tunnel-mode is available as of FortiOS 3.0
Support for IPsec in transport-mode is available as of FortiOS 4.0 MR2

Solution

Diagram

 
The following topology is used:

PC1(.1) - 10.1.1.0/24-port2-[ FGT ]-port1----(198.51.100.1) =======
                                                                  I
                                                 (Internet)       I GRE over IPsec tunnel
                                                                  I
PC2(.2) - 10.2.2.0/24-gi0/0-[ Cisco_RTR ]-gi1/0-(192.0.2.2) =======  
 
 

Design

• Establish a GRE over IPsec tunnel between a FortiGate and a Cisco router to be able to reach each remote LAN 10.x.x.x
• IPsec in transport mode is used since data packets are already tunneled in GRE
• OSPF is used as dynamic routing protocol (multicast traffic, hence the need for GRE-IPsec with some vendors)

Why a GRE over IPsec tunnel instead of a plain IPsec tunnel ?
 
Common reasons are:
 

1-    Multicast over IPsec
 

Some vendors do not support multicast traffic (OSPF, streaming,…) directly inside an IPsec tunnel.
The multicast traffic is therefore tunneled in GRE which itself is protected by IPsec. Only the unicast GRE traffic between the GRE endpoints is exposed to IPsec.
 
FortiOS supports multicast traffic directly inside IPsec.
There is therefore no requirement to use GRE-IPsec to carry multicast traffic between two FortiGates.

 
 
2-    Traffic selector simplification
 

Some vendors do not support negotiating wildcard traffic selectors (i.e., any-any selectors: src-subnet=0.0.0.0/0 and dst-subnet=0.0.0.0/0)
Either they require the exhaustive list of all local-subnets and all remote-subnets,
Or they require specifying all the possible combination of (local <-> remote) subnets.
 
FortiOS supports any-any selectors (src-subnet=0.0.0.0/0 and dst-subnet=0.0.0.0/0).

There is therefore no requirement to use GRE-IPsec to simplify the traffic selector configuration between two FortiGates.

 
 
GRE over IPsec configuration with FortiOS
 
The GRE over IPsec configuration in this article is based on the independent configuration of GRE settings and IPsec settings.
 
A tighter integration between GRE and IPsec (“encapsulation gre”) is available as of FortiOS 5.4

FortiOS 5.4 “encapsulation gre” has however some limitations:
  -       only IPsec in tunnel-mode is supported (no support for IPsec in transport-mode),
  -       the traffic selectors cannot be restricted to the GRE endpoints,
  -       dynamic routing is not supported
 
All the above “encapsulation gre” limitations are removed as of FortiOS 5.6:
  -       IPsec is supported in both transport-mode and tunnel-mode,
  -       traffic selectors can be restricted to the GRE endpoints addresses and GRE protocol (ip/47)
  -       dynamic routing is supported
 
The scenario covered in this article is also available using the “encapsulation gre” configuration. See the Related Articles section.

Limitations

• The inner GRE traffic cannot be hardware offloaded to NPU (NP6, NP4)
• IPsec in transport-mode cannot be offloaded to NPU (NP6, NP4)

Configuration

CLI configuration of the FGT
 
#
# Port1 is the Internet-facing interface
# Port2 is the LAN interface
#
 
config system interface
    edit "port1"
        set ip 198.51.100.1 255.255.255.0
        set alias "Internet"
    next
    edit "port2"
        set ip 10.1.1.254 255.255.255.0
        set alias "LAN"
    next
end
 
#
# IPsec VPN used to protect the GRE traffic
#
 
config vpn ipsec phase1-interface
    edit "ipsec"
        set interface "port1"
        set proposal aes128-sha1
        set dhgrp 14
        set remote-gw 192.0.2.2
        set psksecret fortinet
    next
end
 
config vpn ipsec phase2-interface
    edit "ipsec"
        set phase1name "ipsec"
        set proposal aes128-sha1
        set dhgrp 14
        set protocol 47                     // restrict traffic selectors to GRE protocol (ip/47)
        set auto-negotiate enable
        set encapsulation transport-mode    // transport-mode (GRE is already tunneled)
    next
end
 
#
# Configure a GRE tunnel
#
 
config system gre-tunnel
    edit "toCisco"
        set interface "ipsec"      // the GRE tunnel is protected by IPsec
        set remote-gw 192.0.2.2
        set local-gw 198.51.100.1
    next
end
 
#
# Local and remote overlay IP addresses over the GRE tunnel
#
 
config system interface
    edit "toCisco"
        set ip 10.255.255.1 255.255.255.255
        set remote-ip 10.255.255.2
    next
end
 
#
# Firewall Policies
#
 
config firewall address
    edit "10.1.1.0/24"
        set comment "Local LAN"
        set subnet 10.1.1.0 255.255.255.0
    next
    edit "10.2.2.0/24"
        set comment "Remote LAN"
        set subnet 10.2.2.0 255.255.255.0
    next
end
 
config firewall policy
 
    Allow traffic between the local LAN (port2) and the remote LAN (GRE)
    edit 1
        set name "to remote LAN"
        set srcintf "port2"
        set dstintf "toCisco"
        set srcaddr "10.1.1.0/24"
        set dstaddr "10.2.2.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "local LAN to remote LAN"
    next
    edit 2
        set name "from remote LAN"
        set srcintf "toCisco"
        set dstintf "port2"
        set srcaddr "10.2.2.0/24"
        set dstaddr "10.1.1.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "remote LAN to local LAN"
    next
 
 
    GRE traffic to be IPsec-protected is self-originated, it is not received on an interface
    No forward-policy is therefore needed to allow GRE traffic to enter or leave the IPsec interface
    By FortiOS design, a forward-policy is however required to allow an IPsec negotiation to take place
    An arbitrary forward-policy (e.g., from and to the IPsec interface itself) is therefore used to “activate” IPsec
    edit 3
        set name "Enable IPsec"
        set srcintf "ipsec"
        set dstintf "ipsec"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "Just an \'activator\' for IPsec negotiation. No traffic flowing through this policy since IPsec is used to protect self-originated GRE traffic."
    next
 
 
    Should the remote LAN subnet (10.2.2.0/24) be missing in the routing table (e.g., OSPF adjacency is down)
    packets destined to 10.2.2.0/24 would match the default-route and the ‘Internet Access’ policy  
    This ‘Deny Internet’ policy ensures that packets destined to the remote LAN never match the ‘Internet Access’  
    policy  
    edit 4
        set name "Deny Internet"
        set srcintf "port2"
        set dstintf "port1"
        set srcaddr "all"
        set dstaddr "10.2.2.0/24"
        set schedule "always"
        set service "ALL"
        set logtraffic all
        set comments "Prevent remote LAN access to leak over the Internet"
    next
 
    Internet Access
    edit 5
        set name "Internet Access"
        set srcintf "port2"
        set dstintf "port1"
        set srcaddr "10.1.1.0/24"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "Internet Access"
        set nat enable
    next
 
 
#
# OSPF routing
#
 
config router ospf
    set router-id 10.1.1.254
    config area
        edit 0.0.0.0
        next
    end
    config network
        edit 1
            set prefix 10.1.1.254 255.255.255.255      // enable OSPF on the LAN interface
        next
        edit 2
            set prefix 10.255.255.1 255.255.255.255    // enable OSPF on the GRE tunnel
        next
    end
end
 
 
#
# Static routes
#
 
config router static
    edit 1
        set gateway 198.51.100.254
        set device "port1"
        set comment "default-route to Internet ISP"
    next
 
    After GRE tunneling, GRE packets must be protected by IPsec
    The remote-gw of the gre-tunnel must therefore points toward the IPsec interface
    edit 2
        set dst 192.0.2.2 255.255.255.255
        set device "ipsec"
        set comment "Reach GRE endpoint via IPsec tunnel"
    next
end

CLI configuration of the Cisco Router
 
!
! IPsec configuration
!
 
crypto isakmp policy 10
 encr aes
 authentication pre-share
 group 14
 
crypto isakmp key fortinet address 198.51.100.1
 
crypto ipsec transform-set aes128-sha1-transport esp-aes esp-sha-hmac
 mode transport
 
ip access-list extended encryptionDomain
 permit gre host 192.0.2.2 host 198.51.100.1
 
crypto map gre_over_ipsec 10 ipsec-isakmp
 set peer 198.51.100.1
 set transform-set aes128-sha1-transport
 set pfs group14
 match address encryptionDomain
 
 
!
! GRE tunnel interface
!
 
interface Tunnel0
 ip address 10.255.255.2 255.255.255.252    ! overlay subnet over the GRE tunnel
 tunnel source GigabitEthernet1/0
 tunnel destination 198.51.100.1
 
 
! LAN
 
interface GigabitEthernet0/0
 ip address 10.2.2.254 255.255.255.0
 ip nat inside
 
! Internet
 
interface GigabitEthernet1/0
 ip address 192.0.2.2 255.255.255.0
 ip nat outside
 crypto map gre_over_ipsec        ! apply IPsec to the traffic matching the crypto map
 
 
! OSPF
 
router ospf 1
 router-id 10.2.2.254
 network 10.2.2.254 0.0.0.0 area 0
 network 10.255.255.2 0.0.0.0 area 0
 
 
! SNAT for Internet Access
 
ip nat inside source list natAcl interface GigabitEthernet1/0 overload
 
ip access-list extended natAcl
 permit ip 10.2.2.0 0.0.0.255 any
 
! Static route
 
! default-route to Internet ISP
ip route 0.0.0.0 0.0.0.0 192.0.2.253
 

Verification

 
 
Verify the routing table (RIB)
 

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
O       10.2.2.0/24 [110/101] via 10.255.255.2, toCisco, 00:06:10
O       10.255.255.0/30 [110/1100] via 10.255.255.2, toCisco, 00:06:10
C       10.255.255.1/32 is directly connected, toCisco
C       10.255.255.2/32 is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
S       192.0.2.2/32 [10/0] is directly connected, ipsec
C       198.51.100.0/24 is directly connected, port1

To verify the GRE tunnel status:

FGT# diagnose  sys  gre  keepalive

GRE tunnel: toCisco, Status: alive
period: 5, retries 2/10
Packets sent: 49632  Packets received: 12234

 
Verify that PC1 and PC2 can ping each other

root@PC1:~# ping -c 5 10.2.2.2
PING 10.2.2.2 (10.2.2.2) 56(84) bytes of data.
64 bytes from 10.2.2.2: icmp_seq=1 ttl=62 time=40.7 ms
64 bytes from 10.2.2.2: icmp_seq=2 ttl=62 time=53.5 ms
64 bytes from 10.2.2.2: icmp_seq=3 ttl=62 time=50.0 ms
64 bytes from 10.2.2.2: icmp_seq=4 ttl=62 time=47.1 ms
64 bytes from 10.2.2.2: icmp_seq=5 ttl=62 time=44.9 ms
 
--- 10.2.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4005ms
rtt min/avg/max/mdev = 40.769/47.296/53.577/4.379 ms
 
 
PC2> ping 10.1.1.1
84 bytes from 10.1.1.1 icmp_seq=1 ttl=62 time=87.241 ms
84 bytes from 10.1.1.1 icmp_seq=2 ttl=62 time=80.711 ms
84 bytes from 10.1.1.1 icmp_seq=3 ttl=62 time=46.863 ms
84 bytes from 10.1.1.1 icmp_seq=4 ttl=62 time=46.857 ms
84 bytes from 10.1.1.1 icmp_seq=5 ttl=62 time=46.881 ms
 
5 packets transmitted, 5 received, 0% packet loss

Troubleshooting

 
 
Verify the GRE tunnel interface status
 

FGT # diag sys gre list
 
vd=0 devname=toCisco devindex=15 ifindex=20
saddr=198.51.100.1 daddr=192.0.2.2 ref=0
key=0/0 flags=0/0
 
total tunnel = 1
 
 
FGT # diag netlink interface list | grep -A1 "toCisco"
if=toCisco family=00 type=778 index=20 mtu=1430 link=0 master=0
ref=17 state=off start fw_flags=0 flags=up p2p run noarp multicast
 
 
FGT # get sys interface | grep -A1 "toCisco"
== [ toCisco ]
name: toCisco   ip: 10.255.255.1 255.255.255.255   status: up    netbios-forward: disable    type: tunnel   netflow-sampler: disable    sflow-sampler: disable    scan-botnet-connections: disable    explicit-web-proxy: disable    explicit-ftp-proxy: disable    mtu-override: disable    wccp: disable
 

Verify OSPF status
 

FGT # get router info ospf status
 Routing Process "ospf 0" with ID 10.1.1.254
 Process uptime is 2 minutes
 Process bound to VRF default
 Conforms to RFC2328, and RFC1583Compatibility flag is disabled
 Supports only single TOS(TOS0) routes
 Supports opaque LSA
 Do not support Restarting
 SPF schedule delay 5 secs, Hold time between two SPFs 10 secs
 Refresh timer 10 secs
 Number of incomming current DD exchange neighbors 0/5
 Number of outgoing current DD exchange neighbors 0/5
 Number of external LSA 0. Checksum 0x000000
 Number of opaque AS LSA 0. Checksum 0x000000
 Number of non-default external LSA 0
 External LSA database is unlimited.
 Number of LSA originated 4
 Number of LSA received 21
 Number of areas attached to this router: 1
    Area 0.0.0.0 (BACKBONE)
        Number of interfaces in this area is 2(2)
        Number of fully adjacent neighbors in this area is 1
        Area has no authentication
        SPF algorithm last executed 00:01:35.330 ago
        SPF algorithm executed 21 times
        Number of LSA 2. Checksum 0x0024d5
 
 
FGT # get router info ospf interface
port2 is up, line protocol is up
  Internet Address 10.1.1.254/24, Area 0.0.0.0, MTU 1500
  Process ID 0, Router ID 10.1.1.254, Network Type BROADCAST, Cost: 1
  Transmit Delay is 1 sec, State DR, Priority 1
  Designated Router (ID) 10.1.1.254, Interface Address 10.1.1.254
  No backup designated router on this network
  Timer intervals configured, Hello 10.000, Dead 40, Wait 40, Retransmit 5
    Hello due in 00:00:09
  Neighbor Count is 0, Adjacent neighbor count is 0
  Crypt Sequence Number is 2983
  Hello received 0 sent 16, DD received 0 sent 0
  LS-Req received 0 sent 0, LS-Upd received 0 sent 0
  LS-Ack received 0 sent 0, Discarded 0
toCisco is up, line protocol is up
  Internet Address 10.255.255.1/32, Area 0.0.0.0, MTU 1476
  Process ID 0, Router ID 10.1.1.254, Network Type POINTOPOINT, Cost: 100
  Transmit Delay is 1 sec, State Point-To-Point
  Timer intervals configured, Hello 10.000, Dead 40, Wait 40, Retransmit 5
    Hello due in 00:00:05
  Neighbor Count is 1, Adjacent neighbor count is 1
  Crypt Sequence Number is 2983
  Hello received 15 sent 16, DD received 5 sent 6
  LS-Req received 2 sent 1, LS-Upd received 5 sent 9
  LS-Ack received 5 sent 4, Discarded 0
 
 
FGT # get router info ospf neighbor
 
OSPF process 0:
Neighbor ID     Pri   State           Dead Time   Address         Interface
10.2.2.254        1   Full/ -         00:00:30    10.255.255.2    toCisco
 
 
FGT # get router info ospf database brief
 
                Router Link States (Area 0.0.0.0)
 
Link ID         ADV Router      Age  Seq#     CkSum Flag Link count
10.1.1.254      10.1.1.254      130  80000005 10f5  0031 4
10.2.2.254      10.2.2.254      144  80000003 13e0  0002 3
 
 
FGT # get router info ospf route
C  10.1.1.0/24 [1] is directly connected, port2, Area 0.0.0.0
O  10.2.2.0/24 [101] via 10.255.255.2, toCisco, Area 0.0.0.0
O  10.255.255.0/30 [1100] via 10.255.255.2, toCisco, Area 0.0.0.0
C  10.255.255.1/32 [100] is directly connected, toCisco, Area 0.0.0.0
 

Verify the routing table (RIB)
 

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
O       10.2.2.0/24 [110/101] via 10.255.255.2, toCisco, 00:06:10
O       10.255.255.0/30 [110/1100] via 10.255.255.2, toCisco, 00:06:10
C       10.255.255.1/32 is directly connected, toCisco
C       10.255.255.2/32 is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
S       192.0.2.2/32 [10/0] is directly connected, ipsec
C       198.51.100.0/24 is directly connected, port1
 

 
 
Verify the kernel routes (FIB)
 

FGT # get router info kernel
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.254/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.255/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.1/32 pref=10.255.255.1 gwy=0.0.0.0 dev=20(toCisco)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.1/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.255/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.1/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.255/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->0.0.0.0/0 pref=0.0.0.0 gwy=198.51.100.254 dev=3(port1)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/24 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.2.2.0/24 pref=0.0.0.0 gwy=10.255.255.2 dev=20(toCisco)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.0/30 pref=0.0.0.0 gwy=10.255.255.2 dev=20(toCisco)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.255.255.2/32 pref=10.255.255.1 gwy=0.0.0.0 dev=20(toCisco)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/24 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->192.0.2.2/32 pref=0.0.0.0 gwy=0.0.0.0 dev=15(ipsec)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/24 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
 

 
 
Verify the IPsec tunnel status
 

## phase1 IKE SA

 
FGT # diagnose vpn ike gateway list
 
vd: root/0
name: ipsec
version: 1
interface: port1 3
addr: 198.51.100.1:500 -> 192.0.2.2:500
created: 1598s ago
auto-discovery: 0
IKE SA: created 1/1  established 1/1  time 7230/7230/7230 ms
IPsec SA: created 1/1  established 1/1  time 7380/7380/7380 ms
 
  id/spi: 4 637dd492a91aa3aa/7fce7e98f4817222
  direction: initiator
  status: established 1598-1591s ago = 7230ms
  proposal: aes128-sha1
  key: 60ceeb19c9ff8e7f-29d9ea3fc2be4e2a
  lifetime/rekey: 86400/84508
  DPD sent/recv: 00000000/00000000
 

## phase2 IPsec SA

 
FGT # diagnose vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=ipsec ver=1 serial=1 198.51.100.1:0->192.0.2.2:0
bound_if=3 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/0
proxyid_num=1 child_num=0 refcnt=18 ilast=6 olast=6 auto-discovery=0
stat: rxp=191 txp=231 rxb=29240 txb=22352
dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=3
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=ipsec proto=47 sa=1 ref=2 serial=2 auto-negotiate transport-mode
  src: 47:0.0.0.0/0.0.0.0:0
  dst: 47:0.0.0.0/0.0.0.0:0
  SA: ref=3 options=27 type=00 soft=0 mtu=1454 expire=1979/0B replaywin=2048 seqno=e8 esn=0 replaywin_lastseq=000000c9
  life: type=01 bytes=0/0 timeout=3576/3600
  dec: spi=6ede198b esp=aes key=16 11ed2d9b5665a96f64569a9db743bb8a
       ah=sha1 key=20 ab1074130590c886585d7aebfe319c1bd077eeb0
  enc: spi=e837e17f esp=aes key=16 676c2881a5ea4fb4bb824401da7543f0
       ah=sha1 key=20 b2f5985d9b248acd04e095570ec6fec924be0e28
  dec:pkts/bytes=191/16384, enc:pkts/bytes=231/32536
 

Verify the sniffer trace when PC1 attempts to ping PC2
 

## ICMP traffic between PC1 and PC2

 
FGT # diag sniffer packet any 'host 10.2.2.2 and icmp' 4
interfaces=[any]
filters=[host 10.2.2.2 and icmp]
 
2.831172 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
2.831287 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
2.868716 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
2.868764 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
3.831141 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.831185 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.857989 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
3.858025 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
4.831918 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.831944 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.867633 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
4.867658 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
5.833020 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.833055 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.856450 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
5.856489 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
6.833319 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.833359 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.855880 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
6.855910 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
20 packets received by filter
0 packets dropped by kernel 
 

## GRE traffic (protocol 47) sent and received by the FGT

 
FGT # diagnose sniffer packet any 'ip proto 47' 4
interfaces=[any]
filters=[ip proto 47]
 
1.920502 ipsec out 198.51.100.1 -> 192.0.2.2: gre: length 88 proto-800
1.976693 ipsec in 192.0.2.2 -> 198.51.100.1: gre: length 88 proto-800
 
2.920556 ipsec out 198.51.100.1 -> 192.0.2.2: gre: length 88 proto-800
2.958866 ipsec in 192.0.2.2 -> 198.51.100.1: gre: length 88 proto-800
 
3.921789 ipsec out 198.51.100.1 -> 192.0.2.2: gre: length 88 proto-800
3.972762 ipsec in 192.0.2.2 -> 198.51.100.1: gre: length 88 proto-800
 
4.922061 ipsec out 198.51.100.1 -> 192.0.2.2: gre: length 88 proto-800
4.960529 ipsec in 192.0.2.2 -> 198.51.100.1: gre: length 88 proto-800
 
5.922551 ipsec out 198.51.100.1 -> 192.0.2.2: gre: length 88 proto-800
5.957651 ipsec in 192.0.2.2 -> 198.51.100.1: gre: length 88 proto-800
 
10 packets received by filter
0 packets dropped by kernel 
 

## IPsec traffic (ESP) sent and received by the FGT

 
FGT # diagnose sniffer packet any 'esp' 4
interfaces=[any]
filters=[esp]
 
3.315417 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
3.364389 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
4.316114 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
4.363084 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
5.317221 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
5.360981 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
6.318920 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
6.359161 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
7.319719 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-50 132
7.373217 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-50 132
 
10 packets received by filter
0 packets dropped by kernel

 
 
Verify the debug flow when PC1 attempts to ping PC2
 

FG1 # diag debug flow filter clear
FG1 # diag debug flow show function-name enable
show function name
 
FG1 # diag debug flow show iprope enable
show trace messages about iprope
 
FG1 # diag debug flow filter proto 1
FG1 # diag debug flow filter addr 10.2.2.2
FG1 # diag debug flow show console enable
show trace messages on console
 
FG1 # diag debug flow trace start 1000
FG1 # diag debug enable
  

## ICMP echo-request from PC1 to PC2

 
id=20085 trace_id=9 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.1.1.1:172->10.2.2.2:2048) from port2. type=8, code=0, id=172, seq=1."
id=20085 trace_id=9 func=init_ip_session_common line=4944 msg="allocate a new session-000003d5"
id=20085 trace_id=9 func=iprope_dnat_check line=4659 msg="in-[port2], out-[]"
id=20085 trace_id=9 func=iprope_dnat_check line=4672 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=9 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.255.255.2 via toCisco"
id=20085 trace_id=9 func=iprope_fwd_check line=636 msg="in-[port2], out-[toCisco], skb_flags-02000000, vid-0"
id=20085 trace_id=9 func=__iprope_check line=2049 msg="gnum-100004, check-ffffffffa001e70e"
id=20085 trace_id=9 func=__iprope_check_one_policy line=1823 msg="checked gnum-100004 policy-1, ret-matched, act-accept"
id=20085 trace_id=9 func=__iprope_user_identity_check line=1648 msg="ret-matched"
id=20085 trace_id=9 func=__iprope_check line=2049 msg="gnum-4e20, check-ffffffffa001e70e"
id=20085 trace_id=9 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=9 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=9 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=9 func=__iprope_check line=2068 msg="gnum-4e20 check result: ret-no-match, act-accept, flag-00000000, flag2-00000000"
id=20085 trace_id=9 func=__iprope_check_one_policy line=2020 msg="policy-1 is matched, act-accept"
id=20085 trace_id=9 func=__iprope_check line=2068 msg="gnum-100004 check result: ret-matched, act-accept, flag-08010000, flag2-00004000"
id=20085 trace_id=9 func=iprope_fwd_auth_check line=688 msg="after iprope_captive_check(): is_captive-0, ret-matched, act-accept, idx-1"
id=20085 trace_id=9 func=fw_forward_handler line=697 msg="Allowed by Policy-1:"
 
id=20085 trace_id=9 func=ipsecdev_hard_start_xmit line=157 msg="enter IPsec interface-ipsec"
id=20085 trace_id=9 func=esp_output4 line=859 msg="IPsec encrypt/auth"
id=20085 trace_id=9 func=ipsec_output_finish line=498 msg="send to 198.51.100.254 via intf-port1" 
 

## ICMP echo-reply from PC2 to PC1

 
id=20085 trace_id=10 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.2.2.2:172->10.1.1.1:0) from toCisco. type=0, code=0, id=172, seq=1."
id=20085 trace_id=10 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-000003d5, reply direction"
id=20085 trace_id=10 func=vf_ip_route_input_common line=2586 msg="find a route: flag=00000000 gw-10.1.1.1 via port2"
 

Verify the session
 

FG1 # diag sys session filter clear
 
FG1 # diag sys session filter dst 10.2.2.2
 
FG1 # diag sys session filter proto 1
 
FG1 # diag sys session list
 
session info: proto=1 proto_state=00 duration=4 expire=55 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=ipsec/ vlan_cos=0/255
state=may_dirty
statistic(bytes/packets/allow_err): org=84/1/1 reply=84/1/1 tuples=2
tx speed(Bps/kbps): 19/0 rx speed(Bps/kbps): 19/0
orgin->sink: org pre->post, reply pre->post dev=4->20/20->4 gwy=10.255.255.2/10.1.1.1
hook=pre dir=org act=noop 10.1.1.1:172->10.2.2.2:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.2.2.2:172->10.1.1.1:0(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=000003d5 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
total session 1
 

Example of a decrypted GRE over IPsec packet containing PC1’s Echo-Request
 

## The ESP (IPsec) packet

 
Ethernet II, Src: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02), Dst: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Destination: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Source: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02)
    Type: IPv4 (0x0800)
 
Internet Protocol Version 4, Src: 198.51.100.1, Dst: 192.0.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 152
    Identification: 0x5c6c (23660)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 64
    Protocol: Encap Security Payload (50)
    Header checksum: 0xf190 [correct]
    Source: 198.51.100.1
    Destination: 192.0.2.2
 
Encapsulating Security Payload
    ESP SPI: 0xe837e17f (3895976319)
    ESP Sequence: 367
    ESP IV: 17271258c2b5ebda8ca6dda8b4bfa956
    Pad: 010203040506
    ESP Pad Length: 6
    Next header: Generic Routing Encapsulation (0x2f)
    Authentication Data [correct]  

## The original IP packet carried inside the GRE packet

 
Generic Routing Encapsulation (IP)
    Flags and Version: 0x0000
    Protocol Type: IP (0x0800)
 
Internet Protocol Version 4, Src: 10.1.1.1, Dst: 10.2.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 84
    Identification: 0x3bec (15340)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 63
    Protocol: ICMP (1)
    Header checksum: 0xe8b7 [correct]
    Source: 10.1.1.1
    Destination: 10.2.2.2
 
Internet Control Message Protocol
    Type: 8 (Echo (ping) request)
    Code: 0
    Checksum: 0xbb92 [correct]
    Identifier (BE): 174 (0x00ae)
    Identifier (LE): 44544 (0xae00)
    Sequence number (BE): 1 (0x0001)
    Sequence number (LE): 256 (0x0100)
    Data (48 bytes)

Related Articles

Technical Note: Configuring and verifying a GRE over IPsec tunnel using 'encapsulation gre'