Technical Tip: Configuring and verifying an IP in IP tunnel

[sha-1_FTNT]

Description

This article describes how to configure and troubleshoot an IP in IP tunnel between a FortiGate and a Cisco router

Scope

Support for IP in IP tunneling (RFC 1853) is available as of FortiOS 5.0.3, 5.2 and 5.4.

Solution

Diagram

 
The following topology is used:

PC1(.1) - 10.1.1.0/24-port2-[ FGT ]-port1----(198.51.100.1) =======
                                                                  I
                                                 (Internet)       I IP-in-IP tunnel
                                                                  I
PC2(.2) - 10.2.2.0/24-gi0/0-[ Cisco_RTR ]-gi1/0-(192.0.2.2) =======  
 
 

Design

• Establish an IP in IP tunnel between a FortiGate and a Cisco router to be able to reach each remote LAN 10.x.x.x
• FGT’s IP-in-IP tunnel interface is kept unnumbered (i.e., no overlay IP address is assigned to this interface)
• Static routes are used

Limitations

 

• IP-in-IP tunneling and its inner traffic cannot be hardware offloaded to NPU (NP6, NP4).
• As of version 5.4, FortiOS supports NPx offloading of IP-in-IP tunneling traffic.
  

 

Configuration

CLI configuration of the FGT

 
#
# Port1 is the Internet-facing interface
# Port2 is the LAN interface
#
 
config system interface
    edit "port1"
        set ip 198.51.100.1 255.255.255.0
        set alias "Internet"
    next
    edit "port2"
        set ip 10.1.1.254 255.255.255.0
        set alias "LAN"
    next
end
 
 
#
# IP-in-IP tunnel
#
 
config system ipip-tunnel
    edit "toCisco"
        set interface "port1"
        set remote-gw 192.0.2.2
        set local-gw 198.51.100.1
    next
end
 
 

#
# Firewall Policies
#

config firewall address
    edit "10.1.1.0/24"
        set comment "Local LAN"
        set subnet 10.1.1.0 255.255.255.0
    next
    edit "10.2.2.0/24"
        set comment "Remote LAN"
        set subnet 10.2.2.0 255.255.255.0
    next

end

 

config firewall policy
 
    Allow traffic between the local LAN (port2) and the remote LAN (IP-in-IP)
    edit 1
        set name "to remote LAN"
        set srcintf "port2"
        set dstintf "toCisco"
        set srcaddr "10.1.1.0/24"
        set dstaddr "10.2.2.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "local LAN to remote LAN"
    next
    edit 2
        set name "from remote LAN"
        set srcintf "toCisco"
        set dstintf "port2"
        set srcaddr "10.2.2.0/24"
        set dstaddr "10.1.1.0/24"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "remote LAN to local LAN"
    next
 
    Internet Access
    edit 3
        set name "Internet Access"
        set srcintf "port2"
        set dstintf "port1"
        set srcaddr "10.1.1.0/24"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
        set comments "Internet Access"
        set nat enable
    next
end

 
#
# Static routes
#
 
config router static
    edit 1
        set gateway 198.51.100.254
        set device "port1"
        set comment "default-route to Internet ISP"
    next
    edit 2
        set dst 10.2.2.0 255.255.255.0
        set device "toCisco"
        set comment "Remote LAN via the IP-in-IP tunnel"
    next
end

CLI configuration of the Cisco Router

 
! LAN
 
interface GigabitEthernet0/0
 ip address 10.2.2.254 255.255.255.0
 ip nat inside
 
! Internet
 
interface GigabitEthernet1/0
 ip address 192.0.2.2 255.255.255.0
 ip nat outside
 
! IP-in-IP tunnel interface
 
interface Tunnel0
 ip address 10.255.255.1 255.255.255.255    ! An overlay IP is mandatory for the static route over the tunnel
 tunnel source GigabitEthernet1/0
 tunnel destination 198.51.100.1
 tunnel mode ipip
 
! SNAT for Internet Access
 
ip nat inside source list natAcl interface GigabitEthernet1/0 overload
 
ip access-list extended natAcl
 permit ip 10.2.2.0 0.0.0.255 any
 
! Static routes
 
! default-route to Internet ISP
ip route 0.0.0.0 0.0.0.0 192.0.2.253
 
! Remote LAN via the IP-in-IP tunnel
ip route 10.1.1.0 255.255.255.0 Tunnel0

Verification

Verify the routing table (RIB)

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
S       10.2.2.0/24 [10/0] is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
C       198.51.100.0/24 is directly connected, port1
 

Verify that PC1 and PC2 can ping each other

 

root@PC1:~# ping -c 5 10.2.2.2
PING 10.2.2.2 (10.2.2.2) 56(84) bytes of data.
64 bytes from 10.2.2.2: icmp_seq=1 ttl=62 time=44.8 ms
64 bytes from 10.2.2.2: icmp_seq=2 ttl=62 time=42.4 ms
64 bytes from 10.2.2.2: icmp_seq=3 ttl=62 time=40.8 ms
64 bytes from 10.2.2.2: icmp_seq=4 ttl=62 time=54.1 ms
64 bytes from 10.2.2.2: icmp_seq=5 ttl=62 time=52.6 ms
 
--- 10.2.2.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4000ms
rtt min/avg/max/mdev = 40.820/46.980/54.151/5.427 ms

 
 
PC2> ping 10.1.1.1
84 bytes from 10.1.1.1 icmp_seq=1 ttl=62 time=87.241 ms
84 bytes from 10.1.1.1 icmp_seq=2 ttl=62 time=80.711 ms
84 bytes from 10.1.1.1 icmp_seq=3 ttl=62 time=46.863 ms
84 bytes from 10.1.1.1 icmp_seq=4 ttl=62 time=46.857 ms
84 bytes from 10.1.1.1 icmp_seq=5 ttl=62 time=46.881 ms
 
5 packets transmitted, 5 received, 0% packet loss 

Troubleshooting

Verify the IP-in-IP tunnel interface status
 

FGT # diag netlink interface list | grep -A1 "toCisco"
if=toCisco family=00 type=768 index=15 mtu=1480 link=0 master=0
ref=12 state=off start fw_flags=0 flags=up p2p run noarp multicast
 
FGT # get sys interface | grep -A1 "toCisco"
== [ toCisco ]
name: toCisco   ip: 0.0.0.0 0.0.0.0   status: up    netbios-forward: disable    type: tunnel   netflow-sampler: disable    sflow-sampler: disable    scan-botnet-connections: disable    explicit-web-proxy: disable    explicit-ftp-proxy: disable    wccp: disable
 

Verify the routing table (RIB)

FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
 
S*      0.0.0.0/0 [10/0] via 198.51.100.254, port1
C       10.1.1.0/24 is directly connected, port2
S       10.2.2.0/24 [10/0] is directly connected, toCisco
C       172.16.31.0/24 is directly connected, port10
C       198.51.100.0/24 is directly connected, port1
 

Verify the kernel routes (FIB)
 

FGT # get router info kernel
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.254/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.255/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.1/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.255/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.1/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.255/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->0.0.0.0/0 pref=0.0.0.0 gwy=198.51.100.254 dev=3(port1)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/24 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2)
tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.2.2.0/24 pref=0.0.0.0 gwy=0.0.0.0 dev=15(toCisco)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/24 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10)
tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/24 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1)

 Verify the sniffer trace when PC1 attempts to ping PC2

## ICMP traffic between PC1 and PC2

 
FGT # diag sniffer packet any 'host 10.2.2.2 and icmp' 4
interfaces=[any]
filters=[host 10.2.2.2 and icmp]
 
3.207275 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.208154 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
3.249111 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
3.249239 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
4.208830 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.208853 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
4.246856 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
4.246870 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
5.209664 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.209698 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
5.260851 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
5.260868 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
6.210737 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.210780 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
6.259164 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
6.259182 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
7.212169 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request
7.212212 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request
7.257794 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply
7.257811 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply
 
20 packets received by filter
0 packets dropped by kernel
 

## IP-in-IP traffic (protocol 4) sent and received by the FGT

FGT # diagnose sniffer packet any 'ip proto 4' 4
interfaces=[any]
filters=[ip proto 4]

3.366282 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-4 84
3.417012 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-4 84
 
4.366945 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-4 84
4.436003 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-4 84
 
5.368053 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-4 84
5.413783 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-4 84
 
6.369808 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-4 84
6.423911 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-4 84
 
7.370908 port1 out 198.51.100.1 -> 192.0.2.2:  ip-proto-4 84
7.410080 port1 in 192.0.2.2 -> 198.51.100.1:  ip-proto-4 84
 
10 packets received by filter
0 packets dropped by kernel

 
 
Verify the debug flow when PC1 attempts to ping PC2
 

FG1 # diag debug flow filter clear
FG1 # diag debug flow show function-name enable
show function name
 
FG1 # diag debug flow show iprope enable
show trace messages about iprope
 
FG1 # diag debug flow filter proto 1
FG1 # diag debug flow filter addr 10.2.2.2
FG1 # diag debug flow show console enable
show trace messages on console
 
FG1 # diag debug flow trace start 1000
FG1 # diag debug enable
 
 

## ICMP echo-request from PC1 to PC2

 
id=20085 trace_id=3 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.1.1.1:135->10.2.2.2:2048) from port2. type=8, code=0, id=135, seq=1."
id=20085 trace_id=3 func=init_ip_session_common line=4944 msg="allocate a new session-00000071"
id=20085 trace_id=3 func=iprope_dnat_check line=4659 msg="in-[port2], out-[]"
id=20085 trace_id=3 func=iprope_dnat_check line=4672 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000"
id=20085 trace_id=3 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.2.2.2 via toCisco"
id=20085 trace_id=3 func=iprope_fwd_check line=636 msg="in-[port2], out-[toCisco], skb_flags-02000000, vid-0"
id=20085 trace_id=3 func=__iprope_check line=2049 msg="gnum-100004, check-ffffffffa001e70e"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-100004 policy-1, ret-matched, act-accept"
id=20085 trace_id=3 func=__iprope_user_identity_check line=1648 msg="ret-matched"
id=20085 trace_id=3 func=__iprope_check line=2049 msg="gnum-4e20, check-ffffffffa001e70e"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept"
id=20085 trace_id=3 func=__iprope_check line=2068 msg="gnum-4e20 check result: ret-no-match, act-accept, flag-00000000, flag2-00000000"
id=20085 trace_id=3 func=__iprope_check_one_policy line=2020 msg="policy-1 is matched, act-accept"
id=20085 trace_id=3 func=__iprope_check line=2068 msg="gnum-100004 check result: ret-matched, act-accept, flag-08010000, flag2-00004000"
id=20085 trace_id=3 func=iprope_fwd_auth_check line=688 msg="after iprope_captive_check(): is_captive-0, ret-matched, act-accept, idx-1"
id=20085 trace_id=3 func=fw_forward_handler line=697 msg="Allowed by Policy-1:"
 

## ICMP echo-reply from PC2 to PC1

 
id=20085 trace_id=4 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.2.2.2:135->10.1.1.1:0) from toCisco. type=0, code=0, id=135, seq=1."
id=20085 trace_id=4 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-00000071, reply direction"
id=20085 trace_id=4 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.1.1.1 via port2"
 

 
 
Verify the session
 

FG1 # diag sys session filter clear
 
FG1 # diag sys session filter dst 10.2.2.2
 
FG1 # diag sys session filter proto 1
 
FG1 # diag sys session list
 
session info: proto=1 proto_state=00 duration=5 expire=57 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=may_dirty
statistic(bytes/packets/allow_err): org=84/1/1 reply=84/1/1 tuples=2
tx speed(Bps/kbps): 14/0 rx speed(Bps/kbps): 14/0
orgin->sink: org pre->post, reply pre->post dev=4->15/15->4 gwy=10.2.2.2/10.1.1.1
hook=pre dir=org act=noop 10.1.1.1:140->10.2.2.2:8(0.0.0.0:0)
hook=post dir=reply act=noop 10.2.2.2:140->10.1.1.1:0(0.0.0.0:0)
misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0
serial=00000071 tos=ff/ff app_list=0 app=0 url_cat=0
dd_type=0 dd_mode=0
total session 1
 

 
 
Example of an IP-in-IP packet containing PC1’s Echo-Request
 

## The IP-in-IP packet

 
Ethernet II, Src: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02), Dst: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Destination: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01)
    Source: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02)
    Type: IPv4 (0x0800)
 
Internet Protocol Version 4, Src: 198.51.100.1, Dst: 192.0.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 104
    Identification: 0x5c36 (23606)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 64
    Protocol: IPIP (4)
    Header checksum: 0xf224 [correct]
    Source: 198.51.100.1
    Destination: 192.0.2.2
 

## The original IP packet carried inside the IP-in-IP packet

 
Internet Protocol Version 4, Src: 10.1.1.1, Dst: 10.2.2.2
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 84
    Identification: 0xb7c9 (47049)
    Flags: 0x02 (Don't Fragment)
    Fragment offset: 0
    Time to live: 63
    Protocol: ICMP (1)
    Header checksum: 0x6cda [correct]
    Source: 10.1.1.1
    Destination: 10.2.2.2
 
Internet Control Message Protocol
    Type: 8 (Echo (ping) request)
    Code: 0
    Checksum: 0x1beb [correct]
    Identifier (BE): 147 (0x0093)
    Identifier (LE): 37632 (0x9300)
    Sequence number (BE): 1 (0x0001)
    Sequence number (LE): 256 (0x0100)
    Data (56 bytes)

Related Articles

Technical Note: Configuring and verifying an IP in IP over IPsec tunnel