This article describes how to configure and troubleshoot an IP in IP tunnel between a FortiGate and a Cisco router
Support for IP in IP tunneling (RFC 1853) is available as of FortiOS 5.0.3, 5.2 and 5.4.
Diagram
Design
Limitations
Configuration
CLI configuration of the FGT # # Port1 is the Internet-facing interface # Port2 is the LAN interface # config system interface edit "port1" set ip 198.51.100.1 255.255.255.0 set alias "Internet" next edit "port2" set ip 10.1.1.254 255.255.255.0 set alias "LAN" next end # # IP-in-IP tunnel # config system ipip-tunnel edit "toCisco" set interface "port1" set remote-gw 192.0.2.2 set local-gw 198.51.100.1 next end # # Firewall Policies # config firewall address edit "10.1.1.0/24" set comment "Local LAN" set subnet 10.1.1.0 255.255.255.0 next edit "10.2.2.0/24" set comment "Remote LAN" set subnet 10.2.2.0 255.255.255.0 next end config
firewall policy
Allow traffic between the local LAN (port2) and the remote LAN (IP-in-IP) edit 1 set name "to remote LAN" set srcintf "port2" set dstintf "toCisco" set srcaddr "10.1.1.0/24" set dstaddr "10.2.2.0/24" set action accept set schedule "always" set service "ALL" set comments "local LAN to remote LAN" next edit 2 set name "from remote LAN" set srcintf "toCisco" set dstintf "port2" set srcaddr "10.2.2.0/24" set dstaddr "10.1.1.0/24" set action accept set schedule "always" set service "ALL" set comments "remote LAN to local LAN" next Internet Access edit 3 set name "Internet Access" set srcintf "port2" set dstintf "port1" set srcaddr "10.1.1.0/24" set dstaddr "all" set action accept set schedule "always" set service "ALL" set comments "Internet Access" set nat enable next end # # Static routes # config router static edit 1 set gateway 198.51.100.254 set device "port1" set comment "default-route to Internet ISP" next edit 2 set dst 10.2.2.0 255.255.255.0 set device "toCisco" set comment "Remote LAN via the IP-in-IP tunnel" next end |
CLI configuration of the Cisco Router ! LAN interface GigabitEthernet0/0 ip address 10.2.2.254 255.255.255.0 ip nat inside ! Internet interface GigabitEthernet1/0 ip address 192.0.2.2 255.255.255.0 ip nat outside ! IP-in-IP tunnel interface interface Tunnel0 ip address 10.255.255.1 255.255.255.255 ! An overlay IP is mandatory for the static route over the tunnel tunnel source GigabitEthernet1/0 tunnel destination 198.51.100.1 tunnel mode ipip ! SNAT for Internet Access ip nat inside source list natAcl interface GigabitEthernet1/0 overload ip access-list extended natAcl permit ip 10.2.2.0 0.0.0.255 any ! Static routes ! default-route to Internet ISP ip route 0.0.0.0 0.0.0.0 192.0.2.253 ! Remote LAN via the IP-in-IP tunnel ip route 10.1.1.0 255.255.255.0 Tunnel0 |
Verification
FGT # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default S* 0.0.0.0/0 [10/0] via 198.51.100.254, port1 C 10.1.1.0/24 is directly connected, port2 S 10.2.2.0/24 [10/0] is directly connected, toCisco C 172.16.31.0/24 is directly connected, port10 C 198.51.100.0/24 is directly connected, port1 |
root@PC1:~# ping -c 5 10.2.2.2 PING 10.2.2.2 (10.2.2.2) 56(84) bytes of data. 64 bytes from 10.2.2.2: icmp_seq=1 ttl=62 time=44.8 ms 64 bytes from 10.2.2.2: icmp_seq=2 ttl=62 time=42.4 ms 64 bytes from 10.2.2.2: icmp_seq=3 ttl=62 time=40.8 ms 64 bytes from 10.2.2.2: icmp_seq=4 ttl=62 time=54.1 ms 64 bytes from 10.2.2.2: icmp_seq=5 ttl=62 time=52.6 ms --- 10.2.2.2 ping statistics --- 5 packets transmitted, 5 received, 0% packet loss, time 4000ms rtt min/avg/max/mdev = 40.820/46.980/54.151/5.427 ms PC2> ping 10.1.1.1 84 bytes from 10.1.1.1 icmp_seq=1 ttl=62 time=87.241 ms 84 bytes from 10.1.1.1 icmp_seq=2 ttl=62 time=80.711 ms 84 bytes from 10.1.1.1 icmp_seq=3 ttl=62 time=46.863 ms 84 bytes from 10.1.1.1 icmp_seq=4 ttl=62 time=46.857 ms 84 bytes from 10.1.1.1 icmp_seq=5 ttl=62 time=46.881 ms 5 packets transmitted, 5 received, 0% packet loss |
Troubleshooting
FGT # diag netlink interface list | grep -A1 "toCisco" if=toCisco family=00 type=768 index=15 mtu=1480 link=0 master=0 ref=12 state=off start fw_flags=0 flags=up p2p run noarp multicast FGT # get sys interface | grep -A1 "toCisco" == [ toCisco ] name: toCisco ip: 0.0.0.0 0.0.0.0 status: up netbios-forward: disable type: tunnel netflow-sampler: disable sflow-sampler: disable scan-botnet-connections: disable explicit-web-proxy: disable explicit-ftp-proxy: disable wccp: disable |
FGT # get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default S* 0.0.0.0/0 [10/0] via 198.51.100.254, port1 C 10.1.1.0/24 is directly connected, port2 S 10.2.2.0/24 [10/0] is directly connected, toCisco C 172.16.31.0/24 is directly connected, port10 C 198.51.100.0/24 is directly connected, port1 |
FGT # get router info kernel
tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2) tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.254/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.255/32 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root) tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.0/8 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root) tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.0.0.1/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->127.255.255.255/32 pref=127.0.0.1 gwy=0.0.0.0 dev=13(root) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10) tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.1/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.255/32 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1) tab=255 vf=0 scope=254 type=2 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.1/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1) tab=255 vf=0 scope=253 type=3 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.255/32 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1) tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->0.0.0.0/0 pref=0.0.0.0 gwy=198.51.100.254 dev=3(port1) tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.1.1.0/24 pref=10.1.1.254 gwy=0.0.0.0 dev=4(port2) tab=254 vf=0 scope=0 type=1 proto=11 prio=0 0.0.0.0/0.0.0.0/0->10.2.2.0/24 pref=0.0.0.0 gwy=0.0.0.0 dev=15(toCisco) tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.16.31.0/24 pref=172.16.31.1 gwy=0.0.0.0 dev=12(port10) tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->198.51.100.0/24 pref=198.51.100.1 gwy=0.0.0.0 dev=3(port1) |
## ICMP traffic between PC1 and PC2 FGT # diag sniffer packet any 'host 10.2.2.2 and icmp' 4 interfaces=[any] filters=[host 10.2.2.2 and icmp] 3.207275 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request 3.208154 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request 3.249111 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply 3.249239 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply 4.208830 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request 4.208853 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request 4.246856 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply 4.246870 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply 5.209664 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request 5.209698 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request 5.260851 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply 5.260868 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply 6.210737 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request 6.210780 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request 6.259164 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply 6.259182 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply 7.212169 port2 in 10.1.1.1 -> 10.2.2.2: icmp: echo request 7.212212 toCisco out 10.1.1.1 -> 10.2.2.2: icmp: echo request 7.257794 toCisco in 10.2.2.2 -> 10.1.1.1: icmp: echo reply 7.257811 port2 out 10.2.2.2 -> 10.1.1.1: icmp: echo reply 20 packets received by filter 0 packets dropped by kernel ## IP-in-IP traffic (protocol 4) sent and received by the FGT FGT # diagnose sniffer packet any 'ip proto 4' 4 interfaces=[any] filters=[ip proto 4] 3.366282 port1 out 198.51.100.1 -> 192.0.2.2: ip-proto-4 84 3.417012 port1 in 192.0.2.2 -> 198.51.100.1: ip-proto-4 84 4.366945 port1 out 198.51.100.1 -> 192.0.2.2: ip-proto-4 84 4.436003 port1 in 192.0.2.2 -> 198.51.100.1: ip-proto-4 84 5.368053 port1 out 198.51.100.1 -> 192.0.2.2: ip-proto-4 84 5.413783 port1 in 192.0.2.2 -> 198.51.100.1: ip-proto-4 84 6.369808 port1 out 198.51.100.1 -> 192.0.2.2: ip-proto-4 84 6.423911 port1 in 192.0.2.2 -> 198.51.100.1: ip-proto-4 84 7.370908 port1 out 198.51.100.1 -> 192.0.2.2: ip-proto-4 84 7.410080 port1 in 192.0.2.2 -> 198.51.100.1: ip-proto-4 84 10 packets received by filter 0 packets dropped by kernel |
FG1 # diag debug flow filter clear
FG1 # diag debug flow show function-name enable show function name FG1 # diag debug flow show iprope enable show trace messages about iprope FG1 # diag debug flow filter proto 1 FG1 # diag debug flow filter addr 10.2.2.2 FG1 # diag debug flow show console enable show trace messages on console FG1 # diag debug flow trace start 1000 FG1 # diag debug enable ## ICMP echo-request from PC1 to PC2 id=20085 trace_id=3 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.1.1.1:135->10.2.2.2:2048) from port2. type=8, code=0, id=135, seq=1." id=20085 trace_id=3 func=init_ip_session_common line=4944 msg="allocate a new session-00000071" id=20085 trace_id=3 func=iprope_dnat_check line=4659 msg="in-[port2], out-[]" id=20085 trace_id=3 func=iprope_dnat_check line=4672 msg="result: skb_flags-02000000, vid-0, ret-no-match, act-accept, flag-00000000" id=20085 trace_id=3 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.2.2.2 via toCisco" id=20085 trace_id=3 func=iprope_fwd_check line=636 msg="in-[port2], out-[toCisco], skb_flags-02000000, vid-0" id=20085 trace_id=3 func=__iprope_check line=2049 msg="gnum-100004, check-ffffffffa001e70e" id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-100004 policy-1, ret-matched, act-accept" id=20085 trace_id=3 func=__iprope_user_identity_check line=1648 msg="ret-matched" id=20085 trace_id=3 func=__iprope_check line=2049 msg="gnum-4e20, check-ffffffffa001e70e" id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept" id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept" id=20085 trace_id=3 func=__iprope_check_one_policy line=1823 msg="checked gnum-4e20 policy-6, ret-no-match, act-accept" id=20085 trace_id=3 func=__iprope_check line=2068 msg="gnum-4e20 check result: ret-no-match, act-accept, flag-00000000, flag2-00000000" id=20085 trace_id=3 func=__iprope_check_one_policy line=2020 msg="policy-1 is matched, act-accept" id=20085 trace_id=3 func=__iprope_check line=2068 msg="gnum-100004 check result: ret-matched, act-accept, flag-08010000, flag2-00004000" id=20085 trace_id=3 func=iprope_fwd_auth_check line=688 msg="after iprope_captive_check(): is_captive-0, ret-matched, act-accept, idx-1" id=20085 trace_id=3 func=fw_forward_handler line=697 msg="Allowed by Policy-1:" ## ICMP echo-reply from PC2 to PC1 id=20085 trace_id=4 func=print_pkt_detail line=4793 msg="vd-root received a packet(proto=1, 10.2.2.2:135->10.1.1.1:0) from toCisco. type=0, code=0, id=135, seq=1." id=20085 trace_id=4 func=resolve_ip_tuple_fast line=4857 msg="Find an existing session, id-00000071, reply direction" id=20085 trace_id=4 func=vf_ip_route_input_common line=2586 msg="find a route: flag=04000000 gw-10.1.1.1 via port2" |
FG1 # diag sys session filter clear
FG1 # diag sys session filter dst 10.2.2.2 FG1 # diag sys session filter proto 1 FG1 # diag sys session list session info: proto=1 proto_state=00 duration=5 expire=57 timeout=0 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=3 origin-shaper= reply-shaper= per_ip_shaper= ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255 state=may_dirty statistic(bytes/packets/allow_err): org=84/1/1 reply=84/1/1 tuples=2 tx speed(Bps/kbps): 14/0 rx speed(Bps/kbps): 14/0 orgin->sink: org pre->post, reply pre->post dev=4->15/15->4 gwy=10.2.2.2/10.1.1.1 hook=pre dir=org act=noop 10.1.1.1:140->10.2.2.2:8(0.0.0.0:0) hook=post dir=reply act=noop 10.2.2.2:140->10.1.1.1:0(0.0.0.0:0) misc=0 policy_id=1 auth_info=0 chk_client_info=0 vd=0 serial=00000071 tos=ff/ff app_list=0 app=0 url_cat=0 dd_type=0 dd_mode=0 total session 1 |
## The IP-in-IP packet Ethernet II, Src: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02), Dst: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01) Destination: MS-NLB-PhysServer-09_69:5c:04:01 (02:09:69:5c:04:01) Source: MS-NLB-PhysServer-09_69:5c:04:02 (02:09:69:5c:04:02) Type: IPv4 (0x0800) Internet Protocol Version 4, Src: 198.51.100.1, Dst: 192.0.2.2 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) Total Length: 104 Identification: 0x5c36 (23606) Flags: 0x02 (Don't Fragment) Fragment offset: 0 Time to live: 64 Protocol: IPIP (4) Header checksum: 0xf224 [correct] Source: 198.51.100.1 Destination: 192.0.2.2 ## The original IP packet carried inside the IP-in-IP packet Internet Protocol Version 4, Src: 10.1.1.1, Dst: 10.2.2.2 0100 .... = Version: 4 .... 0101 = Header Length: 20 bytes (5) Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT) Total Length: 84 Identification: 0xb7c9 (47049) Flags: 0x02 (Don't Fragment) Fragment offset: 0 Time to live: 63 Protocol: ICMP (1) Header checksum: 0x6cda [correct] Source: 10.1.1.1 Destination: 10.2.2.2 Internet Control Message Protocol Type: 8 (Echo (ping) request) Code: 0 Checksum: 0x1beb [correct] Identifier (BE): 147 (0x0093) Identifier (LE): 37632 (0x9300) Sequence number (BE): 1 (0x0001) Sequence number (LE): 256 (0x0100) Data (56 bytes) |
Related Articles
Technical Note: Configuring and verifying an IP in IP over IPsec tunnel
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2023 Fortinet, Inc. All Rights Reserved.