Description
This article describes how to monitor the state of GRE tunnels.
Scope
Any supported version of FortiGate.
Solution
The GRE tunnel interface is a virtual interface that will always have the 'up' status, even if the other end is unreachable.
Even if the dead gateway detection is defined for this interface, it will send the traffic to the tunnel but the interface status will always be shown as 'up'.
Gateway detection only deletes the static routes (and leaves the interface up).
The routes are directly connected so it will not be deleted in the case of a GRE tunnel.
The only way to mark the GRE tunnel as 'down' is to administratively set the tunnel interface as down.
To monitor GRE tunnel states, there is no option available in Log & Report from the GUI.
A link-monitor can be configured to monitor the GRE tunnel interface via the following command:
config system link-monitor
edit "1"
set srcintf <GRE-Tunnel-Name>
set server <GRE-Remote-IP>
next
end
In the event of a GRE tunnel failure, the GRE tunnel states can be monitored in the System Events as shown in screenshot below.
The status can also be checked from the CLI with the following command:
diagnose sys link-monitor status
To review the GRE tunnel information from the CLI, the following commands can be used:
diagnose sys gre list
diagnose netlink interface list | grep <GRE-Tunnel-Name>
get system interface | grep <GRE-Tunnel-Name>
diagnose netlink interface list | grep <GRE-Tunnel-Name>
get system interface | grep <GRE-Tunnel-Name>
Example deployment:
Configuration related to the GRE tunnel and link monitor:
Viva-kvm28:
config system gre-tunnel
edit "TEST"
set interface "port1"
set remote-gw 10.47.3.224
set local-gw 10.47.1.180
next
edit "TEST2"
set interface "port1"
set remote-gw 10.47.3.194
set local-gw 10.47.1.180
next
end
config system interface
edit "TEST"
set vdom "VDOM_A"
set ip 10.10.10.1 255.255.255.255
set allowaccess ping
set type tunnel
set remote-ip 10.10.10.2 255.255.255.252
set interface "port1"
next
edit "TEST2"
set vdom "VDOM_A"
set ip 10.10.10.5 255.255.255.255
set allowaccess ping
set type tunnel
set remote-ip 10.10.10.6 255.255.255.252
set interface "port1"
next
end
config system link-monitor
edit "TEST"
set srcintf "TEST"
set server "192.168.10.1"
next
edit "TEST2"
set srcintf "TEST2"
set server "192.168.10.1"
next
end
config router static
edit 1
set gateway 10.47.15.254
set device "port1"
next
edit 2
set dst 192.168.10.0 255.255.255.0
set distance 1
set device "TEST"
next
edit 3
set dst 192.168.10.0 255.255.255.0
set priority 100
set device "TEST2"
next
edit 4
set status disable
set dst 10.47.3.224 255.255.255.255
set distance 1
set blackhole enable
next
end
DC_1:
config system gre-tunnel
edit "TEST"
set interface "port1"
set remote-gw 10.47.1.180
set local-gw 10.47.3.224
next
end
config system interface
edit "TEST" <---
set vdom "root"
set ip 10.10.10.2 255.255.255.255
set allowaccess ping
set type tunnel
set remote-ip 10.10.10.1 255.255.255.252
set snmp-index 15
set interface "port1"
next
end
DC_2:
config system gre-tunnel
edit "TEST"
set interface "port1"
set remote-gw 10.47.1.180
set local-gw 10.47.3.194
next
end
config system interface
edit "TEST" <---
set vdom "root"
set ip 10.10.10.6 255.255.255.255
set allowaccess ping
set type tunnel
set remote-ip 10.10.10.5 255.255.255.252
set snmp-index 12
set interface "port1"
next
end
Result:
- Blackhole disable (primary is up): The static route of the primary (TEST) is active on the routing table.
diag sys link-monitor status
Link Monitor: TEST, Status: alive, Server num(1), HA state: local(alive), shared(alive)
Flags=0x1 init, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST (27)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.1)
Route: 10.10.10.1->192.168.10.1/32, gwy(10.10.10.2)
protocol: ping, state: alive
Latency(Min/Max/Avg): 0.207/0.391/0.268 ms
Jitter(Min/Max/Avg): 0.002/0.126/0.045 ms
Packet lost: 0.000%
Number of out-of-sequence packets: 0
Fail Times(0/5)
Packet sent: 371, received: 371, Sequence(sent/rcvd/exp): 372/372/373
Link Monitor: TEST2, Status: alive, Server num(1), HA state: local(alive), shared(alive)
Flags=0x1 init, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST2 (28)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.5)
Route: 10.10.10.5->192.168.10.1/32, gwy(10.10.10.6)
protocol: ping, state: alive
Latency(Min/Max/Avg): 0.170/0.321/0.230 ms
Jitter(Min/Max/Avg): 0.000/0.091/0.041 ms
Packet lost: 0.000%
Number of out-of-sequence packets: 0
Fail Times(0/5)
Packet sent: 371, received: 371, Sequence(sent/rcvd/exp): 372/372/373
get router info routing-table database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Routing table for VRF=0
S *> 0.0.0.0/0 [10/0] via 10.47.15.254, port1, [1/0]
C *> 10.10.10.0/30 is directly connected, TEST
C *> 10.10.10.1/32 is directly connected, TEST
C *> 10.10.10.4/30 is directly connected, TEST2
C *> 10.10.10.5/32 is directly connected, TEST2
C *> 10.47.0.0/20 is directly connected, port1
S 192.168.10.0/24 [10/0] via 10.10.10.6, TEST2, [100/0]
S *> 192.168.10.0/24 [1/0] via 10.10.10.2, TEST, [1/0]
- Blackhole enabled (Primary tunnel down): The static route for TEST (primary) went inactive on the routing table.
- The secondary static route kicks in.
diag sys link-monitor status
Link Monitor: TEST, Status: dead, Server num(1), HA state: local(dead), shared(dead)
Flags=0x9 init log_downgateway, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST (27)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.1)
Route: 10.10.10.1->192.168.10.1/32, gwy(10.10.10.2)
protocol: ping, state: dead
Packet lost: 26.000%
Number of out-of-sequence packets: 0
Recovery times(0/5) Fail Times(2/5)
Packet sent: 513, received: 485, Sequence(sent/rcvd/exp): 514/486/487
Link Monitor: TEST2, Status: alive, Server num(1), HA state: local(alive), shared(alive)
Flags=0x1 init, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST2 (28)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.5)
Route: 10.10.10.5->192.168.10.1/32, gwy(10.10.10.6)
protocol: ping, state: alive
Latency(Min/Max/Avg): 0.192/0.334/0.258 ms
Jitter(Min/Max/Avg): 0.000/0.109/0.041 ms
Packet lost: 0.000%
Number of out-of-sequence packets: 0
Fail Times(0/5)
Packet sent: 513, received: 513, Sequence(sent/rcvd/exp): 514/514/515
get router info routing-table database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Routing table for VRF=0
S *> 0.0.0.0/0 [10/0] via 10.47.15.254, port1, [1/0]
C *> 10.10.10.0/30 is directly connected, TEST
C *> 10.10.10.1/32 is directly connected, TEST
C *> 10.10.10.4/30 is directly connected, TEST2
C *> 10.10.10.5/32 is directly connected, TEST2
C *> 10.47.0.0/20 is directly connected, port1
S *> 10.47.3.224/32 [1/0] is a summary, Null, [1/0] <--- black hole enabled
S *> 192.168.10.0/24 [10/0] via 10.10.10.6, TEST2, [100/0] <--- Secondary tunnel static route
S 192.168.10.0/24 [1/0] via 10.10.10.2, TEST inactive, [1/0] <--- Inactive due to link-monitor
- Disable the blackhole again to turn up the primary tunnel.
diag sys link-monitor status
Link Monitor: TEST, Status: alive, Server num(1), HA state: local(alive), shared(alive)
Flags=0x1 init, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST (27)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.1)
Route: 10.10.10.1->192.168.10.1/32, gwy(10.10.10.2)
protocol: ping, state: alive
Latency(Min/Max/Avg): 0.219/0.543/0.279 ms
Jitter(Min/Max/Avg): 0.004/0.296/0.050 ms
Packet lost: 0.000%
Number of out-of-sequence packets: 0
Fail Times(0/5)
Packet sent: 891, received: 593, Sequence(sent/rcvd/exp): 892/892/893
Link Monitor: TEST2, Status: alive, Server num(1), HA state: local(alive), shared(alive)
Flags=0x1 init, Create time: Fri Oct 28 14:07:24 2022
Source interface: TEST2 (28)
Interval: 500 ms
Service-detect: disable
Diffservcode: 000000
Class-ID: 0
Peer: 192.168.10.1(192.168.10.1)
Source IP(10.10.10.5)
Route: 10.10.10.5->192.168.10.1/32, gwy(10.10.10.6)
protocol: ping, state: alive
Latency(Min/Max/Avg): 0.158/0.479/0.212 ms
Jitter(Min/Max/Avg): 0.001/0.297/0.044 ms
Packet lost: 0.000%
Number of out-of-sequence packets: 0
Fail Times(0/5)
Packet sent: 891, received: 891, Sequence(sent/rcvd/exp): 892/892/893
get router info routing-table database
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
> - selected route, * - FIB route, p - stale info
Routing table for VRF=0
S *> 0.0.0.0/0 [10/0] via 10.47.15.254, port1, [1/0]
C *> 10.10.10.0/30 is directly connected, TEST
C *> 10.10.10.1/32 is directly connected, TEST
C *> 10.10.10.4/30 is directly connected, TEST2
C *> 10.10.10.5/32 is directly connected, TEST2
C *> 10.47.0.0/20 is directly connected, port1
S 192.168.10.0/24 [10/0] via 10.10.10.6, TEST2, [100/0]
S *> 192.168.10.0/24 [1/0] via 10.10.10.2, TEST, [1/0]
Related article:
