Technical Note: Detecting a link failure using Dead Gateway Detection (ping server) to ensure a link fail-over

Purpose
This article describes how to setup a link fail-over mechanism on a FortiGate unit.

Scope
A FortiGate unit or VDOM running in NAT mode.

Diagram
The following setup illustrates this example:

[                ] --- port1---- ISP1
[  Fortigate     ]
[                ] --- port2---- ISP2

In this scenario both interfaces are used, and egress traffic is shared across port1 and port2 with ECMP; see both Default Routes in the routing table:

get router info routing-table all

S*      0.0.0.0/0 [10/0] via 192.168.183.254, port1
                  [10/0] via 10.160.0.1, port2
C       10.160.0.0/23 is directly connected, port2
S       192.168.0.0/16 [10/0] via 192.168.183.254, port1
C       192.168.182.0/23 is directly connected, port1

Expectations, Requirements
A problem of blackholing egress traffic on a port can arise if there is a routing problem with one of the ISP device, for example the next-hop router that becomes unavailable.

To avoid such a situation, and ensure traffic continuity over the remaining link, the FortiGate unit can detect the failure of a critical network element behind an interface. This is called Dead Gateway Detection used by configuring a ping server.

The Dead Gateway Detection feature will send pings to the ping server at a configured regular interval. The advantage of using ping, is that this is an IP routable packet. The ping server can be located one a remote subnet.

When a ping server fails, the FortiGate unit will withdraw from the routing table all static routes attached to the interface for which the ping-server does not respond.

Note: An important recommendation is to use a reliable network element as Ping server to avoid route flapping.

The interface itself remains up and operational for local traffic. The pings will continue to be sent to check the recovery of the ping server.

When a ping server failure is detected, the FortiGate unit send a log a message to the local (or remote) log and a syslog message similar to the example below. The ping server loss is effectively not part of the SNMP traps that can be configured.

The following is an example of  a syslog message:

date=2009-01-26 time=05:44:07 devname=FGT60B3907500059 device_id=FGT60B3907500059 log_id=0100020001 type=event subtype=system pri=critical vd=root interface="internal" ip=10.160.0.78 status=down msg="Ping peer: 10.160.0.2 is down"

An example of a log message :

1     2009-01-06     16:47:58     critical                 Ping peer: 172.16.1.1 is up
2     2009-01-06     16:46:58     critical                 Ping peer: 172.16.1.1 is down
3     2009-01-06     16:44:38     critical                 Ping peer: 10.160.0.160 is up
4     2009-01-06     16:44:29     critical                 Ping peer: 10.160.0.160 is down

Upon ping server failure, the ping polling will continue on the same interface. The FortiGate unit does not use the routing table to reach a ping server on a remote subnet through another interface. This occurs because a route is programmed in the kernel for the ping server on this interface (see example further below).

Configuration
To configure a ping server, go to System > Network and select Edit for an interface.

To configure from the CLI, enter the following commands:

config system interface
    edit "port1"
        set vdom "root"
        set ip 192.168.182.110 255.255.254.0
        set allowaccess ping https ssh http telnet
        set gwdetect enable
        set detectserver "192.168.183.254"
        set type physical
    next
end

A ping server can also be located on a remote subnet:

config system interface
    edit "port2"
        set vdom "root"
        set ip 10.160.0.110 255.255.254.0
        set allowaccess ping https ssh http telnet
        set gwdetect enable
        set detectserver "172.16.1.1"
        set type physical
    next
end

In this case, a specific route is configured over this interface, which can be seen with the following command :

FGT# diagnose ip route list

You can also configure the ping frequency in the web-based manager by going to System > Network > Options.

Verification
To check the status of a ping server, use the following command:

Fortigate # config system interface
Fortigate (interface) # edit port2
Fortigate (port2) # get

Troubleshooting
To see if Dead Gateway Detection is active and sends ping, use the sniffer command.

Example, assuming the ping server is 172.16.1.1 configured on port 2:

diagnose sniffer packet any "host 172.16.1.1" 4

interfaces=[any]
filters=[host 172.16.1.1]
0.970088 port2 out 10.160.0.110 -> 172.16.1.1: icmp: echo request
1.970081 port2 out 10.160.0.110 -> 172.16.1.1: icmp: echo request
2.970082 port2 out 10.160.0.110 -> 172.16.1.1: icmp: echo request
[...]

Related Articles

Technical Note: Configuring link redundancy - Traffic load-balancing / load-sharing - ECMP (Equal Cost Multiple Path) - Dual Internet or WAN scenario