Technical Tip: Static NAT VIP accessible from 2 external interfaces with E-BGP peerings (dual-homing)

[cgustave]

Description

This article provides an example of architecture to allow the use of a static NAT VIP with an external IP address reachable through multiple interfaces.
In this example, BGP is used to provide a link redundancy (2 EBGP peerings to the same AS), however the same architecture can be applied with peerings on different remote AS.

The design is based on 2 nat-route mode "virtual domains" (vdom) logically interconnected via "inter-vdom links".
The static nat VIP is located on vdom 'root', applied on the inter-vdom link interface.
Vdom 'routing' goal is to manage the dual BGP peering over the 2 distinguished physical interfaces. Vdom 'routing' is configured with option 'asymetric routing' enabled to allow ingress and egress traffic of the same sessions to flow through the 2 interfaces.
One BGP peering is preferred with the AS prepending technique.

This configuration example also provides examples of:

• originate a default route from BGP
• filter out announced subnets
• discriminate one link with the use of 'AS prepending'
• use inter-vdom links
• configure asymmetric routing on a vdom
• configure inter-vdom links and interfaces

Scope

FortiGate.

Solution

Expectations, Requirements:

• Make static VIP external IP address (2.2.2.2) reachable even in case of an uplink failure (external or port1 interfaces of FGT3600A-5).
• Allow internal server to generate traffic natted with static VIP external IP address even in a case of one of the uplink failure.

Configuration:

FGT3600-5:

config vdom
edit root
end

config vdom
edit routing
end

config global
config system global
    set management-vdom "root"
    set vdom-admin enable
end
config system vdom-link
    edit "vdl"
    next
end
config system interface
    edit "port1"
        set vdom "routing"
        set ip 10.10.10.5 255.255.255.252
        set allowaccess ping https ssh http telnet
        set type physical
    next
    edit "internal"
        set vdom "root"
        set ip 10.10.1.2 255.255.255.0
        set allowaccess ping https ssh http telnet
        set type physical
    next
    edit "external"
        set vdom "routing"
        set ip 10.10.10.9 255.255.255.252
        set allowaccess ping https ssh http telnet
        set type physical
    next
    edit "vdl0"
        set vdom "root"
        set ip 10.10.10.2 255.255.255.252
        set allowaccess ping https ssh
        set type vdom-link
    next
    edit "vdl1"
        set vdom "routing"
        set ip 10.10.10.1 255.255.255.252
        set allowaccess ping https ssh
        set type vdom-link
    next
end
config vdom
edit root
config firewall address
    edit "all"
    next
    edit "server-private"
        set subnet 10.10.1.1 255.255.255.255
    next
end
config firewall vip
    edit "server-vip"
        set extip 2.2.2.2
        set extintf "vdl0"
        set mappedip 10.10.1.1
    next
end
config firewall policy
    edit 1
        set srcintf "internal"
        set dstintf "vdl0"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
        set nat enable
    next
    edit 2
        set srcintf "vdl0"
        set dstintf "internal"
            set srcaddr "all"
            set dstaddr "server-vip"
        set action accept
        set schedule "always"
            set service "ANY"
    next
end
config router static
    edit 1
        set device "vdl0"
        set gateway 10.10.10.2
        set weight 50
    next
end
end

config vdom
edit routing
config system settings
    set asymroute enable
end
config firewall address
    edit "all"
    next
end
config firewall policy
    edit 1
        set srcintf "vdl1"
        set dstintf "external"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 2
        set srcintf "vdl1"
        set dstintf "port1"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 3
        set srcintf "external"
        set dstintf "vdl1"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 4
        set srcintf "port1"
        set dstintf "vdl1"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
end
config router access-list
    edit "vip-subnet"
            config rule
                edit 1
                    set prefix 2.2.2.0 255.255.255.0
                    set exact-match enable
                next
            end
    next
end
config router route-map
    edit "EXPORT-EXTERNAL"
            config rule
                edit 1
                    set match-ip-address "vip-subnet"
                next
            end
    next
    edit "EXPORT-PORT1"
            config rule
                edit 1
                    set match-ip-address "vip-subnet"
                        set set-aspath "100"
                next
            end
    next
end
config router static
    edit 4
        set device "vdl1"
        set dst 2.2.2.0 255.255.255.0
        set gateway 10.10.10.2
        set weight 50
    next
end
config router bgp
    set as 100
        config neighbor
            edit "10.10.10.10"
                set remote-as 200
                set route-map-out "EXPORT-EXTERNAL"
                set send-community6 disable
            next
            edit "10.10.10.6"
                set remote-as 200
                set route-map-out "EXPORT-PORT1"
                set send-community6 disable
            next
        end
        config network
            edit 1
                set prefix 2.2.2.0 255.255.255.0
            next
        end
        config redistribute "connected"
        end
        config redistribute "rip"
        end
        config redistribute "ospf"
        end
        config redistribute "static"
            set status enable
        end
        config redistribute6 "connected"
        end
        config redistribute6 "rip"
        end
        config redistribute6 "ospf"
        end
        config redistribute6 "static"
        end
    set router-id 154.32.148.245
end 

FGT400A-2:

config firewall address
    edit "all"
    next
config firewall policy
    edit 1
        set srcintf "port1"
        set dstintf "port5"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 2
        set srcintf "port2"
        set dstintf "port5"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 3
        set srcintf "port5"
        set dstintf "port1"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
    edit 4
        set srcintf "port5"
        set dstintf "port2"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "ANY"
    next
end
config router prefix-list
    edit "only_dflt"
            config rule
                edit 1
                    set prefix 0.0.0.0 0.0.0.0
                    unset ge
                    unset le
                next
            end
    next
end
config router route-map
    edit "only_default_route"
            config rule
                edit 1
                    set match-ip-address "only_dflt"
                next
            end
    next
    edit "EXPORT-PORT1"
            config rule
                edit 1
                    set match-ip-address "only_dflt"
                next
            end
    next
    edit "EXPORT-PORT2"
            config rule
                edit 1
                    set match-ip-address "only_dflt"
                        set set-aspath "200"
                next
            end
    next
end
config router bgp
    set as 200
        config neighbor
            edit "10.10.10.9"
                set capability-default-originate enable
                set remote-as 100
                set route-map-out "EXPORT-PORT1"
            next
            edit "10.10.10.5"
                set capability-default-originate enable
                set remote-as 100
                set route-map-out "EXPORT-PORT2"
            next
        end
        config network
            edit 1
                set prefix 1.1.1.0 255.255.255.0
            next
        end
        config redistribute "connected"
        end
        config redistribute "rip"
        end
        config redistribute "ospf"
        end
        config redistribute "static"
        end
        config redistribute6 "connected"
        end
        config redistribute6 "rip"
        end
        config redistribute6 "ospf"
        end
        config redistribute6 "static"
        end
    set router-id 1.1.1.2
end

Verification:

1. Both links are up (normal situation).
   1. BGP Status.

FG3600-5 (routing) # get router info bgp summary
BGP router identifier 154.32.148.245, local AS number 100
BGP table version is 1
4 BGP AS-PATH entries
0 BGP community entries

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.10.6      4   200       5       5        1    0    0 00:02:12        1
10.10.10.10     4   200       5       5        1    0    0 00:02:12        1




FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 advertised-routes
BGP table version is 1, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.0/24       10.10.10.5                         32768 100 ?

Total number of prefixes 1



FG3600-5 (routing) # get router info bgp neighbors 10.10.10.10 advertised-routes
BGP table version is 1, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.0/24       10.10.10.9                         32768 ?

Total number of prefixes 1



FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 route
BGP table version is 1, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*  0.0.0.0/0        10.10.10.6               0             0 200 200 i

Total number of prefixes 1


FG3600-5 (routing) # get router info bgp neighbors 10.10.10.10 route
BGP table version is 1, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        10.10.10.10              0             0 200 i

Total number of prefixes 1




FG3600-5 (routing) # get router info bgp quote-regexp ^
BGP table version is 1, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        10.10.10.10              0             0 200 i
*                   10.10.10.6               0             0 200 200 i
*  2.2.2.0/24       0.0.0.0                       100  32768 i
*>                  10.10.10.2                         32768 ?

Total number of prefixes 2

2. Routing table.

FG3600-5 (routing) # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

B*      0.0.0.0/0 [20/0] via 10.10.10.10, external, 00:05:03
S       2.2.2.0/24 [10/0] via 10.10.10.2, vdl1, [0/50]
C       10.10.10.0/30 is directly connected, vdl1
C       10.10.10.1/32 is directly connected, vdl1
C       10.10.10.4/30 is directly connected, port1
C       10.10.10.8/30 is directly connected, external



FGT400A-2 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

C       1.1.1.0/24 is directly connected, port5
B       2.2.2.0/24 [20/0] via 10.10.10.9, port1, 00:11:41
C       10.10.10.4/30 is directly connected, port2
C       10.10.10.8/30 is directly connected, port1

3. Sniffer: client pings VIP to reach server (2.2.2.2).

FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4
interfaces=[any]
filters=[icmp]
1.368865 external in 1.1.1.1 -> 2.2.2.2: icmp: echo request
1.368865 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo request
1.368890 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo request
1.369132 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo reply
1.369132 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo reply
1.369144 external out 2.2.2.2 -> 1.1.1.1: icmp: echo reply

4. Sniffer: server pings client (1.1.1.1).

FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4
interfaces=[any]
filters=[icmp]
1.456079 i.nternal in 10.10.1.1 -> 1.1.1.1: icmp: echo request
1.456079 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo request
1.456109 external out 2.2.2.2 -> 1.1.1.1: icmp: echo request
1.456808 external in 1.1.1.1 -> 2.2.2.2: icmp: echo reply
1.456808 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply
1.456820 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo reply

2. Primary link goes down (failure situation).
   1. BGP Status.

FG3600-5 (routing) # get router info bgp summary
BGP router identifier 154.32.148.245, local AS number 100
BGP table version is 2
3 BGP AS-PATH entries
0 BGP community entries

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.10.10.6      4   200      23      23        1    0    0 00:17:57        1
10.10.10.10     4   200      19      24        0    0    0    never Active


FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 advertised-routes
BGP table version is 2, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internales
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.0/24       10.10.10.5                         32768 100 ?

Total number of prefixes 1


FG3600-5 (routing) #  get router info bgp quote-regexp ^
BGP table version is 2, local router ID is 154.32.148.245
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop            Metric LocPrf Weight Path
*> 0.0.0.0/0        10.10.10.6               0             0 200 200 i
*  2.2.2.0/24       0.0.0.0                       100  32768 i
*>                  10.10.10.2                         32768 ?

Total number of prefixes 2

2. Routing Table.

FG3600-5 (routing) #  get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

B*      0.0.0.0/0 [20/0] via 10.10.10.6, port1, 00:02:43
S       2.2.2.0/24 [10/0] via 10.10.10.2, vdl1, [0/50]
C       10.10.10.0/30 is directly connected, vdl1
C       10.10.10.1/32 is directly connected, vdl1
C       10.10.10.4/30 is directly connected, port1


FGT400A-2 #  get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

C       1.1.1.0/24 is directly connected, port5
B       2.2.2.0/24 [20/0] via 10.10.10.5, port2, 00:02:51
C       10.10.10.4/30 is directly connected, port2
C       10.10.10.8/30 is directly connected, port1

3. Sniffer: client pings VIP to reach server (2.2.2.2).

FG3600-5 (routing) #  diagnose sniffer packet any 'icmp' 4
interfaces=[any]
filters=[icmp]
1.525509 port1 in 1.1.1.1 -> 2.2.2.2: icmp: echo request
1.525509 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo request
1.525537 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo request
1.525758 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo reply
1.525758 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo reply
1.525771 port1 out 2.2.2.2 -> 1.1.1.1: icmp: echo reply

4. Sniffer: server pings client (1.1.1.1).

FG3600-5 (routing) #  diagnose sniffer packet any 'icmp' 4
interfaces=[any]
filters=[icmp]
1.082323 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo request
1.082323 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo request
1.082363 port1 out 2.2.2.2 -> 1.1.1.1: icmp: echo request
1.082984 port1 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply
1.082984 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply
1.082998 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo reply

Troubleshooting:

Further troubleshooting can be done using the 'diag debug flow' commands.

Related articles:

• Technical Tip: Configuring BGP on a FortiGate with single-homed eBGP peering, iBGP peering, access-l...
• Technical Tip: FortiGate BGP configuration to announce specific routes and accept only a default rou...
• Technical Tip: BGP dual-home example - solution to avoid RPF problem in asymetric routing scenario
• Technical Tip: How to implement BGP route summary (aggregation) on a FortiGate