Purpose
This technical note provides an example of architecture to allow the use of a static NAT VIP with an external IP address reachable through multiple interfaces.
In this example, BGP is used to provide a link redundancy (2 EBGP peerings to the same AS), however the same architecture can be applied with peerings on different remote AS.
The design is based on 2 nat-route mode "virtual domains" (vdom) logically interconnected via "inter-vdom links".
The static nat VIP is located on vdom 'root', applied on the inter-vdom link interface.
Vdom 'routing' goal is to manage the dual BGP peering over the 2 distinguished physical interfaces. Vdom 'routing' is configured with option 'asymetric routing' enabled to allow ingress and egress traffic of the same sessions to flow through the 2 interfaces.
One BGP peering is preferred with the AS prepending technique.
This configuration example also provide example to :
- originate a default route from BGP
- filter out announced subnets
- discriminate one link with the use of 'AS prepending'
- use inter-vdom links
- configure asymmetric routing on a vdom
- configure inter-vdom links and interfaces
Scope
Diagram
Expectations, Requirements
- Make static VIP external IP address (2.2.2.2) reachable even in case of an uplink failure (external or port1 interfaces of FGT3600A-5)
- Allow internal server to generate traffic natted with static VIP external IP address even in a case of one of the uplink failure
Configuration
FGT3600-5
config vdom edit root end config vdom edit routing end config global config system global set management-vdom "root" set vdom-admin enable end config system vdom-link edit "vdl" next end config system interface edit "port1" set vdom "routing" set ip 10.10.10.5 255.255.255.252 set allowaccess ping https ssh http telnet set type physical next edit "internal" set vdom "root" set ip 10.10.1.2 255.255.255.0 set allowaccess ping https ssh http telnet set type physical next edit "external" set vdom "routing" set ip 10.10.10.9 255.255.255.252 set allowaccess ping https ssh http telnet set type physical next edit "vdl0" set vdom "root" set ip 10.10.10.2 255.255.255.252 set allowaccess ping https ssh set type vdom-link next edit "vdl1" set vdom "routing" set ip 10.10.10.1 255.255.255.252 set allowaccess ping https ssh set type vdom-link next end config vdom edit root config firewall address edit "all" next edit "server-private" set subnet 10.10.1.1 255.255.255.255 next end config firewall vip edit "server-vip" set extip 2.2.2.2 set extintf "vdl0" set mappedip 10.10.1.1 next end config firewall policy edit 1 set srcintf "internal" set dstintf "vdl0" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" set nat enable next edit 2 set srcintf "vdl0" set dstintf "internal" set srcaddr "all" set dstaddr "server-vip" set action accept set schedule "always" set service "ANY" next end config router static edit 1 set device "vdl0" set gateway 10.10.10.2 set weight 50 next end end config vdom edit routing config system settings set asymroute enable end config firewall address edit "all" next end config firewall policy edit 1 set srcintf "vdl1" set dstintf "external" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 2 set srcintf "vdl1" set dstintf "port1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 3 set srcintf "external" set dstintf "vdl1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 4 set srcintf "port1" set dstintf "vdl1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next end config router access-list edit "vip-subnet" config rule edit 1 set prefix 2.2.2.0 255.255.255.0 set exact-match enable next end next end config router route-map edit "EXPORT-EXTERNAL" config rule edit 1 set match-ip-address "vip-subnet" next end next edit "EXPORT-PORT1" config rule edit 1 set match-ip-address "vip-subnet" set set-aspath "100" next end next end config router static edit 4 set device "vdl1" set dst 2.2.2.0 255.255.255.0 set gateway 10.10.10.2 set weight 50 next end config router bgp set as 100 config neighbor edit "10.10.10.10" set remote-as 200 set route-map-out "EXPORT-EXTERNAL" set send-community6 disable next edit "10.10.10.6" set remote-as 200 set route-map-out "EXPORT-PORT1" set send-community6 disable next end config network edit 1 set prefix 2.2.2.0 255.255.255.0 next end config redistribute "connected" end config redistribute "rip" end config redistribute "ospf" end config redistribute "static" set status enable end config redistribute6 "connected" end config redistribute6 "rip" end config redistribute6 "ospf" end config redistribute6 "static" end set router-id 154.32.148.245 end |
FGT400A-2
config firewall address edit "all" next config firewall policy edit 1 set srcintf "port1" set dstintf "port5" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 2 set srcintf "port2" set dstintf "port5" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 3 set srcintf "port5" set dstintf "port1" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next edit 4 set srcintf "port5" set dstintf "port2" set srcaddr "all" set dstaddr "all" set action accept set schedule "always" set service "ANY" next end config router prefix-list edit "only_dflt" config rule edit 1 set prefix 0.0.0.0 0.0.0.0 unset ge unset le next end next end config router route-map edit "only_default_route" config rule edit 1 set match-ip-address "only_dflt" next end next edit "EXPORT-PORT1" config rule edit 1 set match-ip-address "only_dflt" next end next edit "EXPORT-PORT2" config rule edit 1 set match-ip-address "only_dflt" set set-aspath "200" next end next end config router bgp set as 200 config neighbor edit "10.10.10.9" set capability-default-originate enable set remote-as 100 set route-map-out "EXPORT-PORT1" next edit "10.10.10.5" set capability-default-originate enable set remote-as 100 set route-map-out "EXPORT-PORT2" next end config network edit 1 set prefix 1.1.1.0 255.255.255.0 next end config redistribute "connected" end config redistribute "rip" end config redistribute "ospf" end config redistribute "static" end config redistribute6 "connected" end config redistribute6 "rip" end config redistribute6 "ospf" end config redistribute6 "static" end set router-id 1.1.1.2 end |
Verification
I. Both links are up (normal situation)
I.1 BGP Status
FG3600-5 (routing) # get router info bgp summary BGP router identifier 154.32.148.245, local AS number 100 BGP table version is 1 4 BGP AS-PATH entries 0 BGP community entries Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.10.10.6 4 200 5 5 1 0 0 00:02:12 1 10.10.10.10 4 200 5 5 1 0 0 00:02:12 1 FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 advertised-routes BGP table version is 1, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 2.2.2.0/24 10.10.10.5 32768 100 ? Total number of prefixes 1 FG3600-5 (routing) # get router info bgp neighbors 10.10.10.10 advertised-routes BGP table version is 1, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 2.2.2.0/24 10.10.10.9 32768 ? Total number of prefixes 1 FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 route BGP table version is 1, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path * 0.0.0.0/0 10.10.10.6 0 0 200 200 i Total number of prefixes 1 FG3600-5 (routing) # get router info bgp neighbors 10.10.10.10 route BGP table version is 1, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 0.0.0.0/0 10.10.10.10 0 0 200 i Total number of prefixes 1 FG3600-5 (routing) # get router info bgp quote-regexp ^ BGP table version is 1, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 0.0.0.0/0 10.10.10.10 0 0 200 i * 10.10.10.6 0 0 200 200 i * 2.2.2.0/24 0.0.0.0 100 32768 i *> 10.10.10.2 32768 ? Total number of prefixes 2 |
I.2 Routing table
FG3600-5 (routing) # get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default B* 0.0.0.0/0 [20/0] via 10.10.10.10, external, 00:05:03 S 2.2.2.0/24 [10/0] via 10.10.10.2, vdl1, [0/50] C 10.10.10.0/30 is directly connected, vdl1 C 10.10.10.1/32 is directly connected, vdl1 C 10.10.10.4/30 is directly connected, port1 C 10.10.10.8/30 is directly connected, external FGT400A-2 # get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default C 1.1.1.0/24 is directly connected, port5 B 2.2.2.0/24 [20/0] via 10.10.10.9, port1, 00:11:41 C 10.10.10.4/30 is directly connected, port2 C 10.10.10.8/30 is directly connected, port1 |
I.3 Sniffer : client pings VIP to reach server (2.2.2.2)
FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4 interfaces=[any] filters=[icmp] 1.368865 external in 1.1.1.1 -> 2.2.2.2: icmp: echo request 1.368865 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo request 1.368890 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo request 1.369132 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo reply 1.369132 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo reply 1.369144 external out 2.2.2.2 -> 1.1.1.1: icmp: echo reply |
I.4 Sniffer : server pings client (1.1.1.1)
FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4 interfaces=[any] filters=[icmp] 1.456079 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo request 1.456079 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo request 1.456109 external out 2.2.2.2 -> 1.1.1.1: icmp: echo request 1.456808 external in 1.1.1.1 -> 2.2.2.2: icmp: echo reply 1.456808 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply 1.456820 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo reply |
II. Primary link goes down (failure situation)
II.1 BGP Status
FG3600-5 (routing) # get router info bgp summary BGP router identifier 154.32.148.245, local AS number 100 BGP table version is 2 3 BGP AS-PATH entries 0 BGP community entries Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 10.10.10.6 4 200 23 23 1 0 0 00:17:57 1 10.10.10.10 4 200 19 24 0 0 0 never Active FG3600-5 (routing) # get router info bgp neighbors 10.10.10.6 advertised-routes BGP table version is 2, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internales Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 2.2.2.0/24 10.10.10.5 32768 100 ? Total number of prefixes 1 FG3600-5 (routing) # get router info bgp quote-regexp ^ BGP table version is 2, local router ID is 154.32.148.245 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, S Stale Origin codes: i - IGP, e - EGP, ? - incomplete Network Next Hop Metric LocPrf Weight Path *> 0.0.0.0/0 10.10.10.6 0 0 200 200 i * 2.2.2.0/24 0.0.0.0 100 32768 i *> 10.10.10.2 32768 ? Total number of prefixes 2 |
II.2 Routing Table
FG3600-5 (routing) # get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default B* 0.0.0.0/0 [20/0] via 10.10.10.6, port1, 00:02:43 S 2.2.2.0/24 [10/0] via 10.10.10.2, vdl1, [0/50] C 10.10.10.0/30 is directly connected, vdl1 C 10.10.10.1/32 is directly connected, vdl1 C 10.10.10.4/30 is directly connected, port1 FGT400A-2 # get router info routing-table all Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area * - candidate default C 1.1.1.0/24 is directly connected, port5 B 2.2.2.0/24 [20/0] via 10.10.10.5, port2, 00:02:51 C 10.10.10.4/30 is directly connected, port2 C 10.10.10.8/30 is directly connected, port1 |
II.3 Sniffer : client pings VIP to reach server (2.2.2.2)
FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4 interfaces=[any] filters=[icmp] 1.525509 port1 in 1.1.1.1 -> 2.2.2.2: icmp: echo request 1.525509 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo request 1.525537 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo request 1.525758 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo reply 1.525758 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo reply 1.525771 port1 out 2.2.2.2 -> 1.1.1.1: icmp: echo reply |
II.4 Sniffer : server pings client (1.1.1.1)
FG3600-5 (routing) # diagnose sniffer packet any 'icmp' 4 interfaces=[any] filters=[icmp] 1.082323 internal in 10.10.1.1 -> 1.1.1.1: icmp: echo request 1.082323 vdl1 in 2.2.2.2 -> 1.1.1.1: icmp: echo request 1.082363 port1 out 2.2.2.2 -> 1.1.1.1: icmp: echo request 1.082984 port1 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply 1.082984 vdl0 in 1.1.1.1 -> 2.2.2.2: icmp: echo reply 1.082998 internal out 1.1.1.1 -> 10.10.1.1: icmp: echo reply |
Troubleshooting
Further troubleshooting can be done using the 'diag debug flow' commands
Related Articles
Technical Note : BGP dual-home example - solution to avoid RPF problem in asymetric routing scenario
Technical Note : How to implement BGP route summary (aggregation) on a FortiGate
