BGP on Fortigate

[laf]

We have a cluster of two 110C running Virtual Clustering A-P. We run BGP protocol on one VDOM called BGP. We use this VDOM only for routing while the other VDOM root is used for firewall and content inspection. We run eBGP with two ISP: 1. name: DialTelecom mode: static ip: 188.209.a.b 255.255.255.252 status: up netbios-forward: disable type: vlan mtu-override: disable wccp: disable sflow-sampler: disable explicit-web-proxy: disable 2. name: Euroweb mode: static ip: 89.238.m.n 255.255.255.252 status: up netbios-forward: disable type: vlan mtu-override: disable wccp: disable sflow-sampler: disable explicit-web-proxy: disable As we speak the default route is originated from DialTelecom: get router info routing-table bgp B* 0.0.0.0/0 [20/0] via 188.209.a.c, DialTelecom, 2d14h24m The issue I have is that I cannot ping the public IP address of the Euroweb interface from sources that (return) leave Fortigate using the DialTelecom interface. Here is a packet capture for 5 echo-requests: 41.882077 Euroweb in 85.204.227.242 -> 89.238.a.b: icmp: echo request 43.881365 Euroweb in 85.204.227.242 -> 89.238.a.b: icmp: echo request 45.886258 Euroweb in 85.204.227.242 -> 89.238.a.b: icmp: echo request 47.891378 Euroweb in 85.204.227.242 -> 89.238.a.b: icmp: echo request 49.896304 Euroweb in 85.204.227.242 -> 89.238.a.b: icmp: echo request Packets arrive on Euroweb interface, and according to the routing table (0.0.0.0/0) should leave on DialTelecom interface, but it doesn' t work! I mention that I enabled the asymetric routing: ClassIT-EW (BGP) # get sys set comments : Routing only opmode : nat bfd : disable utf8-spam-tagging : enable wccp-cache-engine : disable vpn-stats-log : vpn-stats-period : 0 v4-ecmp-mode : source-ip-based asymroute : enable strict-src-check : disable asymroute6 : disable per-ip-bandwidth : enable sip-helper : enable sip-nat-trace : enable status : enable sip-tcp-port : 5060 sip-udp-port : 5060 sccp-port : 2000 multicast-forward : disable multicast-ttl-notchange: disable allow-subnet-overlap: disable ecmp-max-paths : 10 I run MR 2 patch 8, version 4.0

[emnoc]

Are you specifying the WAN interface connected to or is that ip_address 85.204.227.242? I think you kinda answer your own question with the following;

As we speak the default route is originated from DialTelecom: get router info routing-table bgp B* 0.0.0.0/0 [20/0] via 188.209.a.c, DialTelecom, 2d14h24m

In your setup how are you receiving routes from both upstreams ( partial, full, single default route???????????????) ? Have you validate that the network your sourcing is being sent to Euroweb? Have you queried any ripe route-servers/looking-glas for routing information? or validate that Euroweb has your 85.204.224.0/20 prefix in the route table or whatever prefix your sending? I would start at minimum with RIPE Ris http://www.ripe.net/data-tools/stats/ris/routing-information-service And validate routing information is present within the euroweb router-database.

[laf]

Hello, I receive about 15k prefixes from both ISP and I advertise ONLY my /24 prefix to both ISPs with no extra configuration (BGP free decision). 85.204.227.242 is one IP address located somewhere in Internet and from where I ran tests. I pinged Euroweb interface and packets arrived on Fortigate on Euroweb interface. Now on their way back they have to use the default route originated by Dial. As I disabled asymetric routing I believe it should return, but it doesn' t happen this way. Many thanks, laf.

[emnoc]

When you say ping euroweb interface, isn' t this a directly connected ip_address ? if yes, than the default route out thru Dialtelecom would not be a factor? also you mention earlier and I quote

I mention that I enabled the asymetric routing:

and now you disabled this, Asymetrical routing is going to be required due to the nature of your traffic and the 2 upstream uplinks, where any one of these could be the path for the returned traffic

Now on their way back they have to use the default route originated by Dial. As I disabled asymetric routing I believe it should return, but it doesn' t happen this way.

Not correct, you can' t control how the internet returns traffic to YOU. Your default-route is just a catch-all for traffic leaving your network. Bottom line, does the problem going away when you have asymetrical routing enabled?

[laf]

Asymetric routing is enabled. For the last two months and I won' t disable IT. I ping from whatever source to the IP_address of one interface. If the source I ping FROM is PREFERRED on the return to go out through the other interface it doesn' t work. Didn' t you meet with the same behavior? (do you have any working BGP scenario on Fortigate?)

[emnoc]

Yes, plenty of BGP experince with firewalls & routes. You still haven' t ask the question, what are you trying to ping? is it part of the directly connected network ? is the remote eBGp peers ? draw a map of what your doing or trying to accomplish.

[laf]

I want to ping the public IP assigned to one of my interface. This IP also is one of the two eBGP peer; BGP is running between Fortigate and ISP. ISP A --> Euroweb_interface(Fortigate) ISP B --> DialTelecom_interface(Fortigate). Routing table is using right now the DialTelecom interface: B* 0.0.0.0/0 [20/0] via 188.209.a.c, DialTelecom, 4d4h24m Packets arrive from 5 hops away on Euroweb interface with source 85..... and according to the routing table on the appliance they should leave on the other interface (DialTelecom); but they get stuck. Both Euroweb and DialTelecom interfaces are binded into a zone, called Internet. For this zone, there is no check to block inter-zone traffic: config system zone edit " Internet" set interface " DialTelecom" " Euroweb" set intrazone allow next end So packets should move from Euroweb interface to DialTelecom interface with no problems. But according to the sniffer nothing more happens.

[emnoc]

Packets arrive from 5 hops away on Euroweb interface with source 85..... and according to the routing table on the appliance they should leave on the other interface (DialTelecom); but they get stuck.

Okay clearier ( i think ), so you have a FGT with 2 wan interfaces doing eBGP to 2 different ISP, but only receiving one default route. for all purpose since your cryptic in your ip_address information here' s my scenario; eBgp peer 188.209.1.1 -.2 ( dial telcom ) eBgp peer 77.77.1.1 -.2 ( euroweb ) note: In both cases .2 is your side or the fortigate device ip_address for wan interface 1 & 2 for the respective interfaces. So you trying to ping 77.77.1.2 and hope that the response goes back thru diatelecom? Are these assumptions correct up to this point ? Q1: Does telecom allow for distribution of provider #2 network space? Can you check a looking glass to see how the destination network looks with in that looking-glass and the as_path hops for that network? Q2:Also what happen if you ping the network ( prefix ) that your advertising to the ISP #1 & #2? ( assumption are your sending your 24 to both providers) Does that work? This could be something as simple as provider #1 & #2 don' t know about each others ( your wan interfaces addresses ) and I don' t think zone pairs are going to magically migrate packets in that shape or fashion., nor should be used in that fashion. Also intra-zone-allow doesn' t work in that fashion either, it would be for any traffic from within the zonepair e.g DialTelcom to Euroweb , or Euroweb to DialTelcom, In this case that your describing, you want to use the other wan interface ( dialteleco ) as a transient to the internet. And for the obvious make sure icmp is allow for both wan1 & 2 allowaccess. Outside of that, you have a very strange setup and scenario. if euroweb has a looking glass, i would like to see what your pings and traceroutes looking to each public address http://lg.euroweb.ro/

[laf]

Hello, Your 3 assumptions are right. Q1: why would it matter if the 1st ISP advertise or not the 2nd ISP prefix? Most probably they would do it, but in my case I have the prefix directly connected. Q2: MY /24 prefix I advertise through both ISPs is advertised using both ISPs and can be accessed via both, ONLY if I ENABLE asymmetric routing. Finally the setup isn' t unusual at all, it is just a classic two ISP scenario. If Fortigate would behave LIKE a router there would not be any issues. I started a ticket and hope by Easter it will be solved!

[emnoc]

On q1, with out seeing their bgp policy, they might not accept that prefix, you can query any looking-glass or route-server to verifed. {eg} If you don' t see isp#2 network -prefix in the path of ips#1, then it safe to assume it' s being dropped. On the unusual, I disagree and you last statement is the reason why. Let routers do BGP routing and let firewalls do just that; firewalling/inspection/security. As you can see, issues are more unusual in their function and operation when using BGP on firewalls. fwiw I always try to terminate my internet uplinks to a router-device and then inject a default route into the boundary firewall thru ospf and the local lan into ospf so that the BGP speakers receives whatever networks that your distributing to the internet. Simpler and less frustration. Most other organizations that I consult does exactly the same also. Another advantage with this approach, if you later decided to add physical redundancy, it' s way much easier. (i.e active-passive firewall, dual internet uplink routers ,etc...) Let us know what fortinet support say about the issue at head. And I would like to see what they say about intrazone and the use of zonepairs with packet sources from one wan and going out the other.