Full internet routing table BGP

[lindblom]

Hello! Quick question: Is anyone running a fortigate with a full internet routing table, over BGP? If not: What fortigate would handle it? And is there any documentation regarding this? I' ve read the " maximum values matrix" for fortiOS 4 & 5: http://docs.fortinet.com/fgt/handbook/50/fortigate-max-values-50.pdf I' m not exactly planning to do this, but it would be interesting to know. /Stefan Lindblom

[emnoc]

The full table is over 450K prefix. Unless your multi-homed, I see no real reason to throw that many prefixes into a firewall or even try. If your building a network with a L3 firewalls and think you need BGP routes and full routes, your probably doing it very very wrong. Now to look at some things, and yes we tried doing this with a FGT3610A many years back for shits and grins, and no real benefit or purpose. And the physical memory on the box was tacked, and dropping prefixes all of the time. And almost every other function stopped, or we developed some other issues. If you want to see what will happen, look at your available memory, and then compute how much memory for each bgp prefix. typical formula is for 1k megs of memory per 100K bgp-routes ( note bgp routes b4 the rib is update plus per peer ) Next your hurdle will be, the CPU processing & time processing would be extremely high and frustrating. What that means from a BGP traffic engineer, every update/refresh for anything and all path attributes, will be process. That' s means some thing as simple as communities additive, path length, as_path withdrawn/add,etc...... Your firewall would be consumed with CPU functions related just to BGP. And to answer your question as to what model might handle this , A carrier grade 5K chassis. And I' m highly suspect that even those, don' t carry full BGP paths.

[lindblom]

Thank you for taking the time and answering. I think it answers all my questions! If I get the chance I might try it for the fun of it. /Stefan

[jlindstrom]

Well.. I think there are better boxes than the 5K' s for this :-) Actually in the 3000-series you have devices with loads of cpu & ram, far more than the latest 5K blades. Think of 3600C. But just as a hunch, I would expect even 1K series to pull this off. Naturally it will have a serious penalty on everything else, but it ought to be able to pull it off. But you' re absolutely correct in saying that someone is doing something terribly wrong if they want to run full bgp table on a firewall. Should be restricted to admins laughs only :-) Even in a multi-homed network with FW at edge, I would maybe and that' s MAYBE ask the ISP' s to advertise domestic routes + default. Then adjust weights to select preferred default and let the domestics go the fastest path. //Juha

[Changuelco]

Emnoc, When you said: " 1k megs of memory per 100K bgp-routes" ... thats means 1GB of memory (RAM) for each 100k routes??? So the minimum requirement for full routing table should be 5gb? The FTG-1Kc have 8gb. Thanks, Changuelco

[emnoc]

Let me clarify that some; Take this dual-homed 7600; cr02>show bgp sum | in otal BGP using 86555047 total bytes of memory cr02>show ip bgp sum BGP router identifier x.x.x.x, local AS number 65000 BGP table version is 12615790, main routing table version 12615790 438004 network entries using 49494452 bytes of memory 437994 path entries using 21023712 bytes of memory 143938/71953 BGP path/bestpath attribute entries using 14393800 bytes of memory 61068 BGP AS-PATH entries using 1643224 bytes of memory 90 BGP community entries using 2160 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 86557348 total bytes of memory BGP activity 1752619/1314615 prefixes, 2185932/1747938 paths, scan interval 60 secs Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd 38.104.xx.xx 4 174 4820942 53942 12615758 0 0 2w0d 437991 204.9.2x.xx 4 31846 151261 137928 12615758 0 0 6w5d 0 The best-practices is like 1k byes of memory for every 100K, but that' s only the 1st half of the story. The firewail is suppose to be, will a " firewall" , if you stick a full-bgp feed into a firewall, than it overlaps into what a router does. CPU will be tagged. How' s your current CPU/MEM usage before you add a full-bgp table? Here' s what the above cisco looks ;

[toantrinh_FTNT]

Just an update with some current stats in case anyone is still wondering. One of our guys did this with a FortiGate-VM this year to see how much memory it would consume.

At the time of the test (Feb-2018), full BGP table is around ~600k prefixes (depends which ISP you are peerd with) injected to a FGT-VM, consumed  ~262MB of RAM.

As the others have rightly mentioned, you have to consider that even though you can do it on a FGT, do you really (really) need to?

If you do, what's the currently utilisation of the CPU and memory like, is there enough headroom to process routing changes/updates.

Below is the debug output from the FortiGate-VM to show the memory utilisation.

FGT-VM-1-KVM # get router info bgp summary BGP router identifier 192.168.2.100, local AS number 65000 BGP table version is 1 89183 BGP AS-PATH entries 4569 BGP community entries Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd 192.168.2.250   4      65001 8132714      11        0    0    0 01:11:24   599538 Total number of neighbors 1 FGT-VM-1-KVM # get router info bgp memory ================= XMEM ALLOCATOR INFO ====================== (1)    0x7fbe24b72f00:    4        1493       65536       1   0 (2)    0x7fbe24b72e70:    8       25385       65536       1   0 (3)    0x7fbe24b72de0:   12        8344       65536       1   0 (4)    0x7fbe24b72d50:   16       67281      131072       2   0 (5)    0x7fbe24b72cc0:   20      228621      327680       5   0 (6)    0x7fbe24b72c30:   24     5470557     5832704      89   0 (7)    0x7fbe24b72ba0:   28      598330      720896      11   0 (8)    0x7fbe24b72b10:   32     3308960     3473408      53   0 (9)    0x7fbe24b72a80:   36      170568      196608       3   0 (10)    0x7fbe24b729f0:   40      119726      131072       2   0 (11)    0x7fbe24b72960:   44       79863      131072       2   0 (12)    0x7fbe24b728d0:   48       61865       65536       1   0 (13)    0x7fbe24b72840:   64      273475      327680       5   0 (14)    0x7fbe24b727b0:   80       78452      131072       2   0 (15)    0x7fbe24b72720:   96       78391      131072       2   0 (16)    0x7fbe24b72690:  112     1542556     1638400      25   0 (17)    0x7fbe24b72600:  128    80938108    81723392    1247   0 (18)    0x7fbe24b72570:  160   167971096   198770688    3033   0 (19)    0x7fbe24b724e0:  192      747400      851968      13   0 (20)    0x7fbe24b72450:  224      378721      458752       7   0 (21)    0x7fbe24b723c0:  256      135864      196608       3   0 (22)    0x7fbe24b72330:  384       73825      131072       2   0 (23)    0x7fbe24b722a0:  512        9955       65536       1   0 (24)    0x7fbe24b72210:  640        2164       65536       1   0 (25)    0x7fbe24b72180:  768           0       65536       1   1 (26)    0x7fbe24b720f0:  896         824       65536       1   0 (27)    0x7fbe24b72060: 1024        2048       65536       1   0 (28)    0x7fbe24b71f00: 2048        1167       65536       1   0 (29)    0x7fbe24b71e70: 4096       23632       65536       1   0 (30)    0x7fbe24b71de0: 8192       73816      131072       2   0 (31)    0x7fbe24b71d50:16384           0           0       0   0 (32)    0x7fbe24b71cc0:32768       16608       65536       1   0 Total:                       262489095   296222720    4520   1 ------------------------------------------------ (1)    0x7fbe24b70300:           161688      163840 (2)    0x7fbe24b70840:           161688      163840 Total:                          323376      327680 ------------------------------------------------ Summary: 262812471, 296550400, 13 ================= END OF XMEM ALLOCATOR INFO =============== Timer: pending 6, added 8190304, expired 2379, deleted 8187919 bgp memory usage Memory type                          Alloc count   Alloc bytes  =================================== ============= =============== BGP structure                     :           2          323376 BGP VR structure                  :           2             480 BGP global structure              :           1             112 BGP peer                          :           3           10752 BGP RIB                           :      599538        76740864 BGP attribute                     :      103036        14012896 BGP aspath                        :       89183         2853856 BGP aspath seg                    :       89183         2276438 BGP aspath str                    :       89183        11999236 Community                         :        4569          146208 Community val                     :        4569          156256 Community str                     :        4569          436642 BGP as list master                :           1              32 Community list handler            :           1              32 BGP Damp Reuse List Array         :           2            8192 BGP table                         :          66             528 BGP node                          :     1093747       148749592 ----------------------------------- ------------- --------------- Temporary memory                  :        5226          173989 Hash                              :           7             280 Hash index                        :           7           57344 Hash bucket                       :      196797         4723128 Thread master                     :           1             224 Thread                            :          13            1976 Epoll data                        :           8             640 Link list                         :          37            1480 Link list node                    :          24             576 Show                              :           1             520 Show page                         :           3           12336 Show server                       :           1              64 Prefix IPv4                       :           8              64 Prefix IPv6                       :           4              80 Route table                       :          10             160 Route node                        :          50            4800 Vector                            :        2667           42672 Vector index                      :        2667           47440 Host config                       :           1               2 Message of The Day                :           1             100 IMI Client                        :           1             824 VTY master                        :           1              40 VTY if                            :           9            3312 VTY connected                     :           6             336 Stream sock CB                    :           1             144 Circular queue buf                :           1            4136 Message handler                   :           2             208 NSM Client Handler                :           1           16608 NSM Client                        :           1            2528 Host                              :           2             256 Log information                   :           2              96 Context                           :           1             456 SYS ZEBOS IPC Server              :           1             160 ----------------------------------- ------------- --------------- bgp proto specifc allocations :    257715492 B bgp generic allocations       :      5096979 B bgp total allocations         :    262812471 B

[emnoc]

Very good post. Keep in mind BGP table memory and what's actually in the RIB is not the same. A single bgp ipv4 route is approx 256 bytes, a ipv6 route is approx  4x times bigger

Next the CPU util% in a single 24hour period, ( in my  ISP peer we have had  388493 bgp path updates or notifications  for my topology. 

All of these and more so should be analyze when sizing bgp cpu/memory requirements. Use the following blog for suspect growth model

http://bgphelp.com/2017/01/01/bgpsize/

And use the following  example for soft-reconfgure

http://socpuppet.blogspot.com/2013/09/bgp-table-received-only-demonstration.html