Throughput problem with FGT 60D and PPPoE connection

[andrei123]

The unit is set up with FortiOS 5.2.2 and has the wan1 port connected to the ISP with PPPoE (1Gb subscription).

If I connect the laptop or computer directly with PPPoE to the ISP I get ~800 Mb throughput (tested with speedtest, ISP's own speedtest and torrents). When I connect the Fortigate unit the throughput is capped at ~190 Mb (~140 with 5.2.5) and the unit stops responding (CPU 100%).

I tried the following configurations:

- internal lan in switch mode or in interface mode (hardware switch)

- tried with firmwares 5.0.10 and 5.2.1

The MTU for the PPPoE is 1492 so I also tried with mtu-overrride 1492 and still the same

The unit behaves the same in every situation high cpu and capped througput.

All the UTM features are turned off. All the tests are done with the basic configuration, just a policy from internal to wan1..

Also another strange thing is that when I test with the download limited ~100Mb so that the unit doesn't completely freeze I can see from the top command that the CPU is 50% hogged by the system, however there is no process in the list with that high of a load (if you add all the processes they add up to max 10%).

Any ideas would be greatly appreciated ..

I also noticed that the traffic is not going through the NP4Lite so I guess the 'Supports firewall acceleration across all packet sizes for maximum throughput' on the FGT 60D spec sheet on Fortinet website might be false advertising.

Update: There is no way that I found for a 60D  to reach gigabit speeds on PPPoE connection. Max throughput is 140 Mb.

A workaround is to have another router in front of the 60D to do the PPPoe connection ( i got a Ubiquiti Edgemax Lite router for 100E that works amazing)

Best regards,

Andrei

[Thomas1]

Hello Philippe, I advanced a little on my config: The Edgerouter X works very well in PPOE client on the ONT (MSS 1452) My results speeds in v1.6 ~  250/180 v1.8~370/250 . With my livebox ~500 / 250 In fact the offload is not available on this model. After consultation with the UBNT support (which is very reactive !!!), he confirmed to me that the offload was planned on this model and had to buy on a Edgerouter Lite to use offload pppoe and forward To access FG I'll have to forward ports from UBNT but at least I got hold of it and I can do what I want to like these F..... livebox!

Regards

[Justinb]

PPPoE is very slow, at least on the < 100 series.

With Gigabit Ethernet and an ActionTec device to handle the PPPoE, I get speedtest.net results of over 900Mbps up and down on both my old FWF-60c as well as my current 90D (all IPS / logging disabled for all tests)

When I remove the ActionTec from the path and use PPPoE directly from the 90D, measurements drop down to approximately 280Mbps.  While SSH'd into the 90D and attempting to run diag sys top or diag sys top-summary, the top session doesn't update until the speedtest is complete.

I have a demo 300D that I'll test with over the weekend to see if the NP6 does a better job than NPLite, but the short version is: The lower end Fortigate's don't do well with fast PPPoE connections.

(I ran into the issue and this thread after getting frustrated with implementing IPv6 with the ActionTec in the middle, so I took it out)

[freb · Accepted solution]

I had the same issue with the 60d and gigabit internet with PPPOE. I never found a good solution, so I decided to upgrade. After weighing my options, sticking with an upgraded Fortigate seemed like the best bet (as opposed to going with a PFSense box, which would probably have been at least as expensive, or a Ubiquity EdgeRouter). My only question was would the 60e be able to handle the traffic.

I ended up going with the 80e for the extra ports, but the 60e should perform similarly. And yes, this device can more than handle PPPOE encapsulation and hit gigabit speeds without coming close to maxing out.

Hope that helps anyone considering an upgrade but not wanting to because they don't know if it will solve their bottleneck.

[josh]

I'd be interested to know whether FortiGates of any model support PPPoE off-loading. It appears from the 60E onward the devices are equipped with an NP6-Lite:

fwl-01 # get hardware status
Model name: FortiWiFi-60E
ASIC version: SOC3
ASIC SRAM: 64M
CPU: ARMv7
Number of CPUs: 4
RAM: 1864 MB
EMMC: 3662 MB(MLC) /dev/mmcblk0
Hard disk: not available
USB Flash: not available
Network Card chipset: FortiASIC NP6LITE Adapter (rev.)
WiFi Chipset: Atheros
WiFi firmware version: 0.9.17.1

jf-akl-fwl-01 # diagnose npu np6lite port-list
Chip XAUI Ports Max Cross-chip
Speed offloading
------ ---- ------- ----- ----------
np6lite_0
3 wan1 1000M NO
7 wan2 1000M NO
1 dmz 1000M NO
1 internal1 1000M NO
1 internal2 1000M NO
1 internal3 1000M NO
1 internal4 1000M NO
1 internal5 1000M NO
1 internal6 1000M NO
1 internal7 1000M NO

versus:

fwl-01 # get hardware status
Model name: FortiGate-50E
ASIC version: not available
CPU: ARMv7
Number of CPUs: 2
RAM: 2024 MB
MTD Flash: 128 MB /dev/mtd
Hard disk: not available
USB Flash: not available
Network Card chipset: Marvell NETA Gigabit Ethernet driver 00000010 (rev.)

That being said, I haven't got around to breaking apart the software switching on my 60E at home to confirm, though I can confirm the default configuration of having the software switch enabled does prevent offloading of data for at least the AV/IPS/SSL inspection processing in my experience.

[Philippe_ASTIER]

Hi all !

Since I upgraded to a from my FGT-60C HA cluster to a FortiGate-61E, I wanted to do tests again.

With my LiveBox Pro v4 (ISP provided router), I get approx. 900 / 240 down/up Mb/s, with a 10.8 ms latency to Google.

Through my FGT-61E, I could not get more than 570/240, with an improved 9.9 ms latency.

So 61E can still not cope with a full Gb/s of PPPoE. It just helps reduce the latency by the approximate 2 ms need to go through the router.

Or maybe I'm missing some options to get faster PPPoE ? I haven't seen any...

[mas1971]

Philippe ASTIER wrote:

Through my FGT-61E, I could not get more than 570/240, with an improved 9.9 ms latency.

i would like to know, if the CPU usage is at 100% also, with this speed on FTG 61E, or is it reacting as normal?

Max Speed here is 250/40 in PPPoE, so i hope with SOHO E-Series Modell (60e,80e) or the new F-Series (60F) a PPPoE Connection is possible, without any lag because high CPU usage.

The Workaround by offload the PPPoE Connection to external BOX is a little bit tricky, because the Fortigate will not handle the external WAN IP by itself.

Thank you!

[ede_pfau]

Well, if you can configure the external modem into Bridge Mode (for example, a Draytek Vigor model) then the public IP will be on the FGT's wan port, and the credentials in the wan interface config.

The F series desktop models are based on the SoC4 so the CPU performance may be better. Still, I don't think that PPPoE is offloaded to the NP.

[Ingo__T]

I'm in the evaluation which Fortigate to buy for my home-office. Currently i'm on the 80F in combination with a Draytek Vigor 165 in Bridge Mode.

Connection is currently an 250/40 PPPOE maybe in the future an Fiber with 1000/500 also PPPOE.

Currently i would see not performance problems if i go with the 80F.

The question where i find not real a clear answer is.

Does a modem in bridge mode some kind of pppoe offloading, or does it passthrough (raw) all to the Fortigate, where it has to processed by the FG CPU as there is so far i know no hw offloading for pppoe also in the current models.

[josh]

ithierack wrote:
Does a modem in bridge mode some kind of pppoe offloading, or does it passthrough (raw) all to the Fortigate, where it has to processed by the FG CPU as there is so far i know no hw offloading for pppoe also in the current models.

You will need to use half-bridge mode, versus full-bridge mode.

Half Bridge Mode When the PPP Half Bridge is enabled the modem becomes invisible. The DHCP server will duplicate the WAN IP address from the ISP. When to Use Half Bridge Mode When using a separate firewall that will be protecting the network, half bridge mode will allow the firewall to appear on the internet using the publicly accessible IP address assigned by the ISP. This configuration will allow the dedicated firewall to have full control of the inbound and outbound traffic and is the intended purpose for this mode.

Ref: https://whirlpool.net.au/wiki/hw_model_496

Ref: https://support.netcommwireless.com/sites/default/files/Half_Bridge_Mode_Setup_Guide_03_03_11.pdf

Ref: https://serverfault.com/questions/840395/implementing-pppoe-half-bridge-ip-passthrough-to-suit-ipsec...

Your better alternative would be to find an ISP who provides static IP addressing, or uses DHCP/IPoE address assignment.

[ede_pfau]

Never heard of half bridge mode before, thanks for the links.

As I stated a couple of months ago, you can put a Vigor into modem mode in which

- it handles the PPPoE protocol itself and

- the firewall behind it obtains the public IP address.

So, the FGT will only see ethernet traffic, regardless of the WAN protocol used, like plugged in into a LAN socket. I've been using that combo in a number of places without any problems.

The FG-80F is a really good choice as it not only features a SoC4 with (a lot) more CPU power but 4 GB RAM as well. This will help in v6.4 and future releases as small memory Fortigates will have trouble with RAM shortage.

Just out of curiosity, I'd like to hear which PPPoE throughput is feasable with a Soc4 FGT in comparison to a Soc3 FGT.