Internet Connection Speed

[Brent]

Hi, 

First time poster here, so hopefully I got the most appropriate thread.

I have just purchased a Fortigate 600c firewall to route my home office network to the internet.  I have two ISPs and am on a 1Gb plan with both of them.

When I connect the basic routers that were provided from each ISP I get close to what is promised 900 Mb+.  However, when I run my connections through the Fortigate, I am only getting circa 500 Mb.  (As tested using Speedtest.net)

I previously had a Fortigate 200b, which gave me the same speeds, and while investigating I noted that the CPU of the fortigate was maxing out.  Actually, initially I was getting slower speeds but after turning off logging it increased to around the 500 mark.

I thought okay, the CPU isn't powerful enough to handle what I wanted, so I kept an eye out for a second hand later model when I found the 600c being sold locally.  Checking the specs on the 600c and seeing that it has 2.5 Gbps IPS I thought this would be powerful enough to give me closer to the 1 Gbps speeds.

Is there any way I can tweak the fortigate to get better performance, or does anyone have any ideas as to why I am not getting the speeds as advertised?

I'm running FortiOS 5.4, not using WAN LLB (as I require VPN).  Other than this the Fortigate is functioning as I require;

* Seperate Network for myself and my flatmates (I can't have them accessing the VPN to work)

* Traffic routed through specific ISP for specific tasks (Mail through one, other traffic through another)

* Reverse Proxy

Thanks

Brent

[rwpatterson · Accepted solution]

From a Windows work station:

Get to the command prompt ("CMD" from the start box/globe thing)

In the open window, type:

C:\windows\system32> ping -f -l <packet size> <IP address>

The Ethernet packet size on the WAN maxes out at 1500, so start there and decrease until you get a valid response. The "-f" tells the network to not fragment the packets. This will prevent the ping from happening until the entire PING packet can be transmitted. When you get this number, set this on your WAN interface in the GUI. This will then enable communications over the Internet without either end having to break down and reassemble the traffic to traverse the Internet.

Hope that helps.

[rwpatterson · Accepted solution]

From a Windows work station:

Get to the command prompt ("CMD" from the start box/globe thing)

In the open window, type:

C:\windows\system32> ping -f -l <packet size> <IP address>

The Ethernet packet size on the WAN maxes out at 1500, so start there and decrease until you get a valid response. The "-f" tells the network to not fragment the packets. This will prevent the ping from happening until the entire PING packet can be transmitted. When you get this number, set this on your WAN interface in the GUI. This will then enable communications over the Internet without either end having to break down and reassemble the traffic to traverse the Internet.

Hope that helps.

[tanr]

Also, what UTM features do you have turned on, and are you running in flow-based or proxy-based mode?

Just want to make sure that you're not killing your perf with certain UTM features.  For example, the 600C specs only show 400 Mbps throughput for proxy mode antivirus, probably without the extended database.

[Brent]

tanr wrote:

Also, what UTM features do you have turned on, and are you running in flow-based or proxy-based mode?

Just want to make sure that you're not killing your perf with certain UTM features.  For example, the 600C specs only show 400 Mbps throughput for proxy mode antivirus, probably without the extended database.

I'm in proxy-based mode, though I have tried it in both as I'm not sure I understand the difference.

I don't have any UTM features enabled as the Fortigate is second hand and the licenses have expired.  However, that being said, I had the web filter disabled in the interface, but the feature was still enabled.  After disabling the feature, the speed seemed to improve slightly.  Is that something you would expect?

[tanr]

In general, flow mode is supposed to be a little faster than proxy mode.  It really depends on what sort of traffic you have.  Useful overview is in Life of a Packet https://docs.fortinet.com/d/fortigate-life-of-a-packet-5.4. 

If you're not using any UTM features in your security policies I don't think turning the features on or off will a big difference.  Others with more complete knowledge may know better than me, though.

BTW, before you spend too much time testing, is your ISP guaranteeing the 1Gbps speed?  I know in my area residential 1Gbps is usually nowhere near its advertised speed and can vary wildly, whereas my 1Gbps fiber business line has (supposedly) guaranteed levels of bandwidth and uptime.  And even with that I see more variation than I'd like, especially outside of normal business hours.

[Brent]

tanr wrote:

BTW, before you spend too much time testing, is your ISP guaranteeing the 1Gbps speed?  I know in my area residential 1Gbps is usually nowhere near its advertised speed and can vary wildly, whereas my 1Gbps fiber business line has (supposedly) guaranteed levels of bandwidth and uptime.  And even with that I see more variation than I'd like, especially outside of normal business hours.

No, they don't guarantee it, but when a cheap little give-away router that the ISP provided can give a Speedtest result of 900+ I would have thought a unit such as the Fortigate 600c should be able to match it.  I am testing at different times during the day to eliminate shared usage on the line.

I'd be happy to consistently see 900+ to my ISP's speedtest server, I'm now (after the suggestions above) reasonably constant around 800+ which is up from circa 500.  So I'm not unhappy.

[ede_pfau · Accepted solution]

I would have checked first if this phenomenon is connected to the WAN port and/or protocols.

Please run a performance test between 2 hosts on your LAN, both with GbE ports, AV disabled. I usually use iperf for this as the same exe-file contains the server and the client.

You should see wirespeed here.

If you don't there is a problem within FortiOS such as the traffic offloading to the network ASIC (NP) being disabled. Have you reset the unit after purchase to factory defaults ("exec factoryreset") before configuring it? Beware that this command will delete all of the config.

If you do see wirespeed between LAN ports I would investigate the WAN protocol used. How do you connect to your IPS(s)? DHCP, PPPoE, static IP? There is a known gotcha with PPPoE processing in FortiOS. Cheap but dedicated WAN routers use a special chip to handle the protocol, FortiOS doesn't. It might well suck up the CPU performance if the WAN line speed is in excess of 100 Mbps for desktop models, higher speeds for multi-core FGTs like yours.

The 600C itself is very capable, with decent memory size, content ASIC (CP) and network ASIC (NP) for offloading chores off the CPU. CPU will mostly handle session setup, negotiations (IPsec, SSLVPN, PPPoE, DHCP), logging and GUI. Plus some more but will usually stay out of the way of running session traffic. That's why you would expect wirespeed performance on GbE ports. IMHO the specs on the datasheet come quite close to realworld figures.

Lastly, if you're running FOS v5.4 do update to the latest build (v5.4.6). Each patch version will fix some bugs and possibly improve throughput (while adding features which introduce more bugs...). For a used unit without contract this will be, hm, difficult but you may well ask FTNT for a contract. FortiCare will do (firmware updates, warranty extension) but of course FortiGuard would be more beneficial for you (AV, IPS, botnet IP blacklist, webfilter,...). If such contracts are no longer available from the regular price list you could ask FTNT for a 'coterm quote'.

Happy testing!

[Brent]

ede_pfau wrote:

I would have checked first if this phenomenon is connected to the WAN port and/or protocols.

Please run a performance test between 2 hosts on your LAN, both with GbE ports, AV disabled. I usually use iperf for this as the same exe-file contains the server and the client.

You should see wirespeed here.

If you don't there is a problem within FortiOS such as the traffic offloading to the network ASIC (NP) being disabled. Have you reset the unit after purchase to factory defaults ("exec factoryreset") before configuring it? Beware that this command will delete all of the config.

If you do see wirespeed between LAN ports I would investigate the WAN protocol used. How do you connect to your IPS(s)? DHCP, PPPoE, static IP? There is a known gotcha with PPPoE processing in FortiOS. Cheap but dedicated WAN routers use a special chip to handle the protocol, FortiOS doesn't. It might well suck up the CPU performance if the WAN line speed is in excess of 100 Mbps for desktop models, higher speeds for multi-core FGTs like yours.

The 600C itself is very capable, with decent memory size, content ASIC (CP) and network ASIC (NP) for offloading chores off the CPU. CPU will mostly handle session setup, negotiations (IPsec, SSLVPN, PPPoE, DHCP), logging and GUI. Plus some more but will usually stay out of the way of running session traffic. That's why you would expect wirespeed performance on GbE ports. IMHO the specs on the datasheet come quite close to realworld figures.

Lastly, if you're running FOS v5.4 do update to the latest build (v5.4.6). Each patch version will fix some bugs and possibly improve throughput (while adding features which introduce more bugs...). For a used unit without contract this will be, hm, difficult but you may well ask FTNT for a contract. FortiCare will do (firmware updates, warranty extension) but of course FortiGuard would be more beneficial for you (AV, IPS, botnet IP blacklist, webfilter,...). If such contracts are no longer available from the regular price list you could ask FTNT for a 'coterm quote'.

Happy testing!

So I connected a laptop to my desktop computer via a $30 gigabit switch and IPerf gave me 950Mbs, but then when I connected the laptop to the otherside of the fortigate (same network so no firewall rules etc), it dropped to 850Mbs (haven't ruled out the cable).  If I connect it to my other network it drops even further to 750Mbs.  Though on saying that, I have also had speedtest.net return 950Mbs after making these changes, and it is commonly in the high 800s.  So I'm reasonably happy now.

Though I am curious as to why the local network drops when traversing the fortigate

In answer to your other questions, yes, I did a factory reset.  I'm running FortiOS 5.4.0, I would love to update to 5.4.6, but sadly the FortiCare contract is over $1000 which I'm just not willing to pay (nor can I afford to) :(

[Brent]

rwpatterson wrote:

From a Windows work station:

Get to the command prompt ("CMD" from the start box/globe thing)

In the open window, type:

C:\windows\system32> ping -f -l <packet size> <IP address>

The Ethernet packet size on the WAN maxes out at 1500, so start there and decrease until you get a valid response. The "-f" tells the network to not fragment the packets. This will prevent the ping from happening until the entire PING packet can be transmitted. When you get this number, set this on your WAN interface in the GUI. This will then enable communications over the Internet without either end having to break down and reassemble the traffic to traverse the Internet.

Hope that helps.

I'm not 100% sure that has resolved the issue, but it certainly has done something as I'm not consistently getting around 500Mbps, I'm now getting closer to 700Mbps when I test after first opening a browser window, then on subsequent tests I get higher, I've even seen 900, but I assume that is some sort of caching in the browser as I don't see that on the first test.

Any other tips - I liked that one, but would still like to see it closer to what I'm paying for.

PS. I couldn't find the MTU settings in the GUI.  So I set it via the CLI.  How do you set it via the GUI?

Thanks

[FransUrbo]

There's very little discussion on the few "speed" issues in the forum... :(

I'm having the same problem. I have a FortiGate 50E which is also supposed to have 2.5Gbps but on my 1Gbps fiber broadband from Hyperoptic, I'm getting 356Mbps down and 110Mbps up (on copper).

The router provided by Hyperoptic could manage 400Gbps on the builtin WiFi.

So the question is: What is that cheap piece of c**p they provided me with doing so much better than a FortiGate??

[pireality]

Nobody replied to "how to set this in the gui?".  I did some testing and it looks like with the DF bit set, my default gateway responds @ 1472.  I could also set mtu via CLI, but gui will be easier for noobs.