Slow iPerf (and generall traffic) through firewall

[Brian_M]

Before I start, I want to clarify that the remote iPerf server I’m testing against is on a 10Gbps link.

I have a Fortigate 61F with the wan2 interface connected to a 1Gbps (synchronous) Internet connection. If I run an iPerf test from that interface, I can achieve ~740mbps. However, if I run the same test from a machine on the inside of the firewall, I can never get past 300Mbps.

Version: FortiWiFi-61F v7.2.3,build1262,221109 (GA.F)

Firmware Signature: certified
Virus-DB: 90.08592(2022-12-09 22:26)
Extended DB: 90.08592(2022-12-09 22:25)
AV AI/ML Model: 2.08777(2022-12-09 21:45)
IPS-DB: 22.00454(2022-12-08 00:58)
IPS-ETDB: 0.00000(2001-01-01 00:00)
APP-DB: 22.00454(2022-12-08 00:58)
INDUSTRIAL-DB: 22.00454(2022-12-08 00:58)
IPS Malicious URL Database: 4.00555(2022-12-09 11:23)
IoT-Detect: 22.00454(2022-12-07 17:24)
Serial-Number:
BIOS version: 05000007
System Part-Number: P24307-03
Log hard disk: Available
Hostname:
Private Encryption: Disable
Operation Mode: NAT
Current virtual domain: root
Max number of virtual domains: 10
Virtual domains status: 1 in NAT mode, 0 in TP mode
Virtual domain configuration: disable
FIPS-CC mode: disable
Current HA mode: standalone
Branch point: 1262
Release Version Information: GA

Here is the test from the firewall:

diagnose traffictest run -c x.x.x.x -P 4

[ ID] Interval Transfer Bandwidth Retr
[ 7] 0.00-10.01 sec 204 MBytes 171 Mbits/sec 44 sender
[ 7] 0.00-10.01 sec 204 MBytes 171 Mbits/sec receiver
[ 9] 0.00-10.01 sec 212 MBytes 177 Mbits/sec 138 sender
[ 9] 0.00-10.01 sec 212 MBytes 177 Mbits/sec receiver
[ 11] 0.00-10.01 sec 243 MBytes 204 Mbits/sec 40 sender
[ 11] 0.00-10.01 sec 243 MBytes 204 Mbits/sec receiver
[ 13] 0.00-10.01 sec 224 MBytes 188 Mbits/sec 10 sender
[ 13] 0.00-10.01 sec 224 MBytes 188 Mbits/sec receiver
[SUM] 0.00-10.01 sec 883 MBytes 741 Mbits/sec 232 sender
[SUM] 0.00-10.01 sec 882 MBytes 740 Mbits/sec receiver

Here is the test from the inside machine:

 iperf3 -c x.x.x.x  -P 4

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 83.5 MBytes 70.0 Mbits/sec sender
[ 4] 0.00-10.00 sec 83.5 MBytes 70.0 Mbits/sec receiver
[ 6] 0.00-10.00 sec 83.2 MBytes 69.8 Mbits/sec sender
[ 6] 0.00-10.00 sec 83.2 MBytes 69.8 Mbits/sec receiver
[ 8] 0.00-10.00 sec 83.0 MBytes 69.6 Mbits/sec sender
[ 8] 0.00-10.00 sec 83.0 MBytes 69.6 Mbits/sec receiver
[ 10] 0.00-10.00 sec 83.4 MBytes 69.9 Mbits/sec sender
[ 10] 0.00-10.00 sec 83.4 MBytes 69.9 Mbits/sec receiver
[SUM] 0.00-10.00 sec 333 MBytes 279 Mbits/sec sender
[SUM] 0.00-10.00 sec 333 MBytes 279 Mbits/sec receiver

What’s interesting is that if I increase the threads on the inside client from 4 to 20, I can start realizing those higher speeds.

iperf3 -c x.x.x.x  -P 4

[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 57.0 MBytes 47.8 Mbits/sec sender
[ 4] 0.00-10.00 sec 57.0 MBytes 47.8 Mbits/sec receiver
[ 6] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec sender
[ 6] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec receiver
[ 8] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec sender
[ 8] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec receiver
[ 10] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec sender
[ 10] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec receiver
[ 12] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec sender
[ 12] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec receiver
[ 14] 0.00-10.00 sec 56.8 MBytes 47.6 Mbits/sec sender
[ 14] 0.00-10.00 sec 56.8 MBytes 47.6 Mbits/sec receiver
[ 16] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec sender
[ 16] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec receiver
[ 18] 0.00-10.00 sec 56.1 MBytes 47.1 Mbits/sec sender
[ 18] 0.00-10.00 sec 56.1 MBytes 47.1 Mbits/sec receiver
[ 20] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec sender
[ 20] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec receiver
[ 22] 0.00-10.00 sec 56.4 MBytes 47.3 Mbits/sec sender
[ 22] 0.00-10.00 sec 56.4 MBytes 47.3 Mbits/sec receiver
[ 24] 0.00-10.00 sec 54.2 MBytes 45.5 Mbits/sec sender
[ 24] 0.00-10.00 sec 54.2 MBytes 45.5 Mbits/sec receiver
[ 26] 0.00-10.00 sec 56.0 MBytes 47.0 Mbits/sec sender
[ 26] 0.00-10.00 sec 56.0 MBytes 47.0 Mbits/sec receiver
[ 28] 0.00-10.00 sec 57.0 MBytes 47.8 Mbits/sec sender
[ 28] 0.00-10.00 sec 57.0 MBytes 47.8 Mbits/sec receiver
[ 30] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec sender
[ 30] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec receiver
[ 32] 0.00-10.00 sec 48.6 MBytes 40.8 Mbits/sec sender
[ 32] 0.00-10.00 sec 48.6 MBytes 40.8 Mbits/sec receiver
[ 34] 0.00-10.00 sec 52.9 MBytes 44.4 Mbits/sec sender
[ 34] 0.00-10.00 sec 52.8 MBytes 44.3 Mbits/sec receiver
[ 36] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec sender
[ 36] 0.00-10.00 sec 56.6 MBytes 47.5 Mbits/sec receiver
[ 38] 0.00-10.00 sec 45.2 MBytes 38.0 Mbits/sec sender
[ 38] 0.00-10.00 sec 45.2 MBytes 38.0 Mbits/sec receiver
[ 40] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec sender
[ 40] 0.00-10.00 sec 56.5 MBytes 47.4 Mbits/sec receiver
[ 42] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec sender
[ 42] 0.00-10.00 sec 56.9 MBytes 47.7 Mbits/sec receiver
[SUM] 0.00-10.00 sec 1.08 GBytes 929 Mbits/sec sender
[SUM] 0.00-10.00 sec 1.08 GBytes 928 Mbits/sec receiver

I'm not using a VPN tunnel, I don’t have any strange routing (no PBR), and I don’t even run a routing protocol, just a single static default route. There is no traffic-shaping, and I did not create any software switches. I have a single physical interface is connected to a Cisco switch (no errors on the ports), and it’s configured for VLAN trunking.

I thought maybe the outbound policy might be slowing it down, so I’ve removed all UTM features, set the SSL inspection to “no-inspection”, and even disable traffic logging but it doesn’t make the slightest difference.

config firewall policy
edit 2
set name "outbound default"
set uuid 31503162-25f6-51eb-f268-20a2aadc8c96
set srcintf "Servers" "Workstations" "Lab-1"
set dstintf "Outside"
set action accept
set srcaddr "all"
set dstaddr "all"
set schedule "always"
set service "ALL"
set logtraffic disable
set logtraffic-start enable
next

I’m not sure where to start looking next so any suggestions would be appreciated!

Thanks!!

[nsh]

@Brian_M - It seems to be the same problem as ours. Was the fortinet support able to help?

[Brian_M]

We never called support, but this (combined with other issues), has helped us learn our lesson about Fortinet devices.

[gfleming]

May I ask why you never spoke to support? You could very possibly have a hardware issue on that device. As everything you've mentioned in this thread points to no real explanation as to why you aren't seeing faster speeds. It might be something that requires RMA.

[Brian_M]

---

@gfleming wrote:

May I ask why you never spoke to support?

---

Yes. I would like to refer you to the small amount of tickets that I've opened since owning the device. Not very many of them have been solved by contacting support. Generally, they are set to "Other", "Problem solved by Customer", and one of them they couldn't even figure out (they set the resolution to "Out of scope").

I could definitely go on about the "difficulties" with the Fortigate, but I want to be very careful as to not turn this into a bashing thread. They aren't the worst firewall in the world but I, like many others, feel there are better players out there.

[gfleming]

Well it's unfortunate you didn't follow up with those tickets if they weren't handled to your satisfaction. Escalations are definitely possible. You have the ability to request escalation any time. If you still do not get satisfactory responses your account team can go to bat for you and escalate internally. 

I would still highly recommend you open a ticket. And knowing what you know now, feel free to escalate if you're not getting the response you need.

And please feel free to share your other difficulties. It's good for others to hear about them absolutely. And who knows, you might learn something as well if you never got a resolution to the problems in the first place. Might be best to open it as a separate topic in case it gets buried but I'd be all for sharing them either way.