diagnose traffictest

[localhost]

Since version 5.2.5 Fortinet has added the 'diagnose traffictest' command.

It appears to be an iperf3 running on the Fortigate.

But it seems you can only measure bandwith between interfaces on the Fortigate itself.

I tested some boxes:

FWF60D: 314 Mbit/s (wan1-internalswitch)

FG200b: 1.3GB/s (internal-port9)

FG400D: 15.2 GB/s (port9-port14)

I guess this should proof the firewall throughput? Is there a way to measure troughput between two Fortigates or even another iperf3 instance running on a non-fortinet device?

[AlexFeren]

> Is there a way to measure throughput between two Fortigates

I don't know how, since "diagnose traffictest" is an iPerf v3 client (not server).

> .. or even another iperf3 instance running on a non-fortinet device?

Yes.

1. set (remote) port to iPerf v3 default listening port 5201

2. set server-intf to interface used to reach remote iPerf server

3. use "diagnose traffictest run -c <remote_IPerf_ip>" (Don't ask why Fortinet didn't document "-c <ip>" argument - perhaps they thought it obvious.)

Example:

On Windows host:

d:\Temp\iperf\iperf-3.1.3-win64>iperf3 -s ----------------------------------------------------------- Server listening on 5201 -----------------------------------------------------------

On Fortigate:

FWF61E # diagnose traffictest show server-intf:    lan client-intf:    wan1 port:   5201 proto:  TCP

(Note: it don't matter what interface the client-inf is set to as long as it's got an IP address.)

FWF61E # diagnose traffictest run -c 192.168.3.110 Connecting to host 192.168.3.110, port 5201 [  8] local 192.168.1.13 port 24762 connected to 192.168.3.110 port 5201 [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd [  8]   0.00-1.01   sec  8.87 MBytes  73.9 Mbits/sec    0    209 KBytes        [  8]   1.01-2.00   sec  10.2 MBytes  86.0 Mbits/sec    0    214 KBytes        [  8]   2.00-3.01   sec  10.2 MBytes  85.3 Mbits/sec    0    214 KBytes        [  8]   3.01-4.01   sec  10.1 MBytes  84.8 Mbits/sec    0    214 KBytes        [  8]   4.01-5.01   sec  10.5 MBytes  88.0 Mbits/sec    0    214 KBytes        [  8]   5.01-6.01   sec  10.2 MBytes  85.6 Mbits/sec    0    214 KBytes        [  8]   6.01-7.01   sec  10.1 MBytes  84.4 Mbits/sec    0    214 KBytes        [  8]   7.01-8.01   sec  10.2 MBytes  85.2 Mbits/sec    0    214 KBytes        [  8]   8.01-9.01   sec  9.82 MBytes  82.4 Mbits/sec    0    214 KBytes        [  8]   9.01-10.00  sec  10.2 MBytes  85.9 Mbits/sec    0    214 KBytes        - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval           Transfer     Bandwidth       Retr [  8]   0.00-10.00  sec   100 MBytes  84.1 Mbits/sec    0             sender [  8]   0.00-10.00  sec   100 MBytes  84.1 Mbits/sec                  receiver iperf Done. iperf3: interrupt - the server has terminated

BTW, I think non-zero receiver statistic is a bug - as iPerf client, Fortigate will act as a sender, not receiver. You can verify this using '--get-server-output' argument.

[emnoc]

I found the diag traffictest does not work very good or correct when subinterfaces and LAGs are built.

fwiw to  toggle between TCP/UDP use proto 0 ( tcp )  or 1 ( udp )

YMMV

Ken

[localhost]

Very cool!

The problem that server mode (-s) is not working, is because there are apparently already some arguments passed through by the FortiOS to the iperf binary:

"iperf3: parameter error - cannot be both server and client"

Anyway, with -R you can also test the opposite direction:

FW-1# diagnose traffictest run -c 1.2.3.4 -t 60 -R

Connecting to host 1.2.3.4, port 4430 Reverse mode, remote host 1.2.3.4 is sending

I had no issues with LAG and VLANs on a 400D running 5.2.10.

[emnoc]

Yes you can be a sever/client at the same time

e.g 

FGT-1 (global) # show sys interface lo0 config system interface edit "lo0" set vdom "root" set ip 169.254.1.1 255.255.255.255 set type loopback set snmp-index 52 next end

FGT-1 (global) # diag traffictest run -c 169.254.1.1 -t 10 Connecting to host 169.254.1.1, port 162 [ 8] local 169.254.1.1 port 3037 connected to 169.254.1.1 port 162 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 8] 0.00-1.00 sec 269 MBytes 2.25 Gbits/sec 0 272 KBytes [ 8] 1.00-2.00 sec 266 MBytes 2.23 Gbits/sec 0 288 KBytes [ 8] 2.00-3.00 sec 266 MBytes 2.24 Gbits/sec 0 288 KBytes [ 8] 3.00-4.00 sec 268 MBytes 2.24 Gbits/sec 0 288 KBytes [ 8] 4.00-5.00 sec 269 MBytes 2.26 Gbits/sec 0 288 KBytes [ 8] 5.00-6.00 sec 270 MBytes 2.26 Gbits/sec 0 288 KBytes [ 8] 6.00-7.00 sec 268 MBytes 2.25 Gbits/sec 0 320 KBytes [ 8] 7.00-8.00 sec 266 MBytes 2.24 Gbits/sec 0 320 KBytes [ 8] 8.00-9.00 sec 266 MBytes 2.23 Gbits/sec 0 320 KBytes [ 8] 9.00-10.00 sec 267 MBytes 2.25 Gbits/sec 0 336 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 8] 0.00-10.00 sec 2.61 GBytes 2.24 Gbits/sec 0 sender [ 8] 0.00-10.00 sec 2.61 GBytes 2.24 Gbits/sec receiver

iperf Done. iperf3: interrupt - the server has terminated

FGT-1 (global) # diagnose traffictest show server-intf: lo0 client-intf: lo0 port: 162 proto: TCP

Try this as a client 

(global) # diag traffictest run -c 194.158.119.190 -p 5200 --get-server-output Connecting to host 194.158.119.190, port 5200 [ 8] local x.x.x.x port 9376 connected to 194.158.119.190 port 5200 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 8] 0.00-1.01 sec 293 KBytes 2.38 Mbits/sec 0 66.5 KBytes [ 8] 1.01-2.00 sec 441 KBytes 3.64 Mbits/sec 0 77.8 KBytes [ 8] 2.00-3.01 sec 509 KBytes 4.14 Mbits/sec 0 93.3 KBytes [ 8] 3.01-4.01 sec 648 KBytes 5.31 Mbits/sec 0 112 KBytes ^Ciperf3: interrupt - the server has terminated [ 8] 4.01-4.36 sec 175 KBytes 4.13 Mbits/sec 0 116 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 8] 0.00-4.36 sec 2.02 MBytes 3.88 Mbits/sec 0 sender [ 8] 0.00-4.36 sec 0.00 Bytes 0.00 bits/sec receiver iperf3: interrupt - the client has terminated

[localhost]

Not really sure what you are exactly measuring with a connection from loopback interface to the same loopback interface?

I was looking for a way to run "iperf -s" on one fortigate and "iperf -c" on a second fortigate. To test the bandwidth between two Fortigates without using a client computer.

[emnoc]

I don't think  you can  do that, if you use the client   option run  it against a local attached  iperf3 server  connected to one of your 100/1000mb  ports.

[virtualj]

Hello, I'm also interested in this function to test the traffic between two fortigates. I see two interesting thigs: the command "diag traffictest run" is like a script. It launchs two times the iperf, as you can see with this: FW1_xxx_xx # diagnose traffictest show server-intf:    wan1 client-intf:    wan2 port:   9999 proto:  TCP FW1_xxx_xx # diagnose traffictest run (running in another CLI) FW1_xxx_xx # fnsysctl ps 8931      0       0       R       /bin/iperf -s -B xx.xx.xx.xx -m root -p 9999 8932      0       0       R       /bin/iperf -c xx.xx.xx.xx -B yy.yy.yy.yy -m root -p 9999   You can see also that there is a listening connection on port 9999: FW1_xxx_xx # diagnose sys tcpsock | grep 9999 xx.xx.xx.xx:9999->0.0.0.0:0->state=listen err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:5241->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0 xx.xx.xx.xx:9999->yy.yy.yy.yy:19686->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=774144 tma=0 yy.yy.yy.yy:19686->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=133120 fma=309248 tma=0 yy.yy.yy.yy:5241->xx.xx.xx.xx:9999->state=estabilshed err=0 sockflag=0x1 rma=0 wma=0 fma=0 tma=0   I tried to do the same command on another fortigate with -c xx.xx.xx.xx so it should connect to the other fortigate iperf. Diag sniffer confirm that traffic arrives to the server fortigate, but the syns are dropped. Debug flow shows "iprope_check_failed", like when you are trying to manage the firewall, but don't have trusted hosts or management service enabled. I tried also a firewall local-in policy to accept anything with no result.   Some other ideas? :)

[emnoc]

FWIW if you want to  get real thru-put rates, I would not use the firewall since this traffic would not be offloaded

[virtualj]

My intention is to misure the bandwith between two firewall, so it is important to do the test firewall to firewall. I do not have a client on both side of the firewalls.

Typical example: branch and HQ connected in VPN, I manage the firewalls from internet, but I need the real bandwith between branch and HQ.