Log fech between FAZ appliances is slow.

[michelbergeron]

When fetching logs from this device, the maximum speed of the fetch seems to be approximately 25 mb/s (observed through interface bandwidth delta at time of starting the fetch on the firewall this traffic passes through) while every device in the network should be capable of at least 1 gb/s and the network is not saturated. We would like to increase the speed as a multi-TB fetch will take weeks at this pace.

I was able to gain some improvement by increasing the "config system log-fetch server-settings" parameters of max connections and max sessions, but both are at their maximum of 10. Neither the server or client FAZ are reaching high CPU/memory/disk usage during the fetch.

I had took a look a the uptime but not able post the images on this forum I'm getting errors 
1st image is 
Load average 0.73  0.66 .065
2nd image
Load average 3.19 2.90 2.84

I have attached the exec top results for both the server and client FAZ involved in the Fetch. Neither seem particularly alarming when looking at the overall devices statistics. The first screenshot is the FAZ the logs are being fetched from (server), the second is the client receiving the logs. Overall CPU usage is floating around 1% for server, 7% for client per their System Resources dashboard widget, and "get system performance" results. have attached the exec top results for both the server and client FAZ involved in the Fetch. Neither seem particularly alarming when looking at the overall devices statistics. The first screenshot is the FAZ the logs are being fetched from (server), the second is the client receiving the logs. Overall CPU usage is floating around 1% for server, 7% for client per their System Resources dashboard widget, and "get system performance" results

Any Ideas

[krahemat_FTNT]

Michel,

You have not stated if this is a virtual appliance or a hardware appliance.  If this is a virtual appliance, then I would guess that other VMs in the host are sharing the resources with other VMs.  Also is the storage locally attached or through a SAN?  I think the problem maybe your I/O with the processing of the incoming logs at the same time you are retrieving logs.  I am not sure what your log rate vs insertion rate.

[michelbergeron]

It is a physical device, FAZ-3700F on both sides of the fetch. The serial information provided for the ticket is for the FAZ the logs are being pulled from. I will get the log rate/insertion rate, but I believe logs are not inserted from the fetch until after the fetch is complete, where then a rebuild is done. I will attach screenshots momentarily.

[michelbergeron]

That screenshot is for the FAZ that is fetching the logs, as I presume the receive/forward rate on the FAZ providing the logs is not relevant, and that value is very low as there are very few devices connected to it. I don't believe the fetch impacts the receive/forward until the DB rebuild is started once the fetch is complete.