I have a FAZVM hosted on Dell Compellent (mostly flash) and Dell R630 servers, vSphere 6.5U3. The VM has 16G RAM and 8vCPUs. We have a 2GB/Day license and currently consume ~1.5G/day. As far as I can tell storage capacity and IOPs are not rate limiting but my receive rate never exceeds 150 logs/sec. Insertion rate is consistently less than 60s.
We have two 501Es in A/P HA. We are outputting logs to a Syslog destination in addition to the FAZ. It does not appear that we are dropping logs/traffic but the receive rate appears very flat during what would be peak periods. See attached.
Looking through docs in older FortiOS, it appears there was a min and max buffer setting for FAZ settings on the Fortigate but that is not available in FortiOS 6.0.5. Our FAZ is the same version. We try and log everything including denies so there should be enough volume. Firewall is protecting ~500 devices (desktops/laptops/mobile) and servers such as Exchange. It just seems strange that the receive rate is so flat during business hours. If anyone has any ideas of what to test check, I'd appreciate it.
Partially resolved by killing the miglogd service. Switch that connects FAZ and Gate was rebooted and for some reason that killed DNS logging. Attempted to stop and restart logging, via GUI but DNS flows would not appear. If you log into the Fortigate CLI
diag sys top-summary
look for the miglogd process and note the process ID (PID)
197 320M 0.0 2.0 73 09:19.30 miglogd [x5]
Press "q" to quit the monitoring of sys top.
Now that you have the PID of the miglogd process, enter the following to kill and restart it:
diag sys kill 11 197
Note in my case the PID was 197 as highlighted above. Once we did this, DNS logs started flowing again. Odd issue. Will see what my log volume is over next 24 hours.
Before reading your second post, I thought that the figures look pretty OK.
Assume 100 bytes per log entry, and 1.5 GB/86400s, would mean ~ 186 logs/s. Which wouldn't surprise me in your environment, and is far from any rate limit stated for the FAZ(-VM).
1.5 or 2 GB/day is easily taken by a FAZ-VM, even with just 2 vCPUs. If you look at the receive/insertion rate diagram, you see that everything that is received is promptly stored in the DB (no lag, no steady differences). So that looks OK as well.
Not getting any DNS logs is of course a bug situation. I just wonder why it doesn't show in the load diagrams. I'd ask Support to have a look at this, esp. for DNS logs.
You might (might) rebuild the DBs, or the data partition, with old logs being saved in advance. But, this is definitively a Support issue then.
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.