Explorer III

Question

Estimation of log size per day

Forum|Forum|4 years ago
April 21, 2022
13 replies
40132 views

Hi,

I need to have an estimation of the log sizes generated by my firewall everyday in order to purchase a suitable license for my Fortianalyzer or a similar log solution. The firmware is 6.4.x.

I thought of clearing logs, coming up tomorrow and find the log size on the disk but maybe there are some better ways to find except that clearing valuable logs on my 200 series firewalls.

Regards,

FortiGate

AlexC-FTNT

Staff

Log size can't be estimated as such. Try to obtain a trial FortiAnalyzer license from your local distributor, and set it up to collect the logs. In 1 week you can see the amount of logs collected and based on that, you can make an informed decision.

Every FortiGate passes completely different amount and type of traffic, and has different logging options - making an estimation very difficult.

xsilver_FTNT

Staff

Hi mhdganji,

first set up at least some logging, like to memory or disk (in case you have FortiGate equipped with disk), as close to intended logging filter (config log memory filter) as possible.
Then keep it logging for some time.

And then go CLI to see something like in my example below:

# diag test app miglogd 4
info for vdom: root
memory
traffic: logs=1160137 len=627074265, Sun=89458 Mon=132174 Tue=225162 Wed=239396 Thu=145690 Fri=153707 Sat=60834
event: logs=93493 len=29698750, Sun=1173 Mon=1545 Tue=1372 Wed=1403 Thu=825 Fri=1778 Sat=1107

disk
traffic: logs=1160137 len=627074265, Sun=89458 Mon=132174 Tue=225162 Wed=239396 Thu=145690 Fri=153707 Sat=60834 compressed=37039926
event: logs=82559 len=24792447, Sun=885 Mon=1257 Tue=1084 Wed=1115 Thu=651 Fri=1490 Sat=819 compressed=5176706

I know those are not actual log sizes but amounts of log messages, but they might give some insight to amounts you face.

In case you have a log disk .. show log disk usage
# diag test app miglogd 16

VDOM log disk usage:
root: 58456760B/7556M

Once you have some external log collector (could be simple Syslog on some external machine), then you can see usage in GUI

Debbie_FTNT

Staff & Editor

Hey mhdganji, in addition to Tom's and Alex update - FortiAnalyzer size recommendations are usually made by calcuating with 100 byte per log message and observing over a few days how many log messages are generated.

If you have a rough estimate of the number of logs per day, that times 100 byte would roughly be the daily logging volume, and you can look for a suitable FortiAnalyzer based on that.

You could also go with a VM; the base licence is for one 1GB logs per day, and you can stack up very easily as necessary.

mhdganjiAuthor

Explorer III

With now logging to memory, any query command to find counts of logs in a minute or so?

mhdganjiAuthor

Explorer III

I used demo version of a software to collect the logs and could get a good estimate but as I noticed the size goes about 500-800 bytes per log ! Maybe it's because of the format and the nature of the software functionality (seems it uses elastic search)

Debbie_FTNT

Staff & Editor

Hey mhdganji,

My knowledge may be a bit outdated, but the 100 bytes I mentioned above was how FortiAnalyzer engineers told me they very roughly calculated logging volume.

This might have changed, or it could be that if you're logging to syslog the logs are in a slightly different format than logging to FortiAnalyzer (the logs are compressed when sent to FortiAnalyzer, which would reduce the logging volume).

For more qualified help I would suggest you get in touch with your local Fortinet Sales partner - they should have the resources to more accurately assess and size a FortiAnalyzer to your needs :)

mhdganjiAuthor

Explorer III

The majority (80+ percent) are the allowed forwarding logs. I cannot find the command to disable logging or sending these to remote log server. Is that possible or it should be completely turned off or on per rule (which also I think makes troubleshooting difficult for short and specific periods that a rule is being tested)

regards

ede_pfau

SuperUser

in CLI:

conf log syslogd filter

and get the options by typing

'set ?'

IMHO setting up a FAZ-VM without license would be the most accurate way to see what is coming onto you. The dashboard of the FAZ clearly shows logs/sec, GB/day etc. etc.

Unlicensed VMs run for 14 days for free.

You can even aquire a FAZ license after/during the trial, register it, get the license file and import it into the trial VM - no re-configuration, fully operational.

mhdganjiAuthor

Explorer III

Speaking of FAZ, I've a question about how licensing works. Let's say we have a 1GB/Day license. If the input log size exceeds 1GB in a day, what happens exactly? Collecting logs stops completely? Logs are collected but only 1GB per day are analyzed and available to make reports? or any other method?

Regards,

Debbie_FTNT

Staff & Editor

If logging volume exceeds the licenced volume, FortiAnalzyer does not forcibly drop logs, stop processing them or not use them in reporting, but there can be performance issues which can eventually lead to loss of logs.

See https://community.fortinet.com/t5/FortiAnalyzer/Technical-Note-Minimizing-logging-from-FortiGate-to/ta-p/198018
"Although FortiAnalyzer VM will try its best not to drop logs, consistently running over capacity will eventually lead to undetermined behavior. This is because all FortiAnalyzer VM functions are validated within the licensed limit; the behavior beyond that limit is deemed to be unsupportable."

If you run into issues while exceeding licence, FortiAnalyzer support will not be able to investigate until you have resolved the licencing issue.

Sign up

Login with SSO

Login to the community

Login with SSO