Technical Tip: FortiAnalyzer log forwarding filter using Generic free-text filter

[farhanahmed]

Description

The article describes how to use the generic free-text filter in FortiAnalyzer to filter log forwarding.

Scope

FortiAnalyzer.

Solution

In Log Forwarding the Generic free-text filter is used to match raw log data. It uses POSIX syntax, escape characters should be used when needed.

1. Under FortiAnalyzer -> System Settings -> Advanced -> Log Forwarding, select server and 'Edit' -> Log Forwarding Filters, enable 'Log Filters' and from the drop-down select 'Generic free-text filter'

In this example, FortiAnalyzer is forwarding logs where the policy ID is not equal to 0 (implicit deny).

2. Another example of a Generic free-text filter is to filter logs for where administrator accounts are added or deleted by the user 'admin' only.

Value is set to: user==admin AND (msg ~ "Add" OR msg ~ "Delete").

3. Checking the system event logs on the sender FortiAnalyzer (where log-forwarding is enabled):

4. Checking the system event logs on the receiver FortiAnalyzer:

The sender FortiAnalyzer is only forwarding the logs where the user 'admin' added and deleted administrator accounts.

Note 1: 

The generic free-text filter can also be configured from FortiAnalyzer CLI:

config system log-forward
    edit 1
        set mode forwarding
        set server-name "FAZ"
        set server-addr "172.31.200.138"
        set log-filter-status enable
            config log-filter
                edit 1
                    set field free-text
                    set oper match
                    set value "policyid!=0"
                next

                       edit 2
                    set field free-text
                    set oper match
                    set value "user==admin AND (msg ~ \"Add\" OR msg ~ \"Delete\")" 
                next
                edit 3
                    set field free-text
                    set oper match
                    set value "appcat==Video/Audio"<----- Filtering logs where the application category is Video/Audio.
                next
            end
    next
end

Note 2:

In GUI, quotations can be used for a specific item, not for the whole value (In CLI it automatically inserts the quotation for the whole value).

For example: If cannot use ("appcat==Video/Audio") in GUI it will give an error for an invalid value. But using (appcat=="Video/Audio") will work.

Note 3:

Generic text filters are also used in Event handlers.

To create an event handler using the Generic Text Filter to match raw log data:

• Go to Log View, and select a log type.
• In the toolbar, click Tools > Display Raw.
  The easiest method is to copy the text string you want from the raw log and paste it into the Generic Text Filter field.
  Ensure you insert an escape character when necessary, for example, cfgpath=firewall\.policy.
• Locate and copy the text in the raw log.
• Go to Incidents & Events > Event Monitor > Event Handler List and click Create New.
• In the Generic Text Filter box, paste the text you copied or type the text you want. Ensure you use the raw log field
  names, for example, mem (not memory) and setuprate (not setup-rate).
  For information on text format and operators, hover the cursor over the help icon. The operator ~ means contains
  and !~ means does not contain.
• If you want to be notified of events, configure the Notifications section.
• Configure other settings as required and click OK.

Note 4:

The "equal to" / "==" does not support the subnet filter and "contain" / "~" must be used instead.

Example for acceptable filter for 10.0.0.0/8 subnet using the free-text is: srcip ~ "10\.[0-9]+\.[0-9]+\.[0-9]+"

Related documents:

Technical Note: Use of Operators in Event Handler General Filter (syntax)

Using the Generic Text Filter

Technical Note: How to configure an Event Handler with a generic text filter