dtomic_FTNT
Staff
Staff

FortiSIEM 6.4.0 introduces the ability of enriching Analytics with information from Lookup Tables to provide insight into the data being retrieved from logs. It also introduces the ability to create Correlation Rules based on contents and conditions in one or more Lookup Tables.


 New out of the box LookupTables have been included in this release, which rely on reports to create a baseline of processes and user logins:



  • AWSLoginCountry – Source countries for successful AWS login

  • AzurePortalLoginCountry – Source countries for successful Azure Portal login

  • GSuiteLoginCountry – Source countries for successful GCP login

  • CommonLinuxProcess – Common Linux Process Created

  • CommonWindowsProcess – Common Windows Process Created

  • O365MailLoginCountry – Source countries for successful O365 Mail login

  • ServerLogin – Servers that users login to

  • VPNLoginCountry – Source countries for Successful VPN Login


These are accompanied by new Correlation Rules which will trigger in real-time when FortiSIEM receives an event that doesn’t match the existing baseline.


List of New Rules associated to Lookup Tables:



  • Uncommon AWS Console Login

  • Uncommon Azure Portal Login

  • Uncommon GSuite Login

  • Uncommon Linux process Created

  • Uncommon Office365 Mail Login

  • Uncommon Server Login

  • Uncommon VPN Login

  • Uncommon Windows process Creat


 


What are Lookup Tables?


Lookup Tables are data dictionaries that contain mappings of unique keys to values. They allow you to reference any key or value to enrich or filter the data in your Analytics and Rules Conditions. A single lookup table can contain up to five keys and as many values as you’d like. 


Lookup Tables can be populated manually, by running an on demand or scheduled report, by uploading a CSV file or by using the new Lookup Table API.


Why Should I Use them?


 Lookup Tables are typically used for baselining and for enrichment purposes.


By using them for baselining, we can alert when an unknown occurrence shows up in the logs.


When used for enrichment, they give analysts contextual information about the attribute that they want to lookup (i.e additional user or ip information which is not present in the logs but is contained in the lookup table)


We can also use Lookup Tables to limit the events that are retrieved from the event database by returning only results that match a key in a lookup table.


 


How does FortiSIEM Implement Lookup Tables?


FortiSIEM adds two new functions: LookupTableGet (used for enrichment and filtering purposes) and LookupTableHas (used for filtering purposes).


LookupTableHas can be either True or False. If set to True then it will filter event retrieval and only return data that is present in the Lookup Table. If set to False then it will retrieve all data when the attributes are not present in the LookupTable.


LookupTableGet retrieves a value from a lookup table and can be used as a filter or within Display fields.


FortiSIEM provides a REST API for Lookup Table management, as well as manual creation via GUI and automatic population through running reports or uploading CSVs.


  


Feature Walkthrough


Let’s go through an example of creating a Malware IP Lookup Table manually in the GUI with IP, category, country and confidence information and importing a CSV file with a list of such IPs.


Notice how you can define multiple keys for the lookup table and the type of attribute (LONG, STRING or DOUBLE). 


UploadedImages_BhQXfPK0TzyTDcEeDa9t_Picture 1.png

 


Once table is created, you may import a CSV with this information and do the data mappings in the GUI:


UploadedImages_ohtvZal2TRSVRbOfGfdP_Picture 2.png

Once we import the data, our lookup table will be populated.


UploadedImages_AGQICEnpQQ2khz9AqYef_Picture 3.png

 


We can now analyze permitted traffic events from our firewall and see how we can leverage our new lookup table in a few use cases.


Use Case #1 – Show only Permitted Traffic where either the source or destination IP belongs to our MalwareIPList


 We must specify in the query filter that we only want to return results if either the Source IP or Destination IP are in the MalwareIPList Lookup Table. We use the new function LookupTableHas(MalwareIPList : Source IP) = True OR LookupTableHas(MalwareIPList : Destination IP) to achieve this condition:


UploadedImages_MB9HdoPRhaI32m8gk3Fo_Picture 4.png

When we run the query, it will only return events where either the source or destination IP are in the lookup table:


UploadedImages_dfPqn61WTzuqZVQB3cKA_Picture 5.png


Use Case #2 – Enrich the results of Permitted Traffic IPs that exist in the MalwareIPList by adding Confidence level and Category


 First, we specify in the query filter that we only want to return results if either the Source IP or Destination IP are in the MalwareIPList. We use the new function LookupTableHas(MalwareIPList : Source IP) = True OR LookupTableHas(MalwareIPList : Destination IP) for this:


UploadedImages_x7PcAeO8Q9u1RkYZjSwc_Picture 6.png

 


We must also add the relevant functions to the Display Fields:


LookupTableGet( MalwareIPList : Source IP : Category)


LookupTableGet( MalwareIPList : Source IP : Confidence)


LookupTableGet( MalwareIPList : Destination IP : Category)


 


UploadedImages_I8j9DmtxQ9C6ZYdTJkbQ_Picture 7.png



When we run the query we should see the Source IP and Destination IPs enriched with information from the lookup table:


UploadedImages_8TvS7zeaSEyll5RqMij5_Picture 8.png

 


Use Case #3 – Show only Permitted Traffic where the confidence level of the Source IP is greater than or equal to 87


 To achieve this use case we must leverage the LookupTableGet function by specifying LookupTableGet ( MalwareIPList : Source IP : Confidence ) >= 87


UploadedImages_g7D7VoEjSSOI1GdNK0BB_Picture 9.png

 


The exact same principle can be applied to rules.

4 Comments
martinschmitt1
New Contributor
I am unable to import a CSV. Could you provide a sample how it should look like. I tried a lot of variants but it will not populate my table. I export function would also be nice. I looked for it to make an export to see what format is expected.
dtomic_FTNT
Staff
Staff
Hi Martin,

You can try this file: https://fortinet.egnyte.com/dl/uxy5TIwVEq

Kind Regards,
Dusan
martinschmitt1
New Contributor
Hi Dusan, great this helped. So the name needs to be something ending with .csv otherwise it will not work
martinschmitt1
New Contributor
I also tried to import it with a report. The report has 175 entries but the import only imported 61 entries. Any idea where this difference may come from?