A new FortiSIEM Incident Remediation method based on a custom publishing script leveraging FortiGate Security Fabric External Connectors and/or 3rd party NGFW Connectors

[SilvPufa]

FortiSIEM is a highly flexible solution providing a wide collection of inbuilt Remediation Scripts, integrating FortiSOAR Playbooks or giving the user the ability to create his own custom remediation scripts.

Predefined and custom FortiSIEM scripts can be invoked on-demand (manually) or automatically when the incident happens. A common situation for using remediation scripts includes blocking sources and/or destinations IP address for reported attacks or suspicious activities (ex. port scanning activities, communication with a C&C server, botnets, etc.).

These situations can also be addressed in an easy and reliable way by using a new remediation method based on a FortiSIEM publishing script leveraging Fortinet Security Fabric External Connectors and/or 3^rd party NGFW Connectors. 

Video Presentation & Demo

Main advantages of this method:

• One single publishing script can be used to integrate multiple and different FGT FOS versions and/or 3^rd party firewall devices
• Sources and/or Destinations IP address can be extracted from FortiSIEM Incident XML file and published in a HTML/txt file format in order to be easily fetched by Firewalls and used in specific policy rules
• In case of a firewall firmware upgrade or configuration change, there is no need to update the FortiSIEM script  
• Less computing resources needed, as there is no need to initiate and handle SSH sessions with each specific FortiGate or 3^rd party Firewall
• Fast and easy deployment within Fortinet environment by copying the script on FortiSIEM Supervisor and using FortiManager to configure FortiGate Security Fabric External Connectors
• Sources or Destinations IP address can be fetched from FortiSIEM incidents and used by Firewalls to enforce different policy rules (with block or allow actions)
• This method might be extended to use Hash and URL lists (not just IP address lists)

Download:

• src_ip.py: python script for extracting Source IP Address from Incident XML file and publish it on /var/www/html/src_ip.html file hosted by FortiSIEM Supervisor
• dst_ip.py: python script for extracts Destination IP Address from Incident XML file and publish it on /var/www/html/dst_ip.html file hosted by FortiSIEM Supervisor 
• Additional tools (useful only for advanced testing purposes):
  • send_syslog.py: script for sending sample syslog messages to FortiSIEM 
  • syslog_msg.txt: text file containing sample log messages
  • TestEventParser_A.xml: FortiSIEM parser template for sample logs
  • Report_Event_Type_A.xml: FortiSIEM test event report template for verifying received events
  • Allerting_Rule_Event_Type_A.xml: FortiSIEM rule template for triggering test Incidents 

Implementation:

1. Configure FortiSIEM Supervisor
   • Connect to Supervisor console (ex. using SSH)
   • Install pathlib library (this will be used by the script while automatically creating the IP lists html file):
     • # cd /usr/local
     • # pip2.7 install pathlib
   • Create a local folder by running the following commands:
     • #mkdir /script
   • Copy the src_ip.py script file to the previously created folder or build the file from scratch using vi editor, by running the following commands::
     • # > src_ip.py
     • # vi src_ip.py (insert the script code in the file and save it)
     • src_ip.py script extracts Source IP Address from Incident XML file. If Destination IP Address are needed you have to copy/create the "dst_ip.py" script file.
   • Run the sript using #python src_ip.py command. Upon running this command, HTML file lists will be automatically created, and the script will become executable with admin rights (this way the admin user will be able to trigger/run the script from the web GUI). Ignore the console messages after running the “#python src_ip.py” CLI command. If Destination IP Address are needed you have to run the "dst_ip.py" script file.
     • HTML list file will be automatically created in the /var/www/HTML folder
     • HTML list file will have a similar name as the script name (for example running script "src_ip.py" creates a "src_ip.HTML" file
2. Configure FortiGate Security Fabric External Connectors
   • Connect to FortiManager GUI and select "Security Fabric View":
   • Create a New Fabric Connector:
   • Fill in the "URI of external resources" filed with supervisor FQDN or IP address followed by the generated HTML file list name (ex.https://fortisiem.com/src_ip.html):
   • Press the "OK" button and check if the newly created Threat Feed is listed in the "Connectors" section:
   • Configure appropriate Policy Rules using the newly created Fabric Connector List:
   • Use FortiManager Install Wizard to Install the Policy(during this process, Fortinet Security Fabric Exteral Connectors will be configured on FortiGate Firewalls):

Testing Options
You can test this new remediation method using inbuild FortiGate demo alerts and FortiSIEM inbuild Rules or (if more control is needed), you can use custom Logs/Alerts, Parsers, Rules and Notification Policies.

1). Ad-hoc remediation test using inbuild FortiGate demo alerts and inbuild FortiSIEM Rules

1. Configure FortiGate to send logs to FortiSIEM. Configuration guidelines are available at: https://docs.fortinet.com/document/fortisiem/6.4.0/external-systems-configuration-guide/751381/forti...
2. Access FortiGate CLI and run: #diagnose log test to generate test events
3. Connect to FortiSIEM “Incident tabs” and select Incident “Account Locked: Network Device”
4. Select Actions > Remediate Incident
5. Remediate Incident using src_ip.py publishing script
6. Check if the Incident Source IP address (1.1.1.1) is fetched by FortiGate External Connectors
   
   
   

2). Automatic remediation test based on sample logs and custom Parsers, Rules and Notification Policies

1. Edit "send_syslog.py" script and replace "192.168.0.25" IP address with your Supervisor/Collector IP address
2. Create a FortiSIEM custom parser (to pars the "syslog_msg.txt" logs) using the "TestEventParser_A.xml" file as a reference. More info about working with parser in FortiSIEM is available at: https://help.fortinet.com/fsiem/6-4-0/Online-Help/HTML5_Help/Configuring_parsers.htm
3. Create a new FortiSIEM Rule using as a reference the "Allerting_Rule_Event_Type_A.xml" file. More info about creating a custom Rule is available at https://help.fortinet.com/fsiem/6-4-0/Online-Help/HTML5_Help/Rules.htm
4. Create a new FortiSIEM Notification Policy based on previously created Rule and the needed remediation script (ex. src_ip.py). More info about creating a Notification Policy is available at: https://help.fortinet.com/fsiem/6-4-0/Online-Help/HTML5_Help/Notification_Settings.htm. 
5.  Connect to Supervisor/Collector or 3^rd party computer (the one used in step 1) and run the "python send_syslog.py" command to generate the testing syslog messages (and corresponding Incidents):
   • Check if the generated logs (based on fake IP address written in "syslog_msg.txt" file) are being properly received and parsed by FortiSIEM. You can import the Report_Event_Type_A.xml file and run it as a built-in search, using "Table" or "Link Graph" view options (more information about using FortiSIEM Analytics and Reports as a built-in search is available at https://help.fortinet.com/fsiem/6-4-0/Online-Help/HTML5_Help/Running_a_built-in_search.htm:(
     
   • Check if Incidents based on Type A Events are correctly reported in "Incidents" tab:
   • Check if Fortinet Security Fabric Connectors correctly receives the same IP address lists on all existing Firewalls. Connect on protected Workstations (behind Firewalls) and test (ex. using ping/traceroute/tracepath) if policy enforcement works according to your firewall rules.
   • Check if the IP address received by HQ FortiGate firewall (ex. HQ_FGT1 listed bellow) are identical to those used in "send_syslog.txt" file
   • Check if the IP address received by branch FortiGate firewall (ex. FCT_Branch_1 listed bellow) are identical to those used in "send_syslog.txt" file:
   • Check if the IP address received by the second branch FortiGate firewall (ex. FGT_Branch_2 listed bellow) are identical to those used in "send_syslog.txt" file:
   • 

Miscellaneous:

• Delays must be taken into consideration depending on FGT External Connectors refresh rate. Delays are measured from the moment FSM publishes the IP address list until FGT fetches it.
• HTTP basic authentication should be considered when configuring FortiGate External Connectors
• IP Lists expires could be implemented using an additional scheduled script (to automate deletion of IP address from the list)
• This method might be extended to use(publish) Hash and URL lists (not just IP address lists)

​

[DrWolfgangBeneicke1]

Great job! Thanks a lot.
How can I get the scripts? Currently, a login prompt for a FTNT internal site pops up if I click the .py files.

[SilvPufa]

Hello. Scripts and video presentation are also available at https://github.com/spufan/FSM_publishing_IP

[SilvPufa]

This integration method is now available/implemented by default in the FortiSIEM firmware (starting with 6.6.0 release) by publishing Watchlists. For more info please access the "Watchlist REST API for FortiGate Threat Feed and 3rd Party Devices" section available at  https://docs.fortinet.com/document/fortisiem/6.6.0/release-notes/709914/whats-new-in-6-6-0#New.

[LeandroSgallari]

is there any doc about how to use the watch lists in 6.6.0?

[SilvPufa]

Documentation about using Watch Lists in 6.6.0 is available at https://help.fortinet.com/fsiem/6-6-0/Online-Help/HTML5_Help/Using_a_watch_list.htm 

For 7.7.0 please access https://help.fortinet.com/fsiem/7-0-0/Online-Help/HTML5_Help/Watch_list.htm