Ingest Advisories from PDF, Excel, CSV files

[Parag]

Summary - Most of the Financial Institutes, Insurance Companies, Government departments, etc receive advisories from TIPs and various organisations.  Most advisories contain - IPs, Domain names, Hash, URL, etc. Advisories are generally delivered via email having attachments in PDF, Excel or CSV format. Also the advisories would be defanged example:

• Brackets are added to domain names; for example, www.example.com is replaced with www[.]example[.]com
• Brackets are added to the IP address; for example, 8.8.8.8 is replaced with 8[.]8[.]8[.]8

Automation Use Case

• Monitor dedicated email box for new advisories and ingest new email into FortiSOAR - default data ingestion playbook for exchange
• Extract pdf, excel or csv file attached within the email
• Identify file type - pdf, excel or csv
• Read the file - number of pages, lines, etc
• Extract all the indicators within the file - IP, hash, domains, URL, etc
• Refang the indicators 8[.]8[.]8[.]8 --> 8.8.8.8
• Ingest advisories into FortiSOAR indicators module and run enrichment playbook from IR content pack
• Send email to user with complied report for indicators ingested
• Optional - most clients will ask to push these indicators to firewall, EDR, etc. as a part of automation

Pre-requisite

• Go to linux cmd and login to root
• Install following
  1. pip install tika pycountry
  2. Go to settings --> Modules --> Indicators
  3. Create a New field --> "filePreview" with "RichHTML"
      please check video for step 2 and 3 in the link below
      https://community.fortinet.com/t5/FortiSOAR/Ingest-Advisories-from-PDF-Excel-CSV-files/ta-p/220045
  
  

Process Flow

a703ea9a7303494e8f644fa39850ef07.png

Notes

1. Download the playbook and import them into FortiSOAR playbook module
2. Use default data ingestion playbook for Exchange connector (OOB)
3. Use default playbook for enrichment from IR content pack (OOB)
4. The extraction playbook is on-create and will identify newly ingested email with attachment having a file - pdf, excel and csv. The playbook will run only on this condition
5. Optional - you can modify the playbook initiation trigger from comments, incidents, manual triggers etc. 

Important
** Change email address all the playbook on "Exchange" step
** Configure exchange connector (this will also work with SMTP or gsuite connector) as well.