Technical Tip: How to export Raw Event Logs from the command line interface

The tool to be used to export the logs will depend on the storage type:

• EventDB.
• Clickhouse.
• Elasticsearch.
  
  

The Raw Event Logs will be exported in CSV format.

If using EventDB, use the 'phExportEvent' tool to export events.

Usage:

phExportEvent --src <SourceDir> --dest <DestinationDir> --starttime <startTime> --endtime <endTime>
            --relstarttime <relativeStartTime> --relendtime <relativeEndTime>
            --dev <deviceName> --org <OrganizationName> --tzone TIME_ZONE

The following are valid time limits:

--starttime <StartTime> --endtime <EndTime>

--relstarttime <RelativeStartTime> --endtime <EndTime>

--starttime <StartTime> --relendtime <RelativeEndTime>

<time> : "YYYY-MM-DD hh:mm:ss[+-]TZ" (notice the quotation marks)

<relativeTime> : Num(d|h|m) d/h/m means days/hours/minutes

Example:

phExportEvent --src /data/eventdb --dest /home/admin --starttime "2010-10-18 11:00:00-7" --endtime "2010-10-18 12:00:00-7" --org Super --org customer111

(The -7 timezone is Pacific Daylight-savings Time (PDT) or GMT-7).

If using ClickHouse, use the 'phClickHouseCSVExport' tool to export events:

Usage:

phClickHouseCSVExport --starttime [Start Time] --endtime [End Time]

--outfile [Output file] --deviceip [Reporting IP] --devicename [Reporting Device Name]

--limit [specify the number of records to return, default - no limit] --debug [Dump SQL statement generated for this report]
--orgid [Organization ID (0 - 4294967295)] --orgname [Organization Name] --eventtype [Event Type]

A valid time limit is in the following format:

      [Time] : "YYYY-MM-DD hh:mm:ss
           

The supported timezone is GMT.

Example:

phClickHouseCSVExport --outfile /home/user/report.csv --starttime "2022-01-20 10:10:00" --endtime "2022-01-20 11:10:00"

If using Elasticsearch, use the 'phExportESEvent' tool:

Usage:

phExportESEvent <ESUrl> <ESPort> <ESDeploymentType> "<ESUser>" "<ESPassword>" <ESIndexName> <ReportingDevIp> <destDir> <splitThreads> <LogLevel>

Examples:

Native Elasticsearch Deployment Example:

phExportESEvent https://192.0.2.0 9200 1 "Joe.123--test" "password" fortisiem-event-2021.08.05-1-000001 "192.0.2.4" /archive/ 10 INFO

AWS Elasticsearch Service Deployment Example:

phExportESEvent https://search-eesna78-aaaa4ysukru3ui4ayaz2yya3km.us-east-1.es.amazonaws.com 443 2 "key" "secret" fortisiem-event-2021.09.29-1 "" /archive/ 10 INFO

Elasticsearch Cloud Deployment Example:

phExportESEvent https://cpaagg33-d11e01.es.us-central1.gcp.cloud.es.io 9243 3 "elastic" "password" fortisiem-event-2021.10.01-1-000001 "" /archive/ 10 INFO

Note:

• Events beginning with 'PH_DEV_MON' are FortiSIEM-generated events and thus will not be exported.

Related document:

pHExport2 - FortiSIEM Online Help