Skip to main content
AlexDC
Explorer II
March 26, 2021
Question

FortiSIEM NginxParser Update

  • March 26, 2021
  • 2 replies
  • 790 views
we have found an issue with parsing of nginx logs. The built in parser labeled NginxParser has a hard coded parameter for GMT time that only uses a + value, this will in reality not parse half of the world nginx logs that may use a negative(-) value. sample fix is below. For context we are on version 6.1.1.

Parser name : NginxParser
BEFORE
Section off focus
<collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+\+\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>

AFTER    â€“ adding (?:\+|\-) to parse the time zone as either + or - from GMT time format , seems like only + was hard coded

<collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+(?:\+|\-)\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>

    2 replies

    FSM_FTNT
    Staff
    Staff
    March 29, 2021
    Hi Alex,

    Thanks very much for pointing this out.

    Do you have a sample event, where this was an issue?

    I can use that as a test and then look to improve the built-in parser.

    Thanks

    Dan

    ------------------------------
    Daniel
    FortiSIEM Product Manager
    ------------------------------
    -------------------------------------------
    Original Message:
    Sent: Mar 26, 2021 04:40 PM
    From: Alex D-C
    Subject: FortiSIEM NginxParser Update

    we have found an issue with parsing of nginx logs. The built in parser labeled NginxParser has a hard coded parameter for GMT time that only uses a + value, this will in reality not parse half of the world nginx logs that may use a negative(-) value. sample fix is below. For context we are on version 6.1.1.

    Parser name : NginxParser
    BEFORE
    Section off focus
    <collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+\+\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>

    AFTER    â€“ adding (?:\+|\-) to parse the time zone as either + or - from GMT time format , seems like only + was hard coded

    <collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+(?:\+|\-)\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>
    AlexDC
    AlexDCAuthor
    Explorer II
    March 29, 2021
    Hello Daniel,

    Thank you for your reply. I have engaged support ticket to share the logs in question and discuss update to parser for next release. 

    Regards;
    Alex-------------------------------------------
    Original Message:
    Sent: Mar 29, 2021 03:24 AM
    From: Daniel Hanman
    Subject: FortiSIEM NginxParser Update

    Hi Alex,

    Thanks very much for pointing this out.

    Do you have a sample event, where this was an issue?

    I can use that as a test and then look to improve the built-in parser.

    Thanks

    Dan

    ------------------------------
    Daniel
    FortiSIEM Product Manager
    ------------------------------

    Original Message:
    Sent: Mar 26, 2021 04:40 PM
    From: Alex D-C
    Subject: FortiSIEM NginxParser Update

    we have found an issue with parsing of nginx logs. The built in parser labeled NginxParser has a hard coded parameter for GMT time that only uses a + value, this will in reality not parse half of the world nginx logs that may use a negative(-) value. sample fix is below. For context we are on version 6.1.1.

    Parser name : NginxParser
    BEFORE
    Section off focus
    <collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+\+\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>

    AFTER    â€“ adding (?:\+|\-) to parse the time zone as either + or - from GMT time format , seems like only + was hard coded

    <collectFieldsByRegex src="$_body"> <regex><![CDATA[^<srcIpAddr:gPatIpAddr>\s+-\s+-\s+\[<:gPatDay>/<:gPatMon>/<:gPatYear>:<:gPatTime>\s+(?:\+|\-)\d+\]\s+"<httpMethod:gPatStr>\s+<uriStem:gPatStr>\s+HTTP/<httpVersion:patStrQuoted>"\s+<httpStatusCode:gPatStr>\s+<httpLimitRate:gPatStr>\s+"<httpReferrer:patStrQuoted>"\s+"<httpUserAgent:patStrQuoted>"\s*]]></regex> </collectFieldsByRegex>
    Thought Leadership Security Summit. Outpace New Threats with AI - enhanced defense. Tuesday, Septmeber 15, 8:30 AM - 2:30 PM PT. The Golf Club at Newcastle, WA.
    Virtual event | September 2026. SASE summit. The age of autonomous trust. Register here!