Skip to main content
KarlH
Explorer II
October 9, 2024
Solved

Seeking some code review on parser xml code, failing testing. FortiSIEM 7.1

  • October 9, 2024
  • 2 replies
  • 4914 views
Hello, the below parser xml is failing testing on 7.1, any help is much appreciated. 

<eventFormatRecognizer
>

<![CDATA[.*Vendor _ATTACK\s+]]>
</eventFormatRecognizer>
<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[.*Vendor Name {<_body:gPatMesgBody>}]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="eventType">VendorAlert</setEventAttribute>
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="accountid" key="Account Id"/>
<attrKeyMap attr="VendorAttackModule" key="Attack Module"/>
</collectAndSetAttrByJSON>
</parsingInstructions>
Best answer by Rob_SIEM

Hi Karl,

 

The attached example parser worked for the full sample log you provided. 

 

The important part of this parser is this section:

  <!-- because our json is not ideally formed, you have to strip away the ["my value"] to achieve my value -->
  <setEventAttribute attr="msg">replaceStringByRegex($msg, ":\s*\"\[\\", ":")</setEventAttribute>
  <setEventAttribute attr="msg">replaceStringByRegex($msg, "\\\"\]\",", "\",")</setEventAttribute>
 
Remember that each of the values of the json object are strings, even though those strings are actually an array of json objects. So as a quick workaround we strip off the array designation.
 
"Account Id" : "[\"12345\"]" is a string, so we instead strip this down to "Account Id" : "12345" as a workaround. This isn't perfect however.
 
Ideally the format should have been:
"Account Id" : [ "account1", "account2" , account3" ]
instead of
"Account Id" : "[ \"account1\", \"account2\", \"account3\"] "
The restriction here is we treat entire value as one object instead of separate objects if there happens to be more than one. 
 
Thanks

2 replies

kcanalichio
New Member
October 9, 2024

With out seeing the error message you get when you run the test or the raw event , there is not much anyone will be able to do for you here

KarlH
KarlHAuthor
Explorer II
October 9, 2024
 

failed.png

 

You are right I should have been more complete in my reporting, Sadly the FortisSIEM  test tool knows it failed but neglects to share why.  Is there ever a time when the test tool actually tells one why it failed?  This is 7.1

kcanalichio
New Member
October 9, 2024

The means that the the event didn't match the parser so your event format recognizer didn't match. The used Parser collumn will be filled with what parser its using.  one of the two below should work depending on how specific it need to match

 

<![CDATA[.*_ATTACK\s+]]>

 

<![CDATA[.*MORPHISEC_ATTACK\s+]]>  

 

KarlH
KarlHAuthor
Explorer II
October 22, 2024

Hello,@R

 

When I tried to validate in FortiSIEM 7.1.3 it complained about line 62.   What version did the parser you so kindly created successfully validate in?

 

<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[<_year:gPatYear>-<_mon:gPatMon>-<_day:gPatDay>\s+<_time:gPatTime>\s+(?:<reptDevIpAddr:gPatIpAddr>|<reptDevName:gPatHostName>)\s+<eventSeverityCat:gPatWord>\s+VENDOR_ATTACK\s+<msg:gPatMesgBody>]]></regex>
</collectFieldsByRegex>

<setEventAttribute attr="deviceTime">toDateTime($_mon, $_day, $_year, $_time)</setEventAttribute>

<!-- Give your event types clear names like vendor-model-eventType e.g. Microsoft-Office-LoginFailure -->
<setEventAttribute attr="eventType">Vendor_Attack_Generic</setEventAttribute>
<setEventAttribute attr="eventSeverity">1</setEventAttribute>

<!-- because our json is not ideally formed, you have to strip away the ["my value"] to achieve my value -->
<setEventAttribute attr="msg">replaceStringByRegex($msg, ":\s*\"\[\\", ":")</setEventAttribute>
<setEventAttribute attr="msg">replaceStringByRegex($msg, "\\\"\]\",", "\",")</setEventAttribute>

<collectAndSetAttrByJSON src="$msg">
<attrKeyMap attr="accountId" key="Account Id"/>
<attrKeyMap attr="module" key="Attack Module"/>
<!-- 2024-10-08T14:46:49.769Z must be converted to epoch later in parser for deviceTime -->
<attrKeyMap attr="_eventTime" key="Attack Time"/>
<attrKeyMap attr="errorCode" key="Code Processed"/>
<attrKeyMap attr="command" key="Command Line"/>
<attrKeyMap attr="hostName" key="Computer Name"/>
<attrKeyMap attr="targetHashCode" key="File Hash"/>
<attrKeyMap attr="fileName" key="File Name"/>
<attrKeyMap attr="startModule" key="Last Module Loaded"/>
<attrKeyMap attr="funName" key="Last Stack FunctionCall"/>
<attrKeyMap attr="user" key="Logged In UserName"/>
<attrKeyMap attr="details" key="Message"/>
<attrKeyMap attr="version" key="VENDOR Version"/>
<attrKeyMap attr="parentCommand" key="Parent Process Command Line"/>
<attrKeyMap attr="hashCode" key="Process Signature"/>
<attrKeyMap attr="parentFileHashCode" key="Parent Signature"/>
<attrKeyMap attr="_protectorIp" key="Protector IP"/>
<attrKeyMap attr="tenantId" key="Tenant Id"/>
<attrKeyMap attr="virusId" key="Threat Id"/>
<attrKeyMap attr="description" key="Threat Description"/>
<attrKeyMap attr="threatCategory" key="Threat Module"/>
<attrKeyMap attr="virusName" key="Threat Name"/>
<attrKeyMap attr="_severity" key="Threat Severity"/>
<attrKeyMap attr="subtype" key="Threat Sub-Classification"/>
</collectAndSetAttrByJSON>

<when test="exist _eventTime">
<switch>
<case>
<!-- 2024-10-08T14:46:49.769Z must be converted to epoch later in parser for deviceTime -->
<collectFieldsByRegex src="$_eventTime">
<regex><![CDATA[<_year:gPatYear>-<_mon:gPatMon>-<_day:gPatDay>T<_time:gPatTime>\.\d+(?:<_tz:gPatTimeZone>)?]]></regex>
</collectFieldsByRegex>

<!-- Set Date Time -->
<choose>
<when test="exist _tz">
<setEventAttribute attr="eventTime">toDateTime($_mon, $_day, $_year, $_time, $_tz)</setEventAttribute>
</when>
<otherwise>
<setEventAttribute attr="eventTime">toDateTime($_mon, $_day, $_year, $_time)</setEventAttribute>
</otherwise>
</choose>
</case>
<default/>
</switch>
</when>

<when test="exist _severity">
<switch>
<case>
<!-- %!s(int=5) -->
<collectFieldsByRegex src="$_severity">
<regex><![CDATA[\(int=<eventSeverity:patSeverity>\)]]></regex>
</collectFieldsByRegex>
</case>
<default/>
</switch>
</when>


<when test="exist user">
<switch>
<case>
<!-- DOMAIN/user -->
<collectFieldsByRegex src="$user">
<regex><![CDATA[^\s*<domain:patExceptSlash>/<user:gPatStr>$]]></regex>
</collectFieldsByRegex>
</case>
<default/>
</switch>
</when>

<when test="exist _protectorIp">
<switch>
<case>
<collectFieldsByRegex src="$_protectorIp">
<regex><![CDATA[<hostIpAddr:gPatIpAddr>]]></regex>
</collectFieldsByRegex>
</case>
<default/>
</switch>
</when>

</parsingInstructions>