How to avoid parsing fields = '0'?

[TonyC]

Hello there,

We are getting events from Crowdstrike into FortiSIEM and no many fields are been parsed. I am working in adding the additional fields but I would like to hide when fields ='0'.
Because so many fields are coming over it doesn't make sense to show all of them, I want to show only the fields who are different to 0.


This is an example of Crowdstrike parser:

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer>
<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute>
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="_evtType" key="ExternalApiType"/>
<attrKeyMap attr="allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
</collectAndSetAttrByJSON>



Sample raw log event I want to parse:
2022-01-24 15:30:26 [Falcon-data-replicator] [1] [sqs.us-west-2.amazonaws.com]:{"AllocateVirtualMemoryCount":"0","ArchiveFileWrittenCount":"0","

On this case if I do not want to see the "allocateVirtualMemoryCount" field under Event details because it is = 0 how could I do it within the parser?

Thanks in advance. 

[KenMickeletto1]

Hi Tony,

The best way to handle this is to capture the field into a temporary variable and then only set it to a permanent variable if the value is a desired value.

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer>
<parsingInstructions>
 <collectFieldsByRegex src="$_rawmsg">
  <regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex>
 </collectFieldsByRegex>
 <setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute>
 <collectAndSetAttrByJSON src="$_body">
  <attrKeyMap attr="_evtType" key="ExternalApiType"/>
  <attrKeyMap attr="_allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
 </collectAndSetAttrByJSON>
 <when test="$_allocateVirtualMemoryCount != '0'">
  <setEventAttribute attr="allocateVirtualMemoryCount">$_allocateVirtualMemoryCount</setEventAttribute>
 </when>

Note the underscore on the variable name.  This is a temp variable and we only store it into the real allocateVirtualMemoryCount variable if it does not equal 0.

I hope this helps!

------------------------------
Ken
------------------------------
-------------------------------------------
Original Message:
Sent: Jan 25, 2022 06:07 AM
From: Tony C
Subject: How to avoid parsing fields = '0'?

Hello there,

We are getting events from Crowdstrike into FortiSIEM and no many fields are been parsed. I am working in adding the additional fields but I would like to hide when fields ='0'.
Because so many fields are coming over it doesn't make sense to show all of them, I want to show only the fields who are different to 0.


This is an example of Crowdstrike parser:

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer>
<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute>
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="_evtType" key="ExternalApiType"/>
<attrKeyMap attr="allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
</collectAndSetAttrByJSON>



Sample raw log event I want to parse:
2022-01-24 15:30:26 [Falcon-data-replicator] [1] [sqs.us-west-2.amazonaws.com]:{"AllocateVirtualMemoryCount":"0","ArchiveFileWrittenCount":"0","

On this case if I do not want to see the "allocateVirtualMemoryCount" field under Event details because it is = 0 how could I do it within the parser?

Thanks in advance. 

[KarnGriffen]

I just came to recommend the same.  :)-------------------------------------------
Original Message:
Sent: Jan 26, 2022 09:42 AM
From: Ken Mickeletto
Subject: How to avoid parsing fields = '0'?

Hi Tony,

The best way to handle this is to capture the field into a temporary variable and then only set it to a permanent variable if the value is a desired value.

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer><parsingInstructions> <collectFieldsByRegex src="$_rawmsg">  <regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex> </collectFieldsByRegex> <setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute> <collectAndSetAttrByJSON src="$_body">  <attrKeyMap attr="_evtType" key="ExternalApiType"/>  <attrKeyMap attr="_allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/> </collectAndSetAttrByJSON> <when test="$_allocateVirtualMemoryCount != '0'">  <setEventAttribute attr="allocateVirtualMemoryCount">$_allocateVirtualMemoryCount</setEventAttribute> </when>

Note the underscore on the variable name.  This is a temp variable and we only store it into the real allocateVirtualMemoryCount variable if it does not equal 0.

I hope this helps!

------------------------------
Ken
------------------------------

Original Message:
Sent: Jan 25, 2022 06:07 AM
From: Tony C
Subject: How to avoid parsing fields = '0'?

Hello there,

We are getting events from Crowdstrike into FortiSIEM and no many fields are been parsed. I am working in adding the additional fields but I would like to hide when fields ='0'.
Because so many fields are coming over it doesn't make sense to show all of them, I want to show only the fields who are different to 0.


This is an example of Crowdstrike parser:

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer>
<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute>
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="_evtType" key="ExternalApiType"/>
<attrKeyMap attr="allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
</collectAndSetAttrByJSON>



Sample raw log event I want to parse:
2022-01-24 15:30:26 [Falcon-data-replicator] [1] [sqs.us-west-2.amazonaws.com]:{"AllocateVirtualMemoryCount":"0","ArchiveFileWrittenCount":"0","

On this case if I do not want to see the "allocateVirtualMemoryCount" field under Event details because it is = 0 how could I do it within the parser?

Thanks in advance. 

[TonyC]

Thank you so much Ken, I appreciate it a lot! This is great, It works for me!

Crowdstrike Falcon Data Replicator contains over 200 events and each events approximately 83 fields. There are a lot of "Count" fields so a lot of them are ='0'. See below some of them.

Could you think of anything where I could apply the same idea ( = '0' ) for all the fields for all events types so I don't have to entered those lines for every field within the parser?

Example

<collectAndSetAttrByJSON src="$_body">
1- <attrKeyMap attr="_allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
2- <attrKeyMap attr="_archiveFileWrittenCount" key="ArchiveFileWrittenCount"/>
3- <attrKeyMap attr="_asepWrittenCount" key="AsepWrittenCount"/>
</collectAndSetAttrByJSON>


1- <when test="$_allocateVirtualMemoryCount != '0'">
<setEventAttribute attr="allocateVirtualMemoryCount">$_allocateVirtualMemoryCount</setEventAttribute>
</when>

2- <when test="$_aarchiveFileWrittenCount != '0'">
<setEventAttribute attr="archiveFileWrittenCount">$_archiveFileWrittenCount</setEventAttribute>
</when>

3-<when test="$_asepWrittenCount != '0'">
<setEventAttribute attr="asepWrittenCount">$_asepWrittenCount</setEventAttribute>
</when>



"AllocateVirtualMemoryCount"
"ArchiveFileWrittenCount"
"AsepWrittenCount"
"BinaryExecutableWrittenCount"
"CLICreationCount"
"ConHostId"
"ConHostProcessId"
"ConfigBuild"
"ConfigStateHash"
"ContextData"
"ContextProcessId"
"ContextThreadId"
"ContextTimeStamp"
"CreateProcessCount"
"CycleTime"
"DirectoryCreatedCount"
"DirectoryEnumeratedCount"
"DnsRequestCount"
"DocumentFileWrittenCount"
"EffectiveTransmissionClass"
"Entitlements"
"ExeAndServiceCount"
"ExecutableDeletedCount"
"ExitCode"
"FileDeletedCount"
"GenericFileWrittenCount"
"ImageSubsystem"
"InjectedDllCount"
"InjectedThreadCount"
"KernelTime"
"MaxThreadCount"
"ModuleLoadCount"
"NetworkBindCount"
"NetworkCapableAsepWriteCount"
"NetworkCloseCount"
"NetworkConnectCount"
"NetworkConnectCountUdp"
"NetworkListenCount"
"NetworkModuleLoadCount"
"NetworkRecvAcceptCount"
"NewExecutableWrittenCount"
"ParentProcessId":
"PrivilegedProcessHandleCount"
"ProcessStartTime"
"ProtectVirtualMemoryCount"
"QueueApcCount"
"RawProcessId"
"RegKeySecurityDecreasedCount"
"RemovableDiskFileWrittenCount"
"RunDllInvocationCount"
"SHA256HashData"
"ScreenshotsTakenCount"
"ScriptEngineInvocationCount"
"ServiceEventCount"
"SetThreadContextCount"
"SnapshotFileOpenCount"
"SuspectStackCount"
"SuspiciousCredentialModuleLoadCount"
"SuspiciousDnsRequestCount"
"SuspiciousFontLoadCount"
"SuspiciousRawDiskReadCount"
"TargetProcessId"
"UTCTimestamp"
"UnsignedModuleLoadCount"
"UserMemoryAllocateExecutableCount"
"UserMemoryAllocateExecutableRemoteCount"
"UserMemoryProtectExecutableCount"
"UserMemoryProtectExecutableRemoteCount"
"UserSid"
"UserTime"
-------------------------------------------
Original Message:
Sent: Jan 26, 2022 09:42 AM
From: Ken Mickeletto
Subject: How to avoid parsing fields = '0'?

Hi Tony,

The best way to handle this is to capture the field into a temporary variable and then only set it to a permanent variable if the value is a desired value.

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer><parsingInstructions> <collectFieldsByRegex src="$_rawmsg">  <regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex> </collectFieldsByRegex> <setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute> <collectAndSetAttrByJSON src="$_body">  <attrKeyMap attr="_evtType" key="ExternalApiType"/>  <attrKeyMap attr="_allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/> </collectAndSetAttrByJSON> <when test="$_allocateVirtualMemoryCount != '0'">  <setEventAttribute attr="allocateVirtualMemoryCount">$_allocateVirtualMemoryCount</setEventAttribute> </when>

Note the underscore on the variable name.  This is a temp variable and we only store it into the real allocateVirtualMemoryCount variable if it does not equal 0.

I hope this helps!

------------------------------
Ken
------------------------------

Original Message:
Sent: Jan 25, 2022 06:07 AM
From: Tony C
Subject: How to avoid parsing fields = '0'?

Hello there,

We are getting events from Crowdstrike into FortiSIEM and no many fields are been parsed. I am working in adding the additional fields but I would like to hide when fields ='0'.
Because so many fields are coming over it doesn't make sense to show all of them, I want to show only the fields who are different to 0.


This is an example of Crowdstrike parser:

<eventFormatRecognizer><![CDATA[\[Falcon-data-replicator\]]]></eventFormatRecognizer>
<parsingInstructions>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[Falcon-data-replicator\]\s+\[<phCustId:gPatInt>\]\s+\[<relayDevName:gPatMesgBodyMin>\]:<_body:gPatMesgBody>]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="eventType">Crowdstrike-FDR-Generic</setEventAttribute>
<collectAndSetAttrByJSON src="$_body">
<attrKeyMap attr="_evtType" key="ExternalApiType"/>
<attrKeyMap attr="allocateVirtualMemoryCount" key="AllocateVirtualMemoryCount"/>
</collectAndSetAttrByJSON>



Sample raw log event I want to parse:
2022-01-24 15:30:26 [Falcon-data-replicator] [1] [sqs.us-west-2.amazonaws.com]:{"AllocateVirtualMemoryCount":"0","ArchiveFileWrittenCount":"0","

On this case if I do not want to see the "allocateVirtualMemoryCount" field under Event details because it is = 0 how could I do it within the parser?

Thanks in advance.