FortiSIEM: Parsing key value pairs with multiple strings as a value

[pradarsha]

Dears,

for the below type of log. a key-value pair parser is used. the separator is " " (space) 

in this case w.r.t key log_component, its value string contains space in between, the parser will read only one word (i.e Local) and ignore the remaining(i.e  ACLs). please suggest how we overcome this situation and make the entire string captured in this case.

**The log and the sample parser are as follows.

------------------------------Log----------------------------------------------------

date=2007-12-19 time=15:34:00 timezone="IST" device_name="CR500i" device_id=C010600411-YFK5RL deployment_mode="Route" log_id=010301602001 log_type="Firewall" log_component="Local ACLs" log_subtype="Allowed" priority=Information duration=0 fw_rule_id=”” user_name="" user_gp="" iap=0 application="" application_id=”” in_interface="Port A" out_interface="" src_ip=192.168.15.240 dst_ip=192.168.13.25 protocol="ICMP" icmp_type=8 icmp_code=0 sent_pkts=1 recv_pkts=0 sent_bytes=84 recv_bytes=0 tran_src_ip=0.0.0.0 tran_src_port=0 tran_dst_ip=0.0.0.0 tran_dst_port=0 srczonetype=”LAN” dstzonetype=”Local” dir_disp=”org” connevent=”start” connid=”1425” vconnid=””

-----------------------------------Parser----------------------------------------------

<eventFormatRecognizer><![CDATA[date=<:gPatYear>-<:gPatMon>-<:gPatDay>\s+time=<_time:gPatTime>\s+timezone=<_tz:gPatStr>\s+device_name="CR[^"]*"\s*]]></eventFormatRecognizer>
<parsingInstructions>
<setEventAttribute attr="eventType">CYBERROAM</setEventAttribute>
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[<_body:gPatMesgBody>]]></regex>
</collectFieldsByRegex>
<collectAndSetAttrByKeyValuePair sep=" " src="$_body">
<attrKeyMap attr="_date" key="date="/>
<attrKeyMap attr="reptDevName" key="device_name="/>
<attrKeyMap attr="devId" key="device_id="/>
<attrKeyMap attr="_time" key="time="/>
<attrKeyMap attr="compEventName" key="log_component="/>
<attrKeyMap attr="action" key="log_subtype="/>
<attrKeyMap attr="eventSeverity" key="priority="/>
<attrKeyMap attr="srcIpAddr" key="src_ip="/>
<attrKeyMap attr="destIpAddr" key="dst_ip="/>
<attrKeyMap attr="_proto" key="protocol="/>
<attrKeyMap attr="srcIpPort" key="src_port="/>
<attrKeyMap attr="destIpPort" key="dst_port="/>
<attrKeyMap attr="srcIpPort" key="src_port="/>
</collectAndSetAttrByKeyValuePair>
<collectFieldsByRegex src="$_date">
<regex><![CDATA[<_year1:gPatYear>\-<_mon1:gPatMonNum>\-<_day1:gPatDay>]]></regex>
</collectFieldsByRegex>
<collectFieldsByRegex src="$_time">
<regex><![CDATA[<_time1:gPatTime>]]></regex>
</collectFieldsByRegex>
<setEventAttribute attr="deviceTime">toDateTime($_mon1, $_day1, $_year1, $_time1)</setEventAttribute>
</parsingInstructions>

FortiSIEM 

@Nishtha_Baria 

[Secusaurus]

Hello pradarsha,

The first two things that come to my mind are:

- Either generate a list (RegEx: (\w+=(?:"[^"]*"|[^"\s]+))) and go to this list manually, but I don't really find a "for" or "while" statement in the documentation.

- Or replace the spaces outside quotes with a character you never receive in this log (e.g. "§") and use this one as delimiter (Method: replaceStrInStr, RegEx: \s(?=(?:[^"]*"[^"]*")*[^"]*$))

Best,

Christian

[cdurkin_FTNT]

Hello

I tried your parser above and it actually worked perfectly as is... what version are you testing on?

Two things to note..

1) There is already an out the box Cyberoam Parser in FortiSIEM, check your version to see if it exists.

2) There are two options for the "Collect And Set By Key Value Pair" parser function, you can try both and see which gives the best result.

<collectAndSetAttrByKeyValuePair sep=" " src="$_body">

and

<collectFieldsByKeyValuePair kvsep="=" sep=" " src="$_body">