FortiSIEM Custom JSON Parser

[fortivandis]

Hi all. I'm having issues with this JSON custom parser. I'm trying to test it but I receive an error without any details or parser used. I tried everything and I'm not sure where to go next.

Here is the parser:

<eventFormatRecognizer>     <![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=jamf]]> </eventFormatRecognizer> <patternDefinitions>     <pattern name="patLazyNestedJSON">         <![CDATA[({[\s\S]*?})]]>     </pattern>     <pattern name="patGreedyNestedJSON">         <![CDATA[({[\s\S]*})]]>     </pattern>     <pattern name="patEventType">         <![CDATA[\w+]]>     </pattern> </patternDefinitions> <parsingInstructions>     <!-- Get the raw message and json data objects -->     <collectFieldsByRegex src="$_rawmsg">         <regex>             <![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:<_body:gPatMesgBodyMin>,\[json\]=<_json:gPatMesgBodyMin>\s*$]]>         </regex>     </collectFieldsByRegex>     <!-- parse out nested json objects -->     <collectFieldsByRegex src="$_json">         <regex>             <![CDATA["webhook":\s+<_webhook:patLazyNestedJSON>,\s+"event"]]>         </regex>     </collectFieldsByRegex>          <collectFieldsByRegex src="$_json">         <regex>             <![CDATA["event":\s+<_event:patGreedyNestedJSON>]]>         </regex>     </collectFieldsByRegex>     <!-- Collect attributes in [] -->     <collectAndSetAttrBySymbol sep=",[" src="$_body" symEnd="]=" symStart="["/>     <setEventAttribute attr="eventType">"JAMF-CLOUD-JSON"</setEventAttribute>          <collectAndSetAttrByJSON src="$_webhook">         <attrKeyMap attr="devName" key="name"/>     </collectAndSetAttrByJSON> </parsingInstructions>

And here is the raw JSON file

[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=jamf,[reptModel]=Cloud,[reptDevName]=jamf,[reptDevIpAddr]=10.47.7.9,[json]={ {   "webhook": {     "id": 2,     "name": "SIEM",     "webhookEvent": "ComputerPolicyFinished",     "eventTimestamp": 1628873569515   },   "event": {     "policyId": 219,     "successful": true,     "computer": {       "udid": "20FF5074-46DE-58D3-9702-546ADA164942",       "deviceName": "FVFXV1TWHV2F",       "model": "13-inch Retina MacBook Pro (Mid 2017)",       "macAddress": "38:F9:D3:4D:52:AC",       "alternateMacAddress": "46:00:B8:80:13:01",       "serialNumber": "FVFXV1TWHV2F",       "osVersion": "10.14.6",       "osBuild": "18G8022",       "userDirectoryID": "-1",       "username": "jsmith",       "realName": "John Smith",       "emailAddress": "test@test.com",       "phone": "",       "position": "Sales Analyst",       "department": "Sales",       "building": "New York",       "room": "",       "ipAddress": "100.50.100.200",       "reportedIpAddress": "192.168.1.158",       "jssID": 989     }   },   "event_type": "ComputerPolicyFinished" }

Note that when I test it I make sure to use a single line JSON.

Thanks for all the help and have a nice day

[cdurkin_FTNT]

Hi...

Hard to provide a full parser, but I can provide some tips here..

Assuming this was your test event..

{"webhook":{"id":2,"name":"SIEM","webhookEvent":"ComputerPolicyFinished","eventTimestamp":1628873569515},"event":{"policyId":219,"successful":true,"computer":{"udid":"20FF5074-46DE-58D3-9702-546ADA164942","deviceName":"FVFXV1TWHV2F","model":"13-inch Retina MacBook Pro (Mid 2017)","macAddress":"38:F9:D3:4D:52:AC","alternateMacAddress":"46:00:B8:80:13:01","serialNumber":"FVFXV1TWHV2F","osVersion":"10.14.6","osBuild":"18G8022","userDirectoryID":"-1","username":"jsmith","realName":"John Smith","emailAddress":"test@test.com","phone":"yyy","position":"Sales Analyst","department":"Sales","building":"New York","room":"yyy","ipAddress":"100.50.100.200","reportedIpAddress":"192.168.1.158","jssID":989}},"event_type":"ComputerPolicyFinished"}

Then the JSON Parser Function would be..

<collectAndSetAttrByJSON src="$_body"> <attrKeyMap attr="_id" key="webhook.id"/> <attrKeyMap attr="_name" key="webhook.name"/> <attrKeyMap attr="_webhookEvent" key="webhook.webhookEvent"/> <attrKeyMap attr="_eventTimestamp" key="webhook.eventTimestamp"/> <attrKeyMap attr="_policyId" key="event.policyId"/> <attrKeyMap attr="_successful" key="event.successful"/> <attrKeyMap attr="_udid" key="event.computer.udid"/> <attrKeyMap attr="_deviceName" key="event.computer.deviceName"/> <attrKeyMap attr="_model" key="event.computer.model"/> <attrKeyMap attr="_macAddress" key="event.computer.macAddress"/> <attrKeyMap attr="_alternateMacAddress" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="_serialNumber" key="event.computer.serialNumber"/> <attrKeyMap attr="_osVersion" key="event.computer.osVersion"/> <attrKeyMap attr="_osBuild" key="event.computer.osBuild"/> <attrKeyMap attr="_userDirectoryID" key="event.computer.userDirectoryID"/> <attrKeyMap attr="_username" key="event.computer.username"/> <attrKeyMap attr="_realName" key="event.computer.realName"/> <attrKeyMap attr="_emailAddress" key="event.computer.emailAddress"/> <attrKeyMap attr="_phone" key="event.computer.phone"/> <attrKeyMap attr="_position" key="event.computer.position"/> <attrKeyMap attr="_department" key="event.computer.department"/> <attrKeyMap attr="_building" key="event.computer.building"/> <attrKeyMap attr="_room" key="event.computer.room"/> <attrKeyMap attr="_ipAddress" key="event.computer.ipAddress"/> <attrKeyMap attr="_reportedIpAddress" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="_jssID" key="event.computer.jssID"/> <attrKeyMap attr="_event_type" key="event_type"/> </collectAndSetAttrByJSON>

Obviously, here all the extracts are set to variables which you can change for attributes as required.

Hope this helps.

regards

Chris Durkin

[fortivandis]

Hi Chris,

Thanks for the reply. I tried your solution but I still got an error. Unfortunately FortiSIEM

doesn't tell me what the error is and instead just highlights the test area red.

Thanks again for the help though!

[cdurkin_FTNT]

Can you provide the full parser you tried?

What was the error in RED?

[cdurkin_FTNT]

So based upon what I've seen above, this works... in a test fashion.

Note: You will need to determine the best event format recognizer to use and set the attributes you need.

Test Event {"webhook":{"id":2,"name":"SIEM","webhookEvent":"ComputerPolicyFinished","eventTimestamp":1628873569515},"event":{"policyId":219,"successful":true,"computer":{"udid":"20FF5074-46DE-58D3-9702-546ADA164942","deviceName":"FVFXV1TWHV2F","model":"13-inch Retina MacBook Pro (Mid 2017)","macAddress":"38:F9:D3:4D:52:AC","alternateMacAddress":"46:00:B8:80:13:01","serialNumber":"FVFXV1TWHV2F","osVersion":"10.14.6","osBuild":"18G8022","userDirectoryID":"-1","username":"jsmith","realName":"John Smith","emailAddress":"test@test.com","phone":"yyy","position":"Sales Analyst","department":"Sales","building":"New York","room":"yyy","ipAddress":"100.50.100.200","reportedIpAddress":"192.168.1.158","jssID":989}},"event_type":"ComputerPolicyFinished"}

<eventFormatRecognizer><![CDATA["name":"SIEM","webhookEvent":"ComputerPolicyFinished"]]></eventFormatRecognizer> <parsingInstructions> <collectFieldsByRegex src="$_rawmsg"> <regex><![CDATA[<_body:gPatMesgBody>]]></regex> </collectFieldsByRegex> <collectAndSetAttrByJSON src="$_body"> <attrKeyMap attr="string1" key="webhook.id"/> <attrKeyMap attr="string2" key="webhook.name"/> <attrKeyMap attr="string3" key="webhook.webhookEvent"/> <attrKeyMap attr="string4" key="webhook.eventTimestamp"/> <attrKeyMap attr="string5" key="event.policyId"/> <attrKeyMap attr="string6" key="event.successful"/> <attrKeyMap attr="string7" key="event.computer.udid"/> <attrKeyMap attr="string8" key="event.computer.deviceName"/> <attrKeyMap attr="string9" key="event.computer.model"/> <attrKeyMap attr="string10" key="event.computer.macAddress"/> <attrKeyMap attr="string11" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="string12" key="event.computer.serialNumber"/> <attrKeyMap attr="string13" key="event.computer.osVersion"/> <attrKeyMap attr="string14" key="event.computer.osBuild"/> <attrKeyMap attr="string15" key="event.computer.userDirectoryID"/> <attrKeyMap attr="string16" key="event.computer.username"/> <attrKeyMap attr="string17" key="event.computer.realName"/> <attrKeyMap attr="string18" key="event.computer.emailAddress"/> <attrKeyMap attr="string19" key="event.computer.phone"/> <attrKeyMap attr="string20" key="event.computer.position"/> <attrKeyMap attr="string21" key="event.computer.department"/> <attrKeyMap attr="string22" key="event.computer.building"/> <attrKeyMap attr="string23" key="event.computer.room"/> <attrKeyMap attr="string24" key="event.computer.ipAddress"/> <attrKeyMap attr="string25" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="string26" key="event.computer.jssID"/> <attrKeyMap attr="_event" key="event_type"/> </collectAndSetAttrByJSON> <setEventAttribute attr="eventType">combineMsgId("MyCustomApp-", $_event)</setEventAttribute> </parsingInstructions>

[fortivandis]

Thank you so much! I think the issue then might have been with the event format recognizer. I'll update when I can narrow down the error. I attached an image link of what the error was.

Thank you again!

[fortivandis]

Hi cdurkin,

Thanks for all your help so far. I've narrowed down the problem to the event format recognizer. Unfortunately your solution while it works doesn't work in mine. Apparently, the issue is spaces. I tried using regex but it doesn't work. Is there something I'm doing wrong? My fortisiem is version 6.2.1.

Custom Parser:

<eventFormatRecognizer> <![CDATA["name":\s"SIEM",\s"webhookEvent":\s"ComputerPolicyFinished"]]> </eventFormatRecognizer>

<parsingInstructions> <collectFieldsByRegex src="$_rawmsg">

<regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:<_body:gPatMesgBodyMin>,\[json\]=<_json:gPatMesgBodyMin>\s*$]]></regex> </collectFieldsByRegex>

<collectAndSetAttrByJSON src="$_json"> <attrKeyMap attr="_string1" key="webhook.id"/> <attrKeyMap attr="_string2" key="webhook.name"/> <attrKeyMap attr="_string3" key="webhook.webhookEvent"/> <attrKeyMap attr="_string4" key="webhook.eventTimestamp"/> <attrKeyMap attr="_string5" key="event.policyId"/> <attrKeyMap attr="_string6" key="event.successful"/> <attrKeyMap attr="_string7" key="event.computer.udid"/> <attrKeyMap attr="_string8" key="event.computer.deviceName"/> <attrKeyMap attr="_string9" key="event.computer.model"/> <attrKeyMap attr="_string10" key="event.computer.macAddress"/> <attrKeyMap attr="_string11" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="_string12" key="event.computer.serialNumber"/> <attrKeyMap attr="_string13" key="event.computer.osVersion"/> <attrKeyMap attr="_string14" key="event.computer.osBuild"/> <attrKeyMap attr="_string15" key="event.computer.userDirectoryID"/> <attrKeyMap attr="_string16" key="event.computer.username"/> <attrKeyMap attr="_string17" key="event.computer.realName"/> <attrKeyMap attr="_string18" key="event.computer.emailAddress"/> <attrKeyMap attr="_string19" key="event.computer.phone"/> <attrKeyMap attr="_string20" key="event.computer.position"/> <attrKeyMap attr="_string21" key="event.computer.department"/> <attrKeyMap attr="_string22" key="event.computer.building"/> <attrKeyMap attr="_string23" key="event.computer.room"/> <attrKeyMap attr="_string24" key="event.computer.ipAddress"/> <attrKeyMap attr="_string25" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="_string26" key="event.computer.jssID"/> <attrKeyMap attr="_event" key="event_type"/> </collectAndSetAttrByJSON>

<setEventAttribute attr="eventType">combineMsgId("jamf-Cloud-JSON")</setEventAttribute>

</parsingInstructions>

Raw Log:

[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=jamf,[reptModel]=Cloud,[reptDevName]=jamf,[reptDevIpAddr]=10.47.7.9,[json]={"webhook": {"id": 2, "name": "SIEM", "webhookEvent": "ComputerPolicyFinished", "eventTimestamp": 1628873569515}, "event": {"policyId": 219, "successful": true, "computer": {"udid": "20FF5074-46DE-58D3-9702-546ADA164942", "deviceName": "FVFXV1TWHV2F", "model": "13-inch Retina MacBook Pro (Mid 2017)", "macAddress": "38:F9:D3:4D:52:AC", "alternateMacAddress": "46:00:B8:80:13:01", "serialNumber": "FVFXV1TWHV2F", "osVersion": "10.14.6", "osBuild": "18G8022", "userDirectoryID": "-1", "username": "jsmith", "realName": "John Smith", "emailAddress": "test@test.com", "phone": "", "position": "Strategic Programs Analyst", "department": "Sales", "building": "New York", "room": "", "ipAddress": "100.50.150.200", "reportedIpAddress": "192.168.1.158", "jssID": 989}}, "event_type": "ComputerPolicyFinished"}

I tried to capture any kind of field possible but it doesn't work and keeps the same error I attached before.

Thanks again for all the help!!