Hi all. I'm having issues with this JSON custom parser. I'm trying to test it but I receive an error without any details or parser used. I tried everything and I'm not sure where to go next.
Here is the parser:
<eventFormatRecognizer> <![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=jamf]]> </eventFormatRecognizer> <patternDefinitions> <pattern name="patLazyNestedJSON"> <![CDATA[({[\s\S]*?})]]> </pattern> <pattern name="patGreedyNestedJSON"> <![CDATA[({[\s\S]*})]]> </pattern> <pattern name="patEventType"> <![CDATA[\w+]]> </pattern> </patternDefinitions> <parsingInstructions> <!-- Get the raw message and json data objects --> <collectFieldsByRegex src="$_rawmsg"> <regex> <![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:<_body:gPatMesgBodyMin>,\[json\]=<_json:gPatMesgBodyMin>\s*$]]> </regex> </collectFieldsByRegex> <!-- parse out nested json objects --> <collectFieldsByRegex src="$_json"> <regex> <![CDATA["webhook":\s+<_webhook:patLazyNestedJSON>,\s+"event"]]> </regex> </collectFieldsByRegex> <collectFieldsByRegex src="$_json"> <regex> <![CDATA["event":\s+<_event:patGreedyNestedJSON>]]> </regex> </collectFieldsByRegex> <!-- Collect attributes in [] --> <collectAndSetAttrBySymbol sep=",[" src="$_body" symEnd="]=" symStart="["/> <setEventAttribute attr="eventType">"JAMF-CLOUD-JSON"</setEventAttribute> <collectAndSetAttrByJSON src="$_webhook"> <attrKeyMap attr="devName" key="name"/> </collectAndSetAttrByJSON> </parsingInstructions>
And here is the raw JSON file
[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=jamf,[reptModel]=Cloud,[reptDevName]=jamf,[reptDevIpAddr]=10.47.7.9,[json]={ { "webhook": { "id": 2, "name": "SIEM", "webhookEvent": "ComputerPolicyFinished", "eventTimestamp": 1628873569515 }, "event": { "policyId": 219, "successful": true, "computer": { "udid": "20FF5074-46DE-58D3-9702-546ADA164942", "deviceName": "FVFXV1TWHV2F", "model": "13-inch Retina MacBook Pro (Mid 2017)", "macAddress": "38:F9:D3:4D:52:AC", "alternateMacAddress": "46:00:B8:80:13:01", "serialNumber": "FVFXV1TWHV2F", "osVersion": "10.14.6", "osBuild": "18G8022", "userDirectoryID": "-1", "username": "jsmith", "realName": "John Smith", "emailAddress": "test@test.com", "phone": "", "position": "Sales Analyst", "department": "Sales", "building": "New York", "room": "", "ipAddress": "100.50.100.200", "reportedIpAddress": "192.168.1.158", "jssID": 989 } }, "event_type": "ComputerPolicyFinished" }
Note that when I test it I make sure to use a single line JSON.
Thanks for all the help and have a nice day
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Hi...
Hard to provide a full parser, but I can provide some tips here..
Assuming this was your test event..
{"webhook":{"id":2,"name":"SIEM","webhookEvent":"ComputerPolicyFinished","eventTimestamp":1628873569515},"event":{"policyId":219,"successful":true,"computer":{"udid":"20FF5074-46DE-58D3-9702-546ADA164942","deviceName":"FVFXV1TWHV2F","model":"13-inch Retina MacBook Pro (Mid 2017)","macAddress":"38:F9:D3:4D:52:AC","alternateMacAddress":"46:00:B8:80:13:01","serialNumber":"FVFXV1TWHV2F","osVersion":"10.14.6","osBuild":"18G8022","userDirectoryID":"-1","username":"jsmith","realName":"John Smith","emailAddress":"test@test.com","phone":"yyy","position":"Sales Analyst","department":"Sales","building":"New York","room":"yyy","ipAddress":"100.50.100.200","reportedIpAddress":"192.168.1.158","jssID":989}},"event_type":"ComputerPolicyFinished"}
Then the JSON Parser Function would be..
<collectAndSetAttrByJSON src="$_body"> <attrKeyMap attr="_id" key="webhook.id"/> <attrKeyMap attr="_name" key="webhook.name"/> <attrKeyMap attr="_webhookEvent" key="webhook.webhookEvent"/> <attrKeyMap attr="_eventTimestamp" key="webhook.eventTimestamp"/> <attrKeyMap attr="_policyId" key="event.policyId"/> <attrKeyMap attr="_successful" key="event.successful"/> <attrKeyMap attr="_udid" key="event.computer.udid"/> <attrKeyMap attr="_deviceName" key="event.computer.deviceName"/> <attrKeyMap attr="_model" key="event.computer.model"/> <attrKeyMap attr="_macAddress" key="event.computer.macAddress"/> <attrKeyMap attr="_alternateMacAddress" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="_serialNumber" key="event.computer.serialNumber"/> <attrKeyMap attr="_osVersion" key="event.computer.osVersion"/> <attrKeyMap attr="_osBuild" key="event.computer.osBuild"/> <attrKeyMap attr="_userDirectoryID" key="event.computer.userDirectoryID"/> <attrKeyMap attr="_username" key="event.computer.username"/> <attrKeyMap attr="_realName" key="event.computer.realName"/> <attrKeyMap attr="_emailAddress" key="event.computer.emailAddress"/> <attrKeyMap attr="_phone" key="event.computer.phone"/> <attrKeyMap attr="_position" key="event.computer.position"/> <attrKeyMap attr="_department" key="event.computer.department"/> <attrKeyMap attr="_building" key="event.computer.building"/> <attrKeyMap attr="_room" key="event.computer.room"/> <attrKeyMap attr="_ipAddress" key="event.computer.ipAddress"/> <attrKeyMap attr="_reportedIpAddress" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="_jssID" key="event.computer.jssID"/> <attrKeyMap attr="_event_type" key="event_type"/> </collectAndSetAttrByJSON>
Obviously, here all the extracts are set to variables which you can change for attributes as required.
Hope this helps.
regards
Chris Durkin
Hi Chris,
Thanks for the reply. I tried your solution but I still got an error. Unfortunately FortiSIEM
doesn't tell me what the error is and instead just highlights the test area red.
Thanks again for the help though!
Can you provide the full parser you tried?
What was the error in RED?
So based upon what I've seen above, this works... in a test fashion.
Note: You will need to determine the best event format recognizer to use and set the attributes you need.
Test Event {"webhook":{"id":2,"name":"SIEM","webhookEvent":"ComputerPolicyFinished","eventTimestamp":1628873569515},"event":{"policyId":219,"successful":true,"computer":{"udid":"20FF5074-46DE-58D3-9702-546ADA164942","deviceName":"FVFXV1TWHV2F","model":"13-inch Retina MacBook Pro (Mid 2017)","macAddress":"38:F9:D3:4D:52:AC","alternateMacAddress":"46:00:B8:80:13:01","serialNumber":"FVFXV1TWHV2F","osVersion":"10.14.6","osBuild":"18G8022","userDirectoryID":"-1","username":"jsmith","realName":"John Smith","emailAddress":"test@test.com","phone":"yyy","position":"Sales Analyst","department":"Sales","building":"New York","room":"yyy","ipAddress":"100.50.100.200","reportedIpAddress":"192.168.1.158","jssID":989}},"event_type":"ComputerPolicyFinished"}
<eventFormatRecognizer><![CDATA["name":"SIEM","webhookEvent":"ComputerPolicyFinished"]]></eventFormatRecognizer> <parsingInstructions> <collectFieldsByRegex src="$_rawmsg"> <regex><![CDATA[<_body:gPatMesgBody>]]></regex> </collectFieldsByRegex> <collectAndSetAttrByJSON src="$_body"> <attrKeyMap attr="string1" key="webhook.id"/> <attrKeyMap attr="string2" key="webhook.name"/> <attrKeyMap attr="string3" key="webhook.webhookEvent"/> <attrKeyMap attr="string4" key="webhook.eventTimestamp"/> <attrKeyMap attr="string5" key="event.policyId"/> <attrKeyMap attr="string6" key="event.successful"/> <attrKeyMap attr="string7" key="event.computer.udid"/> <attrKeyMap attr="string8" key="event.computer.deviceName"/> <attrKeyMap attr="string9" key="event.computer.model"/> <attrKeyMap attr="string10" key="event.computer.macAddress"/> <attrKeyMap attr="string11" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="string12" key="event.computer.serialNumber"/> <attrKeyMap attr="string13" key="event.computer.osVersion"/> <attrKeyMap attr="string14" key="event.computer.osBuild"/> <attrKeyMap attr="string15" key="event.computer.userDirectoryID"/> <attrKeyMap attr="string16" key="event.computer.username"/> <attrKeyMap attr="string17" key="event.computer.realName"/> <attrKeyMap attr="string18" key="event.computer.emailAddress"/> <attrKeyMap attr="string19" key="event.computer.phone"/> <attrKeyMap attr="string20" key="event.computer.position"/> <attrKeyMap attr="string21" key="event.computer.department"/> <attrKeyMap attr="string22" key="event.computer.building"/> <attrKeyMap attr="string23" key="event.computer.room"/> <attrKeyMap attr="string24" key="event.computer.ipAddress"/> <attrKeyMap attr="string25" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="string26" key="event.computer.jssID"/> <attrKeyMap attr="_event" key="event_type"/> </collectAndSetAttrByJSON> <setEventAttribute attr="eventType">combineMsgId("MyCustomApp-", $_event)</setEventAttribute> </parsingInstructions>
Thank you so much! I think the issue then might have been with the event format recognizer. I'll update when I can narrow down the error. I attached an image link of what the error was.
Thank you again!
Hi cdurkin,
Thanks for all your help so far. I've narrowed down the problem to the event format recognizer. Unfortunately your solution while it works doesn't work in mine. Apparently, the issue is spaces. I tried using regex but it doesn't work. Is there something I'm doing wrong? My fortisiem is version 6.2.1.
Custom Parser:
<eventFormatRecognizer> <![CDATA["name":\s"SIEM",\s"webhookEvent":\s"ComputerPolicyFinished"]]> </eventFormatRecognizer>
<parsingInstructions> <collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:<_body:gPatMesgBodyMin>,\[json\]=<_json:gPatMesgBodyMin>\s*$]]></regex> </collectFieldsByRegex>
<collectAndSetAttrByJSON src="$_json"> <attrKeyMap attr="_string1" key="webhook.id"/> <attrKeyMap attr="_string2" key="webhook.name"/> <attrKeyMap attr="_string3" key="webhook.webhookEvent"/> <attrKeyMap attr="_string4" key="webhook.eventTimestamp"/> <attrKeyMap attr="_string5" key="event.policyId"/> <attrKeyMap attr="_string6" key="event.successful"/> <attrKeyMap attr="_string7" key="event.computer.udid"/> <attrKeyMap attr="_string8" key="event.computer.deviceName"/> <attrKeyMap attr="_string9" key="event.computer.model"/> <attrKeyMap attr="_string10" key="event.computer.macAddress"/> <attrKeyMap attr="_string11" key="event.computer.alternateMacAddress"/> <attrKeyMap attr="_string12" key="event.computer.serialNumber"/> <attrKeyMap attr="_string13" key="event.computer.osVersion"/> <attrKeyMap attr="_string14" key="event.computer.osBuild"/> <attrKeyMap attr="_string15" key="event.computer.userDirectoryID"/> <attrKeyMap attr="_string16" key="event.computer.username"/> <attrKeyMap attr="_string17" key="event.computer.realName"/> <attrKeyMap attr="_string18" key="event.computer.emailAddress"/> <attrKeyMap attr="_string19" key="event.computer.phone"/> <attrKeyMap attr="_string20" key="event.computer.position"/> <attrKeyMap attr="_string21" key="event.computer.department"/> <attrKeyMap attr="_string22" key="event.computer.building"/> <attrKeyMap attr="_string23" key="event.computer.room"/> <attrKeyMap attr="_string24" key="event.computer.ipAddress"/> <attrKeyMap attr="_string25" key="event.computer.reportedIpAddress"/> <attrKeyMap attr="_string26" key="event.computer.jssID"/> <attrKeyMap attr="_event" key="event_type"/> </collectAndSetAttrByJSON>
<setEventAttribute attr="eventType">combineMsgId("jamf-Cloud-JSON")</setEventAttribute>
</parsingInstructions>
Raw Log:
[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=jamf,[reptModel]=Cloud,[reptDevName]=jamf,[reptDevIpAddr]=10.47.7.9,[json]={"webhook": {"id": 2, "name": "SIEM", "webhookEvent": "ComputerPolicyFinished", "eventTimestamp": 1628873569515}, "event": {"policyId": 219, "successful": true, "computer": {"udid": "20FF5074-46DE-58D3-9702-546ADA164942", "deviceName": "FVFXV1TWHV2F", "model": "13-inch Retina MacBook Pro (Mid 2017)", "macAddress": "38:F9:D3:4D:52:AC", "alternateMacAddress": "46:00:B8:80:13:01", "serialNumber": "FVFXV1TWHV2F", "osVersion": "10.14.6", "osBuild": "18G8022", "userDirectoryID": "-1", "username": "jsmith", "realName": "John Smith", "emailAddress": "test@test.com", "phone": "", "position": "Strategic Programs Analyst", "department": "Sales", "building": "New York", "room": "", "ipAddress": "100.50.150.200", "reportedIpAddress": "192.168.1.158", "jssID": 989}}, "event_type": "ComputerPolicyFinished"}
I tried to capture any kind of field possible but it doesn't work and keeps the same error I attached before.
Thanks again for all the help!!
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1703 | |
1092 | |
752 | |
446 | |
229 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.