Skip to main content
Redbeard
Visitor III
June 6, 2025
Solved

Need help with a Custom Parser config

  • June 6, 2025
  • 3 replies
  • 919 views

The goal is split multiple JSON events sent in a batch into separate events. Going through the docs available to me, I've gotten to this - 

<patternDefinitions> <pattern name="patSplitEventsFromAkamai"><![CDATA[\{.*]]></pattern> </patternDefinitions> <eventFormatRecognizer><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,]]></eventFormatRecognizer> <parsingInstructions> <switch> <case> <!-- Individual event format, process like normal --> <collectFieldsByRegex src="$_rawmsg"> <regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,\[json\]=<_json:patSplitEventsFromAkamai>]]></regex> </collectFieldsByRegex> <when test="exist _json"> <collectAndSetAttrByJSON src="$json"> <attrKeyMap attr="format" key="format"/> <attrKeyMap attr="version" key="version"/> <attrKeyMap attr="eventTime" key="start"/> <attrKeyMap attr="_logId" key="id"/> </collectAndSetAttrByJSON> </when> <when test="exist _logId"> <setEventAttribute attr="eventType">Akamai-Split</setEventAttribute> </when> </case> <case> <!-- monolithic event format, split into individual events --> <collectFieldsByRegex src="$_rawmsg"> <regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,]]></regex> </collectFieldsByRegex> <collectFieldsByKeyValuePair kvsep=":" sep="," src="$_rawmsg"> <attrKeyMap attr="reptDevName" key="reptDevName"/> <attrKeyMap attr="json" key="_json"/> </collectFieldsByKeyValuePair> <!-- input json, path, header, trailer, isDropBool --> <setEventAttribute attr="_resultCount">splitJsonEvent($_json, "type", "[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=Akamai,[reptModel]=Cloud%20Control,Akamai_Individual_Event,json=", "", "true")</setEventAttribute> </case> </switch> </parsingInstructions>

 

I get past validation (its valid XML) but run into this when testing

community post.png

 

 
Best answer by lbahtarliev

Hi there,

You are trying to reinvent the wheel @Redbeard mate ;)

Stick to the default PHCustomJsonParser as a base for your parser.

And just take a look here:
Ingesting JSON Formatted Events Received via HTTP(S) POST 

Pay attention the the optional separator argument you can use in your POST request.

Just define it according to the manual to be 'newline' and you are all set ;)
I use this all the time. 

 

PS: If you have paid attention, your input is not a valid JSON according to the spec. It is two JSON objects, each on a single line. The splitJsonEvent is intended for the cases where you actually upload a valid json array of json objects.

 

PPS: And your parser is messed up. You are referring to $json, but you do not have it defined yet (in the beginning), and so on.

 

Hope this helps,

Lyuben

3 replies

Redbeard
RedbeardAuthor
Visitor III
June 6, 2025

Forgot to add a sample

[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=Akamai,[reptModel]=Cloud%20Control,[reptDevName]=Cloud%20Monitor,[json]={"type":"cloud_monitor","format":"default","version":"1.0","id":"a","start":"1747925675.971","cp":"b","message":{"proto":"https","protoVer":"1.1","status":"200","cliIP":"c","reqPort":"443","reqHost":"d","reqMethod":"GET","reqPath":"f","reqQuery":"g","reqCT":"application%2fjson","sslVer":"TLSv1.3","respCT":"application/json","respLen":"756","bytes":"756","UA":"h","fwdHost":"i"},"content":{"edgeIP":"j","akaReqId":"k","appRules":"l","warn":"m"}} {"type":"cloud_monitor","format":"default","version":"1.0","id":"a","start":"1747925678.198","cp":"c","message":{"proto":"https","protoVer":"1.1","status":"200","cliIP":"c","reqPort":"443","reqHost":"d","reqMethod":"POST","reqPath":"e","reqCT":"application%2fjson","reqLen":31,"sslVer":"TLSv1.3","respCT":"application/json","respLen":"632","bytes":"632","UA":"f","fwdHost":"g"},"content":{"edgeIP":"h","akaReqId":"i","appRules":"j","warn":""}}
lbahtarliev
New Member
June 7, 2025

Hi there,

You are trying to reinvent the wheel @Redbeard mate ;)

Stick to the default PHCustomJsonParser as a base for your parser.

And just take a look here:
Ingesting JSON Formatted Events Received via HTTP(S) POST 

Pay attention the the optional separator argument you can use in your POST request.

Just define it according to the manual to be 'newline' and you are all set ;)
I use this all the time. 

 

PS: If you have paid attention, your input is not a valid JSON according to the spec. It is two JSON objects, each on a single line. The splitJsonEvent is intended for the cases where you actually upload a valid json array of json objects.

 

PPS: And your parser is messed up. You are referring to $json, but you do not have it defined yet (in the beginning), and so on.

 

Hope this helps,

Lyuben

Redbeard
RedbeardAuthor
Visitor III
June 9, 2025

Thanks for the help. Don't know how I missed the separator.