Need help with a Custom Parser config

[Redbeard]

The goal is split multiple JSON events sent in a batch into separate events. Going through the docs available to me, I've gotten to this - 

<patternDefinitions>
<pattern name="patSplitEventsFromAkamai"><![CDATA[\{.*]]></pattern>
</patternDefinitions>
<eventFormatRecognizer><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,]]></eventFormatRecognizer>
<parsingInstructions>
<switch>
<case>
<!-- Individual event format, process like normal -->
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,\[json\]=<_json:patSplitEventsFromAkamai>]]></regex>
</collectFieldsByRegex>
<when test="exist _json">
<collectAndSetAttrByJSON src="$json">
<attrKeyMap attr="format" key="format"/>
<attrKeyMap attr="version" key="version"/>
<attrKeyMap attr="eventTime" key="start"/>
<attrKeyMap attr="_logId" key="id"/>
</collectAndSetAttrByJSON>
</when>
<when test="exist _logId">
<setEventAttribute attr="eventType">Akamai-Split</setEventAttribute>
</when>
</case>
<case>
<!-- monolithic event format, split into individual events -->
<collectFieldsByRegex src="$_rawmsg">
<regex><![CDATA[\[PH_DEV_MON_CUSTOM_JSON\]:\[reptVendor\]=Akamai,\[reptModel\]=Cloud%20Control,]]></regex>
</collectFieldsByRegex>
<collectFieldsByKeyValuePair kvsep=":" sep="," src="$_rawmsg">
<attrKeyMap attr="reptDevName" key="reptDevName"/>
<attrKeyMap attr="json" key="_json"/>
</collectFieldsByKeyValuePair>
<!-- input json, path, header, trailer, isDropBool -->
<setEventAttribute attr="_resultCount">splitJsonEvent($_json, "type", "[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=Akamai,[reptModel]=Cloud%20Control,Akamai_Individual_Event,json=", "", "true")</setEventAttribute>
</case>
</switch>
</parsingInstructions>

I get past validation (its valid XML) but run into this when testing

[Redbeard]

Forgot to add a sample

[PH_DEV_MON_CUSTOM_JSON]:[reptVendor]=Akamai,[reptModel]=Cloud%20Control,[reptDevName]=Cloud%20Monitor,[json]={"type":"cloud_monitor","format":"default","version":"1.0","id":"a","start":"1747925675.971","cp":"b","message":{"proto":"https","protoVer":"1.1","status":"200","cliIP":"c","reqPort":"443","reqHost":"d","reqMethod":"GET","reqPath":"f","reqQuery":"g","reqCT":"application%2fjson","sslVer":"TLSv1.3","respCT":"application/json","respLen":"756","bytes":"756","UA":"h","fwdHost":"i"},"content":{"edgeIP":"j","akaReqId":"k","appRules":"l","warn":"m"}}
{"type":"cloud_monitor","format":"default","version":"1.0","id":"a","start":"1747925678.198","cp":"c","message":{"proto":"https","protoVer":"1.1","status":"200","cliIP":"c","reqPort":"443","reqHost":"d","reqMethod":"POST","reqPath":"e","reqCT":"application%2fjson","reqLen":31,"sslVer":"TLSv1.3","respCT":"application/json","respLen":"632","bytes":"632","UA":"f","fwdHost":"g"},"content":{"edgeIP":"h","akaReqId":"i","appRules":"j","warn":""}}

[lbahtarliev · Accepted solution]

Hi there,

You are trying to reinvent the wheel @Redbeard mate ;)

Stick to the default PHCustomJsonParser as a base for your parser.

And just take a look here:
Ingesting JSON Formatted Events Received via HTTP(S) POST 

Pay attention the the optional separator argument you can use in your POST request.

Just define it according to the manual to be 'newline' and you are all set ;)
I use this all the time. 

PS: If you have paid attention, your input is not a valid JSON according to the spec. It is two JSON objects, each on a single line. The splitJsonEvent is intended for the cases where you actually upload a valid json array of json objects.

PPS: And your parser is messed up. You are referring to $json, but you do not have it defined yet (in the beginning), and so on.

Hope this helps,

Lyuben

[Redbeard]

Thanks for the help. Don't know how I missed the separator.