Openshift parser on FortiSIEM

[Pantashaa]

Hello,
Kindly assist me to develop a parser for the following Openshift logs. I am fairly new to FortiSIEM, so I am struggling to accomplish this.

The following log is what I'm receiving on FortiSIEM from Openshift:
<46>1 2025-10-26T12:49:43.522Z node01.internal.local redgreen-service-deployment-857cb4bb8d-l5v8r_blueyellow-service magenta-7f2a1c9f container - {"@timestamp":"2025-10-26T12:49:43.522728353Z","hostname":"node01.internal.local","kubernetes":{"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n "name": "orangeviolet-net",\n "interface": "eth0",\n "ips": [\n "192.168.42.100"\n ],\n "default": true,\n "dns": {}\n}]","kubectl.kubernetes.io/restartedAt":"2025-09-22T01:16:48+03:00","openshift.io/scc":"anyuid","openshift.openshift.io/restartedAt":"2025-10-06T19:57:19.173Z"},"container_id":"cri-o://pinkteal-7f3c2d1a9b8c4e00a1234b567c89deffedcba123","container_image":"registry.internal.local/colors/redgreen-service:prod-blueyellow-20250916120214","container_image_id":"registry.internal.local/colors/redgreen-service@sha256:cyanmagenta-7ea6a080a9b23d6d28cd63deea38a2df5d3d4297365f76bcaa1fc0af96edf6e8","container_name":"redgreen-service","labels":{"app":"redgreen-service","pod-template-hash":"857cb4bb8d"},"namespace_id":"rainbow-1a2b3c4d-5e6f-7g8h-9i0j-colormix1234","namespace_labels":{"kubernetes_io_metadata_name":"colors","kustomize_toolkit_fluxcd_io_name":"tenants","kustomize_toolkit_fluxcd_io_namespace":"flux-system","pod-security_kubernetes_io_audit":"privileged","pod-security_kubernetes_io_audit-version":"v1.24","pod-security_kubernetes_io_warn":"privileged","pod-security_kubernetes_io_warn-version":"v1.24","toolkit_fluxcd_io_tenant":"artist-team"},"namespace_name":"colors","pod_id":"ultraviolet-f7c2d11a-0e4f-4b6d-8888-neon1234","pod_ip":"192.168.42.100","pod_name":"redgreen-service-deployment-857cb4bb8d-l5v8r","pod_owner":"ReplicaSet/redgreen-service-deployment-857cb4bb8d"},"level":"info","log_source":"container","log_type":"application","message":"2025-10-26 15:49:43 | Severity=INFO | MicroService=redgreen-service | TransactionID=rainbow-89234 | Transaction=null | Process=POST | ProcessDuration=116 | SourceSystem=https://service.internal/api/v1/colors/sync | TargetSystem=PaletteRegistry | ResponseCode=200 | Request=null | RequestXml=null | Response=null | ResponseXml=null | ResponseMsg= ","openshift":{"cluster_id":"prism-7766abcd-1122-3344-5566-colorburst99","sequence":1761482983532549857}}

By following basic tutorials, this is the starting point I was able to come-up with:
<parsingInstructions>
<collectAndSetAttrByRegex src="$_rawmsg">
<regex><![CDATA[<:gPatSyslogPRI>\d+\s+<_year:gPatYear>-<_mon:gPatMonNum>-<_day:gPatDay>\w<_time:gPatTime>\.\w+\s+<_body:gPatMesgBody>]]></regex>
</collectAndSetAttrByRegex>
</parsingInstructions>


Testing this fails. 

Please assist me to create a parser for the Openshift logs.

FortiSIEM @Jean-Philippe_P @lbahtarliev

[cdurkin_FTNT]

Try this ... its hard to build a parser and event format recognizer from one single event sample.

It should work for the above ... but I think it will need work when additional samples are provided.

You would still need to map in the collectAndSetAttrByJSON the relevant attributes for each key.

<eventFormatRecognizer><![CDATA[redgreen-service-deployment-]]></eventFormatRecognizer>

<parsingInstructions>

<collectAndSetAttrByRegex src="$_rawmsg">
  <regex><![CDATA[<_year:gPatYear>-<_mon:gPatMon>-<_day:gPatDay>T<_time:gPatTimeMSec><_tz:gPatTimeZone>\s+<reptDevName:gPatStr>\s+[^ ]+\s+[^ ]+\s+container\s+-\s+<_json:gPatMesgBody>]]></regex>
</collectAndSetAttrByRegex>
<setEventAttribute attr="deviceTime">toDateTime($_mon, $_day, $_year, $_time, $_tz)</setEventAttribute>


<collectAndSetAttrByJSON src="$_json">
  <attrKeyMap attr="_@timestamp" key="@timestamp"/>
  <attrKeyMap attr="_hostname" key="hostname"/>
  <attrKeyMap attr="_io/network-status" key="kubernetes.annotations.k8s\.v1\.cni\.cncf\.io/network-status"/>
  <attrKeyMap attr="_io/restartedAt" key="kubernetes.annotations.kubectl\.kubernetes\.io/restartedAt"/>
  <attrKeyMap attr="_io/scc" key="kubernetes.annotations.openshift\.io/scc"/>
  <attrKeyMap attr="_io/restartedAt" key="kubernetes.annotations.openshift\.openshift\.io/restartedAt"/>
  <attrKeyMap attr="_container_id" key="kubernetes.container_id"/>
  <attrKeyMap attr="_container_image" key="kubernetes.container_image"/>
  <attrKeyMap attr="msg" key="kubernetes.container_image_id"/>
  <attrKeyMap attr="_container_name" key="kubernetes.container_name"/>
  <attrKeyMap attr="_app" key="kubernetes.labels.app"/>
  <attrKeyMap attr="_pod-template-hash" key="kubernetes.labels.pod-template-hash"/>
  <attrKeyMap attr="_namespace_id" key="kubernetes.namespace_id"/>
  <attrKeyMap attr="_kubernetes_io_metadata_name" key="kubernetes.namespace_labels.kubernetes_io_metadata_name"/>
  <attrKeyMap attr="_kustomize_toolkit_fluxcd_io_name" key="kubernetes.namespace_labels.kustomize_toolkit_fluxcd_io_name"/>
  <attrKeyMap attr="_kustomize_toolkit_fluxcd_io_namespace" key="kubernetes.namespace_labels.kustomize_toolkit_fluxcd_io_namespace"/>
  <attrKeyMap attr="_pod-security_kubernetes_io_audit" key="kubernetes.namespace_labels.pod-security_kubernetes_io_audit"/>
  <attrKeyMap attr="_pod-security_kubernetes_io_audit-version" key="kubernetes.namespace_labels.pod-security_kubernetes_io_audit-version"/>
  <attrKeyMap attr="_pod-security_kubernetes_io_warn" key="kubernetes.namespace_labels.pod-security_kubernetes_io_warn"/>
  <attrKeyMap attr="_pod-security_kubernetes_io_warn-version" key="kubernetes.namespace_labels.pod-security_kubernetes_io_warn-version"/>
  <attrKeyMap attr="_toolkit_fluxcd_io_tenant" key="kubernetes.namespace_labels.toolkit_fluxcd_io_tenant"/>
  <attrKeyMap attr="_namespace_name" key="kubernetes.namespace_name"/>
  <attrKeyMap attr="_pod_id" key="kubernetes.pod_id"/>
  <attrKeyMap attr="_pod_ip" key="kubernetes.pod_ip"/>
  <attrKeyMap attr="_pod_name" key="kubernetes.pod_name"/>
  <attrKeyMap attr="_pod_owner" key="kubernetes.pod_owner"/>
  <attrKeyMap attr="_level" key="level"/>
  <attrKeyMap attr="_log_source" key="log_source"/>
  <attrKeyMap attr="_log_type" key="log_type"/>
  <attrKeyMap attr="_message" key="message"/>
  <attrKeyMap attr="_cluster_id" key="openshift.cluster_id"/>
  <attrKeyMap attr="_sequence" key="openshift.sequence"/>
</collectAndSetAttrByJSON>


<setEventAttribute attr="eventType">Openshift-Generic</setEventAttribute>
<setEventAttribute attr="eventSeverity">1</setEventAttribute>

</parsingInstructions>

Note.. the following was used as the Test Event

<46>1 2025-10-26T12:49:43.522Z node01.internal.local redgreen-service-deployment-857cb4bb8d-l5v8r_blueyellow-service magenta-7f2a1c9f container - {"@timestamp":"2025-10-26T12:49:43.522728353Z","hostname":"node01.internal.local","kubernetes":{"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n  \"name\": \"orangeviolet-net\",\n  \"interface\": \"eth0\",\n  \"ips\": [\n    \"192.168.42.100\"\n  ],\n  \"default\": true,\n  \"dns\": {}\n}]","kubectl.kubernetes.io/restartedAt":"2025-09-22T01:16:48+03:00","openshift.io/scc":"anyuid","openshift.openshift.io/restartedAt":"2025-10-06T19:57:19.173Z"},"container_id":"cri-o://pinkteal-7f3c2d1a9b8c4e00a1234b567c89deffedcba123","container_image":"registry.internal.local/colors/redgreen-service:prod-blueyellow-20250916120214","container_image_id":"registry.internal.local/colors/redgreen-service@sha256:cyanmagenta-7ea6a080a9b23d6d28cd63deea38a2df5d3d4297365f76bcaa1fc0af96edf6e8","container_name":"redgreen-service","labels":{"app":"redgreen-service","pod-template-hash":"857cb4bb8d"},"namespace_id":"rainbow-1a2b3c4d-5e6f-7g8h-9i0j-colormix1234","namespace_labels":{"kubernetes_io_metadata_name":"colors","kustomize_toolkit_fluxcd_io_name":"tenants","kustomize_toolkit_fluxcd_io_namespace":"flux-system","pod-security_kubernetes_io_audit":"privileged","pod-security_kubernetes_io_audit-version":"v1.24","pod-security_kubernetes_io_warn":"privileged","pod-security_kubernetes_io_warn-version":"v1.24","toolkit_fluxcd_io_tenant":"artist-team"},"namespace_name":"colors","pod_id":"ultraviolet-f7c2d11a-0e4f-4b6d-8888-neon1234","pod_ip":"192.168.42.100","pod_name":"redgreen-service-deployment-857cb4bb8d-l5v8r","pod_owner":"ReplicaSet/redgreen-service-deployment-857cb4bb8d"},"level":"info","log_source":"container","log_type":"application","message":"2025-10-26 15:49:43 | Severity=INFO | MicroService=redgreen-service | TransactionID=rainbow-89234 | Transaction=null | Process=POST | ProcessDuration=116 | SourceSystem=https://service.internal/api/v1/colors/sync | TargetSystem=PaletteRegistry | ResponseCode=200 | Request=null | RequestXml=null | Response=null | ResponseXml=null | ResponseMsg= ","openshift":{"cluster_id":"prism-7766abcd-1122-3344-5566-colorburst99","sequence":1761482983532549857}}

[Pantashaa]

Thank you for your response @cdurkin_FTNT  . I have now started mapping the relevant attributes for each key.
When I run the test, the relaying IP and reporting IP are set to 127.0.0.1. How do I change that?
I have also mapped the <attrKeyMap attr="srcIpAddr" key="kubernetes.pod_id"/> is that okay? Does it interpret the log correctly?
Lastly, how do I define the Application-Level Log Fields within the 'message' in the parser?, i.e.

"2025-10-26 15:49:43 | Severity=INFO | MicroService=redgreen-service | TransactionID=rainbow-89234 | Transaction=null | Process=POST | ProcessDuration=116 | SourceSystem=https://service.internal/api/v1/colors/sync | TargetSystem=PaletteRegistry | ResponseCode=200 | Request=null | RequestXml=null | Response=null | ResponseXml=null | ResponseMsg= ",

such that I can have the process, the sourceSystem and ResponseCode listed when the log is parsed?
Thank you for your support.

[Pantashaa]

@cdurkin_FTNT 
@Jean-Philippe_P 

Please please assist.

[Secusaurus]

Hi @Pantashaa,

The reporting IP should be mapped automatically to the source of this event. When running the test locally, it's always the local address, so I assume it should insert the correct ip when the parser is running in production. You should be able to override it manually, if you have it in the logs.

For mapping the content of the message, split the the output of the variable $_message into smaller chunks (just as $_json is extracted from the full syslog, you can continue working with the $_message attribute). You could either run a regex on it to find distinct information or try to map using collectFieldsByKeyValuePair, see https://docs.fortinet.com/document/fortisiem/7.3.5/user-guide/956470/collect-and-set-functions#colle...

Best,

Christian

[cdurkin_FTNT]

 Pantashaa

As Christian said the "_message" above variable will contain that data .. you can add a second function to read that variable later in the parser and extract the needed data.

<collectFieldsByKeyValuePair kvsep="=" sep=" " src="$_message">
  <attrKeyMap attr="_Severity" key="Severity"/>
  <attrKeyMap attr="_MicroService" key="MicroService"/>
  <attrKeyMap attr="_TransactionID" key="TransactionID"/>
  <attrKeyMap attr="_Transaction" key="Transaction"/>
  <attrKeyMap attr="_Process" key="Process"/>
  <attrKeyMap attr="_ProcessDuration" key="ProcessDuration"/>
  <attrKeyMap attr="_SourceSystem" key="SourceSystem"/>
  <attrKeyMap attr="_TargetSystem" key="TargetSystem"/>
  <attrKeyMap attr="_ResponseCode" key="ResponseCode"/>
  <attrKeyMap attr="_Request" key="Request"/>
  <attrKeyMap attr="_RequestXml" key="RequestXml"/>
  <attrKeyMap attr="_Response" key="Response"/>
  <attrKeyMap attr="_ResponseXml" key="ResponseXml"/>
  <attrKeyMap attr="_ResponseMsg" key="ResponseMsg"/>
</collectFieldsByKeyValuePair>

Again replace the attr"_abc" values above with the correct attribute names as needed.