Windows Log Parsing Issue

[MebinBaby]

Hi all,

I am facing a partial parsing issue with windows logs. Message, Device Hostname fields are not getting parsed. I am using NxLog method to collect data to the FortiSIEM. Can you help me with any documentations available to create custom parsers? Any guidance will be appreciated.

Thanks in advance!

[AlexDC]

Hello Mebin,

there are great resources provided by Fortinet NSE Training Institute , see below
The NSE 7 Advanced Analytics 5.2 self-paced course is now released 
this course should answer your questions it was on a post a way back - 
https://fusecommunity.fortinet.com/groups/community-home/digestviewer/viewthread?MessageKey=f97db6da-06e7-46e4-b3eb-92f71e205418&CommunityKey=d8119bda-6fd9-4771-87ba-c34eb683ad51&tab=digestviewer#bmf97db6da-06e7-46e4-b3eb-92f71e205418

I hope this helps.

[MebinBaby]

Thanks Alex !

[cdurkin_FTNT]

Hello Mebin

Not sure if you have also tried using "Snare" format as the export method in NxLog?

[RobertEvans]

I second this, snare format is parsed by FortiSIEM, so you can tell nxlog to send in that format. 

Replace your nxlog.conf file with this one, and then restart the nxlog service. Also specify your log target of the collector instead of 1.1.1.1

Panic Soft
#NoFreeOnExit TRUE

define ROOT C:\Program Files (x86)\nxlog
define CERTDIR %ROOT%\cert
define CONFDIR %ROOT%\conf
define LOGDIR %ROOT%\data
define LOGFILE %LOGDIR%\nxlog.log
define OUTPUT_DESTINATION_ADDRESS 1.1.1.1
define OUTPUT_DESTINATION_PORT 514
LogFile %LOGFILE%

Moduledir %ROOT%\modules
CacheDir %ROOT%\data
Pidfile %ROOT%\data\nxlog.pid
SpoolDir %ROOT%\data

<Extension _syslog>
Module xm_syslog
</Extension>

<Extension _charconv>
Module xm_charconv
AutodetectCharsets iso8859-2, utf-8, utf-16, utf-32
</Extension>

<Extension _exec>
Module xm_exec
</Extension>

<Extension _fileop>
Module xm_fileop

# Check the size of our log file hourly, rotate if larger than 5MB
<Schedule>
Every 1 hour
Exec if (file_exists('%LOGFILE%') and \
(file_size('%LOGFILE%') >= 5M)) \
file_cycle('%LOGFILE%', 8);
</Schedule>

# Rotate our log file every week on Sunday at midnight
<Schedule>
When @weekly
Exec if file_exists('%LOGFILE%') file_cycle('%LOGFILE%', 8);
</Schedule>
</Extension>
<Extension json>
Module xm_json
</Extension>


<Input internal>
Module im_internal
</Input>

<Input eventlog>
Module im_msvistalog
Query <QueryList>\
<Query Id="0">\
<Select Path="Application">*</Select>\
<Select Path="System">*</Select>\
<Select Path="Security">*</Select>\
</Query>\
</QueryList>
Exec if ($EventID == 5156) OR ($EventID == 5158) drop();
Exec $HOSTNAME=hostname();
Exec $Message =~ s/(\t|\R)/ /g;
</Input>


<Output out>
Module om_udp
Host %OUTPUT_DESTINATION_ADDRESS%
Port %OUTPUT_DESTINATION_PORT%
Exec to_syslog_snare();
</Output>

<Route 1>
Path eventlog, internal => out
</Route>

[JoeSkinner]

NXLog uses the WinSyslogParser.  Not parsing hostname is a bug that is fixed in 6.3.  It also fixes the random hostnames placed in the Location and Identity table.

[MebinBaby]

Thanks Joe for the update. Appreciate it!

-Mebin

[fbologne]

Hi Mebin, 

I have your same problem. Did you solved it?

Using a similar configuration to the one provided above by Robert and using the to_syslog_snare() directive I get as log, for example, from a Windows server:

2024-11-12T10:56:21+01:00 MYSERVER MSWinEventLog#0111#011System#0111#011Tue Nov 12 10:56:21 2024#0117036#011Service Control Manager#011N/A#011N/A#011Information#011ctx-desk.nso.local#011N/A#011#011The nxlog service entered the stopped state.#011609646

where I have #011 as TAB delimiter and FortiSIEM cannot interpret the log. It can understand that it is a Windows type Log but does not recognize the Event Type..

In the configuration above in theory the directive Exec $Message =~ s/(\t|\R)/ /g; should serve to convert the TABs and CRs to spaces, but it seems not to work, because in fact I find in the logs #011 and #015..

Did you manage to solve the problem somehow? I use CE version 3.2.2329 as an Nxlog agent.

Kind regards,

Fabio.