Description
This article describes how FortiGate sends syslog messages via TCP in FortiOS 6.0 and 6.2 and possible issues related to log length and parsing.
Related document:
https://help.fortinet.com/fos60hlp/60/Content/FortiOS/fortigate-logging-reporting/config-log-advance...
RFC6587:
https://tools.ietf.org/html/rfc6587
Scope
FortiGate.
Solution
When FortiGate sends logs to a syslog server via TCP, it utilizes the RFC6587 standard by default. RFC6587 has two methods to distinguish between individual log messages, 'Octet Counting' and 'Non-Transparent-Framing'.
From the RFC:
3.4.1. Octet Counting:
This framing allows for the transmission of all characters inside a syslog message and is similar to the method used in [RFC5425]. A transport receiver uses the defined message length to delimit a syslog message. As noted in [RFC3164], the upper limit for a legacy syslog message length is 1024 octets. That length has been expanded for standardized syslog.
It can be assumed that octet-counting framing is used if a syslog frame starts with a digit.
All syslog messages can be considered to be TCP 'data' as per the Transmission Control Protocol [RFC0793]. The syslog message stream has the following ABNF [RFC5234] definition:
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = MSG-LEN SP SYSLOG-MSG ; Octet-counting
; method
MSG-LEN = NONZERO-DIGIT *DIGIT
NONZERO-DIGIT = %d49-57
SYSLOG-MSG is defined in the syslog protocol [RFC5424] and may also be considered to be the payload in [RFC3164]. MSG-LEN is the octet count of the SYSLOG-MSG in the SYSLOG-FRAME.
3.4.2. Non-Transparent-Framing:
The non-transparent-framing method inserts a syslog message into a frame and terminates it with a TRAILER character. The TRAILER has usually been a single character and most often is ASCII LF (%d10). However, other characters have also been seen, with ASCII NUL (%d00) being a prominent example. Some devices have also been seen to emit a two-character TRAILER, which is usually CR and LF.
The problem with non-transparent framing comes from the use of a TRAILER character. In that, the traditional TRAILER character is not escaped within the message, which causes problems for the receiver.
For example, a message in the style of [RFC3164] containing one or more LF characters may be misinterpreted as multiple messages by the receiving syslog application.
The ABNF for this is shown here:
TCP-DATA = *SYSLOG-FRAME
SYSLOG-FRAME = SYSLOG-MSG TRAILER ; non-transparent-framing
; method
TRAILER = LF / APP-DEFINED
APP-DEFINED = 1*2OCTET
SYSLOG-MSG is defined in the syslog protocol [RFC5424] and may also be considered to be the payload in [RFC3164]. A transport receiver can assume that non-transparent framing is used if a syslog frame starts with the ASCII character '<' (%d60).
The FortiGate uses 'Octet Counting'. It prefaces the syslog message with a count denoting how long the log is. Some servers or log parsers however, might expect “Non-Transparent-Framing” – they expect a specific character between log messages.
This discrepancy can lead some syslog servers or parsers to interpret the logs sent by FortiGate as one long log message, even when the FortiGate sent multiple logs.
If the syslog server does not support “Octet Counting”, then there are the following options on FortiGate:
- Switch to UDP logging.
- Switch to legacy TCP logging (according to RFC3195).
config log syslogd setting
Set mode <udp|legacy-reliable>
end
Note:
FortiGate v5.6 and lower only support reliable syslog matching RFC3195.
Note:
In the case TCP logging has been used, 'Octet Counting' will be the random number added to the Syslog packet before the priority value, showing as a prefix number which is described in the previous section. It's only added in the SYSLOG packet once FortiGate sends out, and not affect the raw log on FortiGate locally. Hence, those prefix numbers will not show either from the CLI display or view locally on the FortiGate. The below screenshot displays detailed 'Octet Counting' and 'Priority Value' in Wireshark.
