Description
This article describes how to troubleshoot connectivity issues between FortiGate and FortiAnalyzer.
This article additionally describes how the OFTPD protocol is used to create two communication streams between FortiGate and FortiAnalyzer devices.
Scope
OFTP uses TCP/514 for connectivity, health check, file transfer and log display from FortiGate.
Log communication happens over either TCP OR UDP 514:
- TCP/514 is used for log transmission with the reliable option enabled.
- UDP/514 is used for log transmission with the reliable option disabled.
Solution
The following sections describe how to verify and correct FortiAnalyzer connectivity issues.
Section 1: FortiGate and FortiAnalyzer firmware compatibility.
As a general rule, FortiAnalyzer should always be the same firmware release equal to or higher than that running on the FortiGate.
Note: this may not be true at the patch level - for more detail, see 'Compatibility with FortiOS' document for FortiAnalyzer on https://docs.fortinet.com/product/fortianalyzer.
For example:
- FortiAnalyzer on v5.6 and FortiGate on v5.4 or v5.6 will work.
- FortiAnalyzer on v5.4 and FortiGate on v5.6 will not work.
Section 2: Verify FortiAnalyzer configuration on the FortiGate.
The following FortiGate Log settings are used to send logs to the FortiAnalyzer:
# get log fortianalyzer setting
status : enable
ips-archive : enable
server : 10.34.199.143
enc-algorithm : high
conn-timeout : 10
monitor-keepalive-period: 5
monitor-failure-retry-period: 5
certificate :
source-ip :
upload-option : 5-minute -----> Upload logs every 5 minutes.
reliable : disable -----> Logs are sent over UDP.
Note:
Log transmission uses TCP or UDP channels depending on reliable settings. It should be enabled to be encrypted.
The following FortiGate Log filter settings affect the number of logs sent:
(global) # get log fortianalyzer filter
severity : information ---> The number of logs sent depends on the severity level e.g. information, warning, or critical. Different settings may give the impression that no logs are forwarded.
forward-traffic : enable
local-traffic : enable
multicast-traffic : enable
sniffer-traffic : enable
anomaly : enable
voip : enable
dlp-archive : enable
dns : enable
filter : -> Configuring filters can result in less logs being sent. Verify the filter settings to check if logs are being filtered.
filter-type : include -> Will only forward logs matching filter criteria.
To verify the FortiGate event log settings and filters use the folloing commands:
(vdom-name) # get log eventfilter
(vdom-name) # get log setting
(vdom-name) # get sys setting
Note:
Some log settings are set in different parts of the FortiGate configuration.
- Log settings like usernames in uppercase, policy-name and policy-comment are under 'config log setting'.
- VPN tunnel stats information is under 'config system setting'.
- For FortiGate Clusters, configuring a HA-Group name under HA settings is mandatory.
Section 3: Once the settings are verified, check connectivity from the GUI and the CLI of the FortiGate.
CLI:
# exec log fortianalyzer test-connectivity
Look for the following error message:
Failed to get FAZ's status. Authentication Failed. (-19) -> Side effect of FortiGate not being registered in the FortiAnlalyzer. In the FortiAnalyzer GUI under Device manager add the FortiGate.
GUI:
The following will prompt will appear 'FortiGate not authorized. Log in to logging device and confirm registration of this device.'
Verify connectivity when a FortiGate is registered on a FortiAnalyzer.
Use the following commands will verify connectivity:
Successful sending of logs:
# exec log fortianalyzer test-connectivity
FortiAnalyzer Host Name: FAZVM64
FortiGate Device ID: FGT1234567890
Registration: registered
Connection: allow
Disk Space (Used/Allocated): 0/Unlimited MB
Total Free Space: 831949 MB
Log: Tx & Rx (28 logs received since 02:00:18 02/20/18)
IPS Packet Log: Tx & Rx
Content Archive: Tx & Rx
Quarantine: Tx & Rx
Issue with sending the Logs:
# exec log fortianalyzer test-connectivity
FortiAnalyzer Host Name: FAZVM64
FortiGate Device ID: FGT1KD3915802143
Registration: registered
Connection: allow
Disk Space (Used/Allocated): 0/Unlimited MB
Total Free Space: 819502 MB
Log: Tx & Rx (log not received) -> Check if UDP is used (reliable is disabled under log setting).
IPS Packet Log: Tx & Rx
Content Archive: Tx & Rx
Quarantine: Tx & Rx
Basic network connectivity tests using ping, traceroute and telnet tests.
Run the tests from the FortiGate and FortiAnalyzer CLI.
Note:
10.34.199.143 is the FortiAnalyzer IP, use the management IP of the FortiGate when testing from the FortiAnalyzer CLI.
# exec ping 10.34.199.143
PING 10.34.199.143 (10.34.199.143): 56 data bytes
64 bytes from 10.34.199.143: icmp_seq=0 ttl=62 time=0.3 ms
64 bytes from 10.34.199.143: icmp_seq=1 ttl=62 time=0.3 ms
64 bytes from 10.34.199.143: icmp_seq=2 ttl=62 time=0.2 ms
64 bytes from 10.34.199.143: icmp_seq=3 ttl=62 time=0.2 ms
64 bytes from 10.34.199.143: icmp_seq=4 ttl=62 time=0.2 ms
--- 10.34.199.143 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.2/0.2/0.3 ms
# exec traceroute 10.34.199.143
traceroute to 10.34.199.143 (10.34.199.143), 32 hops max, 3 probe packets per hop, 84 byte packets
1 10.107.3.108 0.070 ms 0.060 ms 0.053 ms
2 10.40.31.254 0.083 ms 0.122 ms 0.075 ms
3 10.34.199.143 0.217 ms 0.233 ms 0.120 ms
# exec telnet 10.34.199.143 514
Trying 10.34.199.143...
Connected to 10.34.199.143.
Note:
Although ping and traceroute tests are successful, the connectivity may still fail. If this is the case, verify if TCP/UDP 514 ports are open on the intermediate devices (e.g. firewalls) between FortiGate and FortiAnalyzer.
Section 4: Advanced commands to check connectivity.
Using the sniffer command on the FortiGate and the FortiAnalyzer.
On the FortiGate CLI:
# diag sniffer packet any 'host x.x.x.x and port 514' 6 0 l
x.x.x.x is the IP address of the FortiAnalyzer.
On the FortiAnalyzer CLI:
# diag sniffer packet any 'host y.y.y.y and port 514' 3 0 l
y.y.y.y is the IP address of the FortiGate.
Then select Test Connectivity under Log Setting of the FortiGate GUI or run the command ‘diag log test’ form the CLI, packets received and sent from both devices should be seen.
Note:
Analyze the SYN and ACK numbers in the communication.
Analyzing OFTPD application debugging on the FortiAnalyzer.
Debugging the “OFTPD” deamon for connectivity issues:
# diag debug app oftpd 8 10.40.19.108 -> Or device name can be used. IP is preferable.
# diag debug timestamp enable
# diag debug enable
Then select Test Connectivity under Log Setting of the FortiGate GUI or run the command ‘diag log test’ form the CLI, packets received and sent from both devices should be seen.
A successful attempt will display 'Login Request' messages:
2018-02-20 15:50:51 oftpd_handle_session:3303: sock[29] ip[10.40.19.108] - Handle 'LOGIN_REQUEST' request type=2.
2018-02-20 15:50:51 handle_login:1961: sock[29] ip[10.40.19.108] - host = 'FGT1234567890'
2018-02-20 15:50:51 handle_login:1989: sock[29] ip[10.40.19.108] - Version: FortiGate-1000D v5.6.3,build1547,171204 (GA)
Virus-DB: 1.00123(2015-12-11 13:18)
IPS-DB: 6.00741(2015-12-01 02:30)
APP-DB: 6.00741(2015-12-01 02:30)
Industrial-DB: 6.00741(2015-12-01 02:30)
Serial-Number: FGT1234567890
Botnet DB: 1.00000(2012-05-28 22:51)
Virtual domain configuration: disable
Current HA mode: standalone
Current HA group:
2018-02-20 15:50:51 handle_login:1966: sock[29] ip[10.40.19.108] - vdom = 1
2018-02-20 15:50:51 oftpd_handle_session:3286: sock[29] ip[10.40.19.108] - [oftpd_handle_session] the peer close the connection.
2018-02-20 15:50:51 oftpd_close_session:2600: sock[29] ip[10.40.19.108] - Client connection closed. Reason 8(the peer close the connection)
Disable the debug using below set of commands:
# diag debug disable
# diag debug timestamp disable
# diag debug app oftpd 0
Section 5: If the connectivity issue is still not resolved or isolated, collect the following information for Fortinet TAC to use for further investigation.
On the FortiGate:
- Was there any recent firmware upgrade done on the FortiGate after which connectivity issues occurred? If yes, indicate the upgrade path followed.
- Attach the latest unencrypted configuration backup of the FortiGate.
- Open an ssh session with FortiGate using PUTTY and log all the output to a file (Session -> Logging -> All session output -> Log File name -> Save the file as *.log).
Run the commands and attach the log file to the ticket.
# get sys status
# get sys performance status(run it 4-5 times with an interval of 3 sec)
# diag sys top 1 25(run it for 8-10 seconds and then press ‘q’ to quit)
# get log fortianalyzer setting
# get log fortianalyzer filter
# get log setting
# get log eventfilter
# exec traceroute <FortiAnalyzer IP address>
# exec ping <FortiAnalyzer IP address>
# exec log fortianalyzer test-connectivity
# diag sys flash list
# diag test app miglogd 6
# diag log kernel-stats
# diag debug crashlog read
On the FortiAnalyzer:
- Was there any recent firmware upgrade done on the FortiAnalyzer after which connectivity issues occurred? If yes, indicate the upgrade path followed.
- Attach the latest unencrypted configuration backup of the FortiGate.
- Open an ssh session with FortiGate using PUTTY and log all the output to a file (Session -> Logging -> All session output -> Log File name -> Save the file as *.log).
Run the commands and attach the log file to the ticket.
# get sys status
# get sys performance (run it 4-5 times with an interval of 10 sec)
# exec top (run it for 8-10 seconds and then press ‘q’ to quit)
# diag fortilogd lograte (run it 4-5 times with an interval of 10 sec)
# diag fortilogd msgrate (run it 4-5 times with an interval of 10 sec)
# diag fortilogd msgrate-device (run it 4-5 times with an interval of 10 sec)
# diag fortilogd msgrate-type (run it 4-5 times with an interval of 10 sec)
# diag fortilogd msgrate-total (run it 4-5 times with an interval of 10 sec)
diagnose test application oftp 5
diagnose test application oftp 6
diagnose test application oftp 7
diagnose test application oftp 10
diagnose test application fortilogd 1
diagnose test application fortilogd 2
diagnose test application fortilogd 3
diagnose test application fortilogd 4
diagnose test application fortilogd 7
diagnose test application fortilogd 10
diagnose test application sqllogd 9
Related Articles:
- Technical Note: How to create a log file of a session using PuTTY
- Technical Tip: Ticket Creation via the Support Portal
- Technical Note: Traffic Types and TCP/UDP Ports used by Fortinet Products
- Troubleshooting Tips: No logs received on FortiAnalyzer
- Technical Tip: How to setup a custom certificate regarding OFTP protocol
