Description
This article describes how to troubleshoot connectivity issues between FortiGate and FortiAnalyzer
Scope
This article describes how the OFTPD protocol is used to create two communication streams between FortiGate and FortiAnalyzer devices.
OFTP uses TCP/514 for connectivity, health check, file transfer and log display from FortiGate
Log communication happens over either TCP OR UDP 514 ,
-TCP/514 used for log transmission with the reliable option enabled
-UDP/514 used for log transmission with the reliable option disabled
Solution
The following sections describe how to verify and correct FortiAnalyzer connectivity issues.
Section 1: FortiGate and FortiAnalyzer firmware compatibility
As a general rule, FortiAnalyzer should always be the same firmware release equal to or higher than that running on the FortiGate.
Note: this may not be true at the patch level -- for more detail, see "Compatibility with FortiOS" document for FortiAnalyzer on https://docs.fortinet.com/product/fortianalyzer
For example:
- FortiAnalyzer on v5.6 and FortiGate on v5.4 or v5.6 will workSection 2: Verify FortiAnalyzer configuration on the FortiGate
- FortiAnalyzer on v5.4 and FortiGate on v5.6 will not work
The following FortiGate Log settings are used to send logs to the FortiAnalyzer:
# get log fortianalyzer settingNote.
status : enable
ips-archive : enable
server : 10.34.199.143
enc-algorithm : high
conn-timeout : 10
monitor-keepalive-period: 5
monitor-failure-retry-period: 5
certificate :
source-ip :
upload-option : 5-minute -----> Upload logs every 5 minutes.
reliable : disable -----> Logs are sent over UDP.
Log transmission uses TCP or UDP channel depending on reliable settings. It should be enabled to be encrypted.
The following FortiGate Log filter settings affect the amount of logs sent:
(global) # get log fortianalyzer filterTo verify the FortiGate event log settings and filters use the folloing commands
severity : information ---> The amount of logs sent depend on the severity level e.g. information, warning or critical.
Different settings may give the impression that no logs are forwarded.
forward-traffic : enable
local-traffic : enable
multicast-traffic : enable
sniffer-traffic : enable
anomaly : enable
voip : enable
dlp-archive : enable
dns : enable
filter : ---> Configuring filters can result in less logs being sent
verify the filter settings to check if logs are being filtered
filter-type : include ---> Will only forward logs matching filter criteria
(vdom-name) # get log eventfilterNote: Some log settings are set in different parts of the FortiGate configuration
(vdom-name) # get log setting
(vdom-name) # get sys setting
- Log settings like usernames in uppercase, policy-name and policy-comment are under “config log setting”
- VPN tunnel stats information is under “config system setting”
- For FortiGate Clusters configuring a HA-Group name under HA settings is mandatory
Section 3: Once the settings are verified, check connectivity from the GUI and the CLI of the FortiGate
CLI:
# exec log fortianalyzer test-connectivity
Look for the following error message:
Failed to get FAZ's status. Authentication Failed. (-19) > Side effect of FortiGate not being registered in the FortiAnlalyzer. In the FortiAnalyzer GUI under Device manager add the FortiGateGUI: The following will prompt will appear "FortiGate not authorized. Log in to logging device and confirm registration of this device."
Verify connectivity when a FortiGate is registered on a FortiAnalyzer
Use the following commands will verify connectivity:
Successful sending of logs
#exec log fortianalyzer test-connectivityIssue with sending the Logs:
FortiAnalyzer Host Name: FAZVM64
FortiGate Device ID: FGT1234567890
Registration: registered
Connection: allow
Disk Space (Used/Allocated): 0/Unlimited MB
Total Free Space: 831949 MB
Log: Tx & Rx (28 logs received since 02:00:18 02/20/18)
IPS Packet Log: Tx & Rx
Content Archive: Tx & Rx
Quarantine: Tx & Rx
# exec log fortianalyzer test-connectivityBasic network connectivity tests using ping, traceroute and telnet tests
FortiAnalyzer Host Name: FAZVM64
FortiGate Device ID: FGT1KD3915802143
Registration: registered
Connection: allow
Disk Space (Used/Allocated): 0/Unlimited MB
Total Free Space: 819502 MB
Log: Tx & Rx (log not received) > Check if UDP is used (reliable is disabled under log setting)
IPS Packet Log: Tx & Rx
Content Archive: Tx & Rx
Quarantine: Tx & Rx
Run the tests from the FortiGate and FortiAnalyzer CLI:
Note: 10.34.199.143 is the FortiAnalyzer IP, use the management IP of the FortiGate when testing from the FortiAnalyzer CLI
# exec ping 10.34.199.143
PING 10.34.199.143 (10.34.199.143): 56 data bytes
64 bytes from 10.34.199.143: icmp_seq=0 ttl=62 time=0.3 ms
64 bytes from 10.34.199.143: icmp_seq=1 ttl=62 time=0.3 ms
64 bytes from 10.34.199.143: icmp_seq=2 ttl=62 time=0.2 ms
64 bytes from 10.34.199.143: icmp_seq=3 ttl=62 time=0.2 ms
64 bytes from 10.34.199.143: icmp_seq=4 ttl=62 time=0.2 ms
--- 10.34.199.143 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.2/0.2/0.3 ms# exec traceroute 10.34.199.143
traceroute to 10.34.199.143 (10.34.199.143), 32 hops max, 3 probe packets per hop, 84 byte packets
1 10.107.3.108 0.070 ms 0.060 ms 0.053 ms
2 10.40.31.254 0.083 ms 0.122 ms 0.075 ms
3 10.34.199.143 0.217 ms 0.233 ms 0.120 ms# exec telnet 10.34.199.143 514
Trying 10.34.199.143...
Connected to 10.34.199.143.
Note: Although ping and traceroute tests are successful, the connectivity may still fail. If this is the case, verify if TCP/UDP 514 ports are open on the intermediate devices (e.g. firewalls) between FortiGate and FortiAnalyzer.
Section 4: Advanced commands to check connectivity
Using the sniffer command on the FortiGate and the FortiAnalyzer
On the FortiGate CLI:
# diag sniffer packet any 'host x.x.x.x and port 514' 4 0 lx.x.x.x is the IP address of the FortiAnalyzer
On the FortiAnalyzer CLI:
# diag sniffer packet any 'host y.y.y.y and port 514' 4 0 ly.y.y.y is the IP address of the FortiGate
Then click on Test Connectivity under Log Setting of the FortiGate GUI or run the command ‘diag log test’ form the CLI, you should see packets received and sent from both devices.
Note: Analyze the SYN and ACK numbers in the communication
Analyzing OFTPD application debugging on the FortiAnalyzer
Debugging the “OFTPD” deamon for connectivity issues:
# diag debug app oftpd 8 10.40.19.108 > Or device name can be used. IP is preferableThen click on Test Connectivity under Log Setting of the FortiGate GUI or run the command ‘diag log test’ form the CLI, you should see packets received and sent from both devices.
# diag debug timestamp enable
# diag debug enable
A successful attempt will display "Login Request" messages:
2018-02-20 15:50:51 oftpd_handle_session:3303: sock[29] ip[10.40.19.108] - Handle 'LOGIN_REQUEST' request type=2.Disable the debug using below set of commands
2018-02-20 15:50:51 handle_login:1961: sock[29] ip[10.40.19.108] - host = 'FGT1234567890'
2018-02-20 15:50:51 handle_login:1989: sock[29] ip[10.40.19.108] - Version: FortiGate-1000D v5.6.3,build1547,171204 (GA)
Virus-DB: 1.00123(2015-12-11 13:18)
IPS-DB: 6.00741(2015-12-01 02:30)
APP-DB: 6.00741(2015-12-01 02:30)
Industrial-DB: 6.00741(2015-12-01 02:30)
Serial-Number: FGT1234567890
Botnet DB: 1.00000(2012-05-28 22:51)
Virtual domain configuration: disable
Current HA mode: standalone
Current HA group:
2018-02-20 15:50:51 handle_login:1966: sock[29] ip[10.40.19.108] - vdom = 1
2018-02-20 15:50:51 oftpd_handle_session:3286: sock[29] ip[10.40.19.108] - [oftpd_handle_session] the peer close the connection.
2018-02-20 15:50:51 oftpd_close_session:2600: sock[29] ip[10.40.19.108] - Client connection closed. Reason 8(the peer close the connection)
# diag debug disableSection 5: If the connectivity issue is still not resolved or isolated, collect the following information for Fortinet TAC to use for further investigation
# diag debug timestamp disable
# diag debug app oftpd 0
On the FortiGate:
- Was there any recent firmware upgrade done on the FortiGate after which connectivity issues occurred? If yes, please indicate the upgrade path followed
- Attach the latest unencrypted configuration backup of the FortiGate.
- Open a ssh session with FortiGate using PUTTY and log all the output to a file (Session > Logging > All session output > Log File name > Save the file as *.log).Run the commands and attach the log file to the ticket.On the FortiAnalyzer:get sys status
get sys performance status(run it 4-5 times with an interval of 3 sec)
diag sys top 1 25(run it for 8-10 seconds and then press ‘q’ to quit)
get log fortianalyzer setting
get log fortianalyzer filter
get log setting
get log eventfilter
exec traceroute <FortiAnalyzer IP address>
exec ping <FortiAnalyzer IP address>
exec log fortianalyzer test-connectivity
diag sys flash list
diag test app miglogd 6
diag log kernel-stats
diag debug crashlog read
- Was there any recent firmware upgrade done on the FortiAnalyzer after which connectivity issues occurred? If yes, please indicate the upgrade path followed
- Attach the latest unencrypted configuration backup of the FortiGate.
- Open a ssh session with FortiGate using PUTTY and log all the output to a file (Session > Logging > All session output > Log File name > Save the file as *.log).
Run the commands and attach the log file to the ticket.get sys status
get sys performance (run it 4-5 times with an interval of 10 sec)
exec top (run it for 8-10 seconds and then press ‘q’ to quit)
diag fortilogd lograte (run it 4-5 times with an interval of 10 sec)
diag fortilogd msgrate (run it 4-5 times with an interval of 10 sec)
diag fortilogd msgrate-device (run it 4-5 times with an interval of 10 sec)
diag fortilogd msgrate-type (run it 4-5 times with an interval of 10 sec)
diag fortilogd msgrate-total (run it 4-5 times with an interval of 10 sec)
diagnose test application oftp 5
diagnose test application oftp 6
diagnose test application oftp 7
diagnose test application oftp 10
diagnose test application fortilogd 1
diagnose test application fortilogd 2
diagnose test application fortilogd 3
diagnose test application fortilogd 4
diagnose test application fortilogd 7
diagnose test application fortilogd 10
diagnose test application sqllogd 9
Related Articles
Technical Note: How to create a log file of a session using PuTTY
Technical Tip: Ticket Creation via the Support Portal
Technical Note: Traffic Types and TCP/UDP Ports used by Fortinet Products
