Description
This article explains how the 'FGFM' protocol is used to communicate between FortiGate and FortiManager devices and guides troubleshooting protocol-related issues.
The FGFM protocol operates over SSL (Secure Sockets Layer) using TCP port 541 on IPv4.
Scope
FortiGate and FortiManager.
Solution
Check if 'FMG-Access' is enabled under the FortiGate management port.
Go to Network -> Interfaces -> Double-click the management port -> Administrative access and check 'FMG-Access' is enabled.
Failing that, check the SSL compatibility.
On FortiManager.
config sys global
set fgfm-ssl-protocol
sslv3 <- Set SSLv3 as the lowest version.
tlsv1.0 <- Set TLSv1.0 as the lowest version.
tlsv1.1 <- Set TLSv1.1 as the lowest version.
tlsv1.2 <- Set TLSv1.2 as the lowest version (default).
On FortiGate.
config system global
show full
set ssl-min-protocol-version
SSLv3 SSLv3.
TLSv1 TLSv1.
TLSv1-1 TLSv1.1.
TLSv1-2 TLSv1.2.
TLSv1-3 TLSv1.3.
The TLS versions should match on both FortiGate and FortiManager.
set enc-algorithm
default <----- High strength algorithms and these medium-strength 128-bit key length algorithms: RC4-SHA, RC4-MD5, RC4-MD.
high <----- 128-bit and larger key length algorithms: DHE-RSA-AES256-SHA, AES256-SHA, EDH-RSA-DES-CBC3-SHA, DES-CBC3-SHA, DES-CBC3-MD5, DHE-RSA-AES128-SHA, AES128-SHA.
low <----- 64-bit or 56-bit key length algorithms without export restrictions: EDH-RSA-DES-CDBC-SHA, DES-CBC-SHA, DES-CBC-MD5.
The enc-algorithm can be the same on both or can be higher on FortiGate.
It can be verified on FortiManager with these commands:
get sys global
show full
This can be verified on FortiGate with these commands:
config sys global
show full
Note:
Check also if the DNS servers are reachable from FortiGate under Network -> DNS -> Check the servers are reachable.
Overview.
The 'FGFM' protocol implements a secure communication protocol with the following functions:
FortiGate reachability status (from FortiManager).
FortiManager reachability status (from FortiGate).
Configuration installation and retrieval.
Script push.
JSON monitoring via RTM.
Exceptions:
The following communications between FortiGate and FortiManager units are handled outside of the 'FGFM' protocol and are managed by the FortiGuard protocol:
- FortiGuard package downloads (AV, IPS, Virus Scan, etc.).
- FortiGuard query (WF, AS).
- Firmware Downloads.
The 'FGFM' protocol runs over SSL (Secure Sockets Layer) using TCP port 541 under IPv4. FortiManager 6.2 supports the use of IPv6.
Both FortiGate and FortiManager units have an 'FGFM' daemon running exclusively for FortiGate to FortiManager communication.
The FortiManager unit listens on TCP port 541 for an incoming session request. The FortiGate unit establishes an SSL session with the FortiManager.
Both units use TCP port 541 for sending and receiving messages.
The 'FGFM' daemon handles all FortiGate to FortiManager (and vice versa) authentication, keep-alive messages, and actions resulting from them (such as instructing another daemon on a FortiGate device to update its configuration or various database files).
Debug:
The 'diagnose fdsm central-mgmt-status' command provides the connectivity and registration status of the FortiGate with the FortiManager.
diagnose fdsm central-mgmt-status
Telnet to the FortiManager IP on port 541 to ensure reachability.
execute telnet <FMG-IP> 541
Ensure proper MTU size end to end from FortiGate to FortiManager.
execute ping-options df-bit yes
execute ping-options data-size <1472>
execute ping-options source <FortiGate ip used in fmg-source-ip in central mgmt>
execute ping <fmg IP>
It is possible to confirm the MTU size from FortiGate to FortiManager by executing the following commands on the FortiGate CLI:
execute ping-option df-bit yes -> Do not fragment ICMP packet.
execute ping-option data-size 1500 -> ICMP will add 8 bytes for the ICMP header.
execute ping x.x.x.x -> Where x.x.x.x is FortiManager-IP.
If there was packet loss, change the data size to 1470/1400/1350/1320/1312 and verify on which data size value there was no packet loss.
Then adjust if needed:
Technical Tip: How to adjust the Maximum Transmission Unit (MTU) value on a FortiGate interface
The source IP of the FortiGate can also be configured to use the respective IP of the FortiGate, which is reachable with the FortiManager. This is useful for cases such as VPN access.
config system central-management
set fmg-source-ip <FortiGate-IP>
end
The interface from which the FortiGate initiates the connection can also be specified.
config system central-management
set interface-select-method specify
set interface <interface_name>
end
If FortiGate loses the FGFM tunnel, it may be linked to FortiManager being behind a NAT and sending the wrong IP to FortiGate. To solve this, configure the management address (Configuring the management address) on a FortiManager that is behind a NAT device so the FortiGate can initiate a connection to the FortiManager. Configuring the management address setting in the CLI ensures that FortiManager knows the public IP and can configure it on the FortiGate.
config systems admin setting
set mgmt-fqdn <IP> <FQDN>
end
Note:
Use mgmt-fqdn even if only IP addresses are used and no FQDNs.
Ensure the custom certification is correct when connecting with FortiGate. FortiManager v7.2.5 introduces a new verification of the CN or SAN of a custom certificate uploaded by the FortiGate, where it should include the FortiGate Serial Number either in the CN or SAN.
The tunnel connection may fail, or clusters may show down if a matching serial number is not found.
For more information about this behavior, check this KB article: Technical Tip: Setup custom certificate for FGFM protocol
config system global
set fgfm-peercert-withoutsn enable <----- Removed on v7.2.10/v7.4.6/v7.6.1 onward.
end
Debug on FortiGate.
diagnose debug reset
diagnose debug application fgfmd 255
diagnose debug console timestamp enable
diagnose debug enable
Debug on FortiManager.
diagnose debug reset
diagnose debug application fgfmsd 255 <deviceName> <- Only show (deviceName)-related messages.*
diagnose debug timestamp enable
diagnose debug enable
Note: To use this filter device has to be registered into device database, when troubleshooting registration issues, leave the filter field empty and, once collected, filter the log by device serial-number.
To generate the output in the debugs, re-initiate the connection from the FortiGate (or) from the FortiManager:
- Re-initiate the connection from the FortiGate CLI by restarting the 'FGFM' daemon.
fnsysctl killall fgfmd
- Claim the tunnel from FortiManager CLI using the syntax below.
execute fgfm reclaim-dev-tunnel <device_name>
devicename <-- Optional device name
If this type of message appears in the FortiGate debug output, refer to the following guide for a solution:
2024-11-25 15:44:47 FGFMs: Remote issuer is /C=US/ST=California/L=Sunnyvale/O=Fortinet/OU=Certificate Authority/CN=support/emailAddress=support@fortinet.com.
2024-11-25 15:44:47 FGFMs: issuer matching...try next if not match... localissuer(fortinet-subca2001), remoteissuer(support)
2024-11-25 15:44:47 FGFMs: need change local cert to ISSUER[support]
After v7.2.10, the connection between the VM and FortiManager is restricted for security reasons. Check this article on how to allow VM platform connection in FGFM: Troubleshooting Tip: Unable to add VM into FortiManager.
Sniffer Packets:
If it is possible to ping FortiGate from FortiManager, and if FortiGate is not communicating with FortiManager, it is possible to capture the packets with the commands below.
Run sniffer on FortiGate using Putty with SSH connection and all session output logging:
diagnose sniffer packet any "port 541 and host x.x.x.x" 6 0 a <- Where x.x.x.x is the FortiManager IP address.
At the same time, run a sniffer on FortiManager with the following syntax:
diagnose sniffer packet any "port 541 and host y.y.y.y" 3 0 a <- Where y.y.y.y is the FortiGate IP address.
If the traffic from remote FortiGate is received, but there is no reply from FortiManager, and the sniffer looks like the below:
FMG-VM64 # diagnose sniffer packet any " host 10.4.110.118 " 4
interfaces=[any]
filters=[ host 10.4.110.118 ]
3.407606 10.4.110.118.42754 -> 192.168.4.250.541: syn 3738660390
4.426946 10.4.110.118.42754 -> 192.168.4.250.541: syn 3738660390
6.511686 10.4.110.118.42754 -> 192.168.4.250.541: syn 3738660390
Then check for the local-in policy on FortiManager using the command below:
show system local-in-policy
FMG-VM64 # show system local-in-policy
config system local-in-policy
edit 1
set action accept
set dport 541
set intf "port1"
set protocol TCP
set src 192.168.1.99 255.255.255.255
next
edit 2
set dport 541
set intf "port1"
set protocol TCP
set dport 541
set src 10.4.110.118 255.255.255.255
next
end
Note:
The FortiGate tries to register to the FortiManager cloud, but if the fgfm-deny-unknown setting is enabled, the FortiManager Cloud cannot accept the request from the FortiGate.
Disable the fgfm-deny-unknown on the FortiManager Cloud and restart the FGFM tunnel on the FortiGate. Then, register to the FortiManager again. After this, it will be possible to successfully authorize the FortiGate on the FortiManager Cloud.
Related article:
(global)# set fgfm-deny-unknown disable
(global)# end
FMG-VM64 #
Make sure to set the FortiManager cloud serial number in FortiGate central management settings.
config system central-management
set type fortimanager
set serial-number "FMG cloud serial number"
set fmg "fortimanager.forticloud.com"
end
FortiGate# fnsysctl killall fgfmd <----- Restarts the FGFM tunnel.
FortiGate# execute central-mgmt register-device <FMG Cloud Serial Number> fortinet <----- Register the device to FortiManager Cloud from CLI, 'fortinet' is just a dummy password.
FortiGate# diagnose fdsm central-mgmt-status <----- Check the connection and registration.
Related articles:
Technical Tip: Setup custom certificate for FGFM protocol
Technical Tip: How to verify FortiGate to FortiManager (FGFM) protocol TLS version
Technical Tip: How to create a log file of a session using PuTTY
Troubleshooting Tip: How to troubleshoot connectivity to FortiManager Cloud
Troubleshooting Tip: Understanding 'fgfm diagnose debug' log
