Hello everyone,
FortiOS: 7.2.4
Fortigate: 200E
We have two FGCP clusters and FGSP between them. FGCP clusters are georaphically spaced and RTT between them around 40-50 ms. Session sync is configured over L3 link between FGCP clusters.
We have configured pickup sessions(also expectation and connectionless).
1st FGCP cluster:
config system ha
set group-name "cluster 01"
set mode a-p
set session-pickup enable
set session-pickup-connectionless enable
set session-pickup-expectation enable
set ha-mgmt-status enable
set override disable
config system standalone-cluster
set standalone-group-id 1
set group-member-id 1
config cluster-peer
edit 1
set peerip x.x.x.x
diagnose sys ha standalone-peers
Group=1, ID=1
Detected-peers=1
Kernel standalone-peers: num=1.
peer0: vfid=0, peerip:port = y.y.y.y:708, standalone_id=2
session-type: send=249986, recv=403283
packet-type: send=0, recv=0
2nd FGCP cluster:
config system ha
set group-name "cluster 02"
set mode a-p
set session-pickup enable
set session-pickup-connectionless enable
set session-pickup-expectation enable
set ha-mgmt-status enable
set override disable
config system standalone-cluster
set standalone-group-id 1
set group-member-id 2
config cluster-peer
edit 1
set peerip y.y.y.y
diagnose sys ha standalone-peers
Group=1, ID=2
Detected-peers=1
Kernel standalone-peers: num=1.
peer0: vfid=0, peerip:port = x.x.x.x:708, standalone_id=1
session-type: send=202291, recv=4528433
packet-type: send=0, recv=0
Sessions are synchronized without problem.
So, problem is following:
When traffic symmetrical we have no problem. Symmetrical traffic mean that traffic came out from and came back to the same FGCP cluster (for example 1st FGSP cluster).
But when traffic asymmetrical we have problem: using icmp as an example we have lost first or two packets. Using TCP we have long connection, for example, to smtp services. Using UDP, for example, DNS server sometimes has timeout error. Asymmetrical traffic mean that traffic came out from one FGCP cluster and came back to another FGCP cluster (for example came out from 1st FGSP cluster and came back to 2nd FGSP cluster). So, we have this problem with both TCP and UDP.
For now we have following investigation results:
1) This problem is not related to traffic inspection and observed on both type of rules: with traffic inspection and without traffic inspection.
2) We don't observe this problem when traffic just go through FGCP clusters without NAT.
3) We don't observe this problem when traffic symmetrical.
4) Session synchronization occurs instantly with first packet on one of the FGCP cluster.
So, I suppose problem with NAT. But actually how can I debug this? Maybe some tuning options exist? Has someone encountered such a problem?
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
NAT sessions are not synchronized by default in FGSP. You can enable NAT session synchronization by entering the following command:
config system ha
set session-pickup enable
set session-pickup-nat enable
end
You need a common pool as well.
Ref:
In ForiOS 7.2.4 command "set session-pickup-nat enable" is missing and as I understood
it's enough to use command "set session-pickup enable" to sync NAT session. Moreover I have seen a session replica with status syn_ses on neighbor cluster.
It is available in 7.2.4 and is required as well along with the pool.
# get system status
Version: FortiGate-VM64 v7.2.4,build1396,230131 (GA.F)
(ha) # show
config system ha
set session-pickup enable
set session-pickup-nat enable
set override disable
end
@srajeswaranare you using VM image?
# get system status
Version: FortiGate-200E v7.2.4,build1396,230131
(ha) # session-pickup-nat enable
command parse error before 'session-pickup-nat'
Command fail. Return code -61
The same pool for NAT is created on both clusters
Hi @maxiboom ,
I can see the "session-pickup-nat" option is getting removed when FGCP is active , so what you have observed looks correct.
When you say " using icmp as an example we have lost first or two packets.", does the remaining packets work?
If thats the case, for testing can you disable anti-replay as in below doc and check the behavior?
Hello @srajeswaran,
"When you say " using icmp as an example we have lost first or two packets.", does the remaining packets work?"
Yes, the remaining packet work.
I will read about anti-replay, test it and back with results.
Hello @srajeswaran
No, it doesn't help.
For now I have done traffic dump on path from host to internet through one FGCP cluster and back tracffic from internet to host throught another FGCP cluster.
I've seen packet came out from one site: FGCP cluster > border routers
and came back to another site: border routers > another FGCP cluster. I have seen packet in debug flow in another FGCP cluster. I don't know where I should dig futher ( Maybe back packet is silently dropped by Fortigate, but I can't find debug for this situation.
Created on 02-26-2023 12:14 AM Edited on 02-26-2023 12:26 AM
Actually, I suppose that it's problem with sync session with nat. I started debug flow on both clusters same time and get results below:
1) First cluster where packet came out
2) Second cluster where packet came back
So, session found on the second packet and next packets and I can't explain it.
So packets are matching the session, but not going out. Isn't it?
I think its better to open a TAC ticket (could be a bug).
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.