Support Forum
The Forums are a place to find answers on a range of Fortinet products from peers and product experts.
maxiboom
New Contributor III

Problem with FGSP and FGCP

Hello everyone,

 

FortiOS: 7.2.4

Fortigate: 200E

 

We have two FGCP clusters and FGSP between them. FGCP clusters are georaphically spaced and RTT between them around 40-50 ms. Session sync is configured over L3 link between FGCP clusters.

We have configured pickup sessions(also expectation and connectionless).

 

1st FGCP cluster:

config system ha
  set group-name "cluster 01"
  set mode a-p
  set session-pickup enable
  set session-pickup-connectionless enable
  set session-pickup-expectation enable
  set ha-mgmt-status enable
  set override disable

 

config system standalone-cluster
  set standalone-group-id 1
  set group-member-id 1
  config cluster-peer
    edit 1
      set peerip x.x.x.x

diagnose sys ha standalone-peers
  Group=1, ID=1
  Detected-peers=1
  Kernel standalone-peers: num=1.
    peer0: vfid=0, peerip:port = y.y.y.y:708, standalone_id=2
    session-type: send=249986, recv=403283
    packet-type: send=0, recv=0

 

2nd FGCP cluster:

config system ha
  set group-name "cluster 02"
  set mode a-p
  set session-pickup enable
  set session-pickup-connectionless enable
  set session-pickup-expectation enable
  set ha-mgmt-status enable
  set override disable

 

config system standalone-cluster
  set standalone-group-id 1
  set group-member-id 2
  config cluster-peer
    edit 1
     set peerip y.y.y.y

diagnose sys ha standalone-peers
  Group=1, ID=2
  Detected-peers=1
  Kernel standalone-peers: num=1.
  peer0: vfid=0, peerip:port = x.x.x.x:708, standalone_id=1
  session-type: send=202291, recv=4528433
  packet-type: send=0, recv=0

 

Sessions are synchronized without problem.

 

So, problem is following:

When traffic symmetrical we have no problem. Symmetrical traffic mean that traffic came out from and came back to the same FGCP cluster (for example 1st FGSP cluster).

 

But when traffic asymmetrical we have problem: using icmp as an example we have lost first or two packets. Using TCP we have long connection, for example, to smtp services. Using UDP, for example, DNS server sometimes has timeout error. Asymmetrical traffic mean that traffic came out from one FGCP cluster and came back to another FGCP cluster (for example came out from 1st FGSP cluster and came back to 2nd FGSP cluster). So, we have this problem with both TCP and UDP.

 

For now we have following investigation results:

1) This problem is not related to traffic inspection and observed on both type of rules: with traffic inspection and without traffic inspection.

2) We don't observe this problem when traffic just go through FGCP clusters without NAT.

3) We don't observe this problem when traffic symmetrical.

4) Session synchronization occurs instantly with first packet on one of the FGCP cluster.

 

So, I suppose problem with NAT. But actually how can I debug this? Maybe some tuning options exist? Has someone encountered such a problem?

20 REPLIES 20
srajeswaran

Can you share the complete debug flow from both FGSP members.

Regards,
Suraj
- Have you found a solution? Then give your helper a "Kudos" and mark the solution.
maxiboom
New Contributor III

Screenshots above are complete five ping or what do you mean?

maxiboom
New Contributor III

Yes, session is matched, but only from the second packet. I think that's why the first packet is lost.

 

srajeswaran

do you have the debug for the first packet?

Regards,
Suraj
- Have you found a solution? Then give your helper a "Kudos" and mark the solution.
maxiboom
New Contributor III

yes, I have,

id=65308 trace_id=37 func=print_pkt_detail line=5868 msg="vd-root:0 received a packet(proto=1, x.x.x.x -> y.y.y.y) tun_id=0.0.0.0 from IFNAME. type=0, code=0, id=188, seq=1."
id=65308 trace_id=37 func=vf_ip_route_input_common line=2605 msg="find a route: flag=00000000 gw-z.z.z.z via IFNAME"

id=65308 trace_id=38 func=print_pkt_detail line=5868 msg="vd-root:0 received a packet(proto=1, x.x.x.x -> y.y.y.y) tun_id=0.0.0.0 from IFNAME. type=0, code=0, id=188, seq=2."
id=65308 trace_id=38 func=resolve_ip_tuple_fast line=5956 msg="Find an existing session, id-0577797d, reply direction"
id=65308 trace_id=38 func=vf_ip_route_input_common line=2605 msg="find a route: flag=00000000 gw-z.z.z.z via IFNAME"
id=65308 trace_id=38 func=npu_handle_session44 line=1199 msg="Trying to offloading session from IFNAME to IFNAME, skb.npu_flag=00000000 ses.state=00010280 ses.npu_state=0x04000000"
id=65308 trace_id=38 func=fw_forward_dirty_handler line=414 msg="state=00010280, state2=00000400, npu_state=04000000"
srajeswaran

Is this from the backup FGSP member and for the packet that is dropped? As per this debug the packet is not dropped by flow processing and needs further investigation (may be on the NPU level).

Regards,
Suraj
- Have you found a solution? Then give your helper a "Kudos" and mark the solution.
maxiboom
New Contributor III

Yes, right, it's from backup unit. And yes, packet isn't dropped, but this packet can't match session as I can saw on debug flow in GUI. What should I investigate on NPU level?

srajeswaran

As per the debug the packet is matiching the session, isnt it?

id=65308 trace_id=38 func=resolve_ip_tuple_fast line=5956 msg="Find an existing session, id-0577797d, reply direction"

 

Regards,
Suraj
- Have you found a solution? Then give your helper a "Kudos" and mark the solution.
maxiboom
New Contributor III

Right, but this is the second packet. First packet is trace_id = 37.

AEK
SuperUser
SuperUser

Hello

Any fix for this issue?

 

AEK
AEK
Announcements

Select Forum Responses to become Knowledge Articles!

Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.

Labels
Top Kudoed Authors