|
The FortiGate-6000 distributes traffic across multiple FPCs for scalability.
By default, sessions are hashed to a specific FPC based on Source IP, Destination IP, Protocol, and Port (5-tuple hash).
This ensures that return traffic for the same session goes back to the same FPC.
If all packets of a session land on the same FPC, session state is maintained locally within that FPC
When FortiGate 6000 is configured with ECMP, users may experience random session drops.
This is because different FPCs handle the traffic on Ingress and Egress
The following is an example of a debug output:
[FPC06] id=65308 trace_id=1002 func=print_pkt_detail line=5938 msg="vd-root:0 received a packet(proto=17, 10.X.X.35:35968->10.X.X.50:161) tun_id=0.0.0.0 from ZZZ-Servers. "
[FPC06] id=65308 trace_id=1002 func=init_ip_session_common line=6136 msg="allocate a new session-0817dcd6"
[FPC06] id=65308 trace_id=1002 func=vf_ip_route_input_common line=2612 msg="find a route: flag=040000 gw-10.X.X.165 via port28"
[FPC06] id=65308 trace_id=1002 func=__iprope_tree_check line=539 msg="gnum-100004, use addr/intf hash, len=54"
[FPC06] id=65308 trace_id=1002 func=get_new_addr line=1265 msg="find DNAT: IP-10.X.X.36, port-35968"
[FPC06] id=65308 trace_id=1002 func=fw_forward_handler line=992 msg="Allowed by Policy-1095: SNAT"
[FPC06] id=65308 trace_id=1002 func=__ip_session_run_tuple line=3468 msg="SNAT 10.X.X.35->10.X.X.36:35968"
[FPC06] id=65308 trace_id=1002 func=np6_hif_nturbo_build_vtag line=1237 msg="vtag->magic d153beef, vtag->coretag 33, vtag->vid 0
vtag->sip[0] 2485320a, vtag->sip[1] 0, vtag->sip[2] 0, vtag->sip[3] 0
vtag->sport 32908, vtag->mtu 1500, vtag->flags 32, vtag->np6_flag 0xc0, skb->npu_flag=0xc0880"
[FPC05] id=65308 trace_id=33 func=print_pkt_detail line=5938 msg="vd-root:0 received a packet(proto=17, 10.X.X.50:161->10.X.X.36:35968) tun_id=0.0.0.0 from port27. "
[FPC05] id=65308 trace_id=33 func=init_ip_session_common line=6136 msg="allocate a new session-0130e107"
[FPC05] id=65308 trace_id=33 func=vf_ip_route_input_common line=2612 msg="find a route: flag=80000000 gw-10.X.X.36 via root"
[FPC05] id=65308 trace_id=33 func=fw_local_in_handler line=611 msg="iprope_in_check() check failed on policy 0, drop"
Due to the ECMP route, the traffic is sent to the destination via Port 28, and the return traffic is received on Port 27.
In the debug flow above, the initial packet is processed by FPC06, and the return traffic is handled by FPC05.
To avoid this situation, make 2 changes on the FortiGate6000.
- Enable VDOM-based session tables on the Data Plane (DP) processors.
This is required because the FortiGate-6000 uses a distributed architecture with multiple DP processors, and VDOM-based session tables ensure proper session handling in an ECMP environment.
Enable VDOM-Based Session Tables:
Before configuring ECMP, run the following command in the global CLI configuration mode:
config global
config system settings
set vdom-session-mode enable
end
end
This command ensures that session tables are maintained per-VDOM, which is necessary for proper ECMP operation on the FortiGate-6000.
- Enabling auxiliary session support.
When ECMP is enabled, TCP traffic for the same session can exit and enter the FortiGate on different interfaces.
To allow this traffic to pass through, FortiOS creates auxiliary sessions.
Allowing the creation of auxiliary sessions is handled with the following command:
Apply the changes on the respective VDOMs:
config vdom
edit root
config system settings
set auxiliary-sessions enable
end
end
By default, the auxiliary-session option is disabled. This can block some TCP traffic when ECMP is enabled. If this occurs, enabling auxiliary-session may solve the problem.
|
