| Solution |
When a firewall policy is modified (Locally or pushed by FortiManager), all the existing sessions are dropped by FortiGate with the error 'Anti-Replay Check Fails, Drop' in the debug flow output, and existing sessions are flagged as 'dirty'.
Working scenario debugs:
Debug Flow output:
id=65308 trace_id=1629 func=print_pkt_detail line=5862 msg="vd-root:0 received a packet(proto=6, 172.16.192.130:1521->172.16.192.69:51286) tun_id=0.0.0.0 from 630_database. flag [.], seq 383407350, ack 579585442, win 505"
id=65308 trace_id=1629 func=resolve_ip_tuple_fast line=5950 msg="Find an existing session, id-23735c01, reply direction"
id=65308 trace_id=1629 func=npu_handle_session44 line=1333 msg="Trying to offloading session from 630_database to 601-psoft-app, skb.npu_flag=00000400 ses.state=00010204 ses.npu_state=0x00000400"
id=65308 trace_id=1629 func=ip_session_install_npu_session line=381 msg="npu session installation succeeded"
id=65308 trace_id=1629 func=fw_forward_dirty_handler line=439 msg="state=00010204, state2=00000001, npu_state=00000c00"
Session List output:
session info: proto=6 proto_state=01 duration=17 expire=3596 timeout=3600 flags=00000000 socktype=0 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu f00 log-start
statistic(bytes/packets/allow_err): org=28111/81/1 reply=58770/94/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=74->75/75->74 gwy=0.0.0.0/0.0.0.0
hook=pre dir=org act=noop 172.16.192.69:51286->172.16.192.130:1521(0.0.0.0:0)
hook=post dir=reply act=noop 172.16.192.130:1521->172.16.192.69:51286(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
dst_mac=56:6c:42:5a:00:03
misc=0 policy_id=80 pol_uuid_idx=18303 auth_info=0 chk_client_info=0 vd=0
serial=23735c01 tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=00000000 ngfwid=n/a
npu_state=0x000c00 ofld-O ofld-R
npu info: flag=0x81/0x81, offload=9/9, ips_offload=0/0, epid=179/179, ipid=179/179, vlan=0x0259/0x0276
vlifid=179/179, vtag_in=0x0259/0x0276 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=2/2
Non-working scenario debugs:
Debug Flow output:
id=65308 trace_id=1895 func=print_pkt_detail line=5862 msg="vd-root:0 received a packet(proto=6, 172.16.192.69:51286->172.16.192.130:1521) tun_id=0.0.0.0 from 601-psoft-app. flag [.], seq 580634445, ack 385974823, win 6146"
id=65308 trace_id=1895 func=resolve_ip_tuple_fast line=5950 msg="Find an existing session, id-23735c01, original direction"
id=65308 trace_id=1895 func=tcp_anti_reply line=1042 msg="replay packet(seq_check), suspicious"
id=65308 trace_id=1895 func=ip_session_core_in line=6601 msg="anti-replay check fails, drop"
Session List output:
session info: proto=6 proto_state=01 duration=124 expire=3589 timeout=3600 flags=00000000 socktype=0 sockport=0 av_idx=0 use=3
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log dirty may_dirty npu f00 log-start
statistic(bytes/packets/allow_err): org=1325445/5302/1 reply=2877945/5967/1 tuples=2
tx speed(Bps/kbps): 12773/102 rx speed(Bps/kbps): 29198/233
orgin->sink: org pre->post, reply pre->post dev=74->75/75->74 gwy=0.0.0.0/0.0.0.0
hook=pre dir=org act=noop 172.16.192.69:51286->172.16.192.130:1521(0.0.0.0:0)
hook=post dir=reply act=noop 172.16.192.130:1521->172.16.192.69:51286(0.0.0.0:0)
pos/(before,after) 0/(0,0), 0/(0,0)
dst_mac=56:6f:56:5a:00:08
misc=0 policy_id=80 pol_uuid_idx=18303 auth_info=0 chk_client_info=0 vd=0
serial=23735c01 tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=00000000 ngfwid=n/a
npu_state=00000000
npu info: flag=0x00/0x00, offload=0/0, ips_offload=0/0, epid=0/0, ipid=0/0, vlan=0x0000/0x0000
vlifid=0/0, vtag_in=0x0000/0x0000 in_npu=0/0, out_npu=0/0, fwd_en=0/0, qid=0/0
no_ofld_reason:
Resolution:
Issue #1015616 has been resolved as of the following FortiOS versions:
Workaround: (Each solution works individually. There is no need to apply more than one.)
- Option 1: check only new sessions instead of all sessions (by default).
- Configuration change at the policy level for scenarios where the affected applications and related policies are identifiable.
config system settings
set firewall-session-dirty check-policy-option
end
config firewall policy
edit <policy_id>
set firewall-session-dirty check-new
next
end
- Global configuration for cases where there are too many affected applications and policies to be changed:
config system settings
set firewall-session-dirty check-new
end
- Option 2: Disable anti-reply globally or in the firewall policy.
config system global
set anti-replay disable
end
Or:
config firewall policy
edit <id>
set anti-replay disable
next
end
Note:
- When the global anti-replay option is disabled, the FortiGate does not check TCP flags in packets.
- Existing sessions will still be dropped by anti-replay during traffic revalidation.
- The setting will be applied only for new sessions. Clearing existing session will help to have results directly
- Option 3: Disable offloading on firewall policy for which traffic is affected.
config system settings
set firewall-session-dirty check-policy-option
end
config firewall policy
edit <id>
set auto-asic-offload disable
end
Note:
- Disabling offloading means the primary CPU of the FortiGate will handle the traffic. It may affect global performances. This method is used only for troubleshooting purposes.
- Existing sessions will still be dropped by anti-replay during traffic revalidation.
- The setting will be applied only for new sessions. Clearing existing session will help to have results directly
Logs required by FortiGate TAC for further investigation:
- Debugs:
diagnose sys session list
diagnose debug flow filter addr <IP>
diagnose debug console timestamp enable
diagnose debug flow show function-name enable
diagnose debug flow show iprope enable
diagnose debug flow trace start 1000000000
diagnose debug enable
<reproduce the issue by modifying the firewall policy>
diagnose debug reset
diagnose sys session list
- TAC Report:
execute tac report
- Configuration file of the FortiGate.
|
