Fortigate blocked TCP packets with PSH flag.

[Polikarpov]

Hello community!

Does someone have any idea what is the issue?

PSH flag in TCP packets is rarely used in common life, but our NMEA-to-IP converter is using this.

Fortigate did not allow it to pass and did not logged it as a blocked.

Session was successfully established - SYN, SYN-ACk and ACK passing through firewall,

but PSH-ACK did not want to pass.

I have played with auto-asic-offload config, np-acceleration and inspection proxy/flow mode it did not help.

Wireshark capture:

 

Config:

Firewall (158) # show

config firewall policy

    edit 158

        set name "IT Manager to Nmea converter"

        set uuid fa3c5914-cca8-51e9-819d-57551413207d

        set srcintf "vsw.port16"

        set dstintf "GPS"

        set srcaddr "all"

        set dstaddr "all"

        set action accept

        set schedule "always"

        set service "ALL"

        set profile-protocol-options "NMEA"

        set ssl-ssh-profile "NMEA"

        set logtraffic all

        set fsso disable

    next

end

Aleksandr

[Yurisk]

I wouldn't be sure the reason is PSH flag, as it is a valid flag in TCP sessions, and I see it regularly passing FGs without a problem. I'd suggest : 

1. Making sure it arrives at the FG and is indeed being dropped by it:

FG# dia sniffer packet any 'Ip of oustide host receiving the connection' 4

And see if the same packet enters and leaves the FG or just enters it.

2. If the above shows packet arrives but doesnt leave the FG, run debug flow to see why:

diag debug reset

diag debug flow filter addr <ip of outside host receiving the connection>

diag debug flow show function-name enable

diag debug flow trace start 100

Then initiate the connection and post the result.

[Polikarpov]

Thank you Yuri for prompt and detailed help!

Sniffer shows the packet is receiving by FG, but it does not go out.

The result of debug below:

[size="1"]id=20085 trace_id=1043 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 192.168.202.3:2000->10.49.41.150:56976) from GPS. flag [.], seq 611704, ack 1946171203, win 2920"[/size] id=20085 trace_id=1043 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, reply direction" id=20085 trace_id=1043 func=npu_handle_session44 line=1139 msg="Trying to offloading session from GPS to vsw.port16, skb.npu_flag=00000000 ses.state=00140386 ses.npu_state=0x00000001" id=20085 trace_id=1043 func=fw_forward_dirty_handler line=449 msg="state=00140386, state2=00000001, npu_state=00000001" id=20085 trace_id=1043 func=av_receive line=305 msg="send to application layer" [size="1"]id=20085 trace_id=1044 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 10.49.41.150:56976->192.168.202.3:2000) from local. flag [.], seq 1946171203, ack 611741, win 65535"[/size] id=20085 trace_id=1044 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, original direction" [size="1"]id=20085 trace_id=1045 func=print_pkt_detail line=5501 msg="vd-root:0 received a packet(proto=6, 192.168.202.3:2000->10.49.41.150:56976) from GPS. flag [.], seq 611741, ack 1946171203, win 2920"[/size] id=20085 trace_id=1045 func=resolve_ip_tuple_fast line=5581 msg="Find an existing session, id-098026bf, reply direction" id=20085 trace_id=1045 func=npu_handle_session44 line=1139 msg="Trying to offloading session from GPS to vsw.port16, skb.npu_flag=00000000 ses.state=00140386 ses.npu_state=0x00000001" id=20085 trace_id=1045 func=fw_forward_dirty_handler line=449 msg="state=00140386, state2=00000001, npu_state=00000001" id=20085 trace_id=1045 func=av_receive line=305 msg="send to application layer"

I have temporary put old Cisco 1841 between these networks and problem disappeared.

Is there any chance to get this packet passing through Fortigate?

Aleksandr

[Yurisk]

Strange indeed, in the FG sniffer if you look 3 lines above where 192.168.202.3 returns ACk it also gets blocked (and no PSH flag), seems like FG drops any returning traffic from 192.168.202 except SYN+ACK. On the other hand the traffic is strange as well - first 4 packets establish successfully TCP session, then instead of using this session, the host 10.49 again sends SYN out of the blue ... 

if it is some SCADA/IOT irregular traffic FG may have problems statefully understanding it. 

Is it in proxy mode (I see it is, have you tried FLow mode instead?)? Do you have some fancy VIP/NAT ? What does profile do (why custom, not "default") 

set profile-protocol-options "NMEA" ?

[darwin_FTNT]

id=20085 trace_id=1045 func=av_receive line=305 msg="send to application layer"

It indicates proxy-based utm feature is used.  If it is flow-based utm, it is "send to ips".

To debug proxy-based utm: "diag debug application wad -1"

To debug flow-based utm: "diag debug application ipsengine -1" OR "diag ips debug enable all"

Note that the above commands will print out lot's of debug messages and bring a network traffic to halt.

You can specify filter like ip address to reduce debug slow down. For proxy-based utm, it means 2 connections is done from client to wad daemon, then wad daemon to server (proxy mode).  So it could be the tcp flag is processed by wad daemon but the new connection to server-side didn't use it. Disable all utm features first so either "send to application" or "send to ips" in "diag debug flow..." doesn't show up.

This means the packet is handled by kernel directly or offloaded to npu / hw accel is used/triggered for better performance. Hope it helps.

[emnoc]

I would also check and make note of the . set anti-replay option in gloal cfg 

e.g

   show full sys global | grep -f anti

and adjust and restest.

Ken Felix

[Polikarpov]

I have tried on different Fortigate model, disabled anti-reply changed from proxy to flow and from flow to proxy.

The same result. All ips features are disabled. No additional info was seen then ips debugging was activated.

[size="1"]id=20085 trace_id=276 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.202.3:2000->192.168.100.6:63071) from local. flag [.], seq 1177162469, ack 838393172, win 5840"[/size] id=20085 trace_id=276 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, reply direction" [size="1"]id=20085 trace_id=277 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.202.3:2000->192.168.100.6:63071) from local. flag [F.], seq 1177162469, ack 838393172, win 5840"[/size] id=20085 trace_id=277 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, reply direction" [size="1"]id=20085 trace_id=278 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.100.6:63071->192.168.202.3:2000) from local. flag [F.], seq 1366320358, ack 262568, win 5840"[/size] id=20085 trace_id=278 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, original direction" [size="1"]id=20085 trace_id=279 func=print_pkt_detail line=4479 msg="vd-root received a packet(proto=6, 192.168.100.6:63071->192.168.202.3:2000) from internal3. flag [.], seq 838393172, ack 1177162470, win 8212"[/size] id=20085 trace_id=279 func=resolve_ip_tuple_fast line=4542 msg="Find an existing session, id-02a8a75b, original direction" id=20085 trace_id=279 func=av_receive line=255 msg="send to application layer"

[beacher66]

Hey

Anyone solved this Problem with PSH packet block. I have same issu now...

THX for help

Beat

[boneyard]

good chance you have a different issue because as mentioned before PSH is a quite normal TCP option and gets allowed all the time. please perform the troubleshooting above to show your exact issue and start a new thread then.

also if you want a quicker answer work with Fortinet support, they can guide you through the troubleshooting and work with you on the actual device and traffic.

[ftisxf]

I have solved this problem under the official technical support, share it with you。

# config system settings set sccp-port 1999 end #

本质原因： 思科的SCCP协议端口和会话所用的端口冲突。

这种现象是由于程序代码中用一段知名代码检验网络是否通畅，这段持续恰好用2000端口，所以恰好造成了这样的问题。

[image][/image]