port 80 being blocked for all sorts of destinations, even that it's allowed on policy

[fortinetuser2020]

look at this 

i have 1000's of these blocks for many diferent destinations with many dieferent sources on my network

the one thing they all have in common, they have no session id

why is that?

[EMES]

The firewalls don't, by default, create sessions for dropped sessions. You can disable that by running the following. http://docs.fortinet.com/...sions-to-session-table

[emnoc]

You stated earlier

even that it's allowed on policy

I would  find out what is blocking these and review  your firewall policies

e.g

diag debug reset

diag debug enable

diag debug flow fil dport 80

diag debug flow fil  addr x.x.x.x <--- place one of your internalsources addresss here

diag debug flow show console enable

diag debug flow  trace start 100

Start traffic to the destination and review what's happening

[fortinetuser2020]

thank you

this is some of the debug info

id=20085 trace_id=3246 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 84.39.152.31:80->x.x.x.x:40814) from xxxx_Internet. flag [F.], seq 1891036453, ack 1173756861, win 133" id=20085 trace_id=3246 func=resolve_ip_tuple_fast line=5438 msg="Find an existing session, id-018730d7, reply direction" id=20085 trace_id=3246 func=__ip_session_run_tuple line=3178 msg="DNAT x.x.x.x:40814->192.168.4.170:40814" id=20085 trace_id=3246 func=av_receive line=298 msg="send to application layer" id=20085 trace_id=3247 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:40814->84.39.152.31:80) from local. flag [F.], seq 1173756861, ack 1891036454, win 3918" id=20085 trace_id=3247 func=resolve_ip_tuple_fast line=5438 msg="Find an existing session, id-018730d7, original direction" id=20085 trace_id=3247 func=__ip_session_run_tuple line=3164 msg="SNAT 192.168.4.170->x.x.x.x:40814" id=20085 trace_id=3248 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 84.39.152.31:80->192.168.4.170:40814) from local. flag [F.], seq 916763649, ack 3943954755, win 133" id=20085 trace_id=3248 func=resolve_ip_tuple_fast line=5438 msg="Find an existing session, id-018730d7, reply direction" id=20085 trace_id=3248 func=ip_session_output line=494 msg="send to ips" id=20085 trace_id=3249 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:40814->84.39.152.31:80) from xxxx. flag [.], seq 3943954755, ack 916763650, win 237" id=20085 trace_id=3249 func=resolve_ip_tuple_fast line=5438 msg="Find an existing session, id-018730d7, original direction" id=20085 trace_id=3249 func=ids_receive line=282 msg="send to ips" id=20085 trace_id=3249 func=av_receive line=298 msg="send to application layer" id=20085 trace_id=3250 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 84.39.152.31:80->x.x.x.x:40814) from xxxx_Internet. flag [.], seq 1891036454, ack 1173756862, win 133" id=20085 trace_id=3250 func=resolve_ip_tuple_fast line=5438 msg="Find an existing session, id-018730d7, reply direction" id=20085 trace_id=3250 func=__ip_session_run_tuple line=3178 msg="DNAT x.x.x.x:40814->192.168.4.170:40814" id=20085 trace_id=3250 func=av_receive line=298 msg="send to application layer" id=20085 trace_id=3251 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:31738->84.39.152.32:80) from xxxx. flag [F.], seq 4037772561, ack 830311907, win 245" id=20085 trace_id=3251 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3251 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3252 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:42482->84.39.152.31:80) from xxxx. flag [F.], seq 1800468305, ack 3885295595, win 246" id=20085 trace_id=3252 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3252 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3253 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:31738->84.39.152.32:80) from xxxx. flag [F.], seq 4037772561, ack 830311907, win 245" id=20085 trace_id=3253 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3253 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3254 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:42482->84.39.152.31:80) from xxxx. flag [F.], seq 1800468305, ack 3885295595, win 246" id=20085 trace_id=3254 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3254 func=fw_forward_dirty_handler line=336 msg="no session matched"

i've already found this "forward dirty handler" and acted on fortigate's article but no change

[emnoc]

Something doesn't add up.

Q1: do you have  dual WAN uplinks

Q2: is the same host  always the problem { 192.168.4.170 } is this host dual-homed to 2 o rmore internet ( i.e look at it's route table, eliminate any wifi/mifi or other internet .......

[fortinetuser2020]

no

one uplink

it's a fortigate 100d cluster

this host is under a vdom that is attached to it's on aggregate with it's own separated physical ports and it's own ip pool with the ISP

the host itself is a mail relay under a cluster, a virtual ip actually. but only one is alive in any given time, that's a hot-standby cluster. 

[emnoc]

topology map? With out a topo we can't get an ideal of your layout. The dirty forwarder in my  experiences always equals bad routing, bad ECMP-routing or bad PBRouting issues.

[MikePruett]

Yeah, provide us as much information as possible and we can dig in.

The dirty forwarder logs are of interest to me

[fortinetuser2020]

what do you mean by topology? not the term, but the kind of information you want to know?

basically : vlan with hosts, one of the is this 192.168.4.170 trying to reach out via one of the ISP's IP's in the pool and fails like you've seen in the log

what kind of information i can provide further for you to know the layout?

[michaelbazy_FTNT]

Looks quite straightforward to me :

id=20085 trace_id=3251 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:31738->84.39.152.32:80) from xxxx. flag [F.], seq 4037772561, ack 830311907, win 245" id=20085 trace_id=3251 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3251 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3252 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:42482->84.39.152.31:80) from xxxx. flag [F.], seq 1800468305, ack 3885295595, win 246" id=20085 trace_id=3252 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3252 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3253 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:31738->84.39.152.32:80) from xxxx. flag [F.], seq 4037772561, ack 830311907, win 245" id=20085 trace_id=3253 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3253 func=fw_forward_dirty_handler line=336 msg="no session matched" id=20085 trace_id=3254 func=print_pkt_detail line=5363 msg="vd-xxxx received a packet(proto=6, 192.168.4.170:42482->84.39.152.31:80) from xxxx. flag [F.], seq 1800468305, ack 3885295595, win 246" id=20085 trace_id=3254 func=vf_ip_route_input_common line=2583 msg="find a route: flag=04000000 gw-xxxx via xxxx_Internet" id=20085 trace_id=3254 func=fw_forward_dirty_handler line=336 msg="no session matched"

clients are sending Fin (flag F, in bold... or is it Fin/Ack?) packets to close sessions that are (supposedly?) already closed - maybe they got a timeout.

If you log everything, maybe you can relate the srcIP:srcPort->dstIP:dstPort (in italic) combination to find traffic related to these traffic, that ended because of a timeout. I wouldn't be surprised to see that session traffic got a timeout (default is 3600s, but maybe it's been changed to save RAM?) before receiving a FIN packet from the client.