Skip to main content
chr1zzo
New Member
February 26, 2024
Solved

OnPrem Data Gateway looses Connection

  • February 26, 2024
  • 5 replies
  • 2239 views

Hey all,

 

my on Prem Data Gateway connects to Office 365 via 443(no inspection), but after few minutes it looses connection. 

The issue is definitely the forti. With a direct uplink the connection is persistent.

 

How can i debug? What kind of trace can i perform to find the issue? Any direct ideas?

 

Thanks and best regards

Best answer by hbac

@chr1zzo,

 

Are on Prem Data Gateway connected directly to the FortiGate or through a switch. You can connect directly to rule out switch issue. 

 

Are you using SDWAN? It would be useful to connect packet capture when it is disconnecting. 

 

Regards, 

5 replies

hbac
Staff
Staff
February 26, 2024

Hi @chr1zzo,

 

What is the FortiGate firmware version? Do you notice high CPU/Memory when the connection is lost. Does it only happen to Office 365? 

 

Regards, 

chr1zzo
chr1zzoAuthor
New Member
February 27, 2024

Hi @hbac ,

 

There are two Forti's (601F) in an active passive cluster.

 

The Firmware is "Version: FortiGate-601F v7.0.13,build0566,231024 (GA.M)"

 

There are no CPU/Memory Peaks i noticed.

CPU states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU0 states: 1% user 0% system 0% nice 99% idle 0% iowait 0% irq 0% softirq
CPU1 states: 0% user 1% system 0% nice 99% idle 0% iowait 0% irq 0% softirq
CPU2 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU3 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU4 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU5 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU6 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU7 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU8 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU9 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU10 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
CPU11 states: 0% user 0% system 0% nice 100% idle 0% iowait 0% irq 0% softirq
Memory: 16394768k total, 4162252k used (25.4%), 11440372k free (69.8%), 792144k freeable (4.8%)
Average network usage: 68861 / 68984 kbps in 1 minute, 69618 / 69476 kbps in 10 minutes, 66378 / 66171 kbps in 30 minutes
Maximal network usage: 163840 / 165348 kbps in 1 minute, 244627 / 242009 kbps in 10 minutes, 244627 / 246152 kbps in 30 minutes
Average sessions: 7916 sessions in 1 minute, 6863 sessions in 10 minutes, 6278 sessions in 30 minutes
Maximal sessions: 8424 sessions in 1 minute, 8434 sessions in 10 minutes, 8434 sessions in 30 minutes
Average session setup rate: 81 sessions per second in last 1 minute, 91 sessions per second in last 10 minutes, 90 sessions per second in last 30 minutes
Maximal session setup rate: 150 sessions per second in last 1 minute, 207 sessions per second in last 10 minutes, 256 sessions per second in last 30 minutes
Average NPU sessions: 1005 sessions in last 1 minute, 855 sessions in last 10 minutes, 900 sessions in last 30 minutes
Maximal NPU sessions: 1062 sessions in last 1 minute, 1113 sessions in last 10 minutes, 1390 sessions in last 30 minutes
Average nTurbo sessions: 0 sessions in last 1 minute, 0 sessions in last 10 minutes, 0 sessions in last 30 minutes
Maximal nTurbo sessions: 0 sessions in last 1 minute, 0 sessions in last 10 minutes, 1 sessions in last 30 minutes
Virus caught: 0 total in 1 minute
IPS attacks blocked: 0 total in 1 minute

Office 365 is the only mentioned so far. The is an/are established sessions, but it seems no to be there

 

session info: proto=6 proto_state=06 duration=8 expire=2 timeout=3600 flags=00000000 socktype=0 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu synced f00
statistic(bytes/packets/allow_err): org=563/7/1 reply=2367/3/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=59->9/9->59 gwy=x.x.x.x/0.0.0.0
hook=post dir=org act=snat x.x.x.x:51566->x.x.x.x:80(x.x.x.x:51566)
hook=pre dir=reply act=dnat x.x.x.x:80->x.x.x.x:51566(x.x.x.x:51566)
pos/(before,after) 0/(0,0), 0/(0,0)
src_mac=x:x:x:x:x:x
misc=0 policy_id=22 pol_uuid_idx=1181 auth_info=0 chk_client_info=0 vd=0
serial=013d1fda tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=80000000 ngfwid=n/a
npu_state=0x4000c00 ofld-O ofld-R
npu info: flag=0x81/0x81, offload=9/9, ips_offload=0/0, epid=130/176, ipid=176/130, vlan=0x0028/0x0000
vlifid=176/130, vtag_in=0x0028/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=6/4

 

session info: proto=6 proto_state=06 duration=9 expire=2 timeout=3600 flags=00000000 socktype=0 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu synced f00
statistic(bytes/packets/allow_err): org=707/10/1 reply=2367/3/1 tuples=2
tx speed(Bps/kbps): 0/0 rx speed(Bps/kbps): 0/0
orgin->sink: org pre->post, reply pre->post dev=59->9/9->59 gwy=x.x.x.x/0.0.0.0
hook=post dir=org act=snat x.x.x.x:51563->x.x.x.x:80(x.x.x.x:51563)

hook=pre dir=reply act=dnat x.x.x.x:80->x.x.x.x:51563(x.x.x.x:51563)
pos/(before,after) 0/(0,0), 0/(0,0)
src_mac=x:x:x:x:x:x
misc=0 policy_id=22 pol_uuid_idx=1181 auth_info=0 chk_client_info=0 vd=0
serial=013d1fc9 tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=80000000 ngfwid=n/a
npu_state=0x4000c00 ofld-O ofld-R
npu info: flag=0x81/0x81, offload=9/9, ips_offload=0/0, epid=130/176, ipid=176/130, vlan=0x0028/0x0000
vlifid=176/130, vtag_in=0x0028/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=1/6

 

session info: proto=6 proto_state=01 duration=178 expire=3595 timeout=3600 flags=00000000 socktype=0 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
per_ip_shaper=
class_id=0 ha_id=0 policy_dir=0 tunnel=/ vlan_cos=0/255
state=log may_dirty npu synced f00
statistic(bytes/packets/allow_err): org=3999/31/1 reply=13522/37/1 tuples=2
tx speed(Bps/kbps): 23/0 rx speed(Bps/kbps): 47/0
orgin->sink: org pre->post, reply pre->post dev=59->9/9->59 gwy=x.x.x.x/0.0.0.0
hook=post dir=org act=snat x.x.x.x:51558->x.x.x.x:443(x.x.x.x:51558)
hook=pre dir=reply act=dnat x.x.x.x:443->x.x.x.x:51558(x.x.x.x:51558)
pos/(before,after) 0/(0,0), 0/(0,0)
src_mac=x:x:x:x:x:x
misc=0 policy_id=22 pol_uuid_idx=1181 auth_info=0 chk_client_info=0 vd=0
serial=013cc7ef tos=ff/ff app_list=0 app=0 url_cat=0
rpdb_link_id=80000000 ngfwid=n/a
npu_state=0x4000c00 ofld-O ofld-R
npu info: flag=0x81/0x81, offload=9/9, ips_offload=0/0, epid=130/176, ipid=176/130, vlan=0x0028/0x0000
vlifid=176/130, vtag_in=0x0028/0x0000 in_npu=1/1, out_npu=1/1, fwd_en=0/0, qid=7/6
total session 3

chr1zzo
chr1zzoAuthor
New Member
February 27, 2024

i have seen something strange while performing an debug flow trace:

id=20085 trace_id=5996 func=init_ip_session_common line=6117 msg="allocate a new session-013b9d00, tun_id=0.0.0.0"
id=20085 trace_id=5996 func=__vf_ip_route_input_rcu line=1999 msg="find a route: flag=00000000 gw-x.x.x.x via port3"
id=20085 trace_id=5996 func=get_new_addr line=1227 msg="find SNAT: IP-x.x.x.x(from IPPOOL), port-51547"
id=20085 trace_id=5996 func=fw_forward_handler line=888 msg="Allowed by Policy-22: SNAT"
id=20085 trace_id=5996 func=ip_session_confirm_final line=3185 msg="npu_state=0x4000000, hook=4"
id=20085 trace_id=5996 func=__ip_session_run_tuple line=3529 msg="SNAT x.x.x.x->x.x.x.x:51547"
[2062] fap_fsw_lst_req: buf of https is too small: 581

 

792831

[2062] fap_fsw_lst_req: buf of https is too small: 853 debug message appears in console when upgrading to certain builds.

 

But i can't connect it. Can this be an issue?

Thought Leadership Security Summit. Outpace New Threats with AI - enhanced defense. Tuesday, Septmeber 15, 8:30 AM - 2:30 PM PT. The Golf Club at Newcastle, WA.
Fortinet Flag the Hack. Wednesday, August 26, 9:00 AM - 5:00 PM ET, COSM, Atlanta, GA.
Virtual event | September 2026. SASE summit. The age of autonomous trust. Register here!