Hello,
I've come across fgt100d HA
installation where internal Ip doesn't show in traceroute
What I get.
>tracert -d 8.8.8.8 1 1 ms 2 ms 1 ms 2.2.2.2 2 4 ms 3 ms 3 ms 1.1.1.1
... ^C
What I would like to get
>tracert -d 8.8.8.8 1 1 ms 1 ms 1 ms 192.168.100.1 2 1 ms 2 ms 1 ms 2.2.2.2 3 4 ms 3 ms 3 ms 1.1.1.1 ^C
Setup is quite basic.
there are several Lan interfaces combined under one zone. Policy is
LANzone > wan all traffic is permitted
192.168.100.1 responds to icmp
I'm suspecting fortigate doesn't decrease TTL or it's somehow connected that traffic is processed using NP... Any ideas what could be switched on/off ;) ?
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
1st
is the traceroute udp or icmp based
2nd
The diag debug flow is your best friend
3rd
what's your allowaccess statement on the in-comming interface for the traceroute?
here's a tracerouet unix with udp and then icmp, 10.52.1.1 is my edge-fgt btw, you will see that reported in the ICMP based traceroute
PCNSE
NSE
StrongSwan
Thanks for looking into this. 1st trace issued from windows, should be ICMP 3rd while testing: set allowaccess ping https snmp http fgfm As from your trace with UDP it's obviously blocked, therefore showing * * * In my scenario - hop simply doesn't show up. If I would "translate" it to you screenshot - it would be ... 4. 10.255.255.6 5. 10.255.255.5 ----------10.52.1.1----- missing 6. <public network> nb: I went over similar installations also 100d - first hop is showing properly - direct gateway shows as a first hop(Fortigate internal IP). Only differences I can spot in this case - there are several "Vlan Interfaces" bundled into one "Zone" and then security policies are made from "Zone" to WAN1 Actually this is the first time I've came across this behaviour on Fortigate. I recall over years there were boxes (non Fortigate) where you could artificially manipulate TTL settings to make L3 Firewalls "dissapear" from traceroutes. But in this case customer didn't touch any special knobs or settings. (hopefully :))
What did diag debug flow show? I was also going to suggest a NPU issue but a 100D pretty much does not have any NPU.
FWIW a FGT140D ( similar ) that's running 5.2.4 in my lab works great with udp so I do believe you are on to something with the 1st hop being dropped by the FGT.
e.g
LAB# traceroute 8.8.8.8 port 666 <I tried my own port # for the heck of it traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 40 byte packets 1 10.64.1.1 (10.64.1.1) 0.645 ms 0.943 ms 2.359 ms 2 10.64.18.30 (10.64.18.30) 2.388 ms 0.983 ms 2.294 ms <-FGT 3 1.1.1.1 (1.1.1.1) 2.462 ms 3.677 ms 2.552 ms <- ISP 4 * * * 5 * * * 6 24.175.43.223 (24.175.43.223) 26.671 ms 17.369 ms 17.125 ms 7 24.175.41.46 (24.175.41.46) 20.408 ms 26.485 ms 24.175.41.50 (24.175.41.50) 25.853 ms 8 66.109.6.88 (66.109.6.88) 20.213 ms 22.052 ms 23.949 ms 9 107.14.17.236 (107.14.17.236) 18.466 ms 19.814 ms 107.14.17.232 (107.14.17.232) 23.181 ms 10 66.110.57.97 (66.110.57.97) 41.224 ms 36.653 ms 31.729 ms 11 * *c * 12 *
The TTL is set by the client so I don't see how a FGT or any device will adjust the TTL btw.
PCNSE
NSE
StrongSwan
Hello,
I think you can see the FortiGate as a hop on your traceroute if you disable ASIC offloading on the firewall policy.
It can be done from CLI only:
config firewall policy
edit <policy_ID>
set auto-asic-offload disable
next
end
Or instead of disabling it just wait one minute for the session to be cleared then run traceroute again, you should see the FortiGate as a hop on first traceroute.
Note that it's not recommended to disable it permanently. Just temporary for testing, once test done re-enable it as this will affect the device performance.
ASIC can't generate the TTL exceeded msg back to source, so client report timed out. While on Linux UDP traceroute it will shows Fortigate as hop, since each command has a different source and destination ports which will be processed by the kernel.
Regards,
I think you can see the FortiGate as a hop on your traceroute if you disable ASIC offloading on the firewall policy.
Never had to disable that on a policy. In fact I don't think traceroute offloaded to begin with with ICMP if that was the case every FGT firewall would require what your suggesting.
Either way you can monitor the session table the offload flags from the cli.
Ken
PCNSE
NSE
StrongSwan
Bumping old topic.
After upgrading our 100D from 5.2.5 to current 5.2.11 we have this behavior that when doing trace thru FGT, first packet gets correct reply from FGT but after that, it doesn't respond anymore and hop disappears from trace.
It sounds like first packet is handled by cpu and rest by Asic.
However, I can't find "auto-asic-offload" option anywhere under firewall policy.
When doing 'diag sys session list' there is no NPU tag or similar that would clearly indicate icmp traffic being offloaded.
It seems this 100D model has CP8 Asic.
Is this issue that icmp traffic is offloaded and how to fix it so that it would be handled by cpu?
Maybe some global config?
Clearly there has been some changes since traceroute didn't behave like this with old firmware.
Thanks!
Again.
A windows tracert using ICMP will show the fortigate as 1st hop, a unix traceroute using UDP will never show the fortigate as 1st hop depending on what the policy is set for and how many hops away from the src
Regarding ICMP the fortigate still needs to allow all ICMP or the correct ICMP for this all to happen.
Here's a few demos ( FGT v5.2.11 )
ICMP traceroute ( the same thing windows OSes does by default )
macbook:~ kfelix$ traceroute -I 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 72 byte packets 1 10.11.0.1 (10.11.0.1) 29.633 ms 10.269 ms 4.713 ms 2 10.255.254.22 (10.255.254.22) 1.672 ms 2.913 ms 1.566 ms 3 10.255.255.22 (10.255.255.22) 208.203 ms 201.552 ms 190.007 ms 4 10.255.255.6 (10.255.255.6) 203.844 ms 137.281 ms 79.122 ms 5 10.255.255.5 (10.255.255.5) 68.175 ms 77.159 ms 55.985 ms 6 10.52.1.1 (10.52.1.1) 51.844 ms 18.390 ms 21.070 ms <----FORTIGATE 7 2xxxxxxxxxxxxxx) 26.835 ms 11.950 ms 11.791 ms 8 xxxxxxx.aus3.us.above.net (208.184.x.x) 21.789 ms 16.801 ms 10.877 ms 9 ae3.mpr1.aus1.us.zip.zayo.com (64.125.31.24) 12.339 ms 10.722 ms 8.309 ms 10 ae1.cr1.dfw2.us.zip.zayo.com (64.125.27.29) 22.519 ms 13.803 ms 15.036 ms 11 ae11.er1.dfw2.us.zip.zayo.com (64.125.20.66) 15.649 ms 24.570 ms 22.355 ms 12 72.14.194.53 (72.14.194.53) 14.759 ms 13.649 ms 16.286 ms 13 108.170.252.129 (108.170.252.129) 14.626 ms 17.175 ms 22.475 ms 14 108.170.230.241 (108.170.230.241) 13.945 ms 15.908 ms 13.938 ms 15 google-public-dns-a.google.com (8.8.8.8) 14.039 ms 13.484 ms 13.659 ms
NOW udp.prt 53 to google dns which is allowed by my firewall policy, ( the 1st packet is is port 53 and incremented up by +1 )
macbook:~ kfelix$ traceroute -p 53 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets 1 10.11.0.1 (10.11.0.1) 5.633 ms 19.344 ms 2.172 ms 2 10.255.254.22 (10.255.254.22) 1.961 ms 2.175 ms 1.389 ms 3 10.255.255.22 (10.255.255.22) 424.345 ms 427.892 ms 396.251 ms 4 10.255.255.6 (10.255.255.6) 407.848 ms 433.819 ms 523.280 ms 5 10.255.255.5 (10.255.255.5) 401.046 ms *
6 * * * 7 * * * 8 * *
Now if you set a fwpoilicy for ALL services ports in the firewall the behavior would be different;
e.g ( with all services )
macbook:~ kfelix$ traceroute -p 53 8.8.8.8 traceroute to 8.8.8.8 (8.8.8.8), 64 hops max, 52 byte packets 1 10.11.0.1 (10.11.0.1) 7.658 ms 9.107 ms 2.258 ms 2 10.255.254.22 (10.255.254.22) 1.555 ms 3.262 ms 1.589 ms 3 10.255.255.22 (10.255.255.22) 447.564 ms 399.062 ms 417.110 ms 4 10.255.255.6 (10.255.255.6) 409.758 ms 513.274 ms 397.748 ms 5 10.255.255.5 (10.255.255.5) 182.166 ms * * 6 10.52.1.1 (10.52.1.1) 627.076 ms 428.114 ms 332.773 ms <---THE FGT AGAIN 7 xxxxxxxxxxxxxx 277.824 ms 249.232 ms 251.347 ms 8 xxxxxx.aus3.us.above.net (208.184.xxxx.xxxx) 253.721 ms 258.077 ms 287.120 ms 9 ae3.mpr1.aus1.us.zip.zayo.com (64.125.31.24) 302.571 ms 415.828 ms 366.882 ms 10 ae1.cr1.dfw2.us.zip.zayo.com (64.125.27.29) 357.037 ms 296.428 ms 292.251 ms 11 ae11.er1.dfw2.us.zip.zayo.com (64.125.20.66) 289.884 ms 338.082 ms 303.955 ms 12 * * *
This is pure session-state-tracking and what a firewall does.
Ken
PCNSE
NSE
StrongSwan
Yes I'm aware that FGT doesn't reply to UDP packet unless it's allowed. I'm using Windows with ICMP.
But this wasn't my point.
By "first packet" I didn't mean first hop but rather first traceroute. When doing same traceroute second time, hop has disappeared.
But I just realized that since we use this 100D as vpn endpoint, all destinations are reached thru ipsec tunnels.
And since ipsec is offloaded by CP8 Asic, could that explain behavior I was trying to explain?
All ipsec crypto devices in use: CP8: null: 0 0 des: 0 0 3des: 26092729 19952511 aes: 0 0 aria: 0 0 seed: 0 0 null: 0 0 md5: 0 0 sha1: 26092767 19952511 sha256: 0 0 sha384: 0 0 sha512: 0 0 SOFTWARE: null: 0 0 des: 0 0 3des: 0 0 aes: 0 0 aria: 0 0 seed: 0 0 null: 0 0 md5: 0 0 sha1: 0 0 sha256: 0 0 sha384: 0 0 sha512: 0 0
Could be try disabling the offload ( temporary ) but I don't think that's the issues.
Here's why;
Each traceoute is a new session ( run a diag debug flow trace and capture the :new session-: value } or use diag sys session and a filter than list out the session and look at the serial number.
So I don't see how one traceroute thats hop works ad the other does not, if we think it's a offloading issues. It could be something else going on. It's either offloaded or not but give it a try and the same behavior for that traffic flow.
Can you provide a show full of the policy that your hitting
e.g
show full firewall policy 777
PCNSE
NSE
StrongSwan
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1702 | |
1092 | |
752 | |
446 | |
228 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.