I just overcame an interesting issue last night with a remote office. Unit is a FGT-60C running 4.3.18 with no significant changes made for a long time.
The relevant setup details:
[ul]
What was the problem?
[ul]
What did I initially think was the issue?
[ul]
What was the actual issue and fix?
[ul]Why was this worth posting?
[ul]
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
[ul]The ISP that provides the 2 x PPPoE DSL lines changed the default gateway without any announcement. (I have monitors in place to make sure the interfaces are up from the outside, but the static IP addresses themselves did not change.) After combing through the settings on the FGT, I found the default gateways didn't match in the two policy routes that used the PPPoE DSL lines. I updated the default gateway on the policy routes to match what was showing in interface settings, and traffic immediately started to flow normally again.[/ul]
You had changes in your ISP, so how would you expect this to be a bug?
A diag debug flow output might have shed some light and a diag arp list for the next-hop gateways address.
If I'm understanding your current setup now, you have 2x DSL from the same provider and with next-hop-gateway address being the same?
PCNSE
NSE
StrongSwan
emnoc,
When the ISP settings change, they populate automatically onto the interfaces.
So while the settings for the ISP were correct, and the Domain VLAN passed all traffic normally, the Guest VLAN passed no traffic whatsoever, just because of the policy routes had an incorrect gateway which were only applicable to two specific IPs/devices. I can't believe that is expected behaviour, hence my question if it is a bug.
When the ISP settings change, they populate automatically onto the interfaces.
Qs for clarity;
Q1:So you have DHCP on ISP facing interface?
Q2:Both interface are assigned in the same local broadcast domain and with the same netmask and layer3 next-hop?
Q3: on the PBR and bad gateway was the gateway out of the layer3 netmask?
Q4: in your PBR cfg did you use gateway and output device or just defined one
Q5: I know it's late know, but from a case perspective could you re-create the post ISP PBR route for let's say one destination and run a diag debug flow but also look at the layer2 arp entry diag ip arp list
I think think the bug might be if both interface where dhcp-dynamic-assigned & within the same subnet ( see Q1 ). if this happen that would not be a good thing.
PCNSE
NSE
StrongSwan
emnoc wrote:
Q1:So you have DHCP on ISP facing interface?
Yes. But via PPPoE so it is PPPoE that is selected on the interface, not the DHCP option. But no settings besides the PPPoE username and password are hard coded by me on the FGT.
emnoc wrote:They always had the same next-hop, but prior to the unannounced change they had different default gateways. Now they have the same default gateway also.
Q2:Both interface are assigned in the same local broadcast domain and with the same netmask and layer3 next-hop?
emnoc wrote:With PPPoE I don't see what the netmask of the default gateway is, but by the separation between the IPs I am assigned and the gateway, it is seems to me it is fairly large. In the routing table, the default gateway is /32, as are my IPs. Here is what it shows right now:
Q3: on the PBR and bad gateway was the gateway out of the layer3 netmask?
Connected xxx.28.118.230/32 0 0 0.0.0.0 ppp1
Connected xxx.28.82.108/32 0 0 0.0.0.0 ppp2
Connected xxx.28.124.253/32 0 0 0.0.0.0 ppp1
Connected xxx.28.124.253/32 0 0 0.0.0.0 ppp2
emnoc wrote:
Q4: in your PBR cfg did you use gateway and output device or just defined one
I think what you're asking is if I defined a destination address/mask or not? I only defined the incoming interface, source address (i.e., specific device I wanted to use the PBR), outgoing interface and gateway address.
emnoc wrote:
Q5: I know it's late know, but from a case perspective could you re-create the post ISP PBR route for let's sayone destination and run a diag debug flow but also look at the layer2 arp entry diag ip arp list
Sorry but this office is across an ocean, so it would be irresponsible to try re-create such a serious issue. If I had lost the Domain VLAN as well because of this, I probably would have been getting on a plane to sort this out because everything would have died.
emnoc wrote:
I think think the bug might be if both interface where dhcp-dynamic-assigned & within the same subnet ( see Q1 ). if this happen that would not be a good thing.
I agree with this. I am sure it is a very specific set of circumstances that led to all the traffic dying as it did.
They always had the same next-hop, but prior to the unannounced change they had different default gateways. Now they have the same default gateway also.
FWIW , a next-hop == gateway
You can open a ticket with TAC, provide the before and after cfg and before & after related network changes and see what they say.
PCNSE
NSE
StrongSwan
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1710 | |
1093 | |
752 | |
447 | |
231 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.