Skip to main content
afiroz
Staff
Staff
October 22, 2024

Technical Tip: Interface flapping issue on the FortiGate deployed over Azure

  • October 22, 2024
  • 0 replies
  • 2425 views
Description This article describes a scenario where interfaces of the Firewall deployed over the Azure cloud flap and how to resolve this issue.
Scope FortiGate, Azure.
Solution

This scenario is relevant for Active-passive HA with SDN connector failover deployment.
See HA for FortiGate-VM on Azure for more information on this deployment.

The interface flaps can be identified using the following logs:

  1. Under the HA history, continuous ha port flaps will be observed:

 

diagnose sys ha history read

 

Output will show as below :

 

<2024-09-27 05:02:40> port port2 link status changed: 0->1
<2024-09-27 05:02:40> port port1 link status changed: 0->1
<2024-09-27 05:02:39> port port2 link status changed: 1->0
<2024-09-27 05:02:38> port port1 link status changed: 1->0
<2024-09-27 05:01:40> port port2 link status changed: 0->1
<2024-09-27 05:01:40> port port1 link status changed: 0->1

 

  1. Under the HA log, similar logs will populate:

 

execute log filter category 1

execute log filter field subtype ha

execute log display

 

Output will show as below :

 

date=2024-09-27 time=04:52:40 eventtime=1727430760307635869 tz="-0500" logid="0108035013" type="event" subtype="ha" level="error" vd="root" logdesc="HA failover failed" msg="azd failed to add public ip in nic azprf-fortigate-fw-FGT-A-Nic1"
date=2024-09-27 time=04:51:40 eventtime=1727430700226158351 tz="-0500" logid="0108035013" type="event" subtype="ha" level="error" vd="root" logdesc="HA failover failed" msg="azd failed to add public ip in nic azprf-fortigate-fw-FGT-A-Nic1"
date=2024-09-27 time=04:50:41 eventtime=1727430640819825851 tz="-0500" logid="0108035013" type="event" subtype="ha" level="error" vd="root" logdesc="HA failover failed" msg="azd failed to add public ip in nic azprf-fortigate-fw-FGT-A-Nic1"
date=2024-09-27 time=04:49:40 eventtime=1727430580400252451 tz="-0500" logid="0108035013" type="event" subtype="ha" level="error" vd="root" logdesc="HA failover failed" msg="azd failed to add public ip in nic azprf-fortigate-fw-FGT-A-Nic1"

 

  1. From the crash log, the azd process will be bringing down the interface:

 

diagnose debug crash log read 

 

Output will shows as below : 


16380: 2024-09-27 04:45:40 Interface port2 is brought up. process_id=2424, process_name="azd"
16381: 2024-09-27 04:46:38 Interface port1 is brought down. process_id=2424, process_name="azd"
16382: 2024-09-27 04:46:38 Interface port2 is brought down. process_id=2424, process_name="azd"
16383: 2024-09-27 04:46:40 Interface port1 is brought up. process_id=2424, process_name="azd"
16384: 2024-09-27 04:46:40 Interface port2 is brought up. process_id=2424, process_name="azd"

 

  1. If the above logs match, run the debug for the SDN connector by using the following commands:

diagnose debug application azd -1

diagnose debug enable


Verify if similar API failure logs are populating:

 

azd api failed, url =
https://management.azure.com/subscriptions/ec162d24-afb6-4bb7-9c1d-6b73b5e13791/resourceGroups/hrcaz-pr/providers/Microsoft.Network/publicIPAddresses/AZPR-nuance-lb?api-version=2023-09-01
, rc = 404 {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Network/publicIPAddresses/AZPR-nuance-lb' under resource group 'hrcaz-pr' was not found. For more details please go to
https://aka.ms/ARMResourceNotFoundFix"}}

To disable the debugs:

 

diagnose debug disable

diagnose debug reset

 

The log hrcaz-pr contains the resource group where the firewall is using an API query to obtain the IP address information of AZPR-nuance-lb.

To resolve this issue, it is necessary to map the correct resource group by making changes either on the FortiGate or on Azure.

CLI commands to change the resource group for the Public IP on the FortiGate Firewall for which the errors are populating are as follows:


config system sdn-connector

    edit "AzureHA"

        config nic

            edit "FGT-FGT-A-Nic1"

                config ip

                    edit "ipconfig1"

                        set public-ip "FGTPublicIP"

                        set resource-group ''

                    next

                    edit "Test_IP" 

                        set public-ip "AZPR-nuance-lb" 

                        set resource-group '<resource_group_name as on Azure>'

                    next

                end

            next

        end

 

Running the azd debugs again shows the following output:


2026-01-22 10:55:28 rc=403, {"error":{"code":"LinkedAuthorizationFailed","message":"The client 'b21a4c74-58b4-4361-acb9-69547882d21c' with object id '2a921119-8fc0-4c45-bfa3-e83c40dc88ec' has permission to perform action 'Microsoft.Network/networkInterfaces/write' on scope '/subscriptions/48f3a014-e63e-4328-93e4-4b6fc6a0d065/resourceGroups/RG-EUW-SHAREDVNET-P/providers/Microsoft.Network/networkInterfaces/fweuwfortigate10p-nic-ext-01'; however, it does not have permission to perform action(s) 'Microsoft.Network/networkSecurityGroups/join/action' on the linked scope(s) '/subscriptions/48f3a014-e63e-4328-93e4-4b6fc6a0d065/resourceGroups/rg-euw-sharedfw-p/providers/Microsoft.Network/networkSecurityGroups/nsg-euw-fweuwfg' (respectively) or the linked scope(s) are invalid."}}
2026-01-22 10:55:28 updating nic: fweuwfortigate10p-nic-ext-01, rc: 403


Even though the identity has permission to perform Microsoft.Network/networkInterfaces/write on the NIC .../networkInterfaces/fweuwfortigate10p-nic-ext-01, it lacks the additional linked permission Microsoft.Network/networkSecurityGroups/join/action on the target NSG.

This occurs because associating an NSG with a network interface (NIC) is a separate action from simply writing to the NIC. Azure treats this as a linked authorization requirement.
The principal can modify the NIC but cannot link (join) it to the target NSG because the NSG is in a different resource group (rg-euw-sharedfw-p), and the principal lacks the necessary join permission on that NSG.

The permissions on Azure need to be checked in this case.