Description
This article describes how to fix failover issues in FortiGate Azure-HA configured using AZURE SDN.
Scope
- AZURE Deployed FortiGate in HA with AZURE SDN configured as external connectors.
- When the FortiGate undergoes failover, The Public IP Associated with NIC(.5) should be transferred to NIC(.6).
- Similarly, the LAN Gateway of the AZURE VNET should be moved from NIC(.69) to NIC (.70).
- However, this process fails to happen due to the AZURE SDN fabric connector not working as expected.
Solution
- Log in to the FortiGate Primary and secondary and run the following debug commands:
diag debug application azd -1
diag debug enable
Select the refresh icon on the external fabric GUI page:
- After performing the refresh, wait for 5 minutes and output will appear on the CLI. It will show a failure due to 'AUTHORIZATION FAILED':
-
This output message means that the IAM user that created the FortiGate instances does not have sufficient 'NETWORK LEVEL' authority.
To overcome this, a new application will be created under the AZURE ACTIVE DIRECTORY -> APP REGISTRATIONS.
Once on the APP REGISTRATION PORTAL, select 'NEW REGISTRATION'.
Provide a name and select 'REGISTER'.
- Open the newly created application and select 'Overview'. Information regarding the directory ID and application ID will be visible:
Also, it is necessary to create a secret password to be used in FortiGate’s SDN external connector:
This information is required for the AZURE SDN connector. Login to the FortiGate and match the DIRECTORY ID from AZURE with the SDN external connector Directory ID field and match the APPLICATION ID from AZURE with the SDN external connector APPLICATION ID field in FortiOS along with matching the value field in the 'Certificates & Secrets' section of AZURE with Client SECRET field in FortiGate SDN configuration.
The Subscription-ID information can be retrieved from the 'Subscription' section in AZURE.
Match the resource group name to the one used in AZURE and confirm the configurations have been replicated in the FGT-B.
- It is necessary to give the newly created application 'NETWORK CONTRIBUTOR' access. Go to the SUBSCRIPTIONS -> 'Subscription' -> IAM and select 'Add Role Assignments'.
Choose the 'Network Contributor' option and select 'next':
Add the two FortiGate VMs into the 'Members' section along with the application created in step 3. Select 'Review+Create'.
- It is necessary to configure the FortiGate SDN connector (if not already done) to move the Public IP and the LAN Gateway from one FortiGate to another in case of a failover:
FGT-A:
config system sdn-connector
edit "AzureSDN"
config nic
edit "Fortigat-TAC-FGT-A-Nic1"
config ip
edit "ipconfig1"
set public-ip "Fortigat-TAC-FGT-PIP" <----- Name of the Public IP in AZURE.
next
end
next
end
config route-table
edit "Fortigat-TAC-RouteTable-ProtectedSubnet"<- Route Table in AZURE
config route
edit "toDefault"
set next-hop "172.16.136.68" <----- LAN NIC of FGT-A.
next
end
next
end
next
end
FGT-B:
config system sdn-connector
edit "AzureSDN"
config nic
edit "Fortigat-TAC-FGT-B-Nic1"
config ip
edit "ipconfig1"
set public-ip "Fortigat-TAC-FGT-PIP" <----- Name of the Public IP in AZURE.
next
end
next
end
config route-table
edit "Fortigat-TAC-RouteTable-ProtectedSubnet" <----- Route Table in AZURE.
config route
edit "toDefault"
set next-hop "172.16.136.69" <----- LAN NIC of FGT-A.
next
end
next
end
next
end
Now, refer to the diagram above. These commands will set the logic in FotiGate to tell AZURE to, in the event of a failover, move the Public IP associated with NIC(.5) to NIC(.6) and the LAN Gateway of the AZURE NIC(.69) to NIC (.70) so that the endpoints behind FortiGate use the LAN IP address of FGT-B as their Gateway for traffic and the reverse of it when the Primary comes back-up.
Verification.
Run the following debugs on FGT-B:
diag debug app azd -1
diag debug en
Now, shut down/failover the primary FortiGate. Similar console output to the screenshot shown below should be observed on FGT-B with no service interruption:
When the failover happens, the FortiGate-B pushes an API request to the application created in step 3 to make changes to the Public IP association and the LAN Gateway in the routing table of the VNET in AZURE.
For the cases where the Azure SDN connector is configured to use 'Managed Identity', there is no need for a new application registration. However, for a successful HA failover the following permissions should be configured:
- Reader role for both FortiGate VMs in the cluster to the Subscription ID.
- Contributor role for both FortiGate VMs in the cluster to the Resource Group they belong to.
Related documents:
