I am adding this to the forum for an issue we saw and eventually resolved with input from TAC.
We deployed two new Azure Foritgates (v7.0.6) roughly following the guide; https://github.com/fortinet/fortigate-terraform-deploy/tree/main/azure/7.0/ha-port1-mgmt-crosszone
Whenever we attempted to apply the HA configuration to either firewall:
config system ha
set group-name "AzureHA"
set mode a-p
set hbdev "port4" 255
set session-pickup enable
set session-pickup-connectionless enable
set ha-mgmt-status enable
config ha-mgmt-interfaces
edit 1
set interface "port1"
set gateway x.x.x.x
next
end
set override disable
set priority 255
set unicast-hb enable
set unicast-hb-peerip x.x.x.x
end
We would lose all connectivity to all interfaces the the firewall and from the console, we saw the following error:
[hagwnet_m_vd_policy_set:5047[ pid-234 ERROR args
mlx4_en: sriovslv2: Failed changing HW MAC address
mlx4_en: sriovslv1: Failed changing HW MAC address
mlx4_en: sriovslv0: Failed changing HW MAC address
mlx4_en: sriovslv3: Failed changing HW MAC address
mlx4_en: sriovslv0: Failed changing HW MAC address
port1: can not set mac address(22).
mlx4_en: sriovslv2: Failed changing HW MAC address
port2: can not set mac address(22).
mlx4_en: sriovslv1: Failed changing HW MAC address
port3: can not set mac address(22).
mlx4_en: sriovslv3: Failed changing HW MAC address
port4: can not set mac address(22).
To get the firewall back communicating again, we had to place it back into standalone mode from the console:
config system ha
set mode standalone
end
And sometimes, even that was not enough and we would have to reboot the firewall as well.
The error 'sriovslv' relates to SR-IOV (accelerated networking) and the virtual interfaces this feature creates.
https://docs.fortinet.com/document/fortigate/6.2.0/new-features/651644/sr-iov-support-6-2-1
Originally I thought we had an issue where by our FortiGate's physical interfaces were no tallying up with our virtual ones:
Running the hidden command 'fnsysctl ifconfig' shows all physical and virtual interface MAC addresses. You see that the virtual interfaces use the same MAC as the physical interface they are associated with. But in our case they matched as follows:
Port 1 = sriovslv1
Port 2 = sriovslv2
Port 3 = sriovslv3
Port 4 = sriovslv0
i.e. you'd have expected Port 1 = sriovslv0, Port 2= sriovslv1, etc...
At the same time we discovered this, TAC emailed to say they cannot replicate the same problem in their Azure environment but the only thing they did notice was our interfaces deployment was not the same as the Azure guide. We'd deployed the interfaces in the order:
Private
Public
HA-Sync
Mgmt
But the documentation suggests:
Mgmt
Public
Private
HA-Sync
We shut the Firewall VM down, re-arrange the interfaces and booted it back up. The firewall the accepted the HA config without an issue (or error). Therefore the order of the interface deployment must match the Fortinet documentation if you are looking at a HA deployment.
Running the commands fnsysctl ifconfig now still show some of the interfaces not matching what you'd expect their virtual interface numbers to be:
Port 1 = sriovslv1
Port 2 = sriovslv3
Port 3 = sriovslv0
Port 4 = sriovslv2
But it works so this is where my original logic was wrong...but got us on the right track about interface orders being wrong :)
Hope this help anyone who encounters the same issue. I could not find any information about it on the internet previously.
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Created on 12-29-2022 11:01 AM
Hi
Thanks for sharing this solution with our community
Looks very granular indeed !
Select Forum Responses to become Knowledge Articles!
Select the “Nominate to Knowledge Base” button to recommend a forum post to become a knowledge article.
User | Count |
---|---|
1692 | |
1087 | |
752 | |
446 | |
228 |
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.