FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
Hasnatriad
Staff
Staff
Article Id 363479
Description This article describes why the error 'Authentication type mismatched' is seen on FortiGate OSPF debug and Cisco when establishing OSPF adjacency between FortiGate and Cisco.
Scope FortiGate.
Solution

If authentication is enabled in OSPF and this error 'Authentication type mismatched' is received, follow the steps to resolve the issue.

RFC 2328 (OSPF Version 2) defines the authentication type, which means authentication can be set as per the below table.

 

      AuType             Description    

 ___________________________________________                  

     0                Null authentication                  

     1                Simple password                  

     2                Cryptographic authentication                  

    All others        Reserved for assignment by the IANA (iana@ISI.EDU) Table 20: OSPF authentication types.

 

If neighborship is not established, the first thing to check is whether the traffic flow is happening for OSPF traffic using a sniffer. 

 

diagnose sniffer packet [intf] " host x.x.x.x and proto 89 " 6 0 l

 

Also, OSPF debug shows an authentication mismatch type error as well if the authentication is configured differently. 

 

diagnose ip router ospf all enable
diagnose ip router ospf level info
diagnose debug console timestamp enable
diagnose debug enable

 

2024-12-8 21:49:58 OSPF: -----------------------------------------------------
2024-12-8 21:49:58 OSPF: Header
2024-12-8 21:49:58 OSPF: Version 2
2024-12-8 21:49:58 OSPF: Type 1 (Hello)
2024-12-8 21:49:58 OSPF: Packet Len 48
2024-12-8 21:49:58 OSPF: Router ID 10.99.248.1
2024-12-8 21:49:58 OSPF: Area ID 0.0.0.0
2024-12-8 21:49:58 OSPF: Checksum 0x0
2024-12-8 21:49:58 OSPF: AuType 2
2024-12-8 21:49:58 OSPF: Cryptographic Authentication
2024-12-8 21:49:58 OSPF: Key ID 1
2024-12-8 21:49:58 OSPF: Auth Data Len 16
2024-12-8 21:49:58 OSPF: Sequence number 54898957
2024-12-8 21:49:58 OSPF: Hello
2024-12-8 21:49:58 OSPF: NetworkMask 255.255.255.252
2024-12-8 21:49:58 OSPF: HelloInterval 10
2024-12-8 21:49:58 OSPF: Options 0x2 (*|-|-|-|-|-|E|-)
2024-12-8 21:49:58 OSPF: RtrPriority 1
2024-12-8 21:49:58 OSPF: RtrDeadInterval 40
2024-12-8 21:49:58 OSPF: DRouter 0.0.0.0
2024-12-8 21:49:58 OSPF: BDRouter 0.0.0.0
2024-12-8 21:49:58 OSPF: # Neighbors 1
2024-12-8 21:49:58 OSPF: Neighbor 10.99.248.22
2024-12-8 21:49:58 OSPF: -----------------------------------------------------
2024-12-8 21:49:58 OSPF: NFSM[VL304:10.99.248.17-10.99.248.1]: Full (HelloReceived)
2024-12-8 21:49:58 OSPF: NFSM[VL304:10.99.248.17-10.99.248.1]: nfsm_ignore called
2024-12-8 21:49:58 OSPF: NFSM[VL304:10.99.248.17-10.99.248.1]: Full (2-WayReceived)
2024-12-8 21:49:58 OSPF: RECV[Hello]: From 198.18.11.2 via VL1350:10.99.246.201: Authentication type mismatch

 

Packet Capture: 

Cisco:
Frame 2: 82 bytes on wire (656 bits), 82 bytes captured (656 bits)
Ethernet II, Src: Cisco_f8:19:ff (00:22:bd:xx:xx:xx), Dst: IPv4mcast_05 (01:00:5e:00:00:05)
Destination: IPv4mcast_05 (01:00:5e:00:00:05)
Source: Cisco_f8:19:ff (00:22:bd:xx:xx:xx)
Type: IPv4 (0x0800)
[Stream index: 1]
Internet Protocol Version 4, Src: 10.99.247.18, Dst: 224.0.0.5
Open Shortest Path First
OSPF Header
Version: 2
Message Type: Hello Packet (1)
<Hello: True>
Packet Length: 48
Source OSPF Router: 198.18.0.2
Area ID: 0.0.0.0 (Backbone)
Checksum: 0x6c8a [correct]
Auth Type: Null (0)  ---> authentication type set to none
Auth Data (none): 00 00 00 00 00 00 00 00
OSPF Hello Packet


FortiGate:
Frame 1: 94 bytes on wire (752 bits), 94 bytes captured (752 bits)
Ethernet II, Src: Fortinet_09:00:23 (00:09:0f:yy:yy:yy), Dst: IPv4mcast_05 (01:00:5e:00:00:05)
Destination: IPv4mcast_05 (01:00:5e:00:00:05)
Source: Fortinet_09:00:23 (00:09:0f:yy:yy:yy)
Type: IPv4 (0x0800)
[Stream index: 0]
Internet Protocol Version 4, Src: 10.99.247.17, Dst: 224.0.0.5
Open Shortest Path First
OSPF Header
Version: 2
Message Type: Hello Packet (1)
<Hello: True>
Packet Length: 44
Source OSPF Router: 10.99.248.22
Area ID: 0.0.0.0 (Backbone)
Checksum: 0x0000 (None)
Auth Type: Cryptographic (2) ---> Authentication type set to Cryptographic authentication
Auth Crypt Key id: 1
Auth Crypt Data Length: 16
Auth Crypt Sequence Number: 21420167
Auth Crypt Data: b8 64 7b af 5a 1c 4a 3e 5f 21 43 66 b5 5a c1 ce
OSPF Hello Packet

 

To solve this conflict, either set authentication on the Cisco side or remove authentication on the FortiGate ospf-interface config.

 

config router ospf
    config ospf-interface
        edit [Interface Entry Name]
            set interface [Interface]
            set authentication none
        end