Technical Tip: IPsec site to site not coming up after physical device being replaced (MAC Address issue)

[kaurm]

Description	This article describes the situation when the FortiGate was replaced after restoring the configuration and the IPsec site-to-site tunnel was still not up.
Scope	FortiGate.
Solution	When the devices are replaced, and the configuration is restored after a factory reset or cables are plugged back into an already running modem. Site A(A)------IPsec tunnel -----Site B(B) Running the debugs shows the following output: diagnose debug console timestamp enable diagnose debug application ike -1 diagnose debug enable To stop the debug, run the following commands:    diagnose debug disable    diagnose debug reset 2022-09-22 13:05:12.378979 ike 0:A:411492: sent IKE msg (P1_RETRANSMIT): 192.168.110.12:500->172.16.111.12:500 len=192, id=9432dc4305ddcb99/6f78bfef400060ec 2022-09-22 13:05:12.390287 ike 0: comes 172.16.111.12:500->192.168.110.12:500,ifindex=10.... Initially was getting the error request is in the queue and now getting the error of ike shrank, the sniffer on the remote shows: 22-09-22 13:03:08.312637 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 192 2022-09-22 13:03:18.186185 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 572 2022-09-22 13:03:18.301339 port2 in 172.16.111.12.500 -> 192.168.110.12.500: udp 572 2022-09-22 13:03:18.301391 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 192 2022-09-22 13:03:21.188081 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 572 local site shows 1D5478871D84E4CE000000000000000001100200000000000000023C0D00015400000001000000010000014 8010100080300002801010000800B0001000C00040001518080010007800E00808003000 2022-09-22 13:10:53.519475 ike 0:B_1:2053: sent IKE msg (P1_RETRANSMIT): 172.16.111.12:500->192.168.110.12:500, len=572, id=1d5478871d84e4ce/0000000000000000 2022-09-22 13:11:02.499245 ike 0:B_1:2053: negotiation timeout, deleting 2022-09-22 13:11:02.499447 ike 0:B_1: connection expiring due to phase1 down 2022-09-22 13:11:02.499491 ike 0:B_1: deleting 2022-09-22 13:11:02.499529 ike 0:B_1: deleted 2022-09-22 13:11:02.499560 ike 0:B_1: schedule auto-negotiate 2022-09-22 13:11:03.509281 ike 0:B_1:2060: initiator: main mode is sending 1st message... 2022-09-22 13:11:03.509419 ike 0:B_1:2060: cookie 20bcfa1cd37b39a1/0000000000000000 Line 132: 2022-09-22 12:34:40.068828 ike 0:B:1597: negotiation timeout, deleting From the ike daemon’s perspective, there was no response from the wan1 of site A. The sniffer on both sides shows that port 2 and wan1 were being used on site A and site B, respectively: B_400E (root) # diagnose sniffer packet any 'host 172.16.111.12 and port 500' 4 0 l interfaces=[any] filters=[host 172.16.111.12 and port 500] 2022-09-22 12:36:01.297666 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 192 2022-09-22 12:36:01.307319 port2 in 172.16.111.12.500 -> 192.168.110.12.500: udp 572 2022-09-22 12:36:01.307340 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 192 2022-09-22 12:36:10.347270 port2 in 172.16.111.12.500 -> 192.168.110.12.500: udp 572 2022-09-22 12:36:10.347326 port2 out 192.168.110.12.500 -> 172.16.111.12.500: udp 192 A_60E # diagnose sniffer packet any 'host 192.168.110.12 and port 500' 4 0 l interfaces=[any] filters=[host 192.168.110.12 and port 500] 2022-09-22 13:24:29.548993 wan1 in 192.168.110.12.500 -> 172.16.111.12.500: udp 572 2022-09-22 13:24:30.054244 wan1 out 172.16.111.12.500 -> 192.168.110.12.500: udp 572 2022-09-22 13:24:30.139612 wan1 in 192.168.110.12.500 -> 172.16.111.12.500: udp 192 2022-09-22 13:24:32.548748 wan1 in 192.168.110.12.500 -> 172.16.111.12.500: udp 572 The MAC address of Site A was E8:1C:BA:03:B7:72: The PCAP showed the MAC being used was E8:1C:BA:03:B6:AA: Thus, the endpoint was forced to drop because of which the IKE response packet was never sent to the Ike daemon and the tunnel was never up and working. As the device was replaced and cables were plugged into an already running modem, the old ARP entry was stuck in the arp-cache due to which the tunnel was not working. So, after rebooting the modem, the tunnels were up and traffic was passing.