Skip to main content
EdsonCV
New Member
June 14, 2022
Solved

CMDB lock deadlock is detected - #Fortigate 50E

  • June 14, 2022
  • 3 replies
  • 6135 views

I was checking the fortigate and when I entered the events tab I appeared many times the event of cmdb lock deadlock is detected and when I want to make a change in some policy it does not save anything and it gets stuck. What does this event mean, and why can't I make a configuration?

EdsonCV_0-1655225021586.png

 

Best answer by EdsonCV

Solved.

I had to unplug and plug the Fortigate back into the current and with that everything was solved.

3 replies

ssudhakar
Staff
Staff
June 14, 2022

Hi EdsonCV,

 

Deadlock happens due to amount of time the process was taking to show the config and keeps holding the lock, which is called as a deadlock and will be killed .The system will kill the process and output the message (CMDB lock deadlock is detected) if other process can not acquire the lock in 60 seconds.

 

https://docs.fortinet.com/document/fortigate/6.2.7/fortios-log-message-reference/44555/44555-logid-event-cmdb-deadlock-detected

 

>>> when I want to make a change in some policy it does not save anything and it gets stuck.

-Can you check "diag debug crashlog read" to see if you have any crashes ?

-Do you have any script or tool that is doing a backup of the configuration during that time ?

-Can I know what FortiOS version are you running ?

 

Thank you,

Hope

EdsonCV
EdsonCVAuthor
New Member
June 15, 2022

Hi

Answering the first two questions I am told that someone wanted to make a backup of the Firewall configuration on June 6, 2022. Reviewed the crashlog in the CLI I notice that there is one that corresponds to the date.

They tell me that the backup could not be made from the GUI, they also comment that they reboot the firewall.

This is a log line:

206: 2022-06-06 22:28:40 the killed daemon is /bin/dhcpcd: status=0xff00

 

-Can you check "diag debug crashlog read" to see if you have any crashes ?

-Do you have any script or tool that is doing a backup of the configuration during that time ?

-Can I know what FortiOS version are you running ?

FortiOS v6.2.9 build1234 (GA)

 

I really appreciate the help.

seshuganesh
Staff
Staff
June 15, 2022

Hi Team,

 

Can you copy one entire log and paste it here..
We will have look and keep you posted

EdsonCV
EdsonCVAuthor
New Member
June 15, 2022

FGT50E5620000742 # diagnose debug crashlog read
1: 2022-05-19 14:49:23 <00157> scanunit=manager str="Failed to load anti-virus database."
2: 2022-05-19 14:49:23 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
3: 2022-05-19 14:50:23 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
4: 2022-05-19 14:50:23 last-known good anti-virus database (ATDB)."
5: 2022-05-19 14:50:25 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
6: 2022-05-19 14:50:25 times by updated (pid 160) failed"
7: 2022-05-19 15:11:19 <00157> scanunit=manager str="Failed to load anti-virus database."
8: 2022-05-19 15:11:19 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
9: 2022-05-19 15:11:37 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
10: 2022-05-19 15:11:37 last-known good anti-virus database (ATDB)."
11: 2022-05-19 15:11:49 <00157> scanunit=manager str="Failed to copy /data/etc/vir to try loading
12: 2022-05-19 15:11:49 firmware image anti-virus database (ATDB)."
13: 2022-05-19 15:11:49 <00157> scanunit=manager str="Failed to load any anti-virus database."
14: 2022-05-19 15:11:49 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
15: 2022-05-19 15:11:49 times by updated (pid 160) failed"
16: 2022-05-19 15:13:24 <00157> scanunit=manager str="Failed to load anti-virus database."
17: 2022-05-19 15:13:24 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
18: 2022-05-19 15:13:47 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
19: 2022-05-19 15:13:47 last-known good anti-virus database (ATDB)."
20: 2022-05-19 15:13:51 <00157> scanunit=manager str="Failed to copy /data/etc/vir to try loading
21: 2022-05-19 15:13:51 firmware image anti-virus database (ATDB)."
22: 2022-05-19 15:13:51 <00157> scanunit=manager str="Failed to load any anti-virus database."
23: 2022-05-19 15:13:51 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
24: 2022-05-19 15:13:51 times by updated (pid 160) failed"
25: 2022-05-19 15:15:34 <00157> scanunit=manager str="Failed to load anti-virus database."
26: 2022-05-19 15:15:34 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
27: 2022-05-19 15:15:54 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
28: 2022-05-19 15:15:54 last-known good anti-virus database (ATDB)."
29: 2022-05-19 15:16:01 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
30: 2022-05-19 15:16:01 times by updated (pid 160) failed"
31: 2022-05-19 15:19:01 <00157> scanunit=manager str="Failed to load anti-virus database."
32: 2022-05-19 15:19:01 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
33: 2022-05-19 15:19:31 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
34: 2022-05-19 15:19:31 last-known good anti-virus database (ATDB)."
35: 2022-05-19 15:19:36 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
36: 2022-05-19 15:19:36 times by updated (pid 160) failed"
37: 2022-05-19 16:15:59 <00157> scanunit=manager str="Failed to load anti-virus database."
38: 2022-05-19 16:15:59 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
39: 2022-05-19 16:16:31 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
40: 2022-05-19 16:16:31 last-known good anti-virus database (ATDB)."
41: 2022-05-19 16:16:44 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
42: 2022-05-19 16:16:44 times by updated (pid 160) failed"
43: 2022-05-19 21:10:10 <00157> scanunit=manager str="Failed to load anti-virus database."
44: 2022-05-19 21:10:10 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
45: 2022-05-19 21:10:24 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
46: 2022-05-19 21:10:24 last-known good anti-virus database (ATDB)."
47: 2022-05-19 21:10:26 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
48: 2022-05-19 21:10:26 times by updated (pid 160) failed"
49: 2022-05-19 21:14:38 <00157> scanunit=manager str="Failed to load anti-virus database."
50: 2022-05-19 21:14:38 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
51: 2022-05-19 21:14:53 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
52: 2022-05-19 21:14:53 last-known good anti-virus database (ATDB)."
53: 2022-05-19 21:14:56 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
54: 2022-05-19 21:14:56 times by updated (pid 160) failed"
55: 2022-05-19 22:23:57 <00157> scanunit=manager str="Failed to load anti-virus database."
56: 2022-05-19 22:23:57 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
57: 2022-05-19 22:24:08 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
58: 2022-05-19 22:24:08 last-known good anti-virus database (ATDB)."
59: 2022-05-19 22:24:10 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
60: 2022-05-19 22:24:10 times by updated (pid 160) failed"
61: 2022-05-19 23:14:07 <00157> scanunit=manager str="Failed to load anti-virus database."
62: 2022-05-19 23:14:07 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
63: 2022-05-19 23:14:22 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
64: 2022-05-19 23:14:22 last-known good anti-virus database (ATDB)."
65: 2022-05-19 23:14:23 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
66: 2022-05-19 23:14:23 times by updated (pid 160) failed"
67: 2022-05-19 23:16:04 <00157> scanunit=manager str="Failed to load anti-virus database."
68: 2022-05-19 23:16:04 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
69: 2022-05-19 23:16:35 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
70: 2022-05-19 23:16:35 last-known good anti-virus database (ATDB)."
71: 2022-05-19 23:16:45 <00157> scanunit=manager str="Failed to load firmware image anti-virus
72: 2022-05-19 23:16:45 database (ATDB)."
73: 2022-05-19 23:16:45 <00157> scanunit=manager str="Failed to load any anti-virus database."
74: 2022-05-19 23:16:45 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
75: 2022-05-19 23:16:45 times by updated (pid 160) failed"
76: 2022-05-19 23:17:23 <00157> scanunit=manager str="Failed to load anti-virus database."
77: 2022-05-19 23:17:23 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
78: 2022-05-19 23:17:46 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
79: 2022-05-19 23:17:46 last-known good anti-virus database (ATDB)."
80: 2022-05-19 23:17:52 scanunit=manager pid=157 cause='signal' str="AV database reload requested 2
81: 2022-05-19 23:17:52 times by updated, updated (pid 160, 160) failed"
82: 2022-05-19 23:18:12 <00157> scanunit=manager str="Failed to load anti-virus database."
83: 2022-05-19 23:18:13 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
84: 2022-05-19 23:18:39 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
85: 2022-05-19 23:18:39 last-known good anti-virus database (ATDB)."
86: 2022-05-19 23:18:40 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
87: 2022-05-19 23:18:40 times by updated (pid 160) failed"
88: 2022-05-19 23:19:05 <00157> scanunit=manager str="Failed to load anti-virus database."
89: 2022-05-19 23:19:05 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
90: 2022-05-19 23:19:08 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
91: 2022-05-19 23:19:08 last-known good anti-virus database (ATDB)."
92: 2022-05-19 23:19:10 <00157> scanunit=manager str="Failed to copy /data/etc/vir to try loading
93: 2022-05-19 23:19:10 firmware image anti-virus database (ATDB)."
94: 2022-05-19 23:19:10 <00157> scanunit=manager str="Failed to load any anti-virus database."
95: 2022-05-19 23:19:10 scanunit=manager pid=157 cause='signal' str="AV database reload requested 2
96: 2022-05-19 23:19:10 times by updated, updated (pid 160, 160) failed"
97: 2022-05-19 23:19:37 <00157> scanunit=manager str="Failed to load anti-virus database."
98: 2022-05-19 23:19:37 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
99: 2022-05-19 23:19:41 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
100: 2022-05-19 23:19:41 last-known good anti-virus database (ATDB)."
101: 2022-05-19 23:19:42 <00157> scanunit=manager str="Failed to copy /data/etc/vir to try loading
102: 2022-05-19 23:19:42 firmware image anti-virus database (ATDB)."
103: 2022-05-19 23:19:42 <00157> scanunit=manager str="Failed to load any anti-virus database."
104: 2022-05-19 23:19:42 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
105: 2022-05-19 23:19:42 times by updated (pid 160) failed"
106: 2022-05-19 23:20:09 <00157> scanunit=manager str="Failed to load anti-virus database."
107: 2022-05-19 23:20:10 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
108: 2022-05-19 23:20:34 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
109: 2022-05-19 23:20:34 last-known good anti-virus database (ATDB)."
110: 2022-05-19 23:20:35 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
111: 2022-05-19 23:20:35 times by updated (pid 160) failed"
112: 2022-05-19 23:20:55 <00157> scanunit=manager str="Failed to load anti-virus database."
113: 2022-05-19 23:20:55 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
114: 2022-05-19 23:21:21 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
115: 2022-05-19 23:21:21 last-known good anti-virus database (ATDB)."
116: 2022-05-19 23:21:23 <00157> scanunit=manager str="Failed to load firmware image anti-virus
117: 2022-05-19 23:21:23 database (ATDB)."
118: 2022-05-19 23:21:23 <00157> scanunit=manager str="Failed to load any anti-virus database."
119: 2022-05-19 23:21:23 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
120: 2022-05-19 23:21:23 times by updated (pid 160) failed"
121: 2022-05-19 23:21:58 <00157> scanunit=manager str="Failed to load anti-virus database."
122: 2022-05-19 23:21:58 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
123: 2022-05-19 23:22:13 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
124: 2022-05-19 23:22:13 last-known good anti-virus database (ATDB)."
125: 2022-05-19 23:22:15 scanunit=manager pid=157 cause='signal' str="AV database reload requested 2
126: 2022-05-19 23:22:15 times by updated, updated (pid 160, 160) failed"
127: 2022-05-19 23:23:31 <00157> scanunit=manager str="Failed to load anti-virus database."
128: 2022-05-19 23:23:32 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
129: 2022-05-19 23:23:56 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
130: 2022-05-19 23:23:56 last-known good anti-virus database (ATDB)."
131: 2022-05-19 23:23:57 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
132: 2022-05-19 23:23:57 times by updated (pid 160) failed"
133: 2022-05-20 00:14:11 <00157> scanunit=manager str="Failed to load anti-virus database."
134: 2022-05-20 00:14:11 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
135: 2022-05-20 00:14:19 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
136: 2022-05-20 00:14:19 last-known good anti-virus database (ATDB)."
137: 2022-05-20 00:14:26 <00157> scanunit=manager str="Failed to copy /data/etc/vir to try loading
138: 2022-05-20 00:14:26 firmware image anti-virus database (ATDB)."
139: 2022-05-20 00:14:26 <00157> scanunit=manager str="Failed to load any anti-virus database."
140: 2022-05-20 00:14:26 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
141: 2022-05-20 00:14:26 times by updated (pid 160) failed"
142: 2022-05-20 00:19:25 <00157> scanunit=manager str="Failed to load anti-virus database."
143: 2022-05-20 00:19:26 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
144: 2022-05-20 00:19:37 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
145: 2022-05-20 00:19:37 last-known good anti-virus database (ATDB)."
146: 2022-05-20 00:19:39 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
147: 2022-05-20 00:19:39 times by updated (pid 160) failed"
148: 2022-05-20 01:22:52 <00157> scanunit=manager str="Failed to load anti-virus database."
149: 2022-05-20 01:22:52 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
150: 2022-05-20 01:23:42 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
151: 2022-05-20 01:23:42 last-known good anti-virus database (ATDB)."
152: 2022-05-20 01:23:44 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
153: 2022-05-20 01:23:44 times by updated (pid 160) failed"
154: 2022-05-20 01:25:13 <00157> scanunit=manager str="Failed to load anti-virus database."
155: 2022-05-20 01:25:13 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
156: 2022-05-20 01:25:24 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
157: 2022-05-20 01:25:24 last-known good anti-virus database (ATDB)."
158: 2022-05-20 01:25:27 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
159: 2022-05-20 01:25:27 times by updated (pid 160) failed"
160: 2022-05-20 02:14:32 <00157> scanunit=manager str="Failed to load anti-virus database."
161: 2022-05-20 02:14:33 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
162: 2022-05-20 02:15:30 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
163: 2022-05-20 02:15:30 last-known good anti-virus database (ATDB)."
164: 2022-05-20 02:15:35 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
165: 2022-05-20 02:15:35 times by updated (pid 160) failed"
166: 2022-05-20 02:16:49 <00157> scanunit=manager str="Failed to load anti-virus database."
167: 2022-05-20 02:16:50 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
168: 2022-05-20 02:17:31 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
169: 2022-05-20 02:17:31 last-known good anti-virus database (ATDB)."
170: 2022-05-20 02:17:34 <00157> scanunit=manager str="Failed to load firmware image anti-virus
171: 2022-05-20 02:17:34 database (ATDB)."
172: 2022-05-20 02:17:34 <00157> scanunit=manager str="Failed to load any anti-virus database."
173: 2022-05-20 02:17:34 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
174: 2022-05-20 02:17:34 times by updated (pid 160) failed"
175: 2022-05-20 02:17:54 <00157> scanunit=manager str="Failed to load anti-virus database."
176: 2022-05-20 02:17:54 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
177: 2022-05-20 02:18:04 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
178: 2022-05-20 02:18:04 last-known good anti-virus database (ATDB)."
179: 2022-05-20 02:18:09 <00157> scanunit=manager str="Failed to load firmware image anti-virus
180: 2022-05-20 02:18:09 database (ATDB)."
181: 2022-05-20 02:18:09 <00157> scanunit=manager str="Failed to load any anti-virus database."
182: 2022-05-20 02:18:09 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
183: 2022-05-20 02:18:09 times by updated (pid 160) failed"
184: 2022-05-20 02:18:36 <00157> scanunit=manager str="Failed to load anti-virus database."
185: 2022-05-20 02:18:36 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
186: 2022-05-20 02:19:01 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
187: 2022-05-20 02:19:01 last-known good anti-virus database (ATDB)."
188: 2022-05-20 02:19:03 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
189: 2022-05-20 02:19:03 times by updated (pid 160) failed"
190: 2022-05-20 03:14:22 <00157> scanunit=manager str="Failed to load anti-virus database."
191: 2022-05-20 03:14:22 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
192: 2022-05-20 03:14:38 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
193: 2022-05-20 03:14:38 last-known good anti-virus database (ATDB)."
194: 2022-05-20 03:14:40 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
195: 2022-05-20 03:14:40 times by updated (pid 160) failed"
196: 2022-05-20 03:16:25 <00157> scanunit=manager str="Failed to load anti-virus database."
197: 2022-05-20 03:16:25 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
198: 2022-05-20 03:16:40 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
199: 2022-05-20 03:16:40 last-known good anti-virus database (ATDB)."
200: 2022-05-20 03:16:41 scanunit=manager pid=157 cause='signal' str="AV database reload requested 1
201: 2022-05-20 03:16:41 times by updated (pid 160) failed"
202: 2022-05-20 03:17:08 <00157> scanunit=manager str="Failed to load anti-virus database."
203: 2022-05-20 03:17:08 <00157> scanunit=manager str="Failed to load basic anti-virus database (ATDB)."
204: 2022-05-20 03:17:29 <00157> scanunit=manager str="Failed to copy /tmp/vir.lastgood to try loading
205: 2022-05-20 03:17:29 last-known good anti-virus database (ATDB)."

EdsonCV
EdsonCVAuthorAnswer
New Member
July 21, 2022

Solved.

I had to unplug and plug the Fortigate back into the current and with that everything was solved.