Skip to main content
seadave
New Member
September 29, 2017
Solved

Bad engine update???

  • September 29, 2017
  • 1 reply
  • 28572 views

At 4:23 PST today we started seeing 403 errors when trying to visit sites.  Only way to allow access is simple unfiltered NAT rule.  Searching form shows this happened in the past with a bad AV engine update.  I notice that support.fortinet.com is down with a 500 error so perhaps they self-inflicted the same.  TIME FOR AN ARCHITECTURE MODIFICATION!

 

Anyone else seeing this?

 

I'm seeing these in my debugs:

16330: 2017-09-28 17:03:53 <01449> firmware FortiGate-500D v5.4.5,build1138b1138,170531 (GA) (Release) 16331: 2017-09-28 17:03:53 <01449> application scanunit 16332: 2017-09-28 17:03:53 <01449> *** signal 11 (Segmentation fault) received *** 16333: 2017-09-28 17:03:53 <01449> AVDB 05004000AVDB00201-00052.00000-1709281424 16334: 2017-09-28 17:03:53 <01449> ETDB 05004000AVDB00701-00052.00000-1709281423 16335: 2017-09-28 17:03:53 <01449> EXDB 05004000AVDB00401-00001.00000-1210171547 16336: 2017-09-28 17:03:53 <01449> AVSO 04000000AVEN00701052471705041426 16337: 2017-09-28 17:03:53 <01449> Register dump: 16338: 2017-09-28 17:03:53 <01449> RAX: 0000000000000000 RBX: 00007fff067f82a0 16339: 2017-09-28 17:03:53 <01449> RCX: 00000000000000f8 RDX: 000000001258db56 16340: 2017-09-28 17:03:53 <01449> R8: 00000000000000ff R9: 0000000000000000 16341: 2017-09-28 17:03:53 <01449> R10: 0000000000000002 R11: 00007fb5625a0df0 16342: 2017-09-28 17:03:53 <01449> R12: 0000000000000046 R13: 00000000ffffffff 16343: 2017-09-28 17:03:53 <01449> R14: 00007fff067f82a0 R15: 00007fff067f81f0 16344: 2017-09-28 17:03:53 <01449> RSI: 0000000000000000 RDI: 0000000000000002 16345: 2017-09-28 17:03:53 <01449> RBP: 00000000ffffffff RSP: 00007fff067f80e0 16346: 2017-09-28 17:03:53 <01449> RIP: 00007fb5654af827 EFLAGS: 0000000000010212 16347: 2017-09-28 17:03:53 <01449> CS: 0033 FS: 0000 GS: 0000 16348: 2017-09-28 17:03:53 <01449> Trap: 000000000000000e Error: 0000000000000004 16349: 2017-09-28 17:03:53 <01449> OldMask: 0000000000000000 16350: 2017-09-28 17:03:53 <01449> CR2: 0000000000000014 16351: 2017-09-28 17:03:53 <01449> Backtrace: 16352: 2017-09-28 17:03:53 <01449> [0x7fb5654af827] => /data/lib/libav.so 16353: 2017-09-28 17:03:53 <01449> [0x7fb5654b7f45] => /data/lib/libav.so 16354: 2017-09-28 17:03:53 <01449> [0x7fb5654b86f3] => /data/lib/libav.so 16355: 2017-09-28 17:03:53 <01449> [0x7fb5654ab912] => /data/lib/libav.so 16356: 2017-09-28 17:03:53 <01449> [0x7fb5654b44c3] => /data/lib/libav.so 16357: 2017-09-28 17:03:53 <01449> [0x7fb5654baa39] => /data/lib/libav.so 16358: 2017-09-28 17:03:53 <01449> [0x7fb5654da895] => /data/lib/libav.so 16359: 2017-09-28 17:03:53 <01449> [0x7fb5654d87e7] => /data/lib/libav.so 16360: 2017-09-28 17:03:53 <01449> [0x7fb565494ad7] => /data/lib/libav.so (scanvirFile+0x00000187) 16361: 2017-09-28 17:03:53 <01449> [0x01a07ddf] => /bin/scanunitd 16362: 2017-09-28 17:03:53 <01449> [0x01a455ec] => /bin/scanunitd 16363: 2017-09-28 17:03:53 <01449> [0x01a466db] => /bin/scanunitd 16364: 2017-09-28 17:03:53 <01449> [0x010e54f0] => /bin/scanunitd 16365: 2017-09-28 17:03:53 <01449> [0x010e6599] => /bin/scanunitd 16366: 2017-09-28 17:03:53 <01449> [0x019b1c7c] => /bin/scanunitd 16367: 2017-09-28 17:03:53 <01449> [0x010e734d] => /bin/scanunitd 16368: 2017-09-28 17:03:53 <01449> [0x010e0616] => /bin/scanunitd 16369: 2017-09-28 17:03:53 <01449> [0x010e3fde] => /bin/scanunitd 16370: 2017-09-28 17:03:53 <01449> [0x00427c10] => /bin/scanunitd 16371: 2017-09-28 17:03:53 <01449> [0x0042e5c7] => /bin/scanunitd 16372: 2017-09-28 17:03:53 <01449> [0x0042bcf1] => /bin/scanunitd 16373: 2017-09-28 17:03:53 <01449> [0x0042d881] => /bin/scanunitd 16374: 2017-09-28 17:03:53 <01449> [0x0042deff] => /bin/scanunitd 16375: 2017-09-28 17:03:53 <01449> [0x7fb5690e4475] => /fortidev4-x86_64/lib/libc.so.6 16376: 2017-09-28 17:03:53 (__libc_start_main+0x000000f5) liboffset 00021475 16377: 2017-09-28 17:03:53 <01449> [0x00425065] => /bin/scanunitd 16378: 2017-09-28 17:03:53 [AV Engine <1449>] Last file info: 16379: 2017-09-28 17:03:53 [AV Engine <1449>] filename: bag, filesize: 7151 16380: 2017-09-28 17:03:53 [AV Engine <1449>] Native script imagebase: 0x12546000 16381: 2017-09-28 17:03:53 [AV Engine <1449>] cprl sigid: 489591, bintype: 00000400 16382: 2017-09-28 17:03:53 scanunit=worker pid=1449 exittype=signal code=11 total=7996 free=5679 16383: 2017-09-28 17:03:53 scanunit crash: signal=11, src-ip=172.21.11.126, dst-ip=104.80.89.9, 16384: 2017-09-28 17:03:53 request-uri=http://init-p01st.push.apple.com/bag

 

diag autoupdate ver output:

 

AV Engine --------- Version: 5.00247 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu May 4 14:26:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

Virus Definitions --------- Version: 52.00001 Contract Expiry Date: Mon Jul 16 2018 Last Updated using push update on Thu Sep 28 17:00:11 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: Updates Installed

Extended set --------- Version: 52.00000 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Sep 28 14:23:00 2017 Last Update Attempt: n/a Result: Updates Installed

Extreme set --------- Version: 1.00000 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Wed Oct 17 15:47:00 2012 Last Update Attempt: n/a Result: Updates Installed

Mobile Malware Definitions --------- Version: 52.00000 Contract Expiry Date: Sat Jun 2 2018 Last Updated using push update on Thu Sep 28 17:00:11 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: Updates Installed

Attack Definitions --------- Version: 6.00741 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Tue Dec 1 02:30:00 2015 Last Update Attempt: n/a Result: Updates Installed

Attack Extended Definitions --------- Version: 12.00234 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Sep 28 01:27:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

IPS Malicious URL Database --------- Version: 1.00775 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Sep 28 07:29:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

Flow-based Virus Definitions --------- Version: 52.00000 Contract Expiry Date: Mon Jul 16 2018 Last Updated using push update on Thu Sep 28 17:00:11 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: Updates Installed

Botnet Definitions --------- Version: 4.00058 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Sep 28 10:00:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

IPS Attack Engine --------- Version: 3.00430 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Tue Aug 22 20:13:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

Internet-service Database Apps --------- Version: 2.00702 Contract Expiry Date: n/a Last Updated using manual update on Wed Sep 27 11:15:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

Internet-service Database Maps --------- Version: 2.00702 Contract Expiry Date: n/a Last Updated using manual update on Wed Sep 27 11:15:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

Botnet Domain Database --------- Version: 1.00505 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Aug 11 12:09:00 2016 Last Update Attempt: n/a Result: Updates Installed

Modem List --------- Version: 0.000

Device and OS Identification --------- Version: 1.00061 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Fri Sep 8 17:49:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

IP Geography DB --------- Version: 1.067 Contract Expiry Date: n/a Last Update Date: Fri Aug 4 15:07:26 2017

Certificate Bundle --------- Version: 1.00005 Last Update Date: Thu May 5 10:58:00 2016

FDS Address --------- 208.91.112.78-443

URL White list --------- Version: 1.00810 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu Sep 28 08:05:00 2017 Last Update Attempt: Thu Sep 28 17:00:11 2017 Result: No Updates

    Best answer by seadave

    Not yet, I'll update when my ticket is updated.  If they do so at all.  It seems fairly obvious now that the cause was a bad AV Defs update.  I'm now on 52.00005 with no issues.  I started considering a large purchase of FortiSwitches today.  I guess this is my reward ;)

     

    You can check via the console with the "diag autoupdate ver" command:

     

    AV Engine --------- Version: 5.00247 Contract Expiry Date: Mon Jul 16 2018 Last Updated using manual update on Thu May 4 14:26:00 2017 Last Update Attempt: Thu Sep 28 20:36:17 2017 Result: No Updates

    Virus Definitions --------- Version: 52.00005 Contract Expiry Date: Mon Jul 16 2018 Last Updated using scheduled update on Thu Sep 28 20:36:17 2017 Last Update Attempt: Thu Sep 28 20:36:17 2017 Result: Updates Installed

     

    If you have a FAZ 5.6, we realized today you can configure an event handler to alert you when the "app crash" event fires.

     

    Should give you a small heads up when this is happening instead of the line of people knocking on your door.

    1 reply

    tanr
    New Member
    September 29, 2017

    I see the same thing on a 300D w/ 5.4.5.

    tanr
    New Member
    September 29, 2017

    I ran an "exec update-now".  They've already got a new set of virus definitions, 52.00002 instead of 52.00001.

     

    Unfortunately, I'm still seeing the same sets of crashes, so it's not fixed yet.

     

    tanr
    New Member
    September 29, 2017

    Looks like support.fortinet.com is back up.

     

    Virus definitions have changed from 52.00001 to 52.00002 to 52.00003

    Flow-based virus definitions have moved from 52.00001 to 52.00002.

     

    Haven't seen any more crashes in the 10 minutes since I updated.  Fingers crossed.