Technical Note: [Accelops KB] Informational - Number of Events that are buffered on Collectors when communication between Super/Worker(s) are lost

Summary of Topic

When communication between Collector and Super/Workers is lost due to any issue (eg. Network issue, etc.), AO has a mechanism to buffer events until that communication is restored.

This article describes when you might start losing events.

Background Information

• By default there are a maximum of 10,000 event files that will be buffered on the collector
• Each event file contains 5 seconds worth of events and is limited to 10MB in size (before compression).
• The average event size is estimated to be 200 Bytes.  This will depend on the Device Type and Event.
• NOTE:  File based log events can be much larger.

There are several ways you can view this:

The Total events or amount of time worth of events are calculated below based on the previously described information and assumptions.

Estimated Max # of Events per Event File on Collector
[Based on event size of 200 bytes]

Max event file size / size per event >> 10MB / 200bytes >> 10485760 / 200 = 52,428 events/file

Estimated Max # of Events Total that can be buffered on Collector:

10,000 files x 52,428 events/file = 524,280,000 events

Therefore, 524,280,000 events can be buffered on the collector before any are lost.

Calculating the # of events with the following formula using your own value for EPS:

Avg EPS * 5 sec/file * 10,000 files

• Ex. EPS = 1,000  >>  1000EPS * 5 sec/file * 10,000 files = 50,000,000 events

An Estimated amount of time to reach the maximum buff size on the Collector:

The defaults equate to 2MBPS (megabytes per second), assuming event size of 200 bytes: 10MB / 5 sec = 2MBPS

With 2MBPS as an example, the collector can buffer events for the following amount of time:

(((10,000 files x 10MB / 2MBPS) / 60 min) / 60 sec) = 13.8 Hrs

At 3MBPS >> (((10,000 files x 10MB / 3MBPS) / 60 min) / 60 sec)  = 9.25 Hrs

Version Application

All