[FortiSIEM] Is it possible to check the logs temporarily stored in the Collector?

[Bruce7x2]

Hi Team,

Regarding the disconnection situation between the Collector and the Supervisor, do we have a method to check how many logs are stored in the Collector, thereby ensuring that the Collector will send these logs to the Supervisor after reconnection?

For instance, let’s imagine a scenario where the Supervisor is upgrading and disconnects from the Collector. During this time, the Collector continues to receive logs from network devices/endpoint devices/... . Will the Collector send the logs received during the upgrade to the Supervisor after the Supervisor upgrades and reloads the system?

In summary, two questions:

1. Is there a direct or indirect way to check if the logs still received by the Collector after disconnection from the Supervisor are retained?
2. Will the logs continuously received by the Collector during the Supervisor’s update be transferred to the Supervisor after the update?

[sioannou]

Hi @Bruce7x2 ,

Please see  below:

1) Yes, you can either use tcpdump to check port 443 (Agent logs) and 514 (Device Logs). Also the information is stored in the location /opt/phoenix/cache/parser/events

2) Yes the logs will be transferred to the supervisor after the upgrade but there are limitations. https://help.fortinet.com/fsiem/7-1-1/Online-Help/HTML5_Help/appendix-increasing-collector-event-buf... 

Also I think Advance Analytics FCSS Security Operations on NSE training covers the topic in detail. 

Note here that the logs will appear on the supervisor after the connection is re-establish in the time they where received by the collector. To view the logs are coming in use real time search. The collector will be in Error condition until the buffers are cleared.  

S

[Bruce7x2]

Dear Sioannou,

“According to ‘Increasing Collector Event Buffer Size’, it mentions that ‘Events are stored in compressed format in the following location /opt/phoenix/cache/parser/events before being sent to Worker(s) or Supervisor nodes. By default, a maximum of 10K files are stored and each file has a maximum uncompressed file size of 10MB.’

I’m wondering if the ‘file’ mentioned here refers to a single log entry? Or could it be that the logs received within a certain time period are bundled into a 'file', and the collector’s limit is that it can temporarily store up to 10,000 files? Is my interpretation correct?”

[FSM_FTNT]

Multiple logs are bundled into a single file, but the file count and therefore the max number of cached events can be increased