CA upload interval to Fortigate

[dieter]

We often have users that are in the FSSO Agent list, but ar not known on the fortigate. Sometimes it takes several minutes.

So user comes to work, logs in. Logon event is picked up by FSSO agent. But user has no access to mail (in cloud) or internet. Explain that to the user...

At what interval are logged on users uploaded from FSSO CA to Fortigate ? Can the interval be lowered ?

[xsilver_FTNT]

Hi,

once Collector Agent finishes the checking on received logon event, the user is put to internal user list on Collector, and new logon is then pushed to all connected FortiGate units where group membership of that user match with Group Filter.

This update from Collector to FortiGates is .. usually pretty instant!

AFAIK there is no option to delay it, and there is no need to do so. Therefore there is nothing to lower.

That brings me, and you, to question Why it is delayed? As it takes "several minutes", it brings me to usual sources, or more precisely mistakes in FSSO setup/design,  where  .. 

- Collector do not monitor all the DCs (in polling mode) and so delayed logon is actually another new logon, like to exchange, or shared folder which is spotted and processed by FSSO, rather then original logon to workstation

- Collector operates in DCAgent mode, but workstation made logon towards DC where there is no DCAgent running and listening, so it's missed logon again

- user's group membership did not matched the Group Filters on first logon, but later on it did so later logon was delivered

- Network connection between Collector and FGT is too long or choppy, so first logon update was not received by FGT but some consecutive one was OK

I would suggest to make thorough check of FSSO settings, on FGT and on Collector/DCs as well. That all DCs are polled or monitored by DCAgents. That FGT has right Group Filter on Collector. That group membership of users match intended settings of Group Filters. Have a test user and check with single clean account.

[dieter]

Hi, thank you for the many suggestions.

On a sidenote: In fact a DC without an agent should be no problem, because an agent can collect logon events from logs from different DC's.

So here's our setup:

- I have agents on all DC's. Working in DC Agent mode. All collect logs from all DC's security logs and from on prem exchange (that server is not a DC btw).

- Two of the DC's also operate as collector. Fortigate has both configured (primary and secondary).

Communications between collectors and agents seem to be fine. Fortigate has good (and redundant) connectivity to the collectors.

Group filters are pulled from the fortigate config, I can see them replicated in every agent too ("Filter set by Fortigate").

Yes sometimes there's an issue with a user who was only minutes ago added to a new group, but we know there's a delay there (group membership is only refreshed when logging on).

In fact, because of the issues we've seen for quite some time, I already rebuilt the entire setup (uninstalling and reinstalling all agents/collectors, redoing the config in them, reconfiguring on the fortigate...). Not that easy (needs reboots, looses connectivity, clears cached users...), so I would not like to do it all over again ;)

In regard to "missing" the initial logon (when the user first logs on to the workstation): what logon event ID should I be looking for ? I should be able to match it with the agents logon event log, right ?

Our on prem exchange server is not a DC. Should I install an agent on it ? Currently none installed. But then again, a user logging on should be first seen through a logon event to a DC...

[xsilver_FTNT]

an agent can collect logon events from logs from different DC's.

If we speak about DCAgent, then no. DCAgent collect logons just from local DC as it is hooked directly to LSAS on that DC.

If we speak about Collector Agent, then in Polling mode, yes. The Collector can poll (retrieve) logs from multiple DCs.

As you described you are in DCAgent mode AND you have two Collectors, then make extra sure that all the DCAgents actually do report to both collectors.

Because first, Collector agents act independently of each other and do not share any info.

Second, if you do have more than one collector in FGT's config inside a single connector, which I would do for redundancy purpose, then this FGT actually does talk to only one collector at a time, till this permanent socket connection dies, then next Collector in a row is used, till this one fail, and again next one is used. There is no concept of primary and secondary or HA restore back to primary when reachable again. When last one is used, then first one is used again. List of Collectors inside a single connector on FGT is simply a circular list.

Therefore it is crucial that all the collectors do have their own and accurate list of logons.

FGT # .. that will show you to which collector is FGT connected to in CLI for example .. so debug there first

diag debug en diag debug authd fsso server-status

DCAgents are listed, if connected, in Collector agent's exported config at the bottom.

C:\Program Files (x86)\Fortinet\FSAE\saved_config

Examples:

{DC without agent at all}

ALFA\R24.alfa.xsilver.org     Domain controller is running 64 bits Windows     Status:DC Agent not installed

{DC with agent, but reporting to just one Collector} ALFA\C24.alfa.xsilver.org    Domain controller is running 64 bits Windows    DC Agent version: 5.0.254.0    [HKLM\SOFTWARE\Fortinet\FSAE\dcagent]       enable_log=0       log_file=C:\Program Files\Fortinet\FSAE\dcagentlog.txt       donot_resolve=1       no_keepalive=0       ignore_list=       domain_DNSsuffix=    [ca]       10.109.19.88=8002  <<< address and port to which collector this agent will report, you should have two such records, those can be added via DCAgent GUI or directly to registry keys {see path in [] brackets}.

Q: "what logon event ID should I be looking for ? I should be able to match it with the agents logon event log, right ?"

A: Well, modern Windows uses most often EventID 4624, Kerberos one .. more on other and useful EventIDs for FSSO in KB.fortinet.com

Q: "Our on prem exchange server is not a DC. Should I install an agent on it ?"

A: If you do have users who work from non-domain computers, but use Exchange with domain logons to read their emails/calendar, and you want those user be seen in FSSO, then set collectors to read the events from Exchange as well. However, credential verification of those users should spring Kerberos logons as well, so check WinSec as you might see those already. 

[dieter]

xsilver wrote:

an agent can collect logon events from logs from different DC's.

If we speak about DCAgent, then no. DCAgent collect logons just from local DC as it is hooked directly to LSAS on that DC.

If we speak about Collector Agent, then in Polling mode, yes. The Collector can poll (retrieve) logs from multiple DCs.

You're right, should have specified collector agent. But then again it was just a side note :)

I understand the way Fortigate uses collectors. You can even see in the GUI what collector the Fortigate is using (in the list of FSSO agents, the active collector's IP address is bold). And that matches with CLI query.

Curious thing: On one DC (collector) i have the saved.config.txt with a modified date in 2018. Impossible that is correct, because rebuild was done later. On the other DC, the file does not even exist.

That makes me think the newer agent version (5.0.0271) doesn't generate that file anymore. The old file is probably just that, an old file that was not removed when reinstalling the agent.

So I'll not give too much attention to those files.

I'll have a look at event ID's to make sure I have a match.

Exchange in our case (since no non-domain users use it) is unlikely to enhance user mapping. Will be removed in the end, since we're migrating to cloud mail.

Thanks again !

[xsilver_FTNT]

That saved_config is result of you pressing Collector Agent's GUI bottom menu button "Export config".

It is collection of known configurations and other data like OS versions etc.

It is NOT active configuration. As current config is scattered across MS registry records.

Also note that whenever you press "export" then newer info is appended to the file, so latest collection is on the bottom of the file (so it also could be used as trace/log of config changes).

Non-existing file is also OK, it just mean that no one ever exported that config.

Existing file but prior to "rebuild" is also OK, as during rebuild complete cleanup was probably not performed and so file from previous export was left over and there was no export ordered after rebuild.

And so, make export, have a look to bottom of the file, on both Collector agents and compare. Pay extra attention to mentioned DC Agents and whom they report to.

This exported config, if taken recently, is very helpful for the troubleshooting. So better pay attention and also export it whenever you make changes to config.

[dieter]

Ok, this is where things get interesting (ommitted some info and added some for clarification):

xx\swinadfs01.xx (= 10.252.0.55)
    Domain controller is running 64 bits Windows
    DC Agent version: 5.0.271.0 
    [HKLM\SOFTWARE\Fortinet\FSAE\dcagent]
        log_file=c:\dcagentlog.txt
        enable_log=0
        disable_rdp_override=1
    [ca]
        10.252.0.51=8002
        10.252.0.55=8002
        10.252.0.50=8002

xx\SWINAD01.xx (= 10.252.0.50)
    Domain controller is running 64 bits Windows
    DC Agent version: 5.0.271.0 
    [HKLM\SOFTWARE\Fortinet\FSAE\dcagent]
        log_file=c:\dcagentlog.txt
        enable_log=0
    [ca]
        10.252.0.51=8002
        10.252.0.55=8002

xx\SWINAD02.xx (= 10.252.0.51)
    Domain controller is running 64 bits Windows
    DC Agent version: 5.0.261.0 
    [HKLM\SOFTWARE\Fortinet\FSAE\dcagent]
        log_file=c:\dcagentlog.txt
        enable_log=0
        disable_rdp_override=1
    [ca]
        10.252.0.51=8002
        10.252.0.55=8002
        10.252.0.50=8002

The config export on both CA's are the same. But as you can see, in the [ca] section there's a third Collector. That is the DC (swinad01) that no longer has a collector. So the agents in fact may try to report to a non-existent collector. I could imagine that to cause some delay (waiting for timeout).

You can also see the DC agent reports to the collector agent residing on the same host, does that seem correct ?

The only way I see to alter the collector agent list, is reinstalling: during setup the wizard prompts for collector agent list.

[xsilver_FTNT]

If 10.252.0.50 SWINAD01 do not run Collector Agent, then use simple regedit and edit the rest of the DCs and their registry keys in [HKLM\SOFTWARE\Fortinet\FSAE\dcagent\ca] to correct the listing. No reinstall needed.

If you really have urge to update/install somethign then SWINAD02 runs 5.0.261 .. not a problem, as it is still 5.x it should work pretty well. However it is better to have same version of Collector and Agents.

Having for e.g. 10.252.0.51  SWINAD02 running Collector and having DCAgent reporting to itself is not a problem, it's expected. As Collector and DCAgent are two independent entities/daemons. Well, DCAgent is not exactly daemon, it's sub-authentication package, basically library, so it is not even visible as process. But it does not matter.

Having DCAgent reporting to non-existing Collector should not make delays in minutes range as you stated .. but correct this anyway.

[dieter]

Ok, I'll correct the entries and follow up for a few days.

Strange that one of the DC Agents is still on older version, reinstall was done with the same executable. That and the fact of the old saved_config.txt makes me believe something was not completely uninstalled...

To be safe, I'll try updating it (again).

I did a random sampling for logon events (ID 4624). I do find a correlation between security event log and agent log. In one case however timestamp is about 25 seconds apart (as in: log entry in agent log is 25 s later than the event log). That is acceptable, but it still is a delay. I'll check wether it could be a performance issue on that server...

[xsilver_FTNT]

simple stupid question: Do those servers have time synchronized (via reliable NTP for e.g.) ?