FortiAnalyzer with 5.4.2 firmware fails RADIUS authentication against FortiAuthenticator

[tanr]

I decided to roll the dice and upgrade a FortiAnalyzer 200D to the new 5.4.2 firmware.

I followed the upgrade guide and rebooted before the upgrade, etc.

Everything seemed fine till I noticed that all authentication against RADIUS on the FortiAuthenticator 200D 4.1.2 was failing.  I mostly use this to give me two-factor authentication with FortiTokens for admins on the FAZ.

Checking further, everything seemed to be going through, but the FAC always reported "Invalid Password".  Note that my FortiGate is still working with the FAC without problem.

I tried a number of things, all without success:

[ul]

Re-entering the RADIUS secret for the FAZ and the FAC

Completely recreating the FAC's RADIUS client and the FAZ's Remote Auth (RADIUS) server

Removing and recreating local users on the FAC, removing and recreating the RADIUS users on the FAZ

Updating the FAC to the most recent firmware (4.2.0)

Trying users without two-factor authentication

Trying different EAP methods

Verify the nas-ip was properly set for the FAZ (it was)

Trying RADIUS users (on the FAZ) with wildcard, without wildcard, without the password set, with the password set, etc.

Changing the way the FAC realm (which just refers to local anyway) was referred to from the FAZ[/ul]

So, I'm stumped, and it's looking like time for Wireshark.  It appears that the password just isn't being correctly sent from the FAZ to the FAC's RADIUS server.

Any suggestions?  Besides "Never be the first one to upgrade the firmware" that is...

Thanks.

[emnoc]

Here's what I would do

1: run a radiusdump or radiussniff utility  on capture radius flow while authentication

2: it seems like you did re-keyed the radius secrete but  re-keyed a temporal simple key  on both units ( server/client ) for testing. I'm assuming when you rebuilt it you might have done just that

3: are you using chap/pap/MSchap/etc...( above radius dump/sniff will display that )

4: what does the logs show at the radisuclient and server

5: does 5.4.2 have any new  "diag test application" solution for authorization ( check )

Ken

[hzhao_FTNT]

Hi there, QA is unable to reproduce your issue in lab.  I would suggest you open a ticket for this issue and post the ticket number in forum.

Regards,

hz

[tanr]

@hzhao_FTNT, thanks for looking into this.  I'll open a support ticket later today.

@emnoc,

Thanks for the suggestions.  

1. I'm not familiar with any specific radius dump or sniffer utility.  Any you would suggest?

    Also, any simple radius client I might use to test this scenario?

2. I already tried a different simple key for both client and server, as well as rebuilding.  I haven't tried doing it with a different IP yet, in case this is related to something being cached.  I did already clear the client machine's cache, but I don't know if there is something else needed for the FAZ.

3. I've tried most of the EAP combinations, which give various errors related to EAP before even getting to checking the password.  PAP from the FAZ seems to work the best, except for the invalid password of course. 

4. The logs are pretty brief.  I've included the only items I got, with a test user account on the FAC.  Names and IPs below have been XX'd, but were all correct (so, for instance, the FAC's nas IP matched the NAS IP it was expecting).  I have not yet tried to set the FAC or FAZ to be more verbose.

FAC log shows only:

status=Failed nas=IP.IP.IP.IP itime=2016-12-15 19:54:12 vd=root level=information devid=FAC-XXXXXXX dtime=2016-12-16 03:54:12 logid=20102 subtype=Authentication devname=FAC-XXXX itime_t=1481860452 user=testmgr@XXXX.XXX logdesc=AUTH_FAIL_BADPASS time=03:54:12 date=2016-12-16 type=event action=Authentication msg=Local administrator authentication with no token failed: invalid password

FAZ event log shows only:

016-12-15 19:54:13 log_id=0001010019 type=event subtype=system pri=alert user="testmgr@XXX.XXX" userfrom="GUI(IP.IP.IP.IP)" msg="User 'testmgr@XXXXX.XXX' login failed from GUI(IP.IP.IP.IP), reason:Authentication failure. Please try again..." adminprof=""

5.  FAZ has diag test app but no apps that look associated with RADIUS.  Maybe diag debug service, if I could find a description of the services.  Since I'm not too familiar with CLI on the FAX I'll probably need to go through it with support.

[emnoc]

On the FGT you have at minimum  the   dig test authserver  that would be a simple  tool

e.g

FWHQDCEQXDAL01 (root) # diag test authserver  radius 

<server_name> <chap | pap | mschap | mschap2> <username> <password>

And for radiusdump I like the following ;

macintosh:mail kfelix$ radsniff -h

Usage: radsniff [options][stats options] -- [pcap files]

options:

  -a                    List all interfaces available for capture.

  -c <count>            Number of packets to capture.

  -C                    Enable UDP checksum validation.

  -d <directory>        Set dictionary directory.

  -d <raddb>            Set configuration directory (defaults to /usr/local/Cellar/freeradius-server/3.0.11/etc/raddb).

  -D <dictdir>          Set main dictionary directory (defaults to /usr/local/Cellar/freeradius-server/3.0.11/share/freeradius).

  -e <event>[,<event>]  Only log requests with these event flags.

                        Event may be one of the following:

                        - received - a request or response.

                        - norsp    - seen for a request.

                        - rtx      - of a request that we've seen before.

                        - noreq    - could be matched with the response.

                        - reused   - ID too soon.

                        - error    - decoding the packet.

  -f <filter>           PCAP filter (default is 'udp port <port> or <port + 1> or 3799')

  -h                    This help message.

  -i <interface>        Capture packets from interface (defaults to all if supported).

  -I <file>             Read packets from file (overrides input of -F).

  -l <attr>[,<attr>]    Output packet sig and a list of attributes.

  -L <attr>[,<attr>]    Detect retransmissions using these attributes to link requests.

  -m                    Don't put interface(s) into promiscuous mode.

  -p <port>             Filter packets by port (default is 1812).

  -P <pidfile>          Daemonize and write out <pidfile>.

  -q                    Print less debugging information.

  -r <filter>           RADIUS attribute request filter.

  -R <filter>           RADIUS attribute response filter.

  -s <secret>           RADIUS secret.

  -S                    Write PCAP data to stdout.

  -v                    Show program version information.

  -w <file>             Write output packets to file.

  -x                    Print more debugging information.

stats options:

  -W <interval>         Periodically write out statistics every <interval> seconds.

  -T <timeout>          How many milliseconds before the request is counted as lost (defaults to 5200).

You only need the radius secret and just read in a cap of your radius-client-2-server for udp port 1812/1645 or whatever your using

Since other things are working I bet it's chap or mschap related , I found a few bugs in  the VM FAZ and in fact 5.4.2 just fix one with user-type group and teacakes servers.

Ken

[tanr]

"diag test authserver radius FAC-RADIUS pap UserName Password" works just fine from the FortiGate to the FAC.  No such command in the FAZ cli that I've been able to find.

From the FAZ, the *only* method that works to get to the FAC radius is PAP.  MSv2 (mschap) and CHAP fail (even though I'm allowing all EAP types in the FAC) before they even try to send through the password, with the log in the FAC showing:

   Local administrator authentication(mschap) with no token failed: invalid user parameter

I'll try radsniff if I get a chance before the holidays.

I've been digging around in the documentation trying to find an example of enabling a FortiAnalyzer's local logging (config sys locallog?) and setting it to a debug level, but haven't found any complete example, and there are a lot of options under that heading (disk, fortianalyzer, etc.) so am not sure of the settings.

Any pointers on the CLI commands to set up a FortiAnalyzer for debug logging, and to return it to defaults?

[mel · Accepted solution]

We had the same issue - we had to change the radius key to a 16 character key - 17 characters and above doesn't seem to work anymore

[tanr]

Thanks for the info, I'll try shortening the keys and post how it goes.

Did you report this as a bug to Fortinet?

[tanr]

With FAC 4.2.1, trying with a 16 character radius key still gives the same failure messages as listed above when attempting to use MSv2 (mschap) or CHAP for RADIUS authentication from FAZ 5.4.2.  I can only make it work with PAP.

Now that I'm back on site I've opened a support ticket, and will post the results here.

[tanr]

Okay, went through support tickets for both FAZ and FAC, and now have an understanding of what's been happening.

The short answer is from the FortiAuthenticator documentation: 

http://docs.fortinet.com/uploaded/files/3481/fac-admin-guide-421.pdf on page 62 where it says:

"An account marked as an administrator can be used for RADIUS authentication if Allow RADIUS Authentication is selected. ... These administrator accounts only support Password Authentication Protocol (PAP)."

From the support ticket, "it has been confirmed by engineering that [FAC] admin user passwords have irreversible encryption so they can't be used with mschap in certain situations."

I don't really see how the irreversible encryption would block being able to use mschap.  I can understand they might want to require that admin passwords are only ever passed to them using a specific uni-directional encryption as that would increase security.  But since they DO allow sending admin passwords over EAP PAP (basically clear text) that breaks security anyway....   Ugh.

So, to work around this, I've simply created a user account on the FAC, with associated FortiToken, that I access from the FAZ with MSv2 (mschap-v2).  This works just fine.

Now, for what led me down this rabbit hole in the first place: The FortiAnalyzer documentation:

http://help.fortinet.com/fa/faz50hlp/54/5-4-2/index.htm#FortiAnalyzer_Admin_Guide/900_Admins_Account...

It specifically says to create either an admin or user account on the FAC (and the pictured example is for an admin account) and then says to use whatever EAP methods the FAC uses.

This is incorrect.  The FAC can't handle an EAP access to an admin account with anything other than EAP PAP.

To avoid others running into this the FAZ documentation should be updated to explain why you won't be able to do this with a FAC admin account for anything besides EAP PAP.  It should really also tell you that doing so is a bad idea and just tell you to use a FAC user account.

I'm waiting on one of the tickets to add the documentation correction.