FortiOS CLI question

[rawbar]

We are running an old version of FortiOS 4.3 (patch 6) with a known memory leak.  Obviously it needs to be updated.  In the meantime, once a month one of the network engineers was killing the rogue process to free up the memory.  He has since left the company and didn't document what the process was or how to kill it.  All I have is a Fortinet ticket #.

I connected to the CLI but the only CLI commands available (both via web and ssh) are config, get, show and exit.  I'm looking at the FortiOS Handbook CLI Reference for FortiOS 4.3 and is says the command I should use is "system performance top".  However "system" isn't valid (5499: Unknown action 0 Command fail.  Return code -1)

What might be the reason "system" isn't available?

[Christopher_McMullan · Accepted solution]

The general syntax for the CLI is verb-area-noun, so every command has to start with config, execute, get, show, or diagnose.

In this case, the command to view 'top' data as in Linux would be 'diag sys top'. You can also get a system performance snapshot with 'get sys perf stat'.

If 4.3.6 is suffering from merged_daemons, you would want to run 'diag sys top', and immediately press 'q' afterwards to generate one set of results. Find the process ID for merged_daemons (if that's truly the offending process - but from that build, it likely is), then run 'diag sys kill 11 <PID>'. For instance, if merged_daemons is running with a PID of 50, the command would be 'diag sys kill 11 50'.

The CLI reference guide, except for the bottom sections dealing with the commands beginning with the verbs 'get' and 'execute' all assume an initial verb of 'config'. That may be where the confusion was introduced: every section like 'alertemail...' or 'router....' assumes it begins with 'config'. So, for static routes, the document path would be router > static, but the full command would be 'config router static'.

[Christopher_McMullan · Accepted solution]

The general syntax for the CLI is verb-area-noun, so every command has to start with config, execute, get, show, or diagnose.

In this case, the command to view 'top' data as in Linux would be 'diag sys top'. You can also get a system performance snapshot with 'get sys perf stat'.

If 4.3.6 is suffering from merged_daemons, you would want to run 'diag sys top', and immediately press 'q' afterwards to generate one set of results. Find the process ID for merged_daemons (if that's truly the offending process - but from that build, it likely is), then run 'diag sys kill 11 <PID>'. For instance, if merged_daemons is running with a PID of 50, the command would be 'diag sys kill 11 50'.

The CLI reference guide, except for the bottom sections dealing with the commands beginning with the verbs 'get' and 'execute' all assume an initial verb of 'config'. That may be where the confusion was introduced: every section like 'alertemail...' or 'router....' assumes it begins with 'config'. So, for static routes, the document path would be router > static, but the full command would be 'config router static'.

[rawbar]

Thanks very much for the quick and thorough explanation.  However diag is not a valid command for me nor is system.  Both generate 5499: Unknown action.  

If I hit ? to get a list of valid command, the only ones listed are config, get, show and exit.

'get sys perf stat' also is not valid.  I can do a 'get system status' but for get system, the only valid options I'm shown with 'get system ?' are 'status' and 'system status'.  

Could it be a permission on this account issue?  I'm using what should be a root account, but it's entirely possible someone in our EU team has limited the permission on the US root account.  Update: I just checked and this account is assigned to the 'super_admin' profile, same as the root account.

[rawbar]

The profile is super_admin

[rawbar]

That doesn't seem to be the issue unless something is wrong with the super_admin profile.  My account is assigned to super_admin, and I just checked super_admin permissions and everything is read/write across the board.

[Christopher_McMullan]

It seems like a permissions issue. You might be able to see what profile has been applied to your account:

config system admin

edit <admin_name>

show

end

If the accprofile is prof_admin, or anything other than super_admin, restrictions are likely being applied.

[Christopher_McMullan]

One solution would be to use the maintainer account to recover the super admin's password, if you have the scope to:

config system global

get | grep admin-maintainer

end

If admin-maintainer is enabled, this is equivalent to changing the boot variables for Cisco devices from 0x2102 (from memory, this is normal). Use a console connection, and immediately after gaining the login prompt, you have a short amount of time to login as:

user maintainer

password bcpbserialnumberoffortigate

For instance, my old 80C had the serial number FGT80Cxxxxxx5328. I would enter:

user maintainer

pass bcpbFGT80Cxxxxxx5328 (case sensitive)

This would grant me super user access to the CLI, where I could view and modify the admin accounts, admin profiles, passwords, etc. in order to regain root-level permissions.

[rawbar]

This account is super_admin

[Christopher_McMullan]

I've only seen references to that specific error when an HA cluster was involved.

It may be worth your while to boot into maintainer anyway, to see if you still are locked out of 'diagnose' commands. If 'diag...' is available with maintainer, you could try creating a new admin account to sidestep the issues with the existing admin users. If 'diagnose' is still unavailable, it may point to deeper corruption.

[rawbar]

This is indeed an HA cluster.  I'm ssh'd into the master.