Description
This article describes how to create a BASH script for continuous monitoring of set debug command outputs via SSH connection.
Scope
FortiManager, FortiAnalyzer, FortiGate.
Solution
Prerequisites:
- The following must be installed on the Linux/GNU platform:
- openssh-client.
- sshpass.
- gpg (optional for an encrypted password).
- crontab.
-
‘sshpass’ is used for running SSH using 'keyboard-interactive' password authentication, but in a non-interactive mode.
If no option is selected, 'sshpass' reads the password from the standard input.-
Options:
-f: The password is the first line of the configured file
-e: The password is taken from the environment variable 'SSHPASS'.-
The simplest way of using it is as follows:
-
-
SSHPASS='PassWord' sshpass -e ssh remoteuser@faz.example.com
FAZ_742 #
The password for ‘remoteuser’ is provided through the environment variable.
The downside here is that the password is visible in the bash script file.
-
-
The next option is to use a file to store the password:
-
echo "PassWord" > .hidenpass
sshpass -f .hidenpass ssh remoteuser@faz.example.com
FAZ_742 #
See the Security section below.
The provided script example (below) and the script in the .zip file are based on this example.
-
-
Using 'gpg' to encrypt the password in the file and decrypt it when used.
-
gpg -c .hidenpass
After encrypting the file, a new file is created: .hidenpass.gpg.
gpg -d -q .hidenpass.gpg | sshpass ssh remoteuser@faz.example.com
FAZ_742 #
- Create the BASH script.
Example:
#!/bin/bash
# Author: vraev
# Source: https://community.fortinet.com
USERNAME=remoteuser
HOSTS=('faz.example.com')
# For more hosts use the example: HOSTS=('faz.example.com' ‘12.12.12.1’)
SSHDATE=$(date +"%Y%m%dT%H%M")
remotecommands="remotecommands.txt"
sanit()
{
sed 's/^[ \t]*//;s/[ \t]*$//' ${remotecommands} | tr -s '[:space:]'
}
# Input sanitation: SED will remove all leading and trailing whitespace from the beginning and end of each line;
# TR will “squeeze” every character matching :space: character class to a single one.
retrieve()
{
for HOSTNAME in "${HOSTS[@]}"
do
sshpass -f .hidenpass ssh -tt ${USERNAME}@${HOSTNAME} < <(sanit) >> ${HOSTNAME}_${SSHDATE}.log
done
exit 1
}
#cd ~/ssh_test/
retrieve
- Notes:
-
To review the syntax, use the 'tree' command, as in 'tree diagnose test connection'.
-
All the examples are included in the attached file 'retrieve.zip' below.
The password is: 'fortinet_community'. -
The remote commands should be one per line without additional characters. Use 'exit' as the last command.
-
The limitation of this type of script is that it cannot run live debug commands (i.e., diagnose debug application … ).
- Security:
-
Use chmod 0400 - (read-only) for the files containing the password.
-
Use chmod 0700 - (rwx) for the script.
-
chattr +i is also a good option.
A file with the 'i' attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file, most of the file's metadata cannot be modified, and the file cannot be opened in write mode.
Only the superuser or a process possessing the CAP_LINUX_IMMUTABLE capability can set or clear this attribute.
- Scheduling with crontab:
The last step is to schedule the script at a specific time to retrieve the debug information.
For scheduling, crontab is a very convenient tool:
crontab -e to enter edit mode
field allowed values
----- --------------
minute 0–59
hour 0–23
day of month 1–31
month 1–12
day of week 0–7 (0 or 7 is Sun)
After these fields, add the full path of the script file.
For example:
* */20 * * * /home/user/ssh_test/retrievedata.sh
In this case, the script will be started every 20 minutes.
Use crontab -r - to delete the current configuration of crontab, but only if there is nothing else scheduled.
Note: The default directory for the running script will be the user home directory (/home/user/). The files will be saved there even if the script is in another directory. To change this add in the script, for example: cd ~/ssh_test/
- Troubleshooting:
gpg -d -q .hidenpass.gpg | sshpass ssh remoteuser@10.5.17.152
Pseudo-terminal will not be allocated because stdin is not a terminal.
This error can be mitigated by adding -tt options under ssh client.
If GPG is used, it requires the password for decrypting after a few decryptions. To mitigate this, consider using the following options:
--batch –no-tty
For more information, review the manual of GPG (man gpg).
Connect manually to the remote site at least once to retrieve the key fingerprint.
ssh remoteuser@10.5.17.152
The authenticity of host '10.5.17.152 (10.5.17.152)' can't be established.
ED25519 key fingerprint is SHA256:IsqJAwk3KiR4cIVnP89fC1dhm5WR/gNj8Iz0cpnTulk.
This key is not known by any other name.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.5.17.152' (ED25519) to the list of known hosts.
(remoteuser@10.5.17.152) Password:
FAZ_742 $
If the unit has many IPs/FQDNs, make sure that all of the ones used in the script are added.
ssh remoteuser@faz.example.com
The authenticity of host 'faz.example.com (10.44.1.152)' can't be established.
ED25519 key fingerprint is SHA256:IsqJAwk3KiR4cIVnP89fC1dhm5WR/gNj8Iz0cpnTulk.
This host key is known by the following other names/addresses:
~/.ssh/known_hosts:2: [hashed name]
~/.ssh/known_hosts:5: [hashed name]
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'faz.example.com' (ED25519) to the list of known hosts.
(remoteuser@faz.example.com) Password:
If a Standard_User admin profile is used for the remoteuser, the following error may be observed:
get sys stat
No permission to 'get system.status'
‘remoteuser’ must be assigned a Super_User or equivalent admin profile to allow system access to run diagnostic commands.
Note:
Command parameters are case-sensitive. Quotes are always used around the parameters like in this example: 'my_Account'.
Related articles:
Technical Tip: How to gather information and fix high CPU and Mem utilization conditions.
Technical Tip: BASH script to retrieve debug information periodically with SSH and SSH multiplexing
