Technical Tip: Using rclone to manually upload files from FortiAnalyzer to Azure via Cloud-Out Connector

[jhyde]

Description	This article describes the rclone utility and how it can be used to upload files manually from FortiAnalyzer to a specific folder within Azure Storage Account Container. Note shell access will be required to perform this, and the Azure Cloud-Out Connector must be configured prior. This can be useful in situations where the built-in FortiAnalyzer commands (as below) used to create and upload a test file from the /tmp directory on FortiAnalyzer to the folder location defined within the Azure Storage Container fail with errors: Command Syntax: FAZ # diagnose debug application uploadd 8 FAZ # diagnose debug enable FAZ # diag test app uploadd 62 <azure_connector> <folder_name> Successful Connection: FAZ # diagnose debug application uploadd 8 FAZ # diagnose debug enable FAZ # diagnose test application uploadd 62 fazazureconnector fazlogfolder _cmd_proxy:109: 1729158199 cmd "/usr/local/bin/rclone --config=/drive0/private/rclone.cfg copy /tmp/cloud_storage_test-CzizRY fazazureconnector:fazlogfolder" sent successfully! uuid=3db74df6-8c6c-11ef-8920-006265741201 cloud_storage_add_cmd_proxy_rclone_copy:2095: [CS] uuid[3db74df6-8c6c-11ef-8920-006265741201] src_path[/tmp/cloud_storage_test-CzizRY] remote_name[fazazureconnector][fazazureconnector] remote_path[fazlogfolder][fazlogfolder] s1) copy file. uuid[3db74df6-8c6c-11ef-8920-006265741201] cmd_proxy_fetch_rcode:257: 1729158200 got proc[3db74df6-8c6c-11ef-8920-006265741201] return code: 0 s2) rc=0 message[success]  Unsuccessful Connection: FAZ # diagnose debug application uploadd 8 FAZ # diagnose debug enable FAZ # diagnose test application uploadd 62 fazazureconnecter fazlogfolder _cmd_proxy:109: 1729158257 cmd "/usr/local/bin/rclone --config=/drive0/private/rclone.cfg copy /tmp/cloud_storage_test-6zJqpY fazazureconnecter:fazlogfolder" sent successfully! uuid=605bf398-8c6c-11ef-8920-006265741201 cloud_storage_add_cmd_proxy_rclone_copy:2095: [CS] uuid[605bf398-8c6c-11ef-8920-006265741201] src_path[/tmp/cloud_storage_test-6zJqpY] remote_name[fazazureconnecter][fazazureconnecter] remote_path[fazlogfolder][fazlogfolder] s1) copy file. uuid[605bf398-8c6c-11ef-8920-006265741201] cmd_proxy_fetch_rcode:257: 1729158258 got proc[605bf398-8c6c-11ef-8920-006265741201] return code: 256 s2) rc=256 message[unknown error] Technical Tip: FortiAnalyzer Cloud-out connector
Scope	FortiAnalyzer. Note from v7.6.0, v7.4.4, and v7.2.6 onwards, shell access has been removed.
Solution	Ensure the Azure Cloud-Out Connector is configured, as per Technical Tip: FortiAnalyzer Cloud-out connector. Access the shell on FortiAnalyzer. If this is not enabled, enable it by following the steps in Technical Tip: How to enable backend-shell access in FortiManager/FortiAnalyzer. FAZ # exec shell Enter password: bash$ Create a file on the /tmp directory of FortiAnalyzer. Note the file 'testfile.txt' will be created, and the contents will contain the word 'test': bash$ cd /tmp bash$ echo test > testfile.txt Confirm the file exists in the directory: bash$ ls Confirm the file has the contents defined per the echo command. In this case, 'test': bash$ cat testfile.txt test Issue the following command on the FortiAnalyzer shell. This will manually upload the file created to the folder defined within the Storage Account within Azure, via the Azure Cloud-Out Connector. Note a log file will also be created which will contain info on the result of the command: Command Syntax: bash$ /usr/local/bin/rclone --config=/drive0/private/rclone.cfg copy /tmp/<file_name> <azure_connector>:<folder_name> --log-file=rclone.log -vv Example Command: bash$ /usr/local/bin/rclone --config=/drive0/private/rclone.cfg copy /tmp/testfile.txt fazazureconnector:fazlogfolder --log-file=rclone.log -vv Review the log file 'rclone.log'. In this case, the following shows that the file was sent successfully: bash$ tail -f rclone.log 2024/10/17 12:01:08 INFO : testfile.txt: Copied (new) 2024/10/17 12:01:08 INFO : Transferred: 5 / 5 Bytes, 100%, 7 Bytes/s, ETA 0s Errors: 0 Checks: 0 / 0, - Transferred: 1 / 1, 100% Elapsed time: 600ms 2024/10/17 12:01:08 DEBUG : 4 go routines active 2024/10/17 12:01:08 DEBUG : rclone: Version "v1.44" finishing with parameters ["/usr/local/bin/rclone" "--config=/drive0/private/rclone.cfg " "copy" "/tmp/testfile.txt" "fazazureconnector:fazlogfolder" "--log-file=rclone.log" "-vv"] Review the contents of the Azure Storage Account folder location. The file should appear, as 'testfile.txt' in this case. Note the 'cloud_storage_test-xxxxxx' files are created when issuing the built-in FortiAnalyzer test upload commands: