Good day all,
I was wondering how I could create a dashboard that leverages data collected via a scheduled playbook? The goal is to create a security engineer dashboard where key metrics could be reviewed as part of a shift checklist. I already have several of the jobs running, now I just need to work out how to use the data to create the desired dashboards.
Any assistance here would be greatly appreciated. Thanks!
Nominating a forum post submits a request to create a new Knowledge Article based on the forum post topic. Please ensure your nomination includes a solution within the reply.
Hi,
You can create a custom module in FortiSOAR and save data created by playbook in it. You can use this module records as input data for appropriate widgets in a dashboard.
Hello @tkanade,
Thanks for responding here. Which guide do I review in order to create a custom module?
Perhaps it would be more helpful if I explained what it is that I'm trying to do here. I currently have a script that pulls JSON data from an endpoint running every hour. I want to take an integer value from that output and track the way it's trending for up to a week.
You can check https://docs.fortinet.com/document/fortisoar/7.3.0/administration-guide/97786/application-editor , section module editor . Using module editor , you can create a custom module to store data you need , which you can then use for creating dashboard.
The script you mentioned , what is output format ? Can you tweak script to provide a output in CSV format where columns can be mapped to fields of module in FortiSOAR. The we can use CSV data management solution pack to ingest CSV file and update custom module.
In more advance way , we can make script send output data as a payload directly to FortiSOAR playbook which in turn can update module records.
Hello @tkanade
Apologies for the late reply here as I have been quite busy over the last 2 weeks. If it is possible I'd like to use the JSON format method to take the integer data from certain fields to create timeline trend widgets and utilization widgets like the examples below.
Timeline widget example:
Usage Widget:
I have also attached a redacted copy (credentials removed) of the playbook script that I run to extract this data in addition to the JSON output for the data I need to parse for the widgets.
The output contains a series of metrics for a list of servers. For each server I'd like to display the graphs mentioned above for the following metrics:
- cpu
- hp
- rp
- dup
Playbook:
{
"type": "workflow_collections",
"data": [
{
"@context": "/api/3/contexts/WorkflowCollection",
"@type": "WorkflowCollection",
"name": "001 - CSE Tests",
"description": null,
"visible": true,
"image": null,
"id": 64,
"createDate": 1661333764.460715,
"modifyDate": 1663983252.383072,
"deletedAt": null,
"importedBy": [],
"recordTags": [],
"workflows": [
{
"@type": "Workflow",
"triggerLimit": null,
"name": "Test Elastic Node Stats",
"aliasName": null,
"tag": null,
"description": null,
"isActive": true,
"debug": true,
"singleRecordExecution": false,
"remoteExecutableFlag": false,
"parameters": [],
"synchronous": false,
"lastModifyDate": 1680036269,
"collection": "/api/3/workflow_collections/f48521da-efbc-43b6-85bb-5c9ed5d851f4",
"versions": [],
"triggerStep": "/api/3/workflow_steps/ce02477b-a2ba-400b-9c49-c83658caf67b",
"steps": [
{
"@type": "WorkflowStep",
"name": "Create Elastic Node Stats Record",
"description": null,
"arguments": {
"resource": {
"__replace": "",
"dataSource": "{{vars.steps.Generate_Data_Source.data}}{{vars.steps.Generate_Data_Source.data['code_output']}}",
"dataValueName": "{{vars.steps.Generate_Data_Value_Name.data['code_output']}}",
"elasticNodeStatData": "{{vars.steps.Get_Elastic_Node_Stats.data['code_output']}}"
},
"operation": "Overwrite",
"collection": "/api/3/data_values",
"__recommend": [],
"fieldOperation": {
"recordTags": "Overwrite"
},
"step_variables": []
},
"status": null,
"top": "40",
"left": "1340",
"stepType": "/api/3/workflow_step_types/2597053c-e718-44b4-8394-4d40fe26d357",
"group": null,
"uuid": "2a83730f-deec-4875-9b7e-c2d82f5698f1"
},
{
"@type": "WorkflowStep",
"name": "Generate Data Value Name",
"description": null,
"arguments": {
"config": "3f12d36a-b7c9-42c4-9620-4739b92b284a",
"params": {
"python_function": "print(\"Elastic Node Stats\")"
},
"version": "2.0.3",
"connector": "code-snippet",
"operation": "python_inline_code_editor",
"operationTitle": "Execute Python Code",
"step_variables": []
},
"status": null,
"top": "40",
"left": "360",
"stepType": "/api/3/workflow_step_types/1fdd14cc-d6b4-4335-a3af-ab49c8ed2fd8",
"group": null,
"uuid": "60c0d37f-1070-49b9-a072-7409745c2d59"
},
{
"@type": "WorkflowStep",
"name": "Generate Data Source",
"description": null,
"arguments": {
"config": "3f12d36a-b7c9-42c4-9620-4739b92b284a",
"params": {
"python_function": "print(\"Elastic Cat Nodes API\")"
},
"version": "2.0.3",
"connector": "code-snippet",
"operation": "python_inline_code_editor",
"operationTitle": "Execute Python Code",
"step_variables": []
},
"status": null,
"top": "40",
"left": "680",
"stepType": "/api/3/workflow_step_types/1fdd14cc-d6b4-4335-a3af-ab49c8ed2fd8",
"group": null,
"uuid": "8c88461c-4460-44e2-ab60-0680dea71b7d"
},
{
"@type": "WorkflowStep",
"name": "Get Elastic Node Stats",
"description": null,
"arguments": {
"config": "3f12d36a-b7c9-42c4-9620-4739b92b284a",
"params": {
"python_function": "import json\nimport requests\nfrom requests.structures import CaseInsensitiveDict\nimport time\n\napi_base_url = f\"https://api.endpoint.example.com\"\n\n# API version\nav = f\"v1\"\n\n# Service\nsvc = f\"_cat\"\n\n# Action\nact = f\"nodes?v&h=name,ip,role,master,cpu,rm,rc,rp,hm,hc,hp,dt,du,dup&s=node.role\"\n\nheaders = CaseInsensitiveDict()\n\nheaders[\"Accept\"] = \"application/json\"\n\nheaders[\"Authorization\"] = f\"Basic *****REDACTED*****\"\n\nendpoint = f\"{api_base_url}/{svc}/{act}\"\n\ntimestr = time.strftime(\"%Y-%m-%d_%H:%M:%S\")\n\nsr = requests.get(endpoint, headers=headers)\n\nsr_json = sr.json()\n\nprint(sr_json)"
},
"version": "2.0.3",
"connector": "code-snippet",
"operation": "python_inline_code_editor",
"operationTitle": "Execute Python Code",
"step_variables": []
},
"status": null,
"top": "40",
"left": "1020",
"stepType": "/api/3/workflow_step_types/1fdd14cc-d6b4-4335-a3af-ab49c8ed2fd8",
"group": null,
"uuid": "ae73f819-5b3f-489a-a4cc-5bd9915105e3"
},
{
"@type": "WorkflowStep",
"name": "Start",
"description": null,
"arguments": {
"route": "4f983a73-5c6e-47c3-a48e-ff74d4b3ba7c",
"resources": [
"data_values"
],
"inputVariables": [],
"step_variables": {
"input": {
"params": [],
"records": "{{vars.input.records}}"
}
},
"executeButtonText": "Execute",
"noRecordExecution": true,
"singleRecordExecution": false
},
"status": null,
"top": "40",
"left": "40",
"stepType": "/api/3/workflow_step_types/f414d039-bb0d-4e59-9c39-a8f1e880b18a",
"group": null,
"uuid": "ce02477b-a2ba-400b-9c49-c83658caf67b"
}
],
"routes": [
{
"@type": "WorkflowRoute",
"name": "Start -> Generate Data Value Name",
"targetStep": "/api/3/workflow_steps/60c0d37f-1070-49b9-a072-7409745c2d59",
"sourceStep": "/api/3/workflow_steps/ce02477b-a2ba-400b-9c49-c83658caf67b",
"label": null,
"isExecuted": false,
"uuid": "f0c86d56-2fed-4452-8aea-a0239676cf6b"
},
{
"@type": "WorkflowRoute",
"name": "Generate Data Value Name -> Generate Data Source",
"targetStep": "/api/3/workflow_steps/8c88461c-4460-44e2-ab60-0680dea71b7d",
"sourceStep": "/api/3/workflow_steps/60c0d37f-1070-49b9-a072-7409745c2d59",
"label": null,
"isExecuted": false,
"uuid": "5eb5c03a-539e-4f32-848e-cda7afa4ebbb"
},
{
"@type": "WorkflowRoute",
"name": "Generate Data Source -> Get Elastic Node Stats",
"targetStep": "/api/3/workflow_steps/ae73f819-5b3f-489a-a4cc-5bd9915105e3",
"sourceStep": "/api/3/workflow_steps/8c88461c-4460-44e2-ab60-0680dea71b7d",
"label": null,
"isExecuted": false,
"uuid": "d4e22e4d-32ae-41d0-9878-edd903371b03"
},
{
"@type": "WorkflowRoute",
"name": "Get Elastic Node Stats -> Create Elastic Node Stats Record",
"targetStep": "/api/3/workflow_steps/2a83730f-deec-4875-9b7e-c2d82f5698f1",
"sourceStep": "/api/3/workflow_steps/ae73f819-5b3f-489a-a4cc-5bd9915105e3",
"label": null,
"isExecuted": false,
"uuid": "1b1f9d76-14ec-4386-8992-0e82f5c99ff2"
}
],
"groups": [],
"priority": "/api/3/picklists/2b563c61-ae2c-41c0-a85a-c9709585e3f2",
"uuid": "9b4815f1-1d45-45ec-b403-357192655c4b",
"id": 1086,
"owners": [],
"isPrivate": false,
"deletedAt": null,
"importedBy": [],
"recordTags": []
}
]
}
],
"exported_tags": []
}
Output:
{
"data": {
"code_output": [
{
"dt": "4.5tb",
"du": "4.4tb",
"hc": "21.6gb",
"hm": "30gb",
"hp": "72",
"ip": "172.27.209.249",
"rc": "52gb",
"rm": "60gb",
"rp": "87",
"cpu": "8",
"dup": "98.21",
"name": "instance-0000000093",
"role": "f",
"master": "-"
},
{
"dt": "4.3tb",
"du": "4.2tb",
"hc": "9.6gb",
"hm": "30gb",
"hp": "32",
"ip": "172.27.200.39",
"rc": "52.3gb",
"rm": "60gb",
"rp": "87",
"cpu": "33",
"dup": "97.81",
"name": "instance-0000000107",
"role": "f",
"master": "-"
},
{
"dt": "4.3tb",
"du": "4.2tb",
"hc": "19.2gb",
"hm": "30gb",
"hp": "64",
"ip": "172.27.211.192",
"rc": "52gb",
"rm": "60gb",
"rp": "87",
"cpu": "27",
"dup": "97.81",
"name": "instance-0000000108",
"role": "f",
"master": "-"
},
{
"dt": "4.5tb",
"du": "4.4tb",
"hc": "9gb",
"hm": "30gb",
"hp": "30",
"ip": "172.27.203.191",
"rc": "51.9gb",
"rm": "60gb",
"rp": "87",
"cpu": "7",
"dup": "98.22",
"name": "instance-0000000094",
"role": "f",
"master": "-"
},
{
"dt": "4.3tb",
"du": "4.2tb",
"hc": "14.4gb",
"hm": "30gb",
"hp": "48",
"ip": "172.27.101.171",
"rc": "52.3gb",
"rm": "60gb",
"rp": "87",
"cpu": "95",
"dup": "97.83",
"name": "instance-0000000099",
"role": "f",
"master": "-"
},
{
"dt": "1.7tb",
"du": "230.9gb",
"hc": "7.1gb",
"hm": "30gb",
"hp": "23",
"ip": "172.27.211.40",
"rc": "53.2gb",
"rm": "60gb",
"rp": "89",
"cpu": "8",
"dup": "13.10",
"name": "instance-0000000106",
"role": "hrst",
"master": "-"
},
{
"dt": "1.6tb",
"du": "11gb",
"hc": "4gb",
"hm": "29gb",
"hp": "13",
"ip": "172.27.201.68",
"rc": "31.4gb",
"rm": "58gb",
"rp": "54",
"cpu": "8",
"dup": "0.64",
"name": "instance-0000000064",
"role": "hrst",
"master": "-"
},
{
"dt": "1.6tb",
"du": "11gb",
"hc": "14.7gb",
"hm": "29gb",
"hp": "50",
"ip": "172.27.206.153",
"rc": "31.4gb",
"rm": "58gb",
"rp": "54",
"cpu": "8",
"dup": "0.63",
"name": "instance-0000000067",
"role": "hrst",
"master": "-"
},
{
"dt": "1.6tb",
"du": "10.9gb",
"hc": "16.5gb",
"hm": "29gb",
"hp": "57",
"ip": "172.27.202.244",
"rc": "31.5gb",
"rm": "58gb",
"rp": "54",
"cpu": "6",
"dup": "0.63",
"name": "instance-0000000068",
"role": "hrst",
"master": "-"
},
{
"dt": "1.6tb",
"du": "615.8mb",
"hc": "7.3gb",
"hm": "29gb",
"hp": "25",
"ip": "172.27.206.144",
"rc": "31.4gb",
"rm": "58gb",
"rp": "54",
"cpu": "12",
"dup": "0.03",
"name": "instance-0000000083",
"role": "hrst",
"master": "-"
},
{
"dt": "1.7tb",
"du": "264.8gb",
"hc": "18.8gb",
"hm": "30gb",
"hp": "62",
"ip": "172.27.207.187",
"rc": "53.7gb",
"rm": "60gb",
"rp": "90",
"cpu": "9",
"dup": "15.03",
"name": "instance-0000000104",
"role": "hrst",
"master": "-"
},
{
"dt": "1.7tb",
"du": "246.3gb",
"hc": "19.9gb",
"hm": "30gb",
"hp": "66",
"ip": "172.27.204.91",
"rc": "48.8gb",
"rm": "60gb",
"rp": "81",
"cpu": "7",
"dup": "13.98",
"name": "instance-0000000103",
"role": "hrst",
"master": "-"
},
{
"dt": "1.7tb",
"du": "287.4gb",
"hc": "7.7gb",
"hm": "30gb",
"hp": "25",
"ip": "172.27.210.42",
"rc": "48gb",
"rm": "60gb",
"rp": "80",
"cpu": "6",
"dup": "16.31",
"name": "instance-0000000105",
"role": "hrst",
"master": "-"
},
{
"dt": "225gb",
"du": "237.2mb",
"hc": "2.8gb",
"hm": "7.5gb",
"hp": "38",
"ip": "172.27.201.61",
"rc": "8.6gb",
"rm": "15gb",
"rp": "58",
"cpu": "8",
"dup": "0.10",
"name": "instance-0000000112",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "211.8mb",
"hc": "3.2gb",
"hm": "7.5gb",
"hp": "43",
"ip": "172.27.208.165",
"rc": "8.7gb",
"rm": "15gb",
"rp": "59",
"cpu": "8",
"dup": "0.09",
"name": "instance-0000000116",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "223.2mb",
"hc": "4.5gb",
"hm": "7.5gb",
"hp": "60",
"ip": "172.27.207.215",
"rc": "8.7gb",
"rm": "15gb",
"rp": "58",
"cpu": "8",
"dup": "0.10",
"name": "instance-0000000109",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.5gb",
"hc": "2.8gb",
"hm": "7.5gb",
"hp": "38",
"ip": "172.27.202.192",
"rc": "9gb",
"rm": "15gb",
"rp": "60",
"cpu": "8",
"dup": "3.04",
"name": "instance-0000000016",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.4gb",
"hc": "3.2gb",
"hm": "7.5gb",
"hp": "43",
"ip": "172.27.210.44",
"rc": "9.1gb",
"rm": "15gb",
"rp": "61",
"cpu": "8",
"dup": "2.96",
"name": "instance-0000000015",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.3gb",
"hc": "4.2gb",
"hm": "7.5gb",
"hp": "56",
"ip": "172.27.206.115",
"rc": "9.1gb",
"rm": "15gb",
"rp": "61",
"cpu": "7",
"dup": "2.92",
"name": "instance-0000000033",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.5gb",
"hc": "4.9gb",
"hm": "7.5gb",
"hp": "65",
"ip": "172.27.206.186",
"rc": "8.8gb",
"rm": "15gb",
"rp": "59",
"cpu": "14",
"dup": "3.02",
"name": "instance-0000000017",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "203mb",
"hc": "1.3gb",
"hm": "7.5gb",
"hp": "18",
"ip": "172.27.207.185",
"rc": "8.8gb",
"rm": "15gb",
"rp": "59",
"cpu": "9",
"dup": "0.09",
"name": "instance-0000000110",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.4gb",
"hc": "1.9gb",
"hm": "7.5gb",
"hp": "26",
"ip": "172.27.211.83",
"rc": "9.1gb",
"rm": "15gb",
"rp": "61",
"cpu": "11",
"dup": "2.96",
"name": "instance-0000000031",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.4gb",
"hc": "847.2mb",
"hm": "7.5gb",
"hp": "11",
"ip": "172.27.202.96",
"rc": "9.2gb",
"rm": "15gb",
"rp": "61",
"cpu": "9",
"dup": "2.94",
"name": "instance-0000000032",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "236.6mb",
"hc": "3gb",
"hm": "7.5gb",
"hp": "40",
"ip": "172.27.211.95",
"rc": "8.7gb",
"rm": "15gb",
"rp": "59",
"cpu": "8",
"dup": "0.10",
"name": "instance-0000000117",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4gb",
"hc": "4.4gb",
"hm": "7.5gb",
"hp": "59",
"ip": "172.27.205.118",
"rc": "9gb",
"rm": "15gb",
"rp": "60",
"cpu": "8",
"dup": "2.69",
"name": "instance-0000000065",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "214.9mb",
"hc": "3.1gb",
"hm": "7.5gb",
"hp": "41",
"ip": "172.27.209.242",
"rc": "8.6gb",
"rm": "15gb",
"rp": "58",
"cpu": "8",
"dup": "0.09",
"name": "instance-0000000115",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "4.4gb",
"hc": "1.1gb",
"hm": "7.5gb",
"hp": "14",
"ip": "172.27.200.33",
"rc": "9gb",
"rm": "15gb",
"rp": "60",
"cpu": "8",
"dup": "3.00",
"name": "instance-0000000038",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "201.1mb",
"hc": "4.6gb",
"hm": "7.5gb",
"hp": "62",
"ip": "172.27.203.13",
"rc": "8.8gb",
"rm": "15gb",
"rp": "59",
"cpu": "14",
"dup": "0.09",
"name": "instance-0000000113",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "193mb",
"hc": "1.4gb",
"hm": "7.5gb",
"hp": "19",
"ip": "172.27.203.13",
"rc": "8.7gb",
"rm": "15gb",
"rp": "59",
"cpu": "19",
"dup": "0.08",
"name": "instance-0000000114",
"role": "ir",
"master": "-"
},
{
"dt": "225gb",
"du": "238.3mb",
"hc": "4gb",
"hm": "7.5gb",
"hp": "53",
"ip": "172.27.207.215",
"rc": "8.7gb",
"rm": "15gb",
"rp": "59",
"cpu": "9",
"dup": "0.10",
"name": "instance-0000000111",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "1018mb",
"hc": "1.9gb",
"hm": "7.5gb",
"hp": "25",
"ip": "172.27.210.109",
"rc": "9gb",
"rm": "15gb",
"rp": "60",
"cpu": "7",
"dup": "0.66",
"name": "instance-0000000086",
"role": "ir",
"master": "-"
},
{
"dt": "150gb",
"du": "224.8mb",
"hc": "3.5gb",
"hm": "6gb",
"hp": "59",
"ip": "172.27.207.58",
"rc": "7gb",
"rm": "15gb",
"rp": "47",
"cpu": "5",
"dup": "0.15",
"name": "instance-0000000087",
"role": "lr",
"master": "-"
},
{
"dt": "192gb",
"du": "5mb",
"hc": "3.5gb",
"hm": "6.3gb",
"hp": "56",
"ip": "172.27.202.111",
"rc": "7.3gb",
"rm": "16gb",
"rp": "46",
"cpu": "2",
"dup": "0.00",
"name": "instance-0000000118",
"role": "lr",
"master": "-"
},
{
"dt": "450gb",
"du": "2.5gb",
"hc": "2.5gb",
"hm": "7.5gb",
"hp": "33",
"ip": "172.27.204.62",
"rc": "8.5gb",
"rm": "15gb",
"rp": "57",
"cpu": "2",
"dup": "0.56",
"name": "instance-0000000119",
"role": "mr",
"master": "-"
},
{
"dt": "450gb",
"du": "2.5gb",
"hc": "3.6gb",
"hm": "7.5gb",
"hp": "48",
"ip": "172.27.202.61",
"rc": "8.5gb",
"rm": "15gb",
"rp": "57",
"cpu": "2",
"dup": "0.57",
"name": "instance-0000000121",
"role": "mr",
"master": "-"
},
{
"dt": "960gb",
"du": "5.4gb",
"hc": "8gb",
"hm": "16gb",
"hp": "50",
"ip": "172.27.200.204",
"rc": "17.4gb",
"rm": "32gb",
"rp": "55",
"cpu": "1",
"dup": "0.57",
"name": "instance-0000000100",
"role": "mr",
"master": "-"
},
{
"dt": "204gb",
"du": "2.8gb",
"hc": "7.9gb",
"hm": "8.5gb",
"hp": "93",
"ip": "172.27.201.141",
"rc": "9.7gb",
"rm": "17gb",
"rp": "57",
"cpu": "33",
"dup": "1.42",
"name": "instance-0000000095",
"role": "mr",
"master": "*"
},
{
"dt": "960gb",
"du": "5.4gb",
"hc": "9.6gb",
"hm": "16gb",
"hp": "60",
"ip": "172.27.204.209",
"rc": "17.3gb",
"rm": "32gb",
"rp": "54",
"cpu": "1",
"dup": "0.57",
"name": "instance-0000000102",
"role": "mr",
"master": "-"
},
{
"dt": "10.7tb",
"du": "1.2tb",
"hc": "16.2gb",
"hm": "30gb",
"hp": "54",
"ip": "172.27.208.251",
"rc": "53.1gb",
"rm": "60gb",
"rp": "89",
"cpu": "7",
"dup": "11.65",
"name": "instance-0000000060",
"role": "rw",
"master": "-"
},
{
"dt": "10.7tb",
"du": "1.2tb",
"hc": "10.5gb",
"hm": "30gb",
"hp": "35",
"ip": "172.27.201.96",
"rc": "53.1gb",
"rm": "60gb",
"rp": "89",
"cpu": "17",
"dup": "11.65",
"name": "instance-0000000059",
"role": "rw",
"master": "-"
}
]
},
"status": "Success",
"message": "",
"operation": null
}
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.
Copyright 2024 Fortinet, Inc. All Rights Reserved.