Technical Note: FortiPortal troubleshooting commands

[glebras_FTNT]

Description

This article describes commands that can be used to troubleshoot issues with FortiPortal.

Scope

FortiPortal v4.0

Solution

Web server

FortiPortal uses Apache Tomcat in backend as a Java Servlet Container.

The console output of Apache Tomcat is redirected to a file name catalina.out that can be downloaded on the FortiPortal:

# exec shell
# more /var/tomcat/logs/catalina.out

This file includes:

• Uncaught exceptions printed by java.lang.ThreadGroup.uncaughtException(..)
• Thread dumps

19-Jul-2017 00:00:19.170 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-8443"]
19-Jul-2017 00:00:19.173 INFO [main] org.apache.tomcat.util.net.NioSelectorPool.getSharedSelector Using a shared selector for servlet write/read
19-Jul-2017 00:00:19.182 INFO [main] org.apache.catalina.startup.Catalina.load Initialization processed in 1564 ms
19-Jul-2017 00:00:19.228 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service Catalina
19-Jul-2017 00:00:19.229 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet Engine: Apache Tomcat/8.0.5
19-Jul-2017 00:00:19.296 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive /var/tomcat/webapps/fpc.war
0    [localhost-startStop-1] INFO  org.springframework.web.context.ContextLoader  ---> Root WebApplicationContext: initialization started
139  [localhost-startStop-1] INFO  org.springframework.web.context.support.XmlWebApplicationContext  ---> Refreshing Root WebApplicationContext: startup date [Wed Jul 19 00:00:34 PDT 2017]; root of context hierarchy
178  [localhost-startStop-1] INFO  org.springframework.beans.factory.xml.XmlBeanDefinitionReader  ---> Loading XML bean definitions from class path resource [spring/applicationContext.xml]
2626 [localhost-startStop-1] INFO  org.springframework.beans.factory.xml.XmlBeanDefinitionReader  ---> Loading XML bean definitions from class path resource [spring/applicationSecurity.xml]

Database

Database information for the FortiPortal DB can be found in the database.properties file.

# exec shell
# more /var/tomcat/util/fpc/database.properties

jdbc.driver=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://10.5.57.58:3306
jdbc.user=root
jdbc.password=admin
jdbc.dialect=org.hibernate.dialect.MySQLDialect
fazlite.db=fp_fazlite
collectorStoragePercentageValue = 80
# for Fpc-Wireless projects(db for logs storage)#
jdbc.fpcw.logs.driver=com.mysql.jdbc.Driver
jdbc.fpcw.logs.url=jdbc:mysql://localhost:3306
jdbc.fpcw.logs.user=root
jdbc.fpcw.logs.password=password

The following diagnose commands can be used to get information about the SQL status.

# show system sql
# diagnose sql process list  Id   | User |       Host        |          db          | Command | Time | State |       Info   
    -------+------+-------------------+----------------------+---------+------+-------+-------------------
 214   | root | 10.5.57.70:40822  | fp_fazlite           | Sleep   | 66   |       |                  
 221   | root | 10.5.57.70:40829  | fp_fazlite           | Query   | 0    |       | show  processlist
 382   | root | 10.5.57.70:40998  |                      | Sleep   | 20   |       |                  
 384   | root | 10.5.57.70:41000  |                      | Sleep   | 20   |       |                  
 385   | root | 10.5.57.70:41001  |                      | Sleep   | 20   |       |                  
 386   | root | 10.5.57.70:41002  | ftntpmcdb            | Sleep   | 20   |       |                  
 388   | root | 10.5.57.70:41004  | ftntpmcdb            | Sleep   | 20   |       |                  
 389   | root | 10.5.57.70:41005  | ftntpmcdb_customer_2 | Sleep   | 10   |       |                  
 390   | root | 10.5.57.70:41006  | ftntpmcdb            | Sleep   | 20   |       |                  
 392   | root | 10.5.57.70:41008  | ftntpmcdb            | Sleep   | 20   |       |        
 393   | root | 10.5.57.70:41009  | ftntpmcdb_customer_2 | Sleep   | 0    |       |    
 394   | root | 10.5.57.70:41010  | ftntpmcdb            | Sleep   | 0    |       |    
 1813  | root | 10.5.59.177:57259 | ftntpmcdb            | Sleep   | 16   |       |   
 1819  | root | 10.5.59.177:57266 | fp_fazlite           | Sleep   | 7    |       |   
 1844  | root | 10.5.59.177:57303 | fp_fazlite_2         | Sleep   | 85   |       |    
 43206 | root | 10.5.59.177:51363 | fp_fazlite_2         | Sleep   | 227  |       |   
 43217 | root | 10.5.59.177:51374 | fp_fazlite_2         | Sleep   | 219  |       |   
 43287 | root | 10.5.57.70:58153  | fp_fazlite           | Sleep   | 103  |       |   
 43295 | root | 10.5.57.70:58155  | fp_fazlite           | Sleep   | 84   |       |

# diagnose test application sqllogd 5
    Device           WorkerID  sID  LogFiles DoneFiles  LogRecv     RecvPriv    LastScan    DeviceIP
-------------------------------------------------------------------------------------------------
FPC-VM0000000001          1     0        1       0    1500560912   1500560912   1500560915

Total Devices: 1
Worker 1 handles 1 devices

The following diagnose commands can be used to get information about the SQL log reception on the Collector.

CollectorVM # diagnose fortilogd status
No fortilogd status found.

CollectorVM # diagnose fortilogd msgstat

CollectorVM # diagnose fortilogd msgrate
msgs/sec: 0.0, msgs/30sec: 0.0, msgs/60sec: 0.0

CollectorVM # diagnose fortilogd msgrate-total

Log messages per second
Totals                    Last Hour       Day      Week
-------------------------------------------------------
                        :      0.00      7.32      1.05

CollectorVM # diagnose fortilogd msgrate-device

Log messages per secondTotals                    Last Hour       Day      Week
-------------------------------------------------------
        FGT1KD0000000002:      0.00      7.32      1.05

CollectorVM # diagnose fortilogd msgrate-type

Log messages per second
Totals                    Last Hour       Day      Week
-------------------------------------------------------
    Application Control.:      0.00      2.01      0.29
                 Attack.:      0.00      0.00      0.00
            DLP Archive.:      0.00      0.00      0.00
   Data Leak Prevention.:      0.00      0.00      0.00
           Email Filter.:      0.00      0.00      0.00
                  Event.:      0.00      0.00      0.00
                Generic.:      0.00      0.00      0.00
        Mail Statistics.:      0.00      0.00      0.00
      Instant Messaging.:      0.00      0.00      0.00
       Network Analyzer.:      0.00      0.00      0.00
                Traffic.:      0.00      4.46      0.64
                  Virus.:      0.00      0.00      0.00
                   VoIP.:      0.00      0.00      0.00
             Web Filter.:      0.00      0.85      0.12
           Network Scan.:      0.00      0.00      0.00
                  Event.:      0.00      0.00      0.00
                Traffic.:      0.00      0.00      0.00
           Network Scan.:      0.00      0.00      0.00

CollectorVM # diagnose fortilogd lograte
logs/sec: 0.0, logs/30sec: 0.0, logs/60sec: 0.0

Logs

Raw logs can be found in the /Storage/Log//tlog.log file in the Collector.

# /Storage/Logs/FGT1KD0000000002/tlog.log