Technical Tip: Machine Learning - HMM Learning Stage status 'Unconfirmed' and 'Confirmed' for a parameter
Description
This article describes about the new HMM Learning Stage 'Unconfirmed' and 'Confirmed' status introduced in 6.4 version, and also mentions how to overcome the situation where the parameter gets stuck at 'Unconfirmed' and discarded by the garbage collector eventually.
Scope
For version 6.4.
Solution
Explanation:
When ML anomaly detection learns a new parameter in 6.4 version, it sets the status of that new parameter to 'Unconfirmed' initially.
The status will be changed to 'Confirmed' and then 'Collecting' when ML receives HTTP requests containing the parameter/argument from a minimum number of 3 (ip-expire-cnts) different source IP addresses within the given time period of 4 hours (ip-expire-intval).
Failing to receive HTTP requests containing the learned parameter, in this example 'email', from 3 different IP addresses within 4 hours , the status of the parameter will remain 'Unconfirmed' and discarded by the garbage collector.
An event log gets generated 'Parameter deleted due to being unconfirmed for too long for parameter <submit> of URL </>. Parameter model changed from Running to Discarded by FortiWeb daemon'.
An event log gets generated 'Parameter deleted due to being unconfirmed for too long for parameter <submit> of URL </>. Parameter model changed from Running to Discarded by FortiWeb daemon'.
Problematic case scenario and solution.
If the FortiWeb is placed in an isolated environment where it is learning HTTP requests meant for the web service protected via the server policy (where Machine Learning Anomaly Detection is turned ON) less than 3 different source IP addresses, then it is necessary to decrease the 'ip-expire-cnts' to a lower value, such as '1'.
Denzil-ML-64 # config waf machine-learning-policy
Denzil-ML-64 (machine-learni~g) edit 1
Denzil-ML-64 (1) set ip-expire-cnts 1
Denzil-ML-64 (1) end
Denzil-ML-64 (machine-learni~g) edit 1
Denzil-ML-64 (1) set ip-expire-cnts 1
Denzil-ML-64 (1) end
The status will be changed to 'Confirmed' and then to 'Collecting' immediately as soon as ML sees a HTTP request with that parameter.
Related document:
Machine learning troubleshooting - FortiWeb 8.0.0 administration guide