Technical Tip: Unexpected probe failure / fine tuning probe parameters
Description
This article shows Unexpected probe failure / fine-tuning probe parameters.
Server up-down log messages are received:
Dec 12 11:35:08 Z6JLB001HY acvd[8749]: |w|P HY-Intra-inhit_small2;i HY_SORRY;|02000000: Server Instance server1 in Pool HY-Intra-inhit_small2 has gone TCP ACV DOWN||
Dec 12 11:35:08 Z6JLB001HY acvd[484]: |a|P HY-Intra-inhit_small2;i HY_SORRY;|02000187: State Change: (Server ACV Check) Server Instance server1 in Pool HY-Intra-inhit_small2 has gone TCP ACV DOWN. ||
Dec 12 11:35:09 Z6JLB001HY acvd[8749]: |w|P HY-Intra-inhit_small2;i HY_SORRY;|02000000: Server Instance server1 in Pool HY-Intra-inhit_small2 has gone TCP ACV UP||
Dec 12 11:35:09 Z6JLB001HY acvd[484]: |a|P HY-Intra-inhit_small2;i HY_SORRY;|02000187: State Change: (Server ACV Check) Server Instance server1 in Pool HY-Intra-inhit_small2 has gone TCP ACV UP. ||
Scope
FortiADC
Solution
A server must respond to both TCP and ACV probes before the probe timeout elapses.
In a network configuration where
1) There is high latency between server probes and responses.
or
2) There are many CRC errors.
The probe mechanism may falsely report that a server is down. This is indicated by messages in the event log indicating that a server is down and then comes back up again after a short period of time.
We can fine-tune the ACV health check timer to avoid unnecessary probe failures.
For example, if the probe parameters are set to interval 60, probe_maxtries are set at 2.
- So if a single probe fails and the timing is such that the next probe occurs slightly late, 60 seconds may go by without a good probe, and the SI will be marked down.
- If the next probe succeeds, the server will immediately be marked up.
To reduce the sensitivity of the probes, increase probe_maxtries.
For more information on fine-tuning probes, see the section entitled 'Configuring UDP and TCP Parameters' in the FortiADC Admin Guide.