Technical Tip: Possible reason for a virtual server health check with a content check not working

[kcheng]

Description	This article describes the possible reason for a virtual server health check with a content check not working.  Setup: FortiGate is configured to monitor HTTP content with the word 'master' as below:   FortiGate is configured with a virtual server to forward traffic only to the server that responded with the respective content: Troubleshooting Process: Accessibility to the server has no issue, however, it was observed that the health check failed despite manually accessing the respective URL displays the result correctly: On FortiGate, turn on debug for ldb_monitor daemon to check on the debug output: diag deb app ipldpd -1 diag deb en To further clarify if FortiGate is receiving the packet correctly, it would be necessary to run packet capture to check on the network level: diag sniffer packet any 'host <server_under_monitor> and port <port_under_monitor>" 6 0 a In this scenario, the packet is reaching the FortiGate correctly and the content is visible. However, the health check still failed:
Scope	FortiGate.
Solution	The root cause behind this is that the ldb_monitor feature in FortiGate would rely on the HTTP header 'Content-Length' to determine if there is any presence of the response body. In the above scenario, an HTTP response is observed, however, the response does not contain the 'Content-Length' header and hence, FortiGate does not proceed to check the response body and determine that there is no response body to be checked: -------------------------HTTP Response--------------------------------- HTTP/1.0 200 OK\r\n Server: BaseHTTP/0.6 Python/3.10.12\r\n Date: Mon, 16 Oct 2023 06:54:46 GMT\r\n Content-Type: application/json\r\n \r\n [HTTP response 1/1] [Time since request: 0.000000000 seconds] [Request in frame: 15] [Request URI: http://192.168.40.230:8008/] File Data: 638 bytes -------------------------HTTP Response--------------------------------- Following is an example of packet capture when the health check works with a content check is configured to check the presence of the word 'John': From the packet capture, it has clearly been observed that the 'Content-Length' header is present and hence, ldb_monitor on the FortiGate would proceed to further inspect the response body: -------------------------HTTP Response--------------------------------- Hypertext Transfer Protocol HTTP/1.1 200 OK\r\n X-Powered-By: Express\r\n Vary: Origin, Accept-Encoding\r\n Access-Control-Allow-Credentials: true\r\n Cache-Control: no-cache\r\n Pragma: no-cache\r\n Expires: -1\r\n X-Content-Type-Options: nosniff\r\n Content-Type: application/json; charset=utf-8\r\n Content-Length: 108\r\n ETag: W/"6c-Hh9kOCN9vWZCdbKNu8q2aqEHGRQ"\r\n Date: Fri, 06 Oct 2023 08:59:42 GMT\r\n Connection: keep-alive\r\n \r\n [HTTP response 1/1] [Time since request: 0.003563000 seconds] [Request in frame: 7] [Request URI: http://10.251.2.237:3000/users] File Data: 108 bytes -------------------------HTTP Response--------------------------------- The solution to this is to modify the HTTP response to configure the webserver to respond with 'Content-Length' so that FortiGate is aware of the response body and proceeds to check the content of the response body.