Technical Tip: How to troubleshoot and fix the possible issues when setting up FortiADC Layer-7 HTTPS Server Load Balance Virtual Server with ADFS as the target real server pool

[kmak]

Description	This article describes the possible issues when setting up FortiADC Layer-7 HTTPS Server Load Balance Virtual Server with ADFS as the target real server pool.
Scope	FortiADC.
Solution	Pre-requisite: Layer-7 HTTPS SLB Virtual Server with ADFS as the target real server pool. A standard Layer-7 HTTPS Application profile and client-SSL profile are configured for the proxy frontend. A valid SSL certificate is installed for the ADFS FQDN hostname. Troubleshoot Error 1: The Layer-7 HTTPS SLB Virtual Server of the ADFS page is displaying the 503 Error return code. The error page shown originated from the real server instead of FortiADC. Enable traffic-log setting for the Layer-7 Virtual Server. Check the L7-HTTP log in FortiADC to verify the error messages. Solution: The ADFS page uses HTTPS protocol, which the error page is due to the real server pool configured with Port 80 (HTTP Protocol). Troubleshoot Error 2: The ADFS page shows the error 'ERR_EMPTY_RESPONSE'. The traffic log shows the 'unknown' return code and zero ‘0’ sent Bytes. The error is due to the ADFS real server pool configuration using port ‘443’, but no real server SSL profile is enabled. Solution: Select a real server SSL profile for the real server pool. Troubleshoot Error 3: The ADFS real server pool is configured to port ‘443’ with a real server SSL profile, but the FortiADC SLB Virtual Server frontend is still displaying a 503 error return code. The traffic log shows that the HTTPS request translated correctly to the ADFS real server port 443. There are sent Bytes and received Bytes recorded in the FortiADC. Taking the packet capture in FortiADC to capture the packet transmission between FortiADC and the ADFS server. It shows that the ADFS server sent an RST (RST, ACK) packet after “CLIENT HELLO” from FortiADC. Solution: The ADFS requires the SNI header to be presented in the SSL handshake. Enable the 'SNI Forward Flag' option in the real server SSL profile. Run the packet capture again, and the request should contain the SNI header now. And the ADFS real server no longer sends RST packets, but it is replying to SSL/TLS handshake packets. The ADFS page is showing fine now, proxy through FortiADC SLB Virtual Server. Related document: Configuring real server SSL profiles