Technical Tip: How to block Geo-IP from The United States but allow the search engine crawlers to crawl the website

[gsharma]

Description	This article describes how to Block any IP address from The United States but allow only search engine crawlers from that country to crawl the website.
Scope	FortiAppSec Cloud.
Solution	To achieve this scenario, there are 2 steps to perform: Navigate to Access rules -> IP Protection -> Geo-IP block, and add the United States or any other countries from where the traffic needs to be blocked. This will block all the traffic originating from the United States. The Geo-IP is blocking the US, which could inadvertently disable access to the content for search engine crawlers to the website as well. In this case, get the list of IPs from the official site of Google/Bing( search engine) crawlers. Navigate to Access rules -> IP Protection -> IP list and import that list as CSV or add the IP range too. Select the action/type as Trust IP and click Save. This will allow the search engine crawler IPs from the United States to work while blocking other traffic. Related document:  IP Protection. Note: If the above suggested config does not work as expected, contact support