Skip to main content
gsharma
Staff
Staff
April 9, 2025

Technical Tip: How to block Geo-IP from The United States but allow the search engine crawlers to crawl the website

  • April 9, 2025
  • 0 replies
  • 149 views
Description This article describes how to Block any IP address from The United States but allow only search engine crawlers from that country to crawl the website.
Scope FortiAppSec Cloud.
Solution

To achieve this scenario, there are 2 steps to perform:

  1. Navigate to Access rules -> IP Protection -> Geo-IP block, and add the United States or any other countries from where the traffic needs to be blocked.

 

blockUS.jpg

 

This will block all the traffic originating from the United States.

 

  1. The Geo-IP is blocking the US, which could inadvertently disable access to the content for search engine crawlers to the website as well.

 

In this case, get the list of IPs from the official site of Google/Bing( search engine) crawlers. Navigate to Access rules -> IP Protection -> IP list and import that list as CSV or add the IP range too.

 

Select the action/type as Trust IP and click Save.

 

iplisttrust.jpg

 

This will allow the search engine crawler IPs from the United States to work while blocking other traffic.

 

Related document

IP Protection.

 

Note:

If the above suggested config does not work as expected, contact support