FortiGuard ABP: Not all Bots are Created Equal

[ggenard]

The World and the Internet 

As of October 2023, the world has surpassed a growing internet user of over 5.3 Billion internet users worldwide. These growing users account for 66 percent of the global population. Internet has become a multifaceted mass medium of information that the world has been relying on. Now that the internet has become the primary medium of communication of the world, the internet is full of bots categories as good bots and bad bots. In 2022, 47.4% of all internet traffic came from bots, a 5.1% increase over the previous year with evolving factors of bad bots. In 2018, the increasing trend of bad bots continues to 30.2 percent over the past four years-contributed by malicious automated software applications capable of high-speed abuse, misuse and attacks. Here are some of 2022 notable Bots undertakings:  

• In 2022, the proportion of bad bots classified as “advanced” accounted for 51.2% of all bad bots traffic. In comparison, the level of bad bots sophistication in 2021 was 25.9%. 
• Account takeover (ATO) attacks increased 155% in 2022 and 15% of all login attempts in the past 12 months — across all industries — were classified as account takeover. 
• In 2022, 17% of all attacks on APIs came from bad bots abusing business logic. In addition, 35% of account takeover attacks in 2022 specifically targeted an API. 
• Travel (24.7%), retail (21%) and financial services (12.7%) experienced the highest volume of bots attacks. Gaming (58.7%) and telecommunications (47.7%) had the highest proportion of bad bots traffic on their websites and applications. 
• Of the 13 countries analyzed in the report, seven had bad bots traffic levels that exceeded the global average of 30.2%. Germany (68.6%), Ireland (45.1%) and Singapore (43.1%) ranked in the top three, while the U.S. also exceeded the average at 32.1%. 
• One-in-five bad bots used Mobile Safari as their browser of choice in 2022, up from 16.1% in 2021. 
  

Over the years, the perception of a bots is to automate software application that performs repetitive tasks over a network. Bots are automated software applications that can imitate human behaviors and actions online. Bots follow predefined instructions and operate without human intervention. Bots can perform repetitive, time-consuming tasks much faster and more accurately than humans, like interacting with websites, chatting with visitors, scanning content. These can increase efficiency for organizations specifically used for automated application testing. Today bots carry two faced facets as they can be utilized for good intensions and increase bad intensions is on the rise. 

The initial intention of a bots is to help humans to repetitively perform actions, perform testing repetitively.  However, overtime bots have utilized applications for the dark side as some bots can perform malicious intent to exploit vulnerabilities or overwhelm systems. These "bad bots" can scrape data, spread spam, conduct credential stuffing attacks, overwhelm a network via DDoS attack and more. Bots have become a major security threat as new and sophisticated bots are evolving. 

Good Bots vs Bad Bots  

Judging good and bad bots is not easy! There is no simple way to see the result of a bots until bots’ execution is completed. In a bot’s automation, expect each command line to initiate specific functionalities and process. They inspect variable values and program state as code executes. Every value captured from a bot’s query is materialized and every piece of information is saved for future correlations. Bots can rapidly gather, clean, transform and load data from websites and portals - easing data science and analytics pipelines.  

Bots’ creation has not made it easier.  The development of new bots involves thoughtful process analysis, coding skills and change management when transitioning from manual workflows. These automated are developed with natural language processing, conditional rules, loops, integrations with common web language or external APIs. From the specifics above, bots can be developed to be a bad and good bad. Bots can easily develop within Bots Framework, Dialogflow, AWS Lex, IBM Watson, Pandorabots and these platforms provide templates.  

Although organizations implement bots’ management protection measures such as captchas, rate limiting and more, the misuse of the bots is not easy to predict. Here are the most common techniques to deter from Bots:  

• CAPTCHAs distinguish human vs bots users by testing ability to interpret images/text that would be difficult for bots. However, advancing computer vision approaches are making some CAPTCHAs ineffective. 
• Rate limiting - Throttling levels of requests/traffic from specific IPs or users to protect against resource overload. Prevents denial of service from excessive bots queries. 
• IP reputation databases - Maintaining blacklists of IP addresses known to be associated with malicious bots and blocking them preemptively. Requires constant updates. 
• User behavior analysis - Analyzing usage patterns to detect bots exhibiting inhuman behavior timing, mouse movements etc. However, mimicking human patterns is improving. 
• Browser fingerprinting - Evaluating subtle browser and machine characteristics to identify real human users. Privacy concerns exist though.
  

FortiGuard ABP-Advanced Bots Protection 

As new bots are developed to more advanced and sophisticated bots , bots mitigation systems also evolve to detect and thwart them. Discerning good vs bad bots activity is a key priority as Fortinet Advanced Bots Protection can now utilize. 

Understanding good and bad bots is critical. Good bots should not be denied as good bots are essential in the network landscape. However, the advancement of bot’s development has extended its reach for the bad guys. With Fortinet upcoming FortiGuard Advanced Bots Protection solution incorporates behavioral analysis and deep learning​ feature to protect businesses from financial losses, reputational damage, and regulatory penalties. The FortiABP solution accounts for historical and current transactions of every transaction. Features offered are based on biometric-based detection such mouse movements, keyboard execution and monitor client events. Browser Fingerprinting detection has been an integral attack points and risks that malicious attackers could potentially exploit. With the FortiGuard ABP, browser fingerprint replication is an advanced bots’s attack point that could replicate human browser characteristics to generate fake fingerprints that circumvent fingerprinting checks. This common attack point can now be protected by the FortiGuard ABP solution. The initial release of the FortiGuard ABP will support integration with FortiADC and FortiWEB as a reverse proxy appliance. Transactions will be prepending a JavaScript telemetry script to the HTTP/HTTPS responses for all client requests to collect telemetry data. With client and the FortiADC/Web (via fabric connector) communicate with the Advanced Bots Protection Cloud for data telemetry info (headers, device fingerprinting, and more). With machine learning (ML) the ABP will inspect the request to see if the client is human or a bot. Based on result analytics, the client request can be justified using the FortiADC/Web as a decision stage to either block, allow or require a Captcha challenge. FortiGuard ABP is scheduled to be released in the early Q1, 2024.  

References:  

https://www.statista.com/statistics/209101/age-distribution-internet-users-worldwide-by-region/ 

https://www.securitymagazine.com/articles/99339-47-of-all-internet-traffic-came-from-bots-in-2022