Technical Guide: Selective control & monitoring of GenAI / LLM application traffic with Fortinet Security Fabric

Rudresh_Veerappaji · ‎12-19-2025

Description

This article describes how to manage the control of GenAI/LLM application traffic egressing the network, originating from app users, AI agents, and MCP clients in internal networks. Unrestricted access to AI/LLM servers could expose certain risks, and IT administrators would want to manage access to such applications to meet the security control requirements. There are multiple levels of access control for such application traffic that can be implemented, along with detailed monitoring of the traffic using FortiGate and other Security Fabric components. These are discussed in detail in the article.

Scope FortiGate, Security Fabric, DLP sensor, Application signature, Application control.

Solution

Table of Contents:

Introduction
Use case 1: Block all AI/LLM traffic egressing the network - using Fortiguard category based web fil...
Use case 2: Allow only specific AI/LLM traffic - using static URL filter & FortiGuard category web-f...
Use case 3: Block specific AI/LLM traffic - using static URL filter and Fortiguard category based we...
Use case 4: Force authenticate users accessing AI/LLM applications - using firewall policies
Use case 5: Detect and block sensitive keywords in the LLM chat / conversations - using DLP on Forti...
Use case 6: Allow sanitized (internally tagged) AI traffic - using HTTP header tags and custom appli...
Use case 7: MCP traffic filtering - using FortiGate SSL deep inspection
Use case 8: Monitoring AI/LLM application traffic in the network - using FortiGate or FortiAnalyzer ...
Related documents

Introduction:

With an increase in use of Artificial Intelligence-related applications by various types of users within campus networks like LLM apps, AI agents, MCP clients, etc, IT teams would be expected to ensure the implementation of security control policies to selectively filter and monitor this new type of application traffic.

In this article, some of the most common use cases to implement such security controls are discussed and illustrated with examples, starting with something as simple as detecting and blocking all AI/LLM traffic, to more complex use cases of detecting sensitive keywords used in LLM prompts and using custom HTTP header tags to allow internally sanitized and approved AI/LLM traffic. A combination of these security controls can be used to achieve additional use cases. Client applications accessing these AL/LLM traffic could come from corporate workstations using LLM apps (maybe shadow IT), AI agents hosted on the internal networks, VPN clients (FortiClient), LLM apps on BYOD devices, etc.

Fortinet-Security-Fabric-AI-LLM-applications-traffic-management-v2-Rudresh-Veerappaji.png

Use case 1: Block all AI/LLM traffic egressing the network - using FortiGuard category-based web filtering:

To start with, this is the basic use case, wherein the security control requirement is to identify and block all the known AI/LLM category of websites/applications, whether originating from a particular segment of the internal network or from all internal network. Below is an illustration of how this can be done on the FortiGate, using the new 'Artificial Intelligence Technology' FortiGuard webfilter category (assigned category number is 100).

FortiGate # get webfilter categories | grep "Artificial Intelligence"
    100 Artificial Intelligence Technology
FortiGate #

Configuration examples using this new category to implement a firewall policy to block AL/LLM traffic are shown below.

FortiGate# config webfilter profile
 edit "Block-AL-LLMs-category-webfilter"
        config ftgd-wf
            unset options
            config filters
                edit 1
                    set category 1
                next
                edit 2
                    set category 3
                next
                . . .
                edit 33
                    set category 91
                    set action block
                next
                edit 34
                    set category 100 
                    set action block  <<<<<<<<<<<<< Blocking AI category
                next
            end
        end
    next
end

FortiGate# show firewall policy
config firewall policy
    edit 3
        set name "Block-All-AI-LLM-Traffic"
        set uuid fac30b06-d856-51f0-2043-2641cb12d949
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "Internal-Network"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "Block-AL-LLMs-category-webfilter"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end

When an internal user tries to access an AI/LLM site, the access is blocked with the following error message (the error message can be customized as well).

FortiGate will record this blocked event in the logs as shown below.

date=2025-12-13 time=11:14:49 eventtime=1765653288986595175 tz="-0800" logid="0316013056" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=3 poluuid="fac30b06-d856-51f0-2043-2641cb12d949" policytype="policy" sessionid=96576 srcip=192.168.20.1 srcport=55670 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="www.deepseek.com " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-AL-LLMs-category-webfilter" action="blocked" reqtype="direct" url="https://www.deepseek.com/ " sentbyte=2505 rcvdbyte=5082 direction="outgoing" msg="URL belongs to a denied category in policy" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

date=2025-12-13 time=11:13:46 eventtime=1765653227133832451 tz="-0800" logid="0316013056" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=3 poluuid="fac30b06-d856-51f0-2043-2641cb12d949" policytype="policy" sessionid=94101 srcip=192.168.20.1 srcport=35840 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="chatgpt.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-AL-LLMs-category-webfilter" action="blocked" reqtype="direct" url="https://chatgpt.com/ " sentbyte=3304 rcvdbyte=3103 direction="outgoing" msg="URL belongs to a denied category in policy" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

These logs can be used for monitoring the AI/LLM application traffic in the network using either the 'Log & Report' section of egress FortiGates, or with a FortiAnalyzer for an aggregated & comprehensive view of the application traffic across multiple campuses & remote sites of the company.

Use case 2: Allow only specific AI/LLM traffic - using static URL filter & FortiGuard category web-filter:

If the requirement is to allow a few IT-approved AI/LLM applications and block all the other AI/LLM traffic, this can be implemented with a static URL filter specifying explicitly the FQDNs of the apps that are to be allowed, and blocking the remaining AI/LLM category traffic. The static URL filter takes precedence over the Category filter on the FortiGate and thus enables this use case. Here is a configuration example of how to implement this.

FortiGate # show webfilter urlfilter 
config webfilter urlfilter
    edit 2
        set name "Auto-webfilter-urlfilter_cvwof7u3n"
        config entries
            edit 1
                set url "chatgpt.com"
            next
            edit 2
                set url "gemini.google.com"
            next
        end
    next
end
FortiGate # show webfilter profile Block-AI-LLM-Category-allow-few 
config webfilter profile
    edit "Block-AI-LLM-Category-allow-few"
        config web
            set urlfilter-table 2
        end
        config ftgd-wf
            config filters
                edit 1
                    set category 1
                next
                edit 2
                    set category 3
                next
                . . .
                edit 34
                    set category 100
                    set action block
                next
            end
        end
    next
end
FortiGate #  config firewall policy
Challenger-kvm47 (policy) # edit "1"
Challenger-kvm47 (1) # show
config firewall policy
    edit 1
        set name "Allow-specific-AI-LLM-only"
        set uuid 1fef53c4-d917-51f0-21db-6c30e158d0fb
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "Internal-Network"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "Block-AI-LLM-Category-allow-few"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate #

Static URL filter definition to allow selected URLs, and the 'Artificial Intelligence Technology' category set to Block using FortiGate GUI.

An example of an error log illustrating the configuration taking effect. Traffic to one of the AI/LLM servers is blocked, which is not defined in the static URL, while the other AI/LLM server access is allowed through 'passthrough' mode.

FortiGate# execute log display

date=2025-12-14 time=10:21:28 eventtime=1765736488021454238 tz="-0800" logid="0315012545" type="utm" subtype="webfilter" eventtype="urlfilter" level="information" vd="root" urlfilteridx=2 urlfilterlist="Auto-webfilter-urlfilter_cvwof7u3n" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1144528 srcip=192.168.20.1 srcport=60090 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="Canada" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="gemini.google.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-AI-LLM-Category-allow-few" action="passthrough" reqtype="direct" url="https://gemini.google.com/ " sentbyte=2850 rcvdbyte=6647 direction="outgoing" msg="URL was exempted because it is in the URL filter list"

date=2025-12-14 time=10:21:27 eventtime=1765736487583027473 tz="-0800" logid="0315012545" type="utm" subtype="webfilter" eventtype="urlfilter" level="information" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_cvwof7u3n" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1144511 srcip=192.168.20.1 srcport=47030 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="POST" service="HTTPS" hostname="chatgpt.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-AI-LLM-Category-allow-few" action="passthrough" reqtype="referral" url="https://chatgpt.com/ces/v1/t " referralurl="https://chatgpt.com/ " sentbyte=6789 rcvdbyte=1489 direction="outgoing" msg="URL was exempted because it is in the URL filter list"

date=2025-12-14 time=10:21:26 eventtime=1765736486296681436 tz="-0800" logid="0316013056" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1144519 srcip=192.168.20.1 srcport=56478 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="www.deepseek.com " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-AI-LLM-Category-allow-few" action="blocked" reqtype="direct" url="https://www.deepseek.com/ " sentbyte=2433 rcvdbyte=4810 direction="outgoing" msg="URL belongs to a denied category in policy" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

Note:

Alternatively, two separate firewall policies can be used for this use case - first one to allow specific LLM FQDNs as dstaddr and service 'HTTPS' or 'ALL', and the second policy with a webfilter to block FortiGuard category 'Artificial Intelligence Technology'.

Use case 3: Block specific AI/LLM traffic - using static URL filter and Fortiguard category based web-filter.

This use case is an inverse of the previous use case, wherein the requirement is to block certain AI/LLM traffic and allow the others. This can be achieved again by using the combination of static filters and FortiGuard category-based filters, as shown below. Explicitly define the URLs that are to be blocked in the static filters, and then set the policy to allow in the AI category.

FortiGate # show webfilter urlfilter 
config webfilter urlfilter
    edit 1
        set name "Auto-webfilter-urlfilter_uzyxsa8ai"
        config entries
            edit 1
                set url "chatgpt.com"
                set action block
            next
            edit 2
                set url "deepseek.com"
                set action block
            next
        end
    next
FortiGate # show webfilter profile Block-specific-LLMs-allow-remaining 
config webfilter profile
    edit "Block-specific-LLMs-allow-remaining"
        set feature-set proxy
        config web
            set urlfilter-table 1
        end
        config ftgd-wf
            set exempt-quota g21
            config filters
                edit 1
                    set category 1
                next
                edit 2
                    set category 3
                next
                . . .
                edit 35
                    set category 100
                    set action allow
                next
            end
        end
    next
end
FortiGate # config firewall policy
FortiGate (policy) # edit "1"
FortiGate # show
config firewall policy
    edit 1
        set name "Block-specific-LLMs-allow-remaining"
        set uuid 1fef53c4-d917-51f0-21db-6c30e158d0fb
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "Internal-Network"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "Block-specific-LLMs-allow-remaining"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate #

An internal user trying to access the AI/LLM server that is restricted in the firewall policy will get the following error message:

The block event would be recorded in the logs as shown below.

date=2025-12-14 time=10:41:53 eventtime=1765737712580014808 tz="-0800" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=2 urlfilterlist="Auto-webfilter-urlfilter_uzyxsa8ai" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1161805 srcip=192.168.20.1 srcport=45588 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="www.deepseek.com " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Block-specific-LLMs-allow-remaining" action="blocked" reqtype="direct" url="https://www.deepseek.com/ " sentbyte=2433 rcvdbyte=4810 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

Use case 4: Force authenticate users accessing AI/LLM applications - using firewall policies.

In this use case, IT might want to implement security control by adding explicit authentication for users accessing these applications, thereby limiting access to a subset of internal users, and expanding as needed. This can be done by adding the 'authenticate' action in the FortiGuard category-based filter for category 100, as shown in the example below.

FortiGate # show webfilter profile Auth-for-AI-LLM-traffic 
config webfilter profile
    edit "Auth-for-AI-LLM-traffic"
        set feature-set proxy
        config ftgd-wf
            config filters
                edit 1
                    set category 1
                next
                edit 2
                    set category 3
                next
                . . .
                edit 34
                    set category 100
                    set action authenticate
                    set auth-usr-grp "Internal-Users"
                next
            end
        end
    next
end

FortiGate # config firewall policy
FortiGate (policy) # edit "1"
FortiGate (1) # show
config firewall policy
    edit 1
        set name "Enforce-Auth-for-AI-LLM-traffic"
        set uuid 1fef53c4-d917-51f0-21db-6c30e158d0fb
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "Internal-Network"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "Auth-for-AI-LLM-traffic"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate #

The action is set to 'Authenticate' specifically for the AI category, and allows/blocks for other categories.

The user will see the warning below when trying to access an AI/LLM server. After selecting 'proceed', the user will see an option to provide authentication credentials to access the server.

Once authentication succeeds, the user will see access to the website as shown in the example below.

UC4-user-message-3.jpg.png

Log entries corresponding to the initial block event, and then the subsequent passthrough actions after the user authenticates, are recorded in the FortiGate as shown below:

date=2025-12-14 time=10:58:21 eventtime=1765738701673735411 tz="-0800" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1176900 srcip=192.168.20.1 srcport=36654 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="chatgpt.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Auth-for-AI-LLM-traffic" action="passthrough" reqtype="referral" url="https://chatgpt.com/ces/v1/projects/oai/settings " referralurl="https://chatgpt.com/ " sentbyte=4829 rcvdbyte=72407 direction="outgoing" msg="URL belongs to a category with warnings enabled" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

date=2025-12-14 time=10:58:20 eventtime=1765738700919081459 tz="-0800" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1176900 srcip=192.168.20.1 srcport=36654 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="chatgpt.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Auth-for-AI-LLM-traffic" action="passthrough" reqtype="referral" url="https://chatgpt.com/ " referralurl="https://chatgpt.com:8010/ " sentbyte=3466 rcvdbyte=3662 direction="outgoing" msg="URL belongs to a category with warnings enabled" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

date=2025-12-14 time=10:57:30 eventtime=1765738650911606768 tz="-0800" logid="0316013057" type="utm" subtype="webfilter" eventtype="ftgd_blk" level="warning" vd="root" policyid=1 poluuid="1fef53c4-d917-51f0-21db-6c30e158d0fb" policytype="policy" sessionid=1176900 srcip=192.168.20.1 srcport=36654 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="d79843d6-d855-51f0-4627-b903d4525d2b" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="GET" service="HTTPS" hostname="chatgpt.com" agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101" profile="Auth-for-AI-LLM-traffic" action="blocked" reqtype="direct" url="https://chatgpt.com/ " sentbyte=3336 rcvdbyte=3662 direction="outgoing" msg="URL belongs to a category with warnings enabled" ratemethod="domain" cat=100 catdesc="Artificial Intelligence Technology"

Use case 5: Detect and block sensitive keywords in the LLM chat/conversations - using DLP on FortiGate.

If the requirement is to detect certain sensitive keywords and block those keywords, prompts containing those keywords from reaching the AI/LLM applications, then this can be done using DLP on FortiGate. This could be used as one of the methods to do Input Validation against prompt injection attacks, wherein FortiGate DLP can look for suspicious prompt keywords like 'ignore previous instructions', PII, 'API key', standard system prompts, etc.

To get started, first define the list of sensitive keywords in the DLP dictionary, like as example source code, secret, confidential, or any internal project names, etc. There are several other data types that are supported in the FortiGate DLP dictionary, as shown in the config example below.

FortiGate # config dlp dictionary 
FortiGate  (dictionary) # edit llm-filter 
FortiGate  (llm-filter) # config entries 
FortiGate  (entries) # edit 4
FortiGate  (4) # set type ?
<string>    please input string value
"credit-card"     data-type
"edm-keyword"     data-type
"hex"             data-type
"keyword"         data-type
"regex"           data-type
"ssn-us"          data-type
FortiGate  (4) #

In this example, the concerned sensitive keywords are listed in the DLP dictionary that the DLP sensor will scan for. Next, define a DLP profile with the policy to block the messages containing the sensitive keywords, and finally map the profile to the firewall policy for the AI/LLM applications of interest.

FortiGate # show dlp dictionary
config dlp dictionary
    edit "llm-filter"
        set uuid 3fd30af4-d84b-51f0-4e45-8a62eec945b8
        config entries
            edit 1
                set type "keyword"
                set pattern "source code"
                set ignore-case enable
            next
            edit 2
                set type "keyword"
                set pattern "password"
                set ignore-case enable
            next
            edit 3
                set type "keyword"
                set pattern "API key"
                set ignore-case enable
            next
            edit 4
                set type "keyword"
                set pattern "ignore previous instructions"
                set ignore-case enable
            next
        end
    next
end
FortiGate # show dlp sensor 
config dlp sensor
    edit "llp-filter-sensor"
        config entries
            edit 1
                set dictionary "llm-filter"
            next
        end
    next
end

FortiGate # show dlp profile 
config dlp profile
    edit "default"
        set comment "Default profile."
    next
    edit "sniffer-profile"
        set comment "Log a summary of email and web traffic."
        set summary-proto smtp pop3 imap http-get http-post
    next
    edit "llp-filter-profile"
        set feature-set proxy
        config rule
            edit 1
                set name "llm-filter-1"
                set severity critical
                set proto http-post
                set filter-by sensor
                set sensor "llp-filter-sensor"
                set action block
            next
            edit 2
                set name "llm-filter-2"
                set type message
                set proto http-post
                set filter-by sensor
                set sensor "llp-filter-sensor"
                set action block
            next
        end
    next
end

FortiGate # show firewall address
config firewall address
    edit "1"
        set uuid 4aeb4c16-d84c-51f0-c35c-342d8df71a39
        set type fqdn
        set fqdn "chatgpt.com"
    next
    edit "2"
        set uuid bb47a146-d87c-51f0-c9f6-defff1099e9d
        set type fqdn
        set fqdn "deepseek.com"
    next
    edit "3"
        set uuid 57339582-d84c-51f0-b00d-e7662623e740
        set type fqdn
        set fqdn "gemini.google.com"
    next
end
FortiGate #  show firewall addrgrp
config firewall addrgrp
    edit "llmapps"
        set uuid 77a057f6-d84c-51f0-5cec-6d8bef41e5f5
        set member "1" "2" "3"
    next
end

FortiGate # show firewall policy 1
config firewall policy
    edit 1
        set name "llm-filter-policy"
        set uuid 88f93fc2-d84c-51f0-34ae-03c3446f3e17
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "all"
        set dstaddr "llmapps"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set dlp-profile "llp-filter-profile"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate #

During the user's conversation with an AI/LLM application, if the user inputs a prompt containing sensitive keywords, that message is blocked by the FortiGate, and an error similar to the following is shown to the user.

A log entry corresponding to the DLP event is also recorded in the FortiGate, as shown in the examples below.

date=2025-12-13 time=10:54:04 eventtime=1765652044255377374 tz="-0800" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llm-filter-1" dlpextra="Sensor 'llp-filter-sensor' matching any: ('llm-filter'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="88f93fc2-d84c-51f0-34ae-03c3446f3e17" policytype="policy" sessionid=76006 epoch=842289803 eventid=1 srcip=192.168.20.1 srcport=42242 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="77a057f6-d84c-51f0-5cec-6d8bef41e5f5" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="gemini.google.com" url="https://gemini.google.com/_/BardChatUi/data/assistant.lamda.BardFrontendService/StreamGenerate?bl=boq_assistant-bard-web-server_20251210.04_p2&f.sid=2794338400047601719&hl=en-US&_reqid=1968014&rt=c " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101 Firefox/143.0" httpmethod="POST" referralurl="https://gemini.google.com/ " filename="StreamGenerate" filesize=997 profile="llp-filter-profile"

date=2025-12-13 time=10:51:58 eventtime=1765651918614320058 tz="-0800" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llm-filter-1" dlpextra="Sensor 'llp-filter-sensor' matching any: ('llm-filter'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="88f93fc2-d84c-51f0-34ae-03c3446f3e17" policytype="policy" sessionid=73585 epoch=842289680 eventid=1 srcip=192.168.20.1 srcport=38998 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="77a057f6-d84c-51f0-5cec-6d8bef41e5f5" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="chatgpt.com" url="https://chatgpt.com/backend-anon/f/conversation " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101 Firefox/143.0" httpmethod="POST" referralurl="https://chatgpt.com/ " filename="conversation" filesize=878 profile="llp-filter-profile"

date=2025-12-13 time=10:51:58 eventtime=1765651918473779425 tz="-0800" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llm-filter-1" dlpextra="Sensor 'llp-filter-sensor' matching any: ('llm-filter'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="88f93fc2-d84c-51f0-34ae-03c3446f3e17" policytype="policy" sessionid=73585 epoch=842289679 eventid=1 srcip=192.168.20.1 srcport=38998 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="77a057f6-d84c-51f0-5cec-6d8bef41e5f5" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="chatgpt.com" url="https://chatgpt.com/backend-anon/f/conversation/prepare " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101 Firefox/143.0" httpmethod="POST" referralurl="https://chatgpt.com/ " filename="prepare" filesize=534 profile="llp-filter-profile"

date=2025-12-13 time=10:51:51 eventtime=1765651910724642920 tz="-0800" logid="0954024576" type="utm" subtype="dlp" eventtype="dlp" level="warning" vd="root" ruleid=1 rulename="llm-filter-1" dlpextra="Sensor 'llp-filter-sensor' matching any: ('llm-filter'=1) >= 1; match." filtertype="sensor" filtercat="file" severity="critical" policyid=1 poluuid="88f93fc2-d84c-51f0-34ae-03c3446f3e17" policytype="policy" sessionid=73585 epoch=842289654 eventid=1 srcip=192.168.20.1 srcport=38998 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" dstip=A.B.C.D dstport=443 dstcountry="United States" dstintf="port1" dstintfrole="undefined" dstuuid="77a057f6-d84c-51f0-5cec-6d8bef41e5f5" proto=6 service="HTTPS" filetype="unknown" direction="outgoing" action="block" hostname="chatgpt.com" url="https://chatgpt.com/backend-anon/f/conversation " agent="Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:143.0) Gecko/20100101 Firefox/143.0" httpmethod="POST" referralurl="https://chatgpt.com/ " filename="conversation" filesize=842 profile="llp-filter-profile"

Use case 6: Allow internally sanitized (tagged) AI traffic - Using HTTP header tags and a custom application signature.

Another way to control AI/LLM traffic egressing the network is by 'tagging' approved AI traffic internally, and then allowing traffic to egress the network to the internet only if it has the expected tag at the perimeter Firewall. The HTTP header of the packet from the client machine can be added with a custom tag internally in the path, and as it egresses the network, FortiGate can parse the HTTP headers of the application traffic looking for the custom header tag and allow the traffic only if it is present.

In this example, a custom HTTP header tag called 'X-custom-tag: example12345' is used to illustrate the use case. An application signature can be defined on the FortiGate to match this custom tag, and FortiGate would allow AI/LLM traffic only if it contains these tags, and block all other AI/LLM traffic.

FortiGate # config application custom
FortiGate (custom) # edit "LLM-tag"
FortiGate (LLM-tag) # show
config application custom
    edit "LLM-tag"
        set comment "test"
        set signature "F-SBID( --attack_id 2459; --name \"LLM.Approved\"; --service SSL; --flow from_client; --context header; --pattern \"X-custom-tag: example12345\"; --app_cat 36; )"
        set category 36
    next
end

FortiGate (LLM-tag) # 
FortiGate # config application list
FortiGate (list) # edit "AI-LLM-custom-tag-check"
FortiGate (AI-LLM-custom-tag-check) # show
config application list
    edit "AI-LLM-custom-tag-check"
        set other-application-log enable
        set enforce-default-app-port enable
        config entries
            edit 1
                set application 2459
                set action pass
                set log disable
            next
            edit 2
                set category 2 6 36
            next
        end
        set control-default-network-services enable
    next
end

FortiGate (AI-LLM-custom-tag-check) # 
FortiGate  # config firewall policy
FortiGate (policy) # edit "1"
FortiGate (1) # show
config firewall policy
    edit 1
        set name "Allow-LLM-traffic-with-customHeader"
        set uuid 16d552c2-d9fb-51f0-394b-e6e1754ac1e0
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "Internal-Network"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "default"
        set ips-sensor "default"
        set application-list "AI-LLM-custom-tag-check"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate  (1) #

Testing from a client machine using a curl to an LLM server as shown below, with a custom header tag added in the curl request. The IPS daemon in FortiOS when parsing the header in the packet will check for this custom header tag, and the request will be allowed if it can find this tag. An example debug analysis of this flow is discussed in this article: FortiGate custom application signature troubleshooting with an LLM application example

Linux:~$ curl -k -H "X-Approved-tag: example12345" https://chatgpt.com

Use case 7: MCP traffic filtering - using FortiGate SSL deep inspection.

Model Context Protocol (MCP) is an open standard protocol that allows AI Agents to connect to external tools, data sources, APIs, etc. Depending on the security control objectives on the network, controlling MCP traffic egressing/ingressing the network might be necessary. MCP uses JSON-RPC messages in a client-server model, over standard TLS-encrypted HTTPS. So, one way to filter MCP traffic is to create a list of MCP FQDNs of interest as firewall address objects, enable SSL deep inspection and then block/allow traffic to those MCP servers depending on the requirements. Here is an example wireshark capture of an MCP client to server communication, over TLS-encrypted HTTPS.

More details about MCP are available in the Architecture - Model Context Protocol documentation.

The following is a configuration example to illustrate how this can be implemented. MCP servers typically contain the keyword 'mcp' in the URL to make it clear for the client application that the said URL maps specifically to the MCP server of that site. A few commonly used MCP server URL examples are shown below:

https://day.ai/api/mcp

https://huggingface.co/mcp

https://mcp-server.egnyte.com/mcp

The following is an example to illustrate a client (Postman as an example) testing access to an MCP server by making a tool/call to query list_all_agent_capabilities. This traffic is allowed before the implementation of the firewall policy, and the response is also received by the client, with the list of agent capabilities as shown below.

MCP-allowed-before-applying-security-policy-rudresh-veerappaji.png

To filter the MCP traffic, start by defining a static URL filter with a regex to match URLs containing the keyword 'mcp', with the action set to block. Enable SSL deep inspection in the firewall policy and attach the webfilter profile.

FortiGate # show webfilter urlfilter 3
config webfilter urlfilter
    edit 3
        set name "Auto-webfilter-urlfilter_787s17ejx"
        config entries
            edit 1
                set url ".*mcp.*"
                set type regex
                set action block
            next
        end
    next
end
FortiGate # show webfilter profile Block-MCP-traffic 
config webfilter profile
    edit "Block-MCP-traffic"
        set feature-set proxy
        config web
            set urlfilter-table 3
        end
        config ftgd-wf
            config filters
                edit 1
                    set category 1
                next
                edit 2
                    set category 3
                . . .
                next
            end
        end
    next
end
FortiGate # config firewall policy
FortiGate (policy) # edit "2"
FortiGate (2) # show
config firewall policy
    edit 2
        set name "MCP-traffic-filter"
        set uuid 94be9c32-d9df-51f0-2dbd-d2d7a7a52559
        set srcintf "port2"
        set dstintf "port1"
        set action accept
        set srcaddr "all"
        set dstaddr "all"
        set schedule "always"
        set service "ALL"
        set utm-status enable
        set inspection-mode proxy
        set ssl-ssh-profile "deep-inspection"
        set webfilter-profile "Block-MCP-traffic"
        set application-list "default"
        set logtraffic all
        set logtraffic-start enable
        set nat enable
    next
end
FortiGate #

The client (using the Postman app as an example) is now blocked from access to the same MCP server, due to the configured firewall policy. Postman MCP client returns with an error indicating an issue with the calling method - 'Error POSTing to endpoint (HTTP 403)'.

MCP-blocked-after-applying-security-policy-rudresh-veerappaji.png

A log entry corresponding to the block event is also recorded in the FortiGate, as shown below.

date=2025-12-15 time=10:34:09 eventtime=1765823649260560680 tz="-0800" logid="0315012544" type="utm" subtype="webfilter" eventtype="urlfilter" level="warning" vd="root" urlfilteridx=1 urlfilterlist="Auto-webfilter-urlfilter_787s17ejx" policyid=2 poluuid="94be9c32-d9df-51f0-2dbd-d2d7a7a52559" policytype="policy" sessionid=2287299 srcip=192.168.20.1 srcport=34632 srccountry="Netherlands" srcintf="port2" srcintfrole="undefined" srcuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" dstip=A.B.C.D dstport=443 dstcountry="Germany" dstintf="port1" dstintfrole="undefined" dstuuid="8486dec2-d7b1-51f0-88ae-be591ae2a2fe" proto=6 httpmethod="POST" service="HTTPS" hostname="mcp.openfloor.dev" agent="PostmanClient/11.76.1 (AppId=bb6ea6d4-4e3e-4814-9f5e-09695765684" profile="Block-MCP-traffic" action="blocked" reqtype="direct" url="https://mcp.openfloor.dev/ " sentbyte=1212 rcvdbyte=260 direction="outgoing" urlsource="Local URLfilter Block" msg="URL was blocked because it is in the URL filter list" crscore=30 craction=8 crlevel="high"

These were some of the most common use cases to manage Generative AI/LLM traffic, but a combination of controls can be used as appropriate for a requirement.

Use Case 8: Monitoring the AI/LLM application activity in the network - using FortiGate or FortiAnalyzer Logs & Reporting

FortiGate Logs & Reports, as well as FortiAnalyzer logs and detailed reporting, can be leveraged for extensive monitoring of the usage and activity of GenAI/LLM applications, similar to all the other applications in the network. Furthermore, using the analytics from these logs & reports, FortiGate policies can be further fine-tuned on an ongoing basis.

FortiGate Logs and Reports.

FortiGate with 7.6.4 & later versions have the new dashboard widgets specifically built to monitor AI applications and use cases, as shown in the example below. Select 'Add widget' in the top left corner of the Dashboard in the FortiGate GUI, and add the AI widgets to view the AI application usage summary in the network.

In the 'Reports' section of the FortiGate under 'Log & Report', a detailed application usage report can be created that shows the usage by application categories and bandwidth, including GenAI applications, as shown in the example below.

The report can also show GenAI application usage by top Users, including blocked requests.

FortiGate-Report-example-2.1.jpg

FortiAnalyzer Logs and Reports

For more detailed and aggregated log analysis and Reporting, FortiAnalyzer can be used, which can provide granular and detailed reports of GenAI application traffic in the network. The 'Applications & Websites' section of the 'FortiView' pane in the FortiAnalyzer shows a separate category for 'Artificial Intelligence Technology', with various traffic usage statistics for users in the network, like threat scores, sessions, bytes, etc, as shown in the example below.

FAZ-which-users-accessing-AI-LLM-applications-category.jpg

For a detailed analysis of the logs corresponding to the Artificial Intelligence Technology category traffic, use the 'Logs' section of the 'Log View' pane as shown in the example below, which highlights the GenAI application names/URLs along with the FortiGate policy actions applied to those sessions.

FAZ-detailed-log-analysis-drilldown-Rudresh-veerappaji.png

FortiAnalyzer also provides options to create detailed aggregated reports of GenAI application traffic in the network, across multiple FortiGates. Reports can be customized as well to create very granular views of the intended data. Below is an example of one such report generated for application traffic and Bandwidth usage including GenAI applications. Steps to create custom reports with examples are here: Custom reports in FortiAnalyzer using Chart Builder and Custom reports in FortiAnalyzer for applications & category

FAZ-GenAI-report-example-part-2v2-Rudresh-Veerappaji.jpg

In the FortiAnalyzer 7.6.0 & later versions, FortiAI security assistant is also available, which is a Generative Security AI assistant (powered by FortiGuard) that can be queried on various incident investigation, response, and threat hunting using iterative prompts. More details on the FortiAI assistant can be found here: FortiAnalyzer 7.6.0 New features - FortiAI.

The use cases discussed in this article are some of the most common scenarios for GenAI security control implementations, but it's not an exhaustive list. A combination of the features discussed here, along with other components of the Fortinet Security Fabric, can be utilized to achieve any additional use cases.

Related documents:

Technical Guide: Selective control & monitoring of GenAI / LLM application traffic with Fortinet Security Fabric

You are leaving our website