FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
nevan
Staff
Staff
Article Id 321941
Description This article describes possible solutions to prevent NAT port or socket exhaustion.
Scope FortiGate.
Solution

NAT port exhaustion occurs when the FortiGate does not have enough source ports available to create a session or to NAT traffic to a specific destination since the source ports might already be used by other connections.


The table depicts a common source NAT translation when multiple endpoints go to the same destination over the same source and destination port:


Internal Firewall/Router External
Source 1 (sport 10230 | dport 53) SNAT port translation (10230 -> 44567) Destination 8.8.8.8 (dport 53)
Source 2 (sport 10230 | dport 53) SNAT port translation (10230 -> 44568) Destination 8.8.8.8 (dport 53)


In a scenario where FortiGate uses one public IP to connect to a single destination IP, over one specific destination port then there are 60416 connections supported.  Refer to the following article for the details about this calculation: Technical Note: How FortiOS selects unused NAT ports.

 

If one more connection is needed for the same destination IP on the same destination port then NAT port exhaustion will occur.

 
Port exhaustion can be identified in the System Event Logs, with the message 'NAT port is exhausted’ or with log ID ID 0100020007.


For example:


date="2024-06-09" time="10:57:46" devid="FGT000123456" vd="root" type="event" subtype="system" bid=540114651 devname="FGTXXXX" dstepid=3 dsteuid=3 dstip="8.8.8.8" dstport=53 dvid=2147 epid=3 euid=3 eventtime=1686326266292111978 id=7242716162855799960 level="critical" logdesc="Socket is exhausted" logid="0100020007" logver=604065868 msg="NAT port is exhausted." nat="33.43.111.222" proto=17 service="kernel" srcip="10.200.100.50" srcport=15651 status="failure" tz="-0500" vrf=0

 

It can also be seen in the clash counter in the CLI:


For example:

 

FGT # diag sys session stat

misc info:       session_count=20 setup_rate=0 exp_count=0 reflect_count=0 clash=999

 

Note:

When source ports are lower than 1024 and traffic passes through a firewall policy with NAT enabled, session clashes and port exhaustion occur early and frequently.

Because for NAT'ing privileged ports FortiGate uses the range only 512-1023.

 

There are two scenarios where NAT exhaustion may happen:

 

Scenario 1: (Less likely to occur): NAT exhaustion for local traffic - traffic originating from the FortiGate (For example: SD-WAN SLA, Web ratings):

Increasing the source port range that FortiGate can use for local traffic under the following configuration:

  config system global
      set ip-src-port-range 1024-25000     <- Increase this range.
  end

  • ip-src-port-range - IP source port range used for traffic originating from the FortiGate unit.

 

Scenario 2: NAT exhaustion for forward traffic – User traffic going to the internet:

 

This scenario is more likely to occur if there is a shared resource over the Internet that many users utilize, such as a well-known DNS server. In this DNS example, users will be establishing sessions to the same destination IP(s), the same protocol (UDP, IP Proto 17), and the same destination port (53).

The FortiGate must have unique session entries to identify each session when performing NAT, and so the unique elements of the session that remain are the NAT Source IP of the session (either the FortiGate interface IP or an IP Pool address) and the NAT Source Port (which is allocated by the FortiGate).

 

The ideal solution is to increase the total number of NAT source ports that can be allocated to user sessions. This can be accomplished by creating and applying an IP Pool in the Firewall Policies that have more than one external Public IP configured (the available ports scales linearly with the number of external IPs available in the IP Pool). 

 

Refer to the following article for more information on configuring IP Pools: Technical Tip: How to configure SNAT with IP pool 

 

If this is not an option, then it may be possible to 'optimize' the usage of the limited NAT source ports that are available by removing as many expired/inactive sessions as possible. This is not generally relevant for TCP sessions (since TCP can signal when a session is completed/closed), but it is relevant for DNS and ICMP traffic (since neither protocol can signal the end of a session, they simply stop sending traffic).

 

This can be accomplished by reducing the UDP/ICMP session lifetimes so that inactive sessions are expired by the FortiGate more quickly. For example, UDP sessions have a default session expiration of 180 seconds, but for DNS sessions (which typically complete in a few seconds) this could be reduced substantially to 30 seconds or potentially even shorter. Refer to this KB article for information on adjusting Session TTLs (Time to Live) on the FortiGate: Technical Tip: Session timeout settings.


Related articles:
Technical Note: How FortiOS selects unused NAT ports
Technical Tip: IPpool exhaust