fortimail fail to receive email with URLs, got UriExpander and MediaException error.

[Jackchenwork]

Hi , 

I am wondering if anyone with Fortimail instance can help to confirm if you have same problem as our fortimail.

We just found if a incoming email has more than 1 particular urls, it will go nuts.

Here are two example urls:

https://www.google.com/url?q=https://docs.tenable.com&source=gmail-html&ust=1
https://www.google.com/url?q=https://docs.tenable.com&source=gmail-html&ust=2

As soon as I added the two urls in incoming email ( from gmail or from Exchange Online), Fortimail will fail to receive the email.

I have opened a support ticket with Fortinet, but the response is very slow. I am wondering if any Fortimail user can reproduce the issue.

There are many urls can trigger the UriExpander error, examples:

UriExpander: Broken pipe while connecting to https://t.co/bAMtodzfGI
UriExpander: Broken pipe while connecting to https://goo.gl/maps/VI3i1

[Jackchenwork]

a log example 

[Jackchenwork]

Fortimail version : v6.2.8(GA), build291, 2022.01.20

[Jackchenwork]

Anyone from Fortimail team care to comment ? I have rebooted the server but issue still happen.

If Fortimail drop some emails just because there are some URLs in the email, this is a serious defect.  I have seen several legit business emails got dropped silently.

[jpretli2]

Hi Jackchenwork,

Do you have any blocked connection attempts in your firewall-log?

For public redirects FortiMail connects to the URL, follow the redirect and find out what the destination URL is. This will be rated via FortiGuard afterwards.

thanks,

Jochen

[Jackchenwork]

Thanks @jpretli2 !

I don't have access to the firewall ( Fortigate ) right now, looks like this is related with url redirection.

1. I can access all those URLs from my workstation , so those sites are not blocked by fortigate. 

2. I did test, if I add following urls in the email, it can pass fortimail :

https://www.google.com/search?q=one&rlz=1C1GCEB1111
https://www.google.com/search?q=one&rlz=1C1GCEB1112

3. If I add following urls in the email, it can pass fortimail :

https://www.cnn.com
https://www.abc.com

4. if I add following two urls in the email, fortimail will drop it :

https://www.google.com/url?q=https://www.cnn.com
https://www.google.com/url?q=https://www.abc.com

It's not like fortimail can't reach https://www.google.com/ , or cnn.com, but Fortimail failed to follow through the redirect.

[Jackchenwork]

here is one failed email:

each Broken pipe will cause 2 minutes delay.

[jpretli2 · Accepted solution]

Hi Jackchenwork,

please have a look at the Firewall Policies for your FortiMail and also check your Logfiles.

It is common that FortiMail is allowed to connect to our update- and FortiGuard- Servers to download updates and do some URL rating lookups. But it is also part of good security policies to block all other traffic which is not necessary. I assume that your FortiMail is not allowed to connect to the rest of the internet using http and https just to prevent any XSS and other hacking attempts.

How URL rating works in FortiMail is divided into 2 parts:

(1) all incoming URL are checked against FortiGuard Webfilter database - this seems to work ok in your setup.

(2) URL shortener or redirects cannot be checked in step (1) as at this stage it is only the redirect service. So we need to know to what site the redirect is pointing to. To achieve this the FortiMail URI-Expander you're seeing in your logfile tries to follow the redirect to the URL it is pointing to. Once the destination is known this URL will also be checked using FortiGuard Webilter database for a correct URL rating.

So it is essential that FortiMail can communicate to the internet world.

Please check your log files for blocked attempts and also check your policies.

regards

jochen

[Jackchenwork]

Thanks Jochen, I run traffic capture now I see fortimail did try to connect to www.google.com and there is no return packets, so indeed this is the issue. 

Seems it also shows FortiGuard's algorithm is kind of dumb. In step 1), fortiGuard must respond for URL like https://www.google.com/search?q=one&rlz=1C1GCEB1111 as "good", but "https://t.co/bAMtodzfGI" and "https://www.google.com/url?q=https://www.cnn.com" as "URL redirect, please follow".  Fortiguard must has a predefined/periodically updated internal database for "url redirect/shorter". But what's the point to ask Fortimail to check "https://www.google.com/url?q=https://www.cnn.com" , it should just return the same result as https://www.cnn.com . 

[nillwatson532]

It is common that FortiMail is allowed to connect to our update- and FortiGuard- Servers to download updates and do some URL rating lookups.

But it is also part of good security policies to block all other traffic which is not necessary. I assume that your FortiMail is not allowed to connect to the rest of the internet using HTTP and HTTPS just to prevent any XSS and other hacking attempts.

How URL rating works in FortiMail is divided into 2 parts:

 all incoming URLs are checked against the FortiGuard Web filter database  this seems to work ok in your setup.

(2) URL shortener or redirects cannot be checked in step (1) as at this stage it is only the redirect service.

So we need to know to what site the redirect is pointing. To achieve this the FortiMail URI-Expander you're seeing in your logfile tries to follow the redirect to the URL it is pointing to.

Once the destination is known this URL will also be checked using the FortiGuard Webilter database for a correct URL rating.

Hi-Lo Industrial Trucks Co.