Web filter regular expression does not work as intended

[danielalbrecht]

Hi!

I'm trying to use a web filter profile to allow a certain URL path (the ACME protocol) but block everything else. I have configured the static URL filter, allowing one regular expression and blocking everything else.

I want to allow URLs in the form of "http://<host>/.well-known/acme-challenge/<path>" where <host> should be the hostname (not containing any /) and <path> is the optional rest of the path

For the URL filter I can enter "*/.well-known/acme-challenge/*" but the problem is, it will also match URLs like http://host.example.com/malicious-path/.well-known/acme-challenge/test

So I want to use regular expressions to avoid that. I'm using this regex:

^[^/]+/\.well-known/acme-challenge/

Using this, the <host> part cannot contain a /, so the path must start at the top level. However I noticed this filter will not match http://host.example.com/.well-known/acme-challenge/  but it will match http://host.example.com/.well-known/acme-challenge/a

As long as there is something after the / it will match, but the regex should also match a URL without the "a".

The filter works (because in practice ACME never uses the base URL without the <path> part) but I would have expected the filter to match anyway.

Just wanted to make sure I'm not missing anything. Could it be a bug?

config webfilter profile
    edit "allow-only-acme"
        config web
            set urlfilter-table 1
        end
        # ...
    next
end
config webfilter urlfilter
    edit 1
        set name "Auto-webfilter-urlfilter_ok2a5adjc"
        config entries
            edit 1
                set url "^[^/]+/\\.well-known/acme-challenge/"
                set type regex
                set action monitor
            next
            edit 2
                set url "*"
                set type wildcard
                set action block
            next
        end
    next
end

[saleha]

Hello,

Thank you for reaching out. The regex expression looks correct to me. You can try removing the last "/" character or replacing it with "*" and see if that will improve the matching:
set url "^[^/]+/\\.well-known/acme-challenge"

or

set url "^[^/]+/\\.well-known/acme-challenge*"

Thank you,

saleha

[pminarik]

I wonder if this could be implemented a bit more elegantly with a WAF profile.

Something similar to this: https://community.fortinet.com/t5/FortiAuthenticator/Technical-Tip-Restrict-public-access-to-FortiAu... , where you can filter specific paths, HTTP methods, and even Hosts (if needed).