I got this response from support, though we're hesitant to delete any session helpers:
By design FortiGate looks for invalid/failed DNS traffic and will mark it as action=dns or in the GUI as "Action Deny: DNS error". This happens if the DNS query is not successful returns any other status than NOERROR.
This is an expected behavior in version 5.4 where the firewall logs any invalid DNS traffic. The firewall action itself is allow/pass, but the bad reply from the server is not forwarded back to the requesting client thus showing the "Deny: DNS Error" message. Invalid DNS traffic would be i.e. UDP packets on port 53 that are not DNS traffic, packets are over sized, bad checksum etc.
** Can you try to delete the dns session helper from session-helper configuration:
How session helper works:
The FortiOS firewall can analyze most TCP/IP protocol traffic by comparing packet header information to security policies. This comparison determines whether to accept or deny the packet and the session that the packet belongs to. Some protocols include information in the packet body (or payload) that must be analyzed to successfully process sessions for this protocol. It can happen that the information expected by the security policy is not contained in the header/payload of the packet and that is when the packet is denied or dropped; session helper for DNS is not mandatory for which reason you can delete it and it should work properly after.
#config system session-helper
find the one for DNS and than edit it by giving the number)
#edit 14 <---- I checked on the remote session it is "14"
#set name dns-udp
set protocol 17
set port 53
#delete 14 <------
Explanation on Deny: IP connection error:
This is already known issue in 5.4.X and developers is still working on this issue.
The Fortinet Security Fabric brings together the concepts of convergence and consolidation to provide comprehensive cybersecurity protection for all users, devices, and applications and across all network edges.