[SOLVED] FortiGate DNS server - no response to AAAA queries when there are no AAAA records

[KM]

Hello, Situation: FortiGate 400E running FortiOS 7.0.0 set as DNS server for local networks (recursive, but also forward to system DNS). DNS server IP = interface IP. All networks IPv4. DNS queries type A are answered by FortiGate DNS server, example: "Standard query 0x969a A wp.pl" "Standard query response 0x969a A wp.pl 212.77.98.9" Problem starts when there are AAAA queries, but no AAAA record exists. FortiGate DNS server receives queries: "Standard query 0x8e50 AAAA wp.pl" "Standard query 0x8e50 AAAA wp.pl" "Standard query 0x8e50 AAAA wp.pl" but there is no response to client which causes timeout on client side and unnecessary delay. Is there any solution to this problem? When quering some public DNS server, for example 1.1.1.1, answer to AAAA query is: "Standard query response 0x7b2c AAAA wp.pl SOA ns1.wp.pl" and there is no timeout on clinent side.

[emnoc]

I don't have that issue nor never seen it.

e.g 

supports-MacBook-Pro:~ ken$ host -t a ipv6.hyperfeed.com 192.168.1.99Using domain server:Name: 192.168.1.99Address: 192.168.1.99#53Aliases:  ipv6.hyperfeed.com has address 192.0.2.22  supports-MacBook-Pro:~ ken$ host -t aaaa ipv6.hyperfeed.com 192.168.1.99Using domain server:Name: 192.168.1.99Address: 192.168.1.99#53Aliases:  ipv6.hyperfeed.com has no AAAA recordsupports-MacBook-Pro:~ ken$  and using your example supports-MacBook-Pro:~ ken$ host -t aaaa wp.pl 192.168.1.99Using domain server:Name: 192.168.1.99Address: 192.168.1.99#53Aliases:  wp.pl has no AAAA record and for a recursive lookup;  supports-MacBook-Pro:~ ken$ host -t aaaa www.gmail.com 192.168.1.99Using domain server:Name: 192.168.1.99Address: 192.168.1.99#53Aliases:  www.gmail.com is an alias for mail.google.com.mail.google.com is an alias for googlemail.l.google.com.googlemail.l.google.com has IPv6 address 2607:f8b0:4000:81b::2005 Btw, this is  fortios 7.0.1 Ken Felix

[KM]

Issue noticed on Windows 10 and Ubuntu Server 20.04.

Example from Win 10:

>nslookup wp.pl DNS request timed out.     timeout was 2 seconds. Server:  UnKnown Address:  10.0.0.1 DNS request timed out.     timeout was 2 seconds. DNS request timed out.     timeout was 2 seconds. Non-authoritative answer: DNS request timed out.     timeout was 2 seconds. Name:    wp.pl Address:  212.77.98.9

Example from Ubuntu Server 20.04:

$ nslookup wp.pl Server:         10.0.0.1 Address:        10.0.0.1#53 Non-authoritative answer: Name:   wp.pl Address: 212.77.98.9 ;; connection timed out; no servers could be reached

[emnoc]

Do you have DNS enable on 10.0.0.1?

e.g

host -t txt -c chaos version.bind 192.168.1.99orhost -T -t txt -c chaos version.bind  192.168.1.99 Is dnsproc pid showing "diag sys top "  

Any downstream filters , firewalls, layer2 firewall blocking access to port 53 ? Did you do a diag debug flow?

 diag debug reset  diag debug flow filter dport 53 diag debug flow filter daddr 192.168.1.99 # put your address here diag debug flow trace start 10 diag debug en diag debug flow trace start 10 SOCPUPFGT02 # id=20085 trace_id=2 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=17, 192.168.1.110:55687->192.168.1.99:53) from internal. "id=20085 trace_id=2 func=init_ip_session_common line=5918 msg="allocate a new session-00026a88"id=20085 trace_id=2 func=vf_ip_route_input_common line=2615 msg="find a route: flag=84000000 gw-192.168.1.99 via root"id=20085 trace_id=2 func=__ip_session_run_tuple line=3529 msg="run helper-dns-udp(dir=original)"id=20085 trace_id=3 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag , seq 4203354126, ack 0, win 65535"id=20085 trace_id=3 func=init_ip_session_common line=5918 msg="allocate a new session-00026aa1"id=20085 trace_id=3 func=vf_ip_route_input_common line=2615 msg="find a route: flag=84000000 gw-192.168.1.99 via root"id=20085 trace_id=4 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag [.], seq 4203354127, ack 1720278195, win 2058"id=20085 trace_id=4 func=resolve_ip_tuple_fast line=5827 msg="Find an existing session, id-00026aa1, original direction"id=20085 trace_id=5 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag [.], seq 4203354127, ack 1720278195, win 2058"id=20085 trace_id=5 func=resolve_ip_tuple_fast line=5827 msg="Find an existing session, id-00026aa1, original direction"id=20085 trace_id=6 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag [.], seq 4203354159, ack 1720278247, win 2058"id=20085 trace_id=6 func=resolve_ip_tuple_fast line=5827 msg="Find an existing session, id-00026aa1, original direction"id=20085 trace_id=7 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag [F.], seq 4203354159, ack 1720278247, win 2058"id=20085 trace_id=7 func=resolve_ip_tuple_fast line=5827 msg="Find an existing session, id-00026aa1, original direction"id=20085 trace_id=8 func=print_pkt_detail line=5746 msg="vd-root:0 received a packet(proto=6, 192.168.1.110:64884->192.168.1.99:53) from internal. flag [.], seq 4203354160, ack 1720278248, win 2058"id=20085 trace_id=8 func=resolve_ip_tuple_fast line=5827 msg="Find an existing session, id-00026aa1, original direction" # when done 

diag debug reset 

diag debug disable 

Did you at least do a diag sniffer packet any "host 10.0.0.1 and port 53" and see if our windows or ubuntu clients are hitting the dns-server ip.addr on the fortigate ?

Time-out means exactly that, a time-out due to reachability or the service is not running 

Ken Felix

[KM]

FG DNS server config: # config system dns-server (dns-server) # show full-configuration config system dns-server     edit "vlan201"   //interface IP addr = 10.0.0.1         set mode recursive         set dnsfilter-profile "default"         set doh disable     next     {...}     edit "vlan5"     //interface IP addr = 10.0.5.1         set mode forward-only         set dnsfilter-profile "default"         set doh disable     next     {...} end Output from Ubuntu: $ host -t txt -c chaos version.bind 10.0.0.1 Using domain server: Name: 10.0.0.1 Address: 10.0.0.1#53 Aliases: version.bind descriptive text "Q9-U-7.2" ~$ host -t txt -c chaos version.bind 10.0.5.1 Using domain server: Name: 10.0.5.1 Address: 10.0.5.1#53 Aliases: version.bind descriptive text "Q9-U-7.2" FG: # diag sys top dnsproxy    23589      S       0.1     0.3    1 > Any downstream filters , firewalls, layer2 firewall blocking access to port 53 ? No. From Win 10 side (IP 10.0.1.1): >nslookup wp.pl 10.0.0.1 DNS request timed out.     timeout was 2 seconds. Server:  UnKnown Address:  10.0.0.1 DNS request timed out.     timeout was 2 seconds. DNS request timed out.     timeout was 2 seconds. Non-authoritative answer: DNS request timed out.     timeout was 2 seconds. Name:    wp.pl Address:  212.77.98.9 and FG side diag with additional filter "diagnose debug flow filter saddr 10.0.1.1" for a clearer view: # id=20085 trace_id=11 func=print_pkt_detail line=5723 msg="vd-root:0 received a packet(proto=17, 10.0.1.1:55002->10.0.0.1:53) from vlan201. " id=20085 trace_id=11 func=init_ip_session_common line=5894 msg="allocate a new session-006120b0" id=20085 trace_id=11 func=vf_ip_route_input_common line=2621 msg="find a route: flag=84000000 gw-10.0.0.1 via root" id=20085 trace_id=11 func=__ip_session_run_tuple line=3540 msg="run helper-dns-udp(dir=original)" id=20085 trace_id=12 func=print_pkt_detail line=5723 msg="vd-root:0 received a packet(proto=17, 10.0.1.1:55003->10.0.0.1:53) from vlan201. " id=20085 trace_id=12 func=init_ip_session_common line=5894 msg="allocate a new session-00612104" id=20085 trace_id=12 func=vf_ip_route_input_common line=2621 msg="find a route: flag=84000000 gw-10.0.0.1 via root" id=20085 trace_id=12 func=__ip_session_run_tuple line=3540 msg="run helper-dns-udp(dir=original)" id=20085 trace_id=13 func=print_pkt_detail line=5723 msg="vd-root:0 received a packet(proto=17, 10.0.1.1:55004->10.0.0.1:53) from vlan201. " id=20085 trace_id=13 func=init_ip_session_common line=5894 msg="allocate a new session-00612139" id=20085 trace_id=13 func=vf_ip_route_input_common line=2621 msg="find a route: flag=84000000 gw-10.0.0.1 via root" id=20085 trace_id=13 func=__ip_session_run_tuple line=3540 msg="run helper-dns-udp(dir=original)" id=20085 trace_id=14 func=print_pkt_detail line=5723 msg="vd-root:0 received a packet(proto=17, 10.0.1.1:55005->10.0.0.1:53) from vlan201. " id=20085 trace_id=14 func=init_ip_session_common line=5894 msg="allocate a new session-00612158" id=20085 trace_id=14 func=vf_ip_route_input_common line=2621 msg="find a route: flag=84000000 gw-10.0.0.1 via root" id=20085 trace_id=14 func=__ip_session_run_tuple line=3540 msg="run helper-dns-udp(dir=original)" id=20085 trace_id=15 func=print_pkt_detail line=5723 msg="vd-root:0 received a packet(proto=17, 10.0.1.1:55006->10.0.0.1:53) from vlan201. " id=20085 trace_id=15 func=init_ip_session_common line=5894 msg="allocate a new session-00612159" id=20085 trace_id=15 func=vf_ip_route_input_common line=2621 msg="find a route: flag=84000000 gw-10.0.0.1 via root" id=20085 trace_id=15 func=__ip_session_run_tuple line=3540 msg="run helper-dns-udp(dir=original)" From what I understand there are 5 queries logged. 5 queries were sent from client, so all arrived to DNS server. # diag sniffer packet any "host 10.0.0.1 and port 53" interfaces=[any] filters=[host 10.0.0.1 and port 53] 2.859590 10.0.1.1.51253 -> 10.0.0.1.53: udp 39 3.376995 10.0.0.70.54526 -> 10.0.0.1.53: udp 86 3.377043 10.0.0.1.53 -> 10.0.0.70.54526: udp 90 3.377045 10.0.0.1.53 -> 10.0.0.70.54526: udp 90 3.377045 10.0.0.1.53 -> 10.0.0.70.54526: udp 90 4.592696 10.0.0.70.60546 -> 10.0.0.1.53: udp 86 4.592748 10.0.0.1.53 -> 10.0.0.70.60546: udp 90 4.592749 10.0.0.1.53 -> 10.0.0.70.60546: udp 90 4.592751 10.0.0.1.53 -> 10.0.0.70.60546: udp 90 4.871339 10.0.1.1.51254 -> 10.0.0.1.53: udp 34 6.883311 10.0.1.1.51255 -> 10.0.0.1.53: udp 34 8.892497 10.0.1.1.51256 -> 10.0.0.1.53: udp 23 8.892548 10.0.0.1.53 -> 10.0.1.1.51256: udp 39 8.892550 10.0.0.1.53 -> 10.0.1.1.51256: udp 39 8.892551 10.0.0.1.53 -> 10.0.1.1.51256: udp 39 8.896480 10.0.1.1.51257 -> 10.0.0.1.53: udp 23 9.046379 10.0.0.70.53309 -> 10.0.0.1.53: udp 68 9.066385 10.0.0.1.53 -> 10.0.0.70.53309: udp 84 9.066387 10.0.0.1.53 -> 10.0.0.70.53309: udp 84 9.066389 10.0.0.1.53 -> 10.0.0.70.53309: udp 84 9.667620 10.0.0.70.57252 -> 10.0.0.1.53: udp 64 9.679955 10.0.0.1.53 -> 10.0.0.70.57252: udp 68 9.679957 10.0.0.1.53 -> 10.0.0.70.57252: udp 68 9.679958 10.0.0.1.53 -> 10.0.0.70.57252: udp 68 10.374776 10.0.0.70.38430 -> 10.0.0.1.53: udp 64 Another example: From Ubuntu: $ nslookup wp.pl Server:         10.0.5.1 Address:        10.0.5.1#53 Non-authoritative answer: Name:   wp.pl Address: 212.77.98.9 ;; connection timed out; no servers could be reached and FG sniffer: # diag sniffer packet vlan5 "host 10.0.5.1 and port 53" interfaces=[vlan5] filters=[host 10.0.5.1 and port 53] 3.444542 10.0.5.63.40640 -> 10.0.5.1.53: udp 23 3.444593 10.0.5.1.53 -> 10.0.5.63.40640: udp 39 3.445040 10.0.5.63.41103 -> 10.0.5.1.53: udp 23 8.441238 10.0.5.63.41103 -> 10.0.5.1.53: udp 23 13.441364 10.0.5.63.41103 -> 10.0.5.1.53: udp 23 # diag sniffer packet vlan5 "host 10.0.5.1 and port 53" 2 interfaces=[vlan5] filters=[host 10.0.5.1 and port 53] 4.609092 10.0.5.63.35451 -> 10.0.5.1.53: udp 23 0x0000   4500 0033 9952 0000 4011 c328 0a00 053f        E..3.R..@..(...? 0x0010   0a00 0501 8a7b 0035 001f 2691 4a48 0100        .....{.5..&.JH.. 0x0020   0001 0000 0000 0000 0277 7002 706c 0000        .........wp.pl.. 0x0030   0100 01                                        ... 4.634983 10.0.5.1.53 -> 10.0.5.63.35451: udp 39 0x0000   4500 0043 f473 0000 4011 67f7 0a00 0501        E..C.s..@.g..... 0x0010   0a00 053f 0035 8a7b 002f 0ff8 4a48 8180        ...?.5.{./..JH.. 0x0020   0001 0001 0000 0000 0277 7002 706c 0000        .........wp.pl.. 0x0030   0100 01c0 0c00 0100 0100 0001 2c00 04d4        ............,... 0x0040   4d62 09                                        Mb. 4.635680 10.0.5.63.43620 -> 10.0.5.1.53: udp 23 0x0000   4500 0033 9958 0000 4011 c322 0a00 053f        E..3.X..@.."...? 0x0010   0a00 0501 aa64 0035 001f e692 4f5d 0100        .....d.5....O].. 0x0020   0001 0000 0000 0000 0277 7002 706c 0000        .........wp.pl.. 0x0030   1c00 01                                        ... 9.635190 10.0.5.63.43620 -> 10.0.5.1.53: udp 23 0x0000   4500 0033 9b8f 0000 4011 c0eb 0a00 053f        E..3....@......? 0x0010   0a00 0501 aa64 0035 001f e692 4f5d 0100        .....d.5....O].. 0x0020   0001 0000 0000 0000 0277 7002 706c 0000        .........wp.pl.. 0x0030   1c00 01                                        ... 14.632484 10.0.5.63.43620 -> 10.0.5.1.53: udp 23 0x0000   4500 0033 9f92 0000 4011 bce8 0a00 053f        E..3....@......? 0x0010   0a00 0501 aa64 0035 001f e692 4f5d 0100        .....d.5....O].. 0x0020   0001 0000 0000 0000 0277 7002 706c 0000        .........wp.pl.. 0x0030   1c00 01       So as I understand: 3.444542 10.0.5.63.40640 -> 10.0.5.1.53: udp 23   # A query 3.444593 10.0.5.1.53 -> 10.0.5.63.40640: udp 39   # A answer 3.445040 10.0.5.63.41103 -> 10.0.5.1.53: udp 23   # AAAA query 8.441238 10.0.5.63.41103 -> 10.0.5.1.53: udp 23   # AAAA query 13.441364 10.0.5.63.41103 -> 10.0.5.1.53: udp 23  # AAAA query

and no AAAA answer.

[KM]

Upgrade to FortiOS 7.0.1 build0157 solved the problem.