[Solved] Slow dig

emphire · 2020-03-07 03:18:06

I've noticed that dig and nslookup runs extremely slowly on my machine (takes about 10-45 seconds) while reslovectl query takes less than 10ms. I'm a little perplexed by it. Does any one have any suggestions on how to dig into the problem?

$ time resolvectl query www.google.com
www.google.com: 216.58.193.68                  -- link: mv0
                2607:f8b0:400a:800::2004       -- link: mv0

-- Information acquired via protocol DNS in 918us.
-- Data is authenticated: no

CPU	74%
user	0.003
system	0.000
total	0.004

$ time dig www.google.com A                                
; <<>> DiG 9.16.0 <<>> www.google.com A
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 908
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;www.google.com.			IN	A

;; ANSWER SECTION:
www.google.com.		213	IN	A	216.58.193.68

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53)
;; WHEN: Fri Mar 06 18:58:34 PST 2020
;; MSG SIZE  rcvd: 59

CPU	0%
user	0.006
system	0.000
total	11.458

Last edited by emphire (2020-03-08 04:38:02)

seth · 2020-03-07 08:41:49

Try

time dig @8.8.8.8 www.google.com A

The problem is probably the contents of /etc/resolve.conf, 11.5 seconds sounds suspiciously as if the first entry is not responding (5s timeout, 2 tries before progression)

Edit:otherwise some broken resolver entry in nsswitch.conf (before resolve - mdns, wins or so)

Last edited by seth (2020-03-07 08:45:04)

emphire · 2020-03-07 20:16:31

Thanks for the suggestion seth, my resolve.conf only has one entry and I tested a simplified nsswitch.conf but it didn't seem to do the trick.

My /etc/resolv.conf:

nameserver 127.0.0.53
options edns0

I'm testing with a dumbed-down /etc/nsswitch.conf now:

...
#hosts: files mymachines myhostname libvirt mdns_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] dns
hosts: files resolve [!UNAVAIL=return] dns
...

No luck with specifying the dns server:

$ time dig @8.8.8.8 www.google.com A

; <<>> DiG 9.16.0 <<>> @8.8.8.8 www.google.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 47177
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.google.com.			IN	A

;; ANSWER SECTION:
www.google.com.		39	IN	A	172.217.3.164

;; Query time: 10 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sat Mar 07 12:02:26 PST 2020
;; MSG SIZE  rcvd: 59

CPU	0%
user	0.009
system	0.000
total	24.461

Interestingly, drill is returning results quickly:

time drill A www.google.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 3814
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; www.google.com.	IN	A

;; ANSWER SECTION:
www.google.com.	251	IN	A	172.217.14.196

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 444 msec
;; SERVER: 127.0.0.53
;; WHEN: Sat Mar  7 12:07:53 2020
;; MSG SIZE  rcvd: 48

CPU	0%
user	0.002
system	0.000
total	0.451

I tried nslookup and it is returning very slowly like dig:

$ time nslookup www.google.com 8.8.8.8
Server:		8.8.8.8
Address:	8.8.8.8#53

Non-authoritative answer:
Name:	www.google.com
Address: 172.217.14.228
Name:	www.google.com
Address: 2607:f8b0:400a:801::2004

CPU	0%
user	0.013
system	0.000
total	1:03.52

It does seem likely that there's something timing out before the qiery with dig/nslookup as you said. The dig tool was reporting the query took 10ms but it took 24 seconds to return the result.

seth · 2020-03-07 20:39:14

drill is far more straightforward and importantly doesn't use libresolv or libns.
But since it queries the localhost resolver, systemd-resolved is off the hook (for today… ;-)

=> What if you move dns before resolve/files in nsswitch.conf?

emphire · 2020-03-08 01:52:38

What if you move dns before resolve/files in nsswitch.conf?

I tried putting dns first and still get the slow response. Let me know if you have any other ideas!

I did discover something interesting:

I tried adding the 'single-request' option to my /etc/resolv.conf and got a fast response from dig! Unfortunately, when I attempted a second/thrid time and with a different zone/dns server, it was slow again. I wonder if this could be ipv6 related... perhaps there's a hung connection?

# With single-request
➜  cat  /etc/resolv.conf | grep -v '^#'
nameserver 127.0.0.53
options edns0 single-request

➜  time dig @8.8.8.8 www.google.com A  # First try with single-request (FAST!)
...
total	0.026

➜  time dig @8.8.8.8 www.google.com A  # Trying again (slow)
total	34.912
➜  time dig @8.8.8.8 www.yahoo.com A  # Trying a different zone (slow)
total	1:40.27
➜  time dig @8.8.4.4 www.google.com A  # Trying a different dns server (slow)
total	13.723

# With single-request-reopen
➜  cat  /etc/resolv.conf | grep -v '^#'
nameserver 127.0.0.53
options edns0 single-request-reopen
➜  time dig @8.8.4.4 www.google.com A  # Trying a different dns server (slow)
total	23.324

emphire · 2020-03-08 04:38:57

Well, it seems that a pacman upgrade and reboot fixed the issue. Hopefully it doesn't come back. Thanks for your assistance seth!

Arch Linux

#1 2020-03-07 03:18:06

[Solved] Slow dig

#2 2020-03-07 08:41:49

Re: [Solved] Slow dig

#3 2020-03-07 20:16:31

Re: [Solved] Slow dig

#4 2020-03-07 20:39:14

Re: [Solved] Slow dig

#5 2020-03-08 01:52:38

Re: [Solved] Slow dig

#6 2020-03-08 04:38:57

Re: [Solved] Slow dig

Board footer