[SOLVED] DNS resolver(s) issues

0BADC0DE · 2024-12-12 08:05:53

I am having issues with some programs doing DNS queries through "standard" resolvers.
First things first.
This is /etc/resolv.conf:

# Generated by NetworkManager
search 8.8.8.8 1.1.1.1 208.67.222.222

This is my current systemd-resolved configuration:

$ systemd-analyze cat-config systemd/resolved.conf
# /etc/systemd/resolved.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file (or a copy of it placed in
# /etc/ if the original file is shipped in /usr/), or by creating "drop-ins" in
# the /etc/systemd/resolved.conf.d/ directory. The latter is generally
# recommended. Defaults can be restored by simply deleting the main
# configuration file and all drop-ins located in /etc/.
#
# Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config.
#
# See resolved.conf(5) for details.

[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
# Google:     8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.google
# Quad9:      9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
#DNS=
#FallbackDNS=1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net 8.8.8.8#dns.google 2606:4700:4700::1111#cloudflare-dns.com 2620:fe::9#dns.quad9.net 2001:4860:4860::8888#dns.google
#Domains=
#DNSSEC=no
#DNSOverTLS=no
#MulticastDNS=yes
#LLMNR=yes
#Cache=yes
#CacheFromLocalhost=no
#DNSStubListener=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no
#StaleRetentionSec=0

This is where systemd-resolve is listening to:

$ sudo ss -lunp | grep systemd-resolve
UNCONN 0      0            0.0.0.0:5353       0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=15))
UNCONN 0      0            0.0.0.0:5355       0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=11))
UNCONN 0      0         127.0.0.54:53         0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=24))
UNCONN 0      0      127.0.0.53%lo:53         0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=22))
UNCONN 0      0               [::]:5353          [::]:*    users:(("systemd-resolve",pid=5554,fd=16))
UNCONN 0      0               [::]:5355          [::]:*    users:(("systemd-resolve",pid=5554,fd=13))
$ sudo ss -ltnp | grep systemd-resolve
LISTEN 0      4096      127.0.0.54:53         0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=25))
LISTEN 0      4096   127.0.0.53%lo:53         0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=23))
LISTEN 0      4096         0.0.0.0:5355       0.0.0.0:*    users:(("systemd-resolve",pid=5554,fd=12))
LISTEN 0      4096            [::]:5355          [::]:*    users:(("systemd-resolve",pid=5554,fd=14))

And this is my simplest way to reproduce the problem:

$ nslookup www.google.com
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to 127.0.0.1#53: connection refused
;; no servers could be reached

There is also aur/tenv-bin which fails in a similar way:

$ tenv tf list-remote
Fetching all releases information from https://releases.hashicorp.com/terraform/index.json
Error: Get "https://releases.hashicorp.com/terraform/index.json": dial tcp: lookup releases.hashicorp.com on [::1]:53: read udp [::1]:50283->[::1]:53: read: connection refused

There is neither a [::1] not a 127.0.0.1 listener.

Of course, all of my browsers can resolve names as well as pacman, paru and other network-related tools.
I suspect they are running their own resolution code with several different strategies.

Is there anyone having my same issue?

Any hint?

Last edited by 0BADC0DE (2024-12-12 08:50:40)

0BADC0DE · 2024-12-12 08:29:51

I think have found the culprit!
In my networkmanager configuration (I used KDE and related applet) I had defined my main network connection as "Automatic (Only addresses)" (aka DHCP w/o DNS servers) with a list of DNS servers as shown in the OP.
Now I have brought it back to "Automatic" (aka Full DHCP) so also DNS servers come from the DHCP server.

Now my /etc/resolv.conf looks like this:

# Generated by NetworkManager
search 8.8.8.8 1.1.1.1 208.67.222.222
nameserver 8.8.8.8

The search list looks weird to me: it should be a list of domains, not IPs!
Quoting from "resolv.conf(5)":

       search Search list for host-name lookup.
              By  default, the search list contains one entry, the local domain
              name.  It is determined from the local hostname returned by geth‐
              ostname(2); the local domain name is taken to be everything after
              the first '.'.  Finally, if the hostname does not contain a  '.',
              the root domain is assumed as the local domain name.

              This  may  be  changed  by listing the desired domain search path
              following the search keyword with spaces or tabs  separating  the
              names.  Resolver queries having fewer than ndots dots (default is
              1)  in  them will be attempted using each component of the search
              path in turn until a match is found.  For environments with  mul‐
              tiple  subdomains please read options ndots:n below to avoid man-
              in-the-middle attacks and unnecessary traffic for  the  root-dns-
              servers.   Note that this process may be slow and will generate a
              lot of network traffic if the servers for the listed domains  are
              not  local, and that queries will time out if no server is avail‐
              able for one of the domains.

              If there are multiple search directives,  only  the  search  list
              from the last instance is used.

              In  glibc 2.25 and earlier, the search list is limited to six do‐
              mains with a total of 256  characters.   Since  glibc  2.26,  the
              search list is unlimited.

              The domain directive is an obsolete name for the search directive
              that handles one search list entry only.

Is this a networkmanager bug? What do you (all) think?

-thc · 2024-12-12 08:32:24

No I'm not having this issue because I did use NetworkManager for years on it's own and after thinking to integrate systemd-resolved i did RTFW: https://wiki.archlinux.org/title/Networ … d-resolved.

In short: Make NetworkManager aware that systemd-resolved is responsible for DNS and NM should not do it.

Last edited by -thc (2024-12-12 08:35:04)

0BADC0DE · 2024-12-12 08:41:27

-thc wrote:

No I'm not having this issue because I did use NetworkManager for years on it's own and after thinking to integrate systemd-resolved i did RTFW: https://wiki.archlinux.org/title/Networ … d-resolved.
In short: Make NetworkManager aware that systemd-resolved is responsible for DNS and NM should not do it.

Hmmm...
I have read this specific chapter.
I have created that file with the same exact content.
Then have disconnected and reconnected the link.
Now my /etc/resolv.conf is like this:

# Generated by NetworkManager
search 8.8.8.8 1.1.1.1 208.67.222.222

But I still get the error:

user@Feynman [  1]  ~ nslookup www.google.com.
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to ::1#53: connection refused
;; communications error to 127.0.0.1#53: connection refused
;; no servers could be reached

user@Feynman [  1]  ~ nslookup www.google.com. 1.1.1.1
Server:         1.1.1.1
Address:        1.1.1.1#53

Non-authoritative answer:
Name:   www.google.com
Address: 142.250.180.132
Name:   www.google.com
Address: 2a00:1450:4002:414::2004

Let me try a reboot (I really hate reboots!).

[UPDATE]
It worked indeed.
I think a simple " sudo systemctl restart NetworkManager" would have sufficed after the file "/etc/NetworkManager/conf.d/dns.conf" had been created!

Last edited by 0BADC0DE (2024-12-12 08:49:49)

0BADC0DE · 2024-12-12 08:51:59

I would say that this is a bug in the installation of the NetworkManager package.
What do you think?

-thc · 2024-12-12 09:02:03

Let's get into the details: You can either tell NetworkManager directly (conf drop-in file) to use system-resolved or let it discover that "/etc/resolv.conf" is a link to "/run/systemd/resolve/stub-resolv.conf" and it will switch to systemd-resolved.

lrwxrwxrwx 1 root root 37 Oct 30  2023 /etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf

Depending on what you did and when, a restart of systemd-resolved and NetworkManager should™ have worked. No, I don't think that's a bug.

0BADC0DE · 2024-12-12 09:12:56

-thc wrote:

Let's get into the details: You can either tell NetworkManager directly (conf drop-in file) to use system-resolved or let it discover that "/etc/resolv.conf" is a link to "/run/systemd/resolve/stub-resolv.conf" and it will switch to systemd-resolved.
lrwxrwxrwx 1 root root 37 Oct 30  2023 /etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf
Depending on what you did and when, a restart of systemd-resolved and NetworkManager should™ have worked. No, I don't think that's a bug.

When I install ArchLinux' essential packages with "pacstrap" I get systemd-resolved by default.
So, installing Networkmanager should take that into consideration by default or at least warn users that some changes are needed in order to integrate it with systemd-resolved.
Being that a symlink or a drop-in file is irrelevant.
For sure NetworkManager won't work properly without that change. Will it?

So, maybe I can agree it is not a bug, but I would say there is a missing warning/notice with NetworkManager installation.

Lone_Wolf · 2024-12-12 09:43:36

When I install ArchLinux' essential packages with "pacstrap" I get systemd-resolved by default.

systemd-resolved, systemd-networkd , systemd-timesyncd etc are disabled by default and inactive.

NetworkManager has its own DHCP-client that uses DHCP provided DNS servers by default and DNS resolution works fine with them.

seth · 2024-12-12 11:16:02

Also

Being that a symlink or a drop-in file is irrelevant.

Nope. Unfortunately systemd is exatly that insane: https://man.archlinux.org/man/systemd-r … ESOLV.CONF
The filetype has *drastic* impact on resolvd's behavior.

0BADC0DE · 2024-12-12 15:12:14

seth wrote:

Also
Being that a symlink or a drop-in file is irrelevant.
Nope. Unfortunately systemd is exatly that insane: https://man.archlinux.org/man/systemd-r … ESOLV.CONF
The filetype has *drastic* impact on resolvd's behavior.

Alternatively, /etc/resolv.conf may be managed by other packages, in which case systemd-resolved will read it for DNS configuration data. In this mode of operation systemd-resolved is consumer rather than provider of this configuration file.

This very part proved not to work at all.
In my case "/etc/resolv.conf" was a correct file managed by NetworkManager and pointing at working and reachable servers.
Nonetheless, it wasn't working.

Last edited by 0BADC0DE (2024-12-12 15:12:40)

-thc · 2024-12-12 17:52:16

You had multiple issues that influenced one another.

1. Your network configuration mode ("Automatic (Only addresses)") was lacking the DNS servers - this could not work. You've changed this.

2. The DNS domain search list contains DNS server IP addresses which is wrong. Check if you entered those manually - otherwise your DHCP server (router?) spouts nonsense.

3. After obtaining the DNS servers per DHCP your "resolv.conf" contained at least one "nameserver" entry - systemd-resolved accepts this and switches to "foreign" mode and should work - but only on it's own (without a concurrent management). NetworkManager "sees" that "resolv.conf" is not a link and "thinks" systemd-resolved isn't handling DNS. Concurrent network managements leads to all kinds of shenanigans.

4. After you created the drop-in file NetworkManager expects systemd-resolved to handle DNS - and leaves "resolv.conf" alone. You can even create your own "resolv.conf" - systemd-resolved will switch to "foreign" mode and NetworkManager doesn't care.

seth · 2024-12-12 17:57:50

Edit: cross talk - the below is an answer to #10

seth wrote:

In which case the localhost :53 does not act as DNS server
Assuming the claimed contents of /etc/resolv.conf were permanent (the file wasn't constantly rewritten beween NM and resolved) you'd want to check you nsswitch.conf and maybe strace why 127.0.0.1:53 has been used at all.

Last edited by seth (2024-12-12 17:59:26)

seth · 2024-12-16 21:40:53

Please use [code][/code] tags. Edit your post in this regard.

You're not running resolved - w/ the 2nd variant, what's the output of

ping  -c1 google.com; ping -4  -c1 google.com; ping -6 -c1 google.com

Arch Linux

#1 2024-12-12 08:05:53

[SOLVED] DNS resolver(s) issues

#2 2024-12-12 08:29:51

Re: [SOLVED] DNS resolver(s) issues

#3 2024-12-12 08:32:24

Re: [SOLVED] DNS resolver(s) issues

#4 2024-12-12 08:41:27

Re: [SOLVED] DNS resolver(s) issues

#5 2024-12-12 08:51:59

Re: [SOLVED] DNS resolver(s) issues

#6 2024-12-12 09:02:03

Re: [SOLVED] DNS resolver(s) issues

#7 2024-12-12 09:12:56

Re: [SOLVED] DNS resolver(s) issues

#8 2024-12-12 09:43:36

Re: [SOLVED] DNS resolver(s) issues

#9 2024-12-12 11:16:02

Re: [SOLVED] DNS resolver(s) issues

#10 2024-12-12 15:12:14

Re: [SOLVED] DNS resolver(s) issues

#11 2024-12-12 17:52:16

Re: [SOLVED] DNS resolver(s) issues

#12 2024-12-12 17:57:50

Re: [SOLVED] DNS resolver(s) issues

#13 2024-12-16 21:40:53

Re: [SOLVED] DNS resolver(s) issues

Board footer