[SOLVED] DNS resolution failing inside containers and VMs when on 4G

iduoad · 2021-07-23 02:51:38

Hello Everyone

When I am connected to my mobile phone network through a hotspot (My phone is on 4G LTE), all the containers Fail DNS resolution, although they can reach internet (ping works).

This does not happen (the resolution always succeeds) as long as I am connected to my home or my work router via wifi.

This happens with podman (rootless nad rootful) and LXC containers, and VMs created with libvirt/Qemu.

I am stuck and can't figure out what to do to start troubleshooting.

Can you please guide me through troubleshooting this problem ?

Edit: Leads to start troubleshooting, and debugging the problem are Okey. Even if you've never had such a problem, you can tell me what could be wrong and what should I try/ How should I think.

Last edited by iduoad (2021-07-29 01:16:13)

Lone_Wolf · 2021-07-23 12:28:02

Does dns resolution still work on the host when connected through the 4g hotspot ?

iduoad · 2021-07-23 21:20:24

Yes it does work in the host !

Lone_Wolf · 2021-07-24 11:13:13

Does a container started when using the 4G hotspot have the same issue ?

iduoad · 2021-07-25 00:12:57

Yes, whenever I connect to my 4G, the problem occurs. For containers already started and for new containers.

Lone_Wolf · 2021-07-25 12:46:13

That suggests the containers / VMs use a static dns setup that isn't changed when host dns changes .
Troubleshooting can be tricky since podman / lxc and libvirt/qemu all have their own network settings.

from within a few containers / VMs run

drill archlinux.org
drill @1.1.1.1 archlinux.org

The first will likely fail, the 2nd may succeed.

What are you using to manage your network / switch from wifi to 4G ?

post output of

$ find /etc/systemd -name *.service -exec stat -c %N '{}' \+

iduoad · 2021-07-28 23:20:29

I could install drill quickly inside the container since it can't resolve dns so I used dig. and Still does DNS does not work. Aslso tried to change the nameserver in resolv.conf file

/ # dig +trace @1.1.1.1 archlinux.org

; <<>> DiG 9.16.11 <<>> +trace @1.1.1.1 archlinux.org
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

BTW Ping is working

ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=255 time=79.5 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=255 time=69.0 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=255 time=66.9 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=255 time=63.3 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 63.311/69.665/79.517/6.037 ms

Also I tried running podman container as root with host net, but I got the same results.

Last edited by iduoad (2021-07-29 00:14:06)

progandy · 2021-07-29 00:12:36

your carrier might block third party dns servers? Maybe try DoH or the dns ip provided by your hotspot.

curl --doh-url https://1.1.1.1/dns-query example.com

Maybe run your own (stub or DoH) resolver on the host or in a container and use it in the other containers.

Last edited by progandy (2021-07-29 00:15:17)

iduoad · 2021-07-29 00:22:53

Yes curl with doh works !! Also using the hotspot's ip.

Please, can you explain to me how can my carrier block custom DNS server, and why my browser was working with the 8.8.8.8 as server (or share with me some web resources) ?

Thank you so much !!

Last edited by iduoad (2021-07-29 00:41:52)

progandy · 2021-07-29 05:37:19

iduoad wrote:

Please, can you explain to me how can my carrier block custom DNS server, and why my browser was working with the 8.8.8.8 as server (or share with me some web resources) ?

Chrom(ium) has a short list of known servers that it automatically upgrades to DoH, 8.8.8.8 is among them. Firefox uses DoH automatically in some countries and simply ignores your system resolver in that case unless you disable DoH.

Blocking standard dns is not difficult: Unencrypted DNS uses port 53. The carrier simply blocks connection attempts to that port for all IPs except its own DNS servers.

Last edited by progandy (2021-07-29 05:40:44)

Arch Linux

#1 2021-07-23 02:51:38

[SOLVED] DNS resolution failing inside containers and VMs when on 4G

#2 2021-07-23 12:28:02

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#3 2021-07-23 21:20:24

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#4 2021-07-24 11:13:13

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#5 2021-07-25 00:12:57

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#6 2021-07-25 12:46:13

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#7 2021-07-28 23:20:29

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#8 2021-07-29 00:12:36

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#9 2021-07-29 00:22:53

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

#10 2021-07-29 05:37:19

Re: [SOLVED] DNS resolution failing inside containers and VMs when on 4G

Board footer