You are not logged in.

Pages: 1

#1 2023-07-24 12:09:07

ollosh
Member
Registered: 2022-09-20
Posts: 44

Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Hello,

I've been having some issues recently with DNS and can't wrap my head around what might be going on. So I'm using NetworkManager with systemd-resolvd and I'm also using cloudflare-warp client for work from this package https://aur.archlinux.org/packages/cloudflare-warp-bin.
I don't run it always so I'm not sure if that's the culprit, but what happens is that after using my pc for a while I keep running into some name resolution problems. I don't know what actually to look into, but my current setup is

lrwxrwxrwx 1 root root 37 Jul 24 13:47 /etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .

resolvctl output (No idea where that 192.168.0.1 is coming from, is it my router? Should/Can I have the dns set to Google dns there as well?)

Global
           Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
    resolv.conf mode: stub
  Current DNS Server: 8.8.8.8
         DNS Servers: 8.8.8.8
Fallback DNS Servers: 8.8.4.4

Link 2 (enp1s0f0)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (wwp4s0f3u4)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported


Link 4 (wlp2s0)
    Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6 mDNS/IPv4 mDNS/IPv6
         Protocols: +DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 2a02:8109:88c0:5fe8:8a71:b1ff:fe7b:71f9
       DNS Servers: 192.168.0.1 2a02:8109:88c0:5fe8:8a71:b1ff:fe7b:71f9

/etc/systemd/resolved.conf

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it under the
#  terms of the GNU Lesser General Public License as published by the Free
#  Software Foundation; either version 2.1 of the License, or (at your option)
#  any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the resolved.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config.
#
# See resolved.conf(5) for details.

[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
# Google:     8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.google
# Quad9:      9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
DNS=8.8.8.8
FallbackDNS=8.8.4.4
#FallbackDNS=1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net 8.8.8.8#dns.google 2606:4700:4700::1111#cloudflare-dns.com 2620:fe::9#dns.quad9.net 2001:4860:4860::8888#dns.google
#Domains=
#DNSSEC=no
#DNSOverTLS=no
#MulticastDNS=yes
#LLMNR=yes
#Cache=yes
#CacheFromLocalhost=no
#DNSStubListener=yes
#DNSStubListenerExtra=
#ReadEtcHosts=yes
ResolveUnicastSingleLabel=yes

Offline

#2 2023-07-24 13:47:48

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

I keep running into some name resolution problems

And what are those DNS issues exactly?
Symptoms?

Assuming an unresolvable domain is google.com, can you still
- nslookup google.com # bind package
-  getent ahosts google.com
- drill google.com # ldns package, drill ignores nsswitch
- dig @8.8.8.8 google.com # bind, ask 8.8.8.8 directly
And what are the outputs?

is it my router? Should/Can I have the dns set to Google dns there as well?

likely, whether you can configure its DNS depends on ho much locked down it is.
You'll get that DNS via dhcp, but can configure NM to use a specific DNS for that connection.

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#3 2023-07-24 23:08:38

ollosh
Member
Registered: 2022-09-20
Posts: 44

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Well tbh it's pretty random and symptoms are weird. Like after a while my browser starts showing ERR_NETWORK_CHANGED for a second and then refreshes the site and sometimes the site loads and other times the issue persits.
When there's a lot of AJAX calls on the web some randomly fail and the page seems to be half loaded and half the interactions are broken which leads to frustrating retries and then the interaction works eventually.
The only thing I've noticed is helping clear the issue for a while is running `nmcli radio wifi off/on`.

So basically once it starts failing it shows this screen randomly
https://i.imgur.com/50BEEoy.png

Otputs from flakey websites:
github.com

~> nslookup github.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	github.com
Address: 140.82.121.3
~> getent ahosts github.com
140.82.121.4    STREAM github.com
140.82.121.4    DGRAM  
140.82.121.4    RAW   
~> drill github.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 54549
;; flags: qr rd ra ; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; github.com.	IN	A

;; ANSWER SECTION:
github.com.	38	IN	A	140.82.121.4

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 0 msec
;; SERVER: 127.0.0.53
;; WHEN: Tue Jul 25 00:52:21 2023
;; MSG SIZE  rcvd: 44
~> dig @8.8.8.8 github.com

; <<>> DiG 9.18.16 <<>> @8.8.8.8 github.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 54546
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;github.com.			IN	A

;; ANSWER SECTION:
github.com.		40	IN	A	140.82.121.4

;; Query time: 66 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Tue Jul 25 00:52:33 CEST 2023
;; MSG SIZE  rcvd: 55

reddit.com

~> nslookup reddit.com
Server:		127.0.0.53
Address:	127.0.0.53#53

Non-authoritative answer:
Name:	reddit.com
Address: 151.101.129.140
Name:	reddit.com
Address: 151.101.65.140
Name:	reddit.com
Address: 151.101.1.140
Name:	reddit.com
Address: 151.101.193.140
Name:	reddit.com
Address: 2a04:4e42:400::396
Name:	reddit.com
Address: 2a04:4e42:600::396
Name:	reddit.com
Address: 2a04:4e42::396
Name:	reddit.com
Address: 2a04:4e42:200::396
~> getent ahosts reddit.com
2a04:4e42::396  STREAM reddit.com
2a04:4e42::396  DGRAM  
2a04:4e42::396  RAW    
2a04:4e42:600::396 STREAM 
2a04:4e42:600::396 DGRAM  
2a04:4e42:600::396 RAW    
2a04:4e42:400::396 STREAM 
2a04:4e42:400::396 DGRAM  
2a04:4e42:400::396 RAW    
2a04:4e42:200::396 STREAM 
2a04:4e42:200::396 DGRAM  
2a04:4e42:200::396 RAW    
151.101.1.140   STREAM 
151.101.1.140   DGRAM  
151.101.1.140   RAW    
151.101.65.140  STREAM 
151.101.65.140  DGRAM  
151.101.65.140  RAW    
151.101.193.140 STREAM 
151.101.193.140 DGRAM  
151.101.193.140 RAW    
151.101.129.140 STREAM 
151.101.129.140 DGRAM  
151.101.129.140 RAW  
~> drill reddit.com
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 54213
;; flags: qr rd ra ; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0 
;; QUESTION SECTION:
;; reddit.com.	IN	A

;; ANSWER SECTION:
reddit.com.	142	IN	A	151.101.65.140
reddit.com.	142	IN	A	151.101.129.140
reddit.com.	142	IN	A	151.101.193.140
reddit.com.	142	IN	A	151.101.1.140

;; AUTHORITY SECTION:

;; ADDITIONAL SECTION:

;; Query time: 0 msec
;; SERVER: 127.0.0.53
;; WHEN: Tue Jul 25 01:07:52 2023
;; MSG SIZE  rcvd: 92
~> dig @8.8.8.8 reddit.com

; <<>> DiG 9.18.16 <<>> @8.8.8.8 reddit.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 1018
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;reddit.com.			IN	A

;; ANSWER SECTION:
reddit.com.		113	IN	A	151.101.1.140
reddit.com.		113	IN	A	151.101.129.140
reddit.com.		113	IN	A	151.101.193.140
reddit.com.		113	IN	A	151.101.65.140

;; Query time: 20 msec
;; SERVER: 8.8.8.8#53(8.8.8.8) (UDP)
;; WHEN: Tue Jul 25 01:08:11 CEST 2023
;; MSG SIZE  rcvd: 103

Last edited by ollosh (2023-07-25 07:04:39)

Offline

#4 2023-07-25 06:53:21

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Please replace the oversized image with a link, the board has a 250²px² max rule (and ERR_NETWORK_CHANGED is the only relevant information here anyway)

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#5 2023-07-25 07:05:11

ollosh
Member
Registered: 2022-09-20
Posts: 44

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Ah sure didn't kno wabout the embedded images

~> find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
bluetooth.service                        | bluetooth.target.wants
cups.path                                | multi-user.target.wants
cups.service                             | multi-user.target.wants
cups.service                             | printer.target.wants
cups.socket                              | sockets.target.wants
dbus-org.bluez.service                   | system
dbus-org.freedesktop.home1.service       | system
dbus-org.freedesktop.nm-dispatcher.service | system
dbus-org.freedesktop.resolve1.service    | system
dbus-org.freedesktop.timesync1.service   | system
display-manager.service                  | system
gcr-ssh-agent.socket                     | sockets.target.wants
getty@tty1.service                       | getty.target.wants
gnome-keyring-daemon.socket              | sockets.target.wants
NetworkManager.service                   | multi-user.target.wants
NetworkManager-wait-online.service       | network-online.target.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire-media-session.service           | pipewire.service.wants
pipewire-pulse.socket                    | sockets.target.wants
pipewire-session-manager.service         | user
pipewire.socket                          | sockets.target.wants
remote-fs.target                         | multi-user.target.wants
sleep.service                            | suspend.target.wants
sleep.service                            | system
systemd-homed-activate.service           | systemd-homed.service.wants
systemd-homed.service                    | multi-user.target.wants
systemd-resolved.service                 | sysinit.target.wants
systemd-timesyncd.service                | sysinit.target.wants
systemd-userdbd.socket                   | sockets.target.wants

Offline

#6 2023-07-25 07:13:02

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

There're at least no competing network services.
Please post your complete system journal for a boot covering those incidentes, eg.  

sudo journalctl -b -1 | curl -F 'f:1=<-' ix.io

for the previous boot ("-b -1" is what controls that)

Have you ever perceived any related issues outside the browser (firefox?)?
Have you tried the behavior of a different browser?
Are you using DoH?
https://wiki.archlinux.org/title/Domain … -level_DNS

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#7 2023-07-25 08:27:42

ollosh
Member
Registered: 2022-09-20
Posts: 44

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

In the most recent boot I didn't get to experience the issues and I've rebooted immediately so not sure if it'll be relevant
http://ix.io/4Brc

I'm sending -b -2 when I did experience the issues. Unfortunately couldn't push to ix.io most probably cause it's 5mb so sharing a drive link
https://drive.google.com/file/d/111U-UO … phtMB/view


Have you ever perceived any related issues outside the browser (firefox?)?

I'll actually try to curl next time, but I do think I had the same issues. Will keep an eye on this.

Have you tried the behavior of a different browser?

I'll try it again, but I believe I've tried it already recently where firefox would also show the same issues

Are you using DoH?

I've explicitly disabled DoH in all my chrome accounts

Offline

#8 2023-07-25 15:26:41

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Jul 24 09:33:13 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:17:51 makina warp-svc[304549]: 2023-07-24T09:17:51.311Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:17:51 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:17:52 makina warp-svc[304549]: 2023-07-24T09:17:52.230Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:08 makina warp-svc[304549]: 2023-07-24T09:18:08.234Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:08 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:18:09 makina warp-svc[304549]: 2023-07-24T09:18:09.165Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:25 makina warp-svc[304549]: 2023-07-24T09:18:25.937Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:25 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:18:26 makina warp-svc[304549]: 2023-07-24T09:18:26.872Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:43 makina warp-svc[304549]: 2023-07-24T09:18:43.630Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:18:43 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:18:44 makina warp-svc[304549]: 2023-07-24T09:18:44.539Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:00 makina warp-svc[304549]: 2023-07-24T09:19:00.757Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:00 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:19:01 makina warp-svc[304549]: 2023-07-24T09:19:01.668Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:17 makina warp-svc[304549]: 2023-07-24T09:19:17.597Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:17 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:19:18 makina warp-svc[304549]: 2023-07-24T09:19:18.530Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:35 makina warp-svc[304549]: 2023-07-24T09:19:35.354Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:35 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:19:36 makina warp-svc[304549]: 2023-07-24T09:19:36.293Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:50 makina warp-svc[304549]: 2023-07-24T09:19:50.114Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:19:50 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:19:51 makina warp-svc[304549]: 2023-07-24T09:19:51.034Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:06 makina warp-svc[304549]: 2023-07-24T09:20:06.693Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:06 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:20:07 makina warp-svc[304549]: 2023-07-24T09:20:07.640Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:23 makina warp-svc[304549]: 2023-07-24T09:20:23.884Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:24 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:20:24 makina warp-svc[304549]: 2023-07-24T09:20:24.790Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:40 makina warp-svc[304549]: 2023-07-24T09:20:40.738Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:40 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:20:41 makina warp-svc[304549]: 2023-07-24T09:20:41.644Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:55 makina warp-svc[304549]: 2023-07-24T09:20:55.697Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:20:55 makina systemd[1]: Starting Network Name Resolution...
Jul 24 11:20:56 makina warp-svc[304549]: 2023-07-24T09:20:56.614Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 11:21:07 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:01:04 makina warp-svc[425132]: 2023-07-24T10:01:04.657Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:04 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:01:05 makina warp-svc[425132]: 2023-07-24T10:01:05.554Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:23 makina warp-svc[425132]: 2023-07-24T10:01:23.427Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:23 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:01:24 makina warp-svc[425132]: 2023-07-24T10:01:24.372Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:40 makina warp-svc[425132]: 2023-07-24T10:01:40.267Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:40 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:01:41 makina warp-svc[425132]: 2023-07-24T10:01:41.203Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:56 makina warp-svc[425132]: 2023-07-24T10:01:56.410Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:01:56 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:01:57 makina warp-svc[425132]: 2023-07-24T10:01:57.330Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:13 makina warp-svc[425132]: 2023-07-24T10:02:13.520Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:13 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:02:14 makina warp-svc[425132]: 2023-07-24T10:02:14.450Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:29 makina warp-svc[425132]: 2023-07-24T10:02:29.993Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:30 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:02:30 makina warp-svc[425132]: 2023-07-24T10:02:30.915Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:46 makina warp-svc[425132]: 2023-07-24T10:02:46.347Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:02:46 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:02:47 makina warp-svc[425132]: 2023-07-24T10:02:47.272Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:02 makina warp-svc[425132]: 2023-07-24T10:03:02.720Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:02 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:03:03 makina warp-svc[425132]: 2023-07-24T10:03:03.654Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:19 makina warp-svc[425132]: 2023-07-24T10:03:19.553Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:19 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:03:20 makina warp-svc[425132]: 2023-07-24T10:03:20.459Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:36 makina warp-svc[425132]: 2023-07-24T10:03:36.583Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:36 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:03:37 makina warp-svc[425132]: 2023-07-24T10:03:37.511Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:54 makina warp-svc[425132]: 2023-07-24T10:03:54.106Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:03:54 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:03:55 makina warp-svc[425132]: 2023-07-24T10:03:55.041Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:10 makina warp-svc[425132]: 2023-07-24T10:04:10.467Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:10 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:04:11 makina warp-svc[425132]: 2023-07-24T10:04:11.382Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:27 makina warp-svc[425132]: 2023-07-24T10:04:27.537Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:27 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:04:28 makina warp-svc[425132]: 2023-07-24T10:04:28.441Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:45 makina warp-svc[425132]: 2023-07-24T10:04:45.557Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:04:45 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:04:46 makina warp-svc[425132]: 2023-07-24T10:04:46.480Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:05:02 makina warp-svc[425132]: 2023-07-24T10:05:02.800Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:05:02 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:05:03 makina warp-svc[425132]: 2023-07-24T10:05:03.708Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:05:10 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:12:15 makina warp-svc[456432]: 2023-07-24T10:12:15.047Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:12:15 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:12:15 makina warp-svc[456432]: 2023-07-24T10:12:15.980Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:12:24 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:15:55 makina warp-svc[466611]: 2023-07-24T10:15:55.523Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:15:55 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:15:56 makina warp-svc[466611]: 2023-07-24T10:15:56.459Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:12 makina warp-svc[466611]: 2023-07-24T10:16:12.993Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:13 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:16:13 makina warp-svc[466611]: 2023-07-24T10:16:13.921Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:28 makina warp-svc[466611]: 2023-07-24T10:16:28.980Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:29 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:16:29 makina warp-svc[466611]: 2023-07-24T10:16:29.937Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:46 makina warp-svc[466611]: 2023-07-24T10:16:46.493Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:16:46 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:16:47 makina warp-svc[466611]: 2023-07-24T10:16:47.430Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:03 makina warp-svc[466611]: 2023-07-24T10:17:03.573Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:03 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:17:04 makina warp-svc[466611]: 2023-07-24T10:17:04.497Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:20 makina warp-svc[466611]: 2023-07-24T10:17:20.646Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:20 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:17:21 makina warp-svc[466611]: 2023-07-24T10:17:21.572Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:38 makina warp-svc[466611]: 2023-07-24T10:17:38.182Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:38 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:17:39 makina warp-svc[466611]: 2023-07-24T10:17:39.082Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:55 makina warp-svc[466611]: 2023-07-24T10:17:55.153Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 12:17:55 makina systemd[1]: Starting Network Name Resolution...
Jul 24 12:17:56 makina warp-svc[466611]: 2023-07-24T10:17:56.071Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 13:43:51 makina systemd[1]: Starting Network Name Resolution...
Jul 24 13:47:19 makina warp-svc[742549]: 2023-07-24T11:47:19.235Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 13:47:19 makina systemd[1]: Starting Network Name Resolution...
Jul 24 13:47:20 makina warp-svc[742549]: 2023-07-24T11:47:20.163Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 13:47:27 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:12:50 makina warp-svc[812476]: 2023-07-24T12:12:50.825Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:12:50 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:12:51 makina warp-svc[812476]: 2023-07-24T12:12:51.753Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:07 makina warp-svc[812476]: 2023-07-24T12:13:07.779Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:07 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:13:08 makina warp-svc[812476]: 2023-07-24T12:13:08.736Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:24 makina warp-svc[812476]: 2023-07-24T12:13:24.575Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:24 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:13:25 makina warp-svc[812476]: 2023-07-24T12:13:25.453Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:43 makina warp-svc[812476]: 2023-07-24T12:13:43.798Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:13:43 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:13:44 makina warp-svc[812476]: 2023-07-24T12:13:44.751Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:01 makina warp-svc[812476]: 2023-07-24T12:14:01.281Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:01 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:14:02 makina warp-svc[812476]: 2023-07-24T12:14:02.225Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:17 makina warp-svc[812476]: 2023-07-24T12:14:17.974Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:18 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:14:18 makina warp-svc[812476]: 2023-07-24T12:14:18.881Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:35 makina warp-svc[812476]: 2023-07-24T12:14:35.281Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:35 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:14:36 makina warp-svc[812476]: 2023-07-24T12:14:36.219Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:51 makina warp-svc[812476]: 2023-07-24T12:14:51.784Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:14:51 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:14:52 makina warp-svc[812476]: 2023-07-24T12:14:52.719Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:09 makina warp-svc[812476]: 2023-07-24T12:15:09.558Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:09 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:15:10 makina warp-svc[812476]: 2023-07-24T12:15:10.480Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:26 makina warp-svc[812476]: 2023-07-24T12:15:26.811Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:26 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:15:27 makina warp-svc[812476]: 2023-07-24T12:15:27.693Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:44 makina warp-svc[812476]: 2023-07-24T12:15:44.318Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:44 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:15:45 makina warp-svc[812476]: 2023-07-24T12:15:45.231Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:59 makina warp-svc[812476]: 2023-07-24T12:15:59.118Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:15:59 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:16:00 makina warp-svc[812476]: 2023-07-24T12:16:00.027Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:15 makina warp-svc[812476]: 2023-07-24T12:16:15.648Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:15 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:16:16 makina warp-svc[812476]: 2023-07-24T12:16:16.582Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:32 makina warp-svc[812476]: 2023-07-24T12:16:32.541Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:32 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:16:33 makina warp-svc[812476]: 2023-07-24T12:16:33.457Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:48 makina warp-svc[812476]: 2023-07-24T12:16:48.638Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:16:48 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:16:49 makina warp-svc[812476]: 2023-07-24T12:16:49.562Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:17:05 makina warp-svc[812476]: 2023-07-24T12:17:05.421Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:17:05 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:17:06 makina warp-svc[812476]: 2023-07-24T12:17:06.352Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:17:22 makina warp-svc[812476]: 2023-07-24T12:17:22.181Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:17:22 makina systemd[1]: Starting Network Name Resolution...
Jul 24 14:17:23 makina warp-svc[812476]: 2023-07-24T12:17:23.107Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 14:17:37 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:15:50 makina warp-svc[1007288]: 2023-07-24T13:15:50.741Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:15:50 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:15:51 makina warp-svc[1007288]: 2023-07-24T13:15:51.644Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:09 makina warp-svc[1007288]: 2023-07-24T13:16:09.201Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:09 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:16:10 makina warp-svc[1007288]: 2023-07-24T13:16:10.113Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:25 makina warp-svc[1007288]: 2023-07-24T13:16:25.697Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:25 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:16:26 makina warp-svc[1007288]: 2023-07-24T13:16:26.634Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:43 makina warp-svc[1007288]: 2023-07-24T13:16:43.037Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:43 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:16:43 makina warp-svc[1007288]: 2023-07-24T13:16:43.946Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:59 makina warp-svc[1007288]: 2023-07-24T13:16:59.454Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:16:59 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:17:00 makina warp-svc[1007288]: 2023-07-24T13:17:00.384Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:15 makina warp-svc[1007288]: 2023-07-24T13:17:15.781Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:15 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:17:16 makina warp-svc[1007288]: 2023-07-24T13:17:16.691Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:29 makina warp-svc[1007288]: 2023-07-24T13:17:29.894Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:29 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:17:30 makina warp-svc[1007288]: 2023-07-24T13:17:30.806Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:46 makina warp-svc[1007288]: 2023-07-24T13:17:46.991Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:17:47 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:17:47 makina warp-svc[1007288]: 2023-07-24T13:17:47.934Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:05 makina warp-svc[1007288]: 2023-07-24T13:18:05.394Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:05 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:18:06 makina warp-svc[1007288]: 2023-07-24T13:18:06.324Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:21 makina warp-svc[1007288]: 2023-07-24T13:18:21.681Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:21 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:18:22 makina warp-svc[1007288]: 2023-07-24T13:18:22.605Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:38 makina warp-svc[1007288]: 2023-07-24T13:18:38.717Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:38 makina systemd[1]: Starting Network Name Resolution...
Jul 24 15:18:39 makina warp-svc[1007288]: 2023-07-24T13:18:39.607Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 15:18:50 makina systemd[1]: Starting Network Name Resolution...
Jul 24 17:18:53 makina warp-svc[1438317]: 2023-07-24T15:18:53.327Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 17:18:53 makina systemd[1]: Starting Network Name Resolution...
Jul 24 17:18:53 makina warp-svc[1438317]: 2023-07-24T15:18:53.670Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:32:49 makina warp-svc[1641106]: 2023-07-24T21:32:49.445Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:32:49 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:32:50 makina warp-svc[1641106]: 2023-07-24T21:32:50.360Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:06 makina warp-svc[1641106]: 2023-07-24T21:33:06.782Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:06 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:33:07 makina warp-svc[1641106]: 2023-07-24T21:33:07.739Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:21 makina warp-svc[1641106]: 2023-07-24T21:33:21.812Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:21 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:33:22 makina warp-svc[1641106]: 2023-07-24T21:33:22.712Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:38 makina warp-svc[1641106]: 2023-07-24T21:33:38.579Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:38 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:33:39 makina warp-svc[1641106]: 2023-07-24T21:33:39.482Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:55 makina warp-svc[1641106]: 2023-07-24T21:33:55.995Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:33:56 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:33:56 makina warp-svc[1641106]: 2023-07-24T21:33:56.899Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:34:14 makina warp-svc[1641106]: 2023-07-24T21:34:14.915Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:34:14 makina systemd[1]: Starting Network Name Resolution...
Jul 24 23:34:15 makina warp-svc[1641106]: 2023-07-24T21:34:15.848Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 24 23:34:22 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:27:19 makina warp-svc[1988112]: 2023-07-24T23:27:19.014Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:27:19 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:27:19 makina warp-svc[1988112]: 2023-07-24T23:27:19.959Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:27:37 makina warp-svc[1988112]: 2023-07-24T23:27:37.714Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:27:37 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:27:38 makina warp-svc[1988112]: 2023-07-24T23:27:38.662Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:27:54 makina warp-svc[1988112]: 2023-07-24T23:27:54.331Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:27:54 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:27:54 makina warp-svc[1988112]: 2023-07-24T23:27:54.971Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:28:10 makina warp-svc[1988112]: 2023-07-24T23:28:10.971Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:28:11 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:28:11 makina warp-svc[1988112]: 2023-07-24T23:28:11.908Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:28:25 makina warp-svc[1988112]: 2023-07-24T23:28:25.625Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"
Jul 25 01:28:25 makina systemd[1]: Starting Network Name Resolution...
Jul 25 01:28:26 makina warp-svc[1988112]: 2023-07-24T23:28:26.568Z DEBUG main_loop: warp::warp_service: Entering main loop arm arm="network_changed"

This looks bad, systemd-resolved constantly restarts and it smells like this is indeed related to warp-svc wich also has a lot of

Jul 24 11:18:32 makina warp-svc[304549]: 2023-07-24T09:18:32.296Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Jul 24 11:18:33 makina warp-svc[304549]: 2023-07-24T09:18:33.298Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Jul 24 11:18:34 makina warp-svc[304549]: 2023-07-24T09:18:34.308Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Jul 24 11:18:34 makina warp-svc[304549]: 2023-07-24T09:18:34.308Z ERROR main_loop: warp::warp::connectivity_check: DNS connectivity check failed to resolve host="warp-svc."
Jul 24 11:18:34 makina warp-svc[304549]: 2023-07-24T09:18:34.308Z  WARN main_loop: warp::warp: Connectivity checks failed, retrying in 1165ms error=Dns

and looks like it basically doesn't work at all?

See whether you ever encounter the original issue w/o warp and then check why the resolution of warp-svc fails (you may have to adjust the config or add a static entry?)

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#9 2023-07-26 11:21:21

ollosh
Member
Registered: 2022-09-20
Posts: 44

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Yeah warp-svc is flakey and there are some people experiencing same issues with it so I assume part of the issues is caused by it. I'll keep following the issue on the package

However, today I haven't used it and there are no entries like that in journal, but here and there I do experience this weird issue. So far I've only seen it in chrome, but I'll keep testing in firefox and figure out if I can find anything about it.
Here's my journal from today http://ix.io/4BwF

EDIT: When posting this answer, it resulted in ERR_NETWORK_CHANGED. It's so random I can't wrap my head around the possible issue

EDIT 2: Ok definitely something strange happening with Chrome since I can't normally open reddit while it works perfectly fine in Firefox and pinging reddit.com results in no packet loss while github.com resulted in ~2% one time and then 0 all the other times so not sure if there's anything to it.

Last edited by ollosh (2023-07-26 11:42:38)

Offline

#10 2023-07-26 13:00:25

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

What is chrome? Docker or flatpak?
There're issues starting xdg-portal-gtk and docker certainly adds A LOT of Veth's…

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#11 2023-07-27 08:01:28

ollosh
Member
Registered: 2022-09-20
Posts: 44

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Tbh I don't know which underlying technology is used I'm pretty sure it's not docker . I'm using this package from aur https://aur.archlinux.org/packages/google-chrome

Offline

#12 2023-07-27 13:00:22

seth
Member
Registered: 2012-09-03
Posts: 52,534

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Looks like it's a debian package.
No idea whether chrome would respond to the virtual device juggling but you could disable and not use docker for a while and see whether chrome misbehaves and then for a cross-test cause a lot of docker activity to see whether it reacts to that.

Otherwise the problem hasn't shown up so far?

How to upload text · How to boot w/o GUI · Disable Windows Fast-Start! · Fix your xinitrc

Offline

#13 2023-08-02 19:35:04

chmodx
Member
Registered: 2023-08-02
Posts: 2

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Just want to chime in with some extra information as I'm experiencing the same problem. 
I'm guessing it's the same because I get identical log messages:

Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z  WARN main_loop: warp::warp::connectivity_check: DNS lookup failed error=Custom { kind: Uncategorized, error: "failed to lookup address information: Name or service not known" } host="warp-svc."
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z ERROR main_loop: warp::warp::connectivity_check: DNS connectivity check failed to resolve host="warp-svc."
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z DEBUG main_loop: warp::warp::tunnel: Trace status: Err(Dns)
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z DEBUG main_loop: warp::warp::warp_connection: start_status=Err(FailedConnectivityCheck(DNSLookupFailed))
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.409Z DEBUG warp_tun::completion::wrapper: Stopping drive_writer
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.410Z DEBUG warp::warp_service::network_change: Routes changed:
Aug 02 17:50:42 archpad warp-svc[21901]: DelNeighbour; Destination: 0.0.0.0;
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.413Z DEBUG warp::warp_service::network_change: Routes changed:
Aug 02 17:50:42 archpad warp-svc[21901]: DelNeighbour; Destination: ff02::1:3;
Aug 02 17:50:42 archpad warp-svc[21901]: DelNeighbour; Destination: ff02::16;
Aug 02 17:50:42 archpad warp-svc[21901]: DelNeighbour; Destination: ff02::2;
Aug 02 17:50:42 archpad warp-svc[21901]: DelNeighbour; Destination: ff02::fb;
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.414Z DEBUG main_loop: warp::warp::dns_recovery::unix: Reverting DNS settings old_dns=RestoreDNS { action: OverwriteResolv([35, 32, 71, 101, 110, 101, 114, 97, 116, 101, 100, 32, 98, 121, 32, 78, 101, 116, 119, 111, 114, 107, 77, 97, 110, 97, 103, 101, 114, 10, 115, 101, 97, 114, 99, 104, 32, 72, 111, 109, 101, 10, 110, 97, 109, 101, 115, 101, 114, 118, 101, 114, 32, 49, 57, 50, 46, 49, 54, 56, 46, 49, 46, 50, 53, 52, 10, 110, 97, 109, 101, 115, 101, 114, 118, 101, 114, 32, 49, 57, 50, 46, 49, 54, 56, 46, 49, 48, 46, 49, 10]), owner: NetworkManager, previous_nameservers: Some([192.168.1.254, 192.168.10.1]), warp_nameservers: [127.0.2.2, 127.0.2.3, fd01:db8:1111::2, fd01:db8:1111::3] }
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.414Z DEBUG main_loop: warp::warp::dns_recovery::unix: DNS settings reverted (<resolv.conf restored>)
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.415Z DEBUG main_loop: firewall: Firewall reset to defaults
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.426Z DEBUG main_loop: firewall::linux: Firewall rules loaded
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.426Z DEBUG main_loop: warp::warp::dns_recovery::unix: Reverting Tunnel settings
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.426Z DEBUG main_loop: warp::warp::dns_recovery::unix: Deleted FWMARK rules table with status: Ok(())
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.426Z DEBUG main_loop: firewall: Firewall reset to defaults
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.502Z DEBUG main_loop: firewall::linux: Firewall rules loaded
Aug 02 17:50:42 archpad warp-svc[21901]: 2023-08-02T15:50:42.503Z  WARN main_loop: warp::warp_service: Tunnel failed to start error=FailedConnectivityCheck(DNSLookupFailed)

However what I've also found is that my /etc/resolv.conf keeps changing between:

# Generated by NetworkManager
nameserver 1.1.1.1
nameserver 1.0.0.1
nameserver 2606:4700:4700::1111
# NOTE: the libc resolver may not support more than 3 nameservers.
# The nameservers listed below may not be recognized.
nameserver 2606:4700:4700::1001

and

# This file was generated by cloudflare-warp.
nameserver 127.0.2.2
nameserver 127.0.2.3
nameserver fd01:db8:1111::2
nameserver fd01:db8:1111::3

It does so every few seconds (5-10).
It seems to coincide with the following journal messages from NetworkManager:

Aug 02 21:25:26 archpad NetworkManager[803]: <info>  [1691004326.2159] device (CloudflareWARP): state change: activated -> unmanaged (reason 'unmanaged', sys-iface-state: 'removed')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6038] manager: (CloudflareWARP): new Tun device (/org/freedesktop/NetworkManager/Devices/709)
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6133] device (CloudflareWARP): state change: unmanaged -> unavailable (reason 'connection-assumed', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6137] device (CloudflareWARP): state change: unavailable -> disconnected (reason 'connection-assumed', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6140] device (CloudflareWARP): Activation: starting connection 'CloudflareWARP' (6d77c5e1-8310-4c43-8a30-73bd39dcc0a9)
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6141] device (CloudflareWARP): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6142] device (CloudflareWARP): state change: prepare -> config (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6143] device (CloudflareWARP): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6144] device (CloudflareWARP): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6185] device (CloudflareWARP): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6187] device (CloudflareWARP): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Aug 02 21:25:31 archpad NetworkManager[803]: <info>  [1691004331.6189] device (CloudflareWARP): Activation: successful, device activated.

So when

device (CloudflareWARP): state change: activated -> unmanaged (reason 'unmanaged', sys-iface-state: 'removed')

appears, my /etc/resolv.conf changes to  the one with

# Generated by NetworkManager

.
And once the other set of log messages appear (5s later) it changes to the one with

# This file was generated by cloudflare-warp.

This seems strange behavior to me.

If I disable warp-svc.service it doesn't change anymore, but persists the one generated by NetworkManager.

Hope it gives someone an idea of how to fix this.

Last edited by chmodx (2023-08-02 19:39:20)

Offline

#14 2023-09-20 12:46:10

msherman13
Member
Registered: 2023-09-20
Posts: 1

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

Same issue here. Exact same symptoms. I will try and dig more but my feeling is that either:

1. Network manager is fighting with warp over dns config
2. Warp-svc is problematic and it keeps registering and deregistering the dns config

Offline

#15 2023-12-01 15:54:11

chmodx
Member
Registered: 2023-08-02
Posts: 2

Re: Flakey DNS with NetworkManager, systemd-resolvd and warp-cli

I found that if I restart systemd-resolved, then I (usually, sometimes I need a couple of tries) keeps the etc/resolv.conf generated by cloudflare-warp.

❯ warp-cli status
Status update: Connecting
Success
❯ sudo systemctl restart systemd-resolved.service
❯ warp-cli status
Status update: Connected
Success
❯ cat /etc/resolv.conf
# This file was generated by cloudflare-warp.
nameserver 127.0.2.2
nameserver 127.0.2.3

So "fixes" the issue.

Offline

Pages: 1

Board footer

Powered by FluxBB