[SOLVED] Delays while accessing servers on the internet IPv6

capthookb · 2024-12-26 14:37:50

Ok, I have a complicated issue with my archlinux, and I'm not sure if the issue is OS's fault or Internet conneciton.
It seems to me like a routing problem. I'm using dual stack, IPv4 and IPv6

When I want to visit a website (I'm using chromium), I get a delay of seconds. Not with all websites and not always.
For example, bbs.archlinux.org is one of those that struggle to visit the first time. After that it is fine.

I have a small script that I'm using to diagnose the problem, using curl:

#!/bin/bash
curl -w "@curl-format.txt" -o /dev/null -s "https:/dynv6.com"
curl -w "@curl-format.txt" -o /dev/null -s "https://bbs.archlinux.org"
curl -w "@curl-format.txt" -o /dev/null -s "https://help.mikrotik.com"

and curl-format.txt is this:

time_namelookup: %{time_namelookup}\n
time_connect: %{time_connect}\n
time_appconnect: %{time_appconnect}\n
time_pretransfer: %{time_pretransfer}\n
time_starttransfer: %{time_starttransfer}\n
time_total: %{time_total}\n

Half an hour ago, I run this script and got these results:

# https:/dynv6.com
time_namelookup: 0.068546
time_connect: 0.113837
time_appconnect: 8.641314
time_pretransfer: 8.641524
time_starttransfer: 8.835134
time_total: 8.836265

# bbs.archlinux.org
time_namelookup: 0.023922
time_connect: 0.069442
time_appconnect: 2.221849
time_pretransfer: 2.221930
time_starttransfer: 2.340663
time_total: 2.432246

# help.mikrotik.com
time_namelookup: 0.031263
time_connect: 0.106612
time_appconnect: 50.947436
time_pretransfer: 50.947633
time_starttransfer: 0.000000
time_total: 50.947755

I run the script again now, in order to put the times in this post and this is what i got:

time_namelookup: 0.051757
time_connect: 0.097299
time_appconnect: 0.162261
time_pretransfer: 0.162368
time_starttransfer: 0.254884
time_total: 0.255172

time_namelookup: 0.012866
time_connect: 0.058626
time_appconnect: 0.114109
time_pretransfer: 0.114224
time_starttransfer: 0.174911
time_total: 0.220081

time_namelookup: 0.013727
time_connect: 0.088614
time_appconnect: 0.176376
time_pretransfer: 0.176516
time_starttransfer: 0.251826
time_total: 0.251888

A minute ago, while writhing this topic, I clicked on the Preview button and it timed out. I had to refresh again.
Do you have any idea of what could be the issue? Is it my network connection's fault?
Something else?

Of course while doing those tests, my bandwidth is at idle. I am not downloading or uploading anything.
How can I pinpoin the problem?

Last edited by capthookb (2025-01-10 20:01:56)

-thc · 2024-12-26 23:31:48

While this looks like a serious DNS problem lets check the basics first:

ip a
ip r
ip -6 r
ls -lh /etc/resolv.conf
cat /etc/resolv.conf
find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-45s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

seth · 2024-12-26 23:49:29

I'm not sure if the issue is OS's fault or Internet conneciton.

Are you sure it's DNS/the dual stack?
Does the problem go away when you disable IPv6?
Is strict IP access immune?
Do you get timeouts when running

ping google.com

but not for

ping 8.8.8.8

or even

ping -n google.com # no reverse lookups of the response

?
Does it help to set your DNS server to 8.8.8.8 or 1.1.1.1 (google resp. cloudflare)?

To catch the usual error, please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

capthookb · 2024-12-27 17:15:43

Thank you for the responses guys.

I don't think it is a DNS problem, since the name resolution is fast. I have a Pi-Hole dns server on my router, which is reachable at 10.0.0.2.
At the same time, the Upstream DNS Servers on PiHole are ipv4 and ipv6 dns servers of google and cloudflare

$ ip a                                                                                                                                                                   [19:06:59]
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 48:..:..:..:..:53 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.10/24 brd 192.168.1.255 scope global noprefixroute wlan0
       valid_lft forever preferred_lft forever
    inet6 2a02:...:....:....:....:....:c5c2:2d28/64 scope global dynamic noprefixroute
       valid_lft 2591826sec preferred_lft 604626sec
    inet6 2a02:...:....:....:....:....:ed73:f3a/64 scope global deprecated dynamic noprefixroute
       valid_lft 2591826sec preferred_lft 0sec
    inet6 fe80::9e3f:9990:c2fa:ce7b/64 scope link noprefixroute
       valid_lft forever preferred_lft forever

user@USB-arch: ~
$ ip r                                                                                                                                                                   [19:06:59]
default via 192.168.1.1 dev wlan0 proto static metric 600
192.168.1.0/24 dev wlan0 proto kernel scope link src 192.168.1.10 metric 600

user@USB-arch: ~
$ ip -6 r                                                                                                                                                                [19:07:03]
2a02:...:....:2200::/64 dev wlan0 proto ra metric 600 pref medium
2a02:...:....:8600::/64 dev wlan0 proto ra metric 600 pref medium
fe80::/64 dev wlan0 proto kernel metric 1024 pref medium
default via fe80::1afd:74ff:fefb:7236 dev wlan0 proto ra metric 600 pref medium

user@USB-arch: ~
$ ls -lh /etc/resolv.conf                                                                                                                                                [19:07:08]
-rw-r--r-- 1 root root 78 Δεκ  27 19:04 /etc/resolv.conf

user@USB-arch: ~
$ cat /etc/resolv.conf                                                                                                                                                   [19:07:13]
# Generated by NetworkManager
nameserver 10.0.0.2
nameserver fd9f:3416:d29a::

user@USB-arch: ~
$ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-45s | %s\n", $(NF-0), $(NF-1)) }' | sort -f                                             [19:07:20]
bluetooth.service                             | bluetooth.target.wants
cups.path                                     | multi-user.target.wants
cups.service                                  | multi-user.target.wants
cups.service                                  | printer.target.wants
cups.socket                                   | sockets.target.wants
dbus-org.bluez.service                        | system
dbus-org.freedesktop.nm-dispatcher.service    | system
dbus-org.freedesktop.timesync1.service        | system
disable-USB-wakeup.service                    | multi-user.target.wants
display-manager.service                       | system
fstrim.timer                                  | timers.target.wants
getty@tty1.service                            | getty.target.wants
ipp-usb.service                               | multi-user.target.wants
named.service                                 | multi-user.target.wants
NetworkManager.service                        | multi-user.target.wants
NetworkManager-wait-online.service            | network-online.target.wants
p11-kit-server.socket                         | sockets.target.wants
pipewire-pulse.socket                         | sockets.target.wants
pipewire-session-manager.service              | user
pipewire.socket                               | sockets.target.wants
plocate-updatedb.timer                        | timers.target.wants
remote-fs.target                              | multi-user.target.wants
rpcbind.service                               | multi-user.target.wants
sshd.service                                  | multi-user.target.wants
systemd-timesyncd.service                     | sysinit.target.wants
wireplumber.service                           | pipewire.service.wants
xdg-user-dirs-update.service                  | default.target.wants

The delays are presented when the application tries to connect to the http server.
Seth, no I'm not sure if it is dual stack's fault. This is why I'm asking here.

-thc · 2024-12-27 17:44:55

Your config looks O.K. - but there is a second nameserver line

capthookb wrote:

$ cat /etc/resolv.conf                                                                                                                                                   [19:07:13]
# Generated by NetworkManager
nameserver 10.0.0.2
nameserver fd9f:3416:d29a::

that points to an IPv6 ULA address space that your host isn't part of. Where does it come from? DHCP?

capthookb · 2024-12-29 10:19:54

The second nameserver is the local-static IPv6 address of the host running PiHole.

-thc · 2024-12-29 10:33:24

Yes - where does it come from? Did you configure your "wlan0" connection yourself? Is it configured via DHCP?

nameserver fd9f:3416:d29a::

Using this DNS server entry on your machine makes absolutely no sense. And if it's acquired via DHCP other boxes may suffer as well.

Even if it's not related to the cause of your problem: Remove this DNS entry (at least on your machine).

capthookb · 2024-12-29 17:00:00

The DNS server in IPv6 is configured to be "advertised" through the ND settings (Neighbour Discovery). No DHCP is used.
For IPv4, there is a DHCP but I'm using static address, and static DNS settings (10.0.0.2) as it is depicted at the /etc/resolv.conf file.
Well, it makes sense to have another dns server, that is reachable through IPv6. How should the following command work otherwise?

$ dig -6 bbs.archlinux.org

Anyway, the problem is NOT related to DNS name resolution. The time required to resolve a hostname is 10msec the first time, and thereafter it is 3msec at the most.

seth · 2024-12-29 20:27:31

seth wrote:

Does the problem go away when you disable IPv6?
Is strict IP access immune?

And to challenge your assertion a bit, does it go away when using a local DNS cache (*sigh* - resolved)?

-thc · 2024-12-29 21:33:11

capthookb wrote:

The DNS server in IPv6 is configured to be "advertised" through the ND settings (Neighbour Discovery). No DHCP is used.

O.K. - I know that as "SLAAC with RDNSS" - it's just ominous.

capthookb wrote:

How should the following command work otherwise?
$ dig -6 bbs.archlinux.org

O.K. - I checked that scenario. An IPv6 host can reach ULA addresses even if itself has no such ULA address. It's just weird then.

Last edited by -thc (2024-12-29 21:52:13)

capthookb · 2024-12-31 17:16:27

Dear -thc and seth, thank you for your efforts to help me.

Right now I'm playing with settings in gai.conf .But the thing is that it does not happen every time. Mostly the first time. It is intermittent.
The Issue has not been resolved.
I tried tracerouting both with -4 and -6 and the -6 seems faster.
I want to try the following:

Connect via my 5G phone
Try to see if the problem exists in Windows as well (dual boot). I have another laptop for my work, and using the same internet connection (Widnows 11) the pages load instantly.
Try some other linux distributions in VirtualBox and see what happens there

Another thing that I'm noticing, and I believe that is due to the same issue, is that when I'm trying to do a speedtest (ookla), the first time, the process of finding optimal server takes a long time and it is not instant. The speed however max outs my VDSL line.

I just wish I could find a metric/way to reproduce the problem every time in order to identify what's wrong.
I'm wishing a happy new year to everyone!
May 2025 have fewer bugs and problems

seth · 2024-12-31 19:43:40

Try to see if the problem exists in Windows as well (dual boot)

3rd link below. Mandatory.
Disable it (it's NOT the BIOS setting!) and reboot windows and linux twice for voodo reasons.

capthookb · 2025-01-10 20:01:23

seth wrote:

Try to see if the problem exists in Windows as well (dual boot)
3rd link below. Mandatory.
Disable it (it's NOT the BIOS setting!) and reboot windows and linux twice for voodo reasons.

Seth thank you but that is irrelevant. In fact I didn't even boot to windows to try. As a matter of fact I hardly ever boot to windows.
Nonetheless I did manage to solve the problem.

What I did was to dig a little more into the situation. I wrote a python script to measure the time required to connect to a "good" site and some "bad" sites, at the same time, in order to avoid any network congestion reasons. The script tried to download the first page of each site, using IPv4, IPv6 and automatic mode every 5 minutes.
This is what I found out:

https://google.gr
Whether ipv4 or ipv6 there were hardly any problems, which I knew. Consider it like a baseline
https://ibb.co/gdfvZ0r
https://ibb.co/1KCb7MK
https://ibb.co/h2BBWB7

https://bbs.archlinux.org
It had problems only with IPv6
https://ibb.co/t8F50j1
https://ibb.co/syjJhY0
https://ibb.co/wr7ghGy

htts://help.mikrotik.com
Problems as well only with IPv6. Perhaps bigger issues. There are some negative values in the graph. These were OpenSSL SSL_read: SSL_ERROR_SYSCALL errors (pycurl)
https://ibb.co/qm6v6yD
https://ibb.co/xM0r2db
https://ibb.co/D8nx518

The issue is clear. When using IPv6 some of the requests were timing out to 60sec, which was the timeout I had set in my script.
But with google I had no issues!

Then I started a chat with chatgpt and It suggested that I look the MTU values of my router interfaces.
And then I came accross a Greek forum suggesting a solution to I believe a similar issue:
https://www.adslgr.com/forum/threads/12 … ost7203506
I look my router mtu of pppoe-client and it was set to auto, but the actual value was around 1410-1420.
This caused some packets of IPv6 to be fragmented and dropped. However IPv4 was ok, since IPv4 packets are smaller than IPv6 and no issues were caused.
To fix that the user posted a solution of adding an firewall rule to the IPv6 router (mikrotik) on the mangle table. The rule will make sure that IPv6 packets are no longer than what is should be, respecting the MTU value.

And bang!!!!
All sites now are blazing fast.

https://google.gr
https://ibb.co/XS5fZMv
https://ibb.co/McVTkkT
https://ibb.co/gJyvZZY

https://bbs.archlinux.org
https://ibb.co/RgwXcYs
https://ibb.co/vLPsWmw
https://ibb.co/BqtcgTJ

htts://help.mikrotik.com
https://ibb.co/1KPcZCb
https://ibb.co/tJbZLFw
https://ibb.co/qdhNQCN

The data collected is only within a few hours, but I do not need to gather more.
The problem is solved!
And I can feel the speed even in sites with IPv4 only. For example when I was trying to do a speedtest with ookla, it was taking too long to find the best server.
But now in 1-2 seconds the server is chosen and the speedtest is instant!

Thank you guys for your input. I appreciate it.

Last edited by capthookb (2025-01-12 11:05:31)

seth · 2025-01-10 21:12:47

However IPv4 was ok, since IPv4 packets are smaller than IPv6 and no issues were caused.

No. Which is also why

And I can feel the speed even in sites with IPv4 only.

The packages are fragmented as defined by the local MTU (defaults to 1500), ICMP will typically be much smaller, everyhing else rather not.
Everytime larger packages don't make it through they have to be re-send in smaller fragments.

MTU mismatches are a frequent issue w/ VPNs (since they add an overhead) but usually™
- they will hit you across the board
- also cause a complete failure of ssl connections, while
- pings remain completely unaffected
"PPPoE" would have been a good hint because it adds 8 bytes and the maximum MTU becomes 1492 (which is exceeded by the posted ip output)

I look my router mtu of pppoe-client and it was set to auto, but the actual value was around 1410-1420.

Also means the issue is likely

For IPv4, there is a DHCP but I'm using static address, and static DNS settings (10.0.0.2) as it is depicted at the /etc/resolv.conf file.

because the router should™ tell your system to lower the MTU via dhcp otherwise?

Well, we all learned in this thread

Please replace the oversized images w/ links and just for the records:
interference from a hibernating (what fast-start actually is) windows is never irrelevant and causes the most obscure issues across the board (though mostly radio)
I'm not saying that to hate on windows, I'm saying that because it's a link in my signature.

-thc · 2025-01-11 07:32:55

seth wrote:

Also means the issue is likely [...] because the router should™ tell your system to lower the MTU via dhcp otherwise?

AFAIK there are only (uncommon/advanced) DHCP options for setting the MSS window size - which is thematically related but not the MTU itself.

"MTU path discovery" relies completely on ICMP messaging between your machine and all route hops and the target.

There is a second contributing factor with VPN providers here: Some of them consider ICMP a non-acceptable security risk and block it completely. This breaks "MTU path discovery" and misleads your machine about the "correct" MTU size.

In all my years with the Linux netfilter subsystem I can also add:

The netfilter is extremely reliable in (connection tracking-) related ICMPv4 traffic. Allowing that and blocking only unsolicited ICMPv4 traffic will not hamper other communications.

IPv6 on the other hand is quite "ICMP-chatty" and netfilter categorizes some of them as unsolicited. For the time being do not block ICMPv6 at all. Since local machines may also use their globally routeable IPv6 address (2000::/3) for local communication and "by design" there is no longer a "NAT inside" (192.168.x.y) filtering ICMPv6 traffic is quite challenging.

Arch Linux

#1 2024-12-26 14:37:50

[SOLVED] Delays while accessing servers on the internet IPv6

#2 2024-12-26 23:31:48

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#3 2024-12-26 23:49:29

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#4 2024-12-27 17:15:43

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#5 2024-12-27 17:44:55

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#6 2024-12-29 10:19:54

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#7 2024-12-29 10:33:24

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#8 2024-12-29 17:00:00

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#9 2024-12-29 20:27:31

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#10 2024-12-29 21:33:11

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#11 2024-12-31 17:16:27

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#12 2024-12-31 19:43:40

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#13 2025-01-10 20:01:23

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#14 2025-01-10 21:12:47

Re: [SOLVED] Delays while accessing servers on the internet IPv6

#15 2025-01-11 07:32:55

Re: [SOLVED] Delays while accessing servers on the internet IPv6

Board footer