Slow download speeds with wget, firefox and browsers

xenobro · 2023-11-28 02:46:48

I get 190~200Mbps dowload and 100Mbps upload on fast and speedtest-cli, and +20MB/sec downloads in pacman and curl.
But the problem is that in wget, firefox or any other browser, I consistently get either 500KB/sec or up to 11.5MB/sec downloads.
To test it, I've used the largest file (1.4GB) on the fastest arch mirror I could get for my region:

http://archlinux.c3sl.ufpr.br/extra/os/ … kg.tar.zst

curl and pacman ALWAYS get +20MB/sec, while wget and firefox ALWAYS get the speeds mentioned.
An interesting thing is that after testing it sometimes, wget/firefox dl speeds seem to alternate between up to 11.5MB/sec and up to 500KB/sec.

I should mention that I'm using an extremely light wm (evilwm) and that these things happen even on console, so it isn't a question of wm bloat slowing down the dl speeds.
I've also stopped testing using that server constantly to avoid abusing it.

Anyway, here are the results of some commands from this script by seth:

[x@archlinux ~]$ ./abbs net show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 98:83:89:fd:b9:f4 brd ff:ff:ff:ff:ff:ff
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 80:b6:55:b6:fb:0c brd ff:ff:ff:ff:ff:ff
    altname wlp0s20f3
    inet 192.168.15.X/24 brd 192.168.15.255 scope global dynamic noprefixroute wlo1
       valid_lft 10632sec preferred_lft 8832sec
    inet6 X:X:X:X/64 scope global dynamic mngtmpaddr noprefixroute 
       valid_lft 43176sec preferred_lft 43176sec
    inet6 fe80::9a34:1b28:28ca:181a/64 scope link 
       valid_lft forever preferred_lft forever
   ---
default via 192.168.15.1 dev wlo1 proto dhcp src 192.168.15.X metric 3003 
192.168.15.0/24 dev wlo1 proto dhcp scope link src 192.168.15.X metric 3003 
   ---
95.217.163.246  STREAM archlinux.org
95.217.163.246  DGRAM  
95.217.163.246  RAW    
   ---
2a01:4f9:c010:6b1f::1 STREAM archlinux.org
2a01:4f9:c010:6b1f::1 DGRAM  
2a01:4f9:c010:6b1f::1 RAW    
   ---
./abbs: linha 130: nslookup: comando não encontrado
   ---

[x@archlinux ~]$ ./abbs net show resolv
Global
           Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
    resolv.conf mode: foreign
  Current DNS Server: 192.168.15.1
         DNS Servers: 192.168.15.1
Fallback DNS Servers: 1.1.1.1#cloudflare-dns.com 9.9.9.9#dns.quad9.net 8.8.8.8#dns.google 2606:4700:4700::1111#cloudflare-dns.com
                      2620:fe::9#dns.quad9.net 2001:4860:4860::8888#dns.google

Link 2 (enp2s0)
    Current Scopes: none
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported

Link 3 (wlo1)
    Current Scopes: LLMNR/IPv4 LLMNR/IPv6 mDNS/IPv4 mDNS/IPv6
         Protocols: -DefaultRoute +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
   ---
-rw-r--r-- 1 root root 92 nov 27 18:10 /etc/resolv.conf
   ---
# Generated by resolvconf
nameserver 192.168.15.1
nameserver fe80::c23d:d9ff:fe62:9d88%wlo1

[x@archlinux ~]$ sudo ./abbs net show dhcp
sudo é /usr/bin/sudo
nmap é /usr/bin/nmap
Starting Nmap 7.94 ( https://nmap.org ) at 2023-11-27 23:19 -03
Pre-scan script results:
| broadcast-dhcp-discover: 
|   Response 1 of 1: 
|     Interface: wlo1
|     IP Offered: 192.168.15.42
|     DHCP Message Type: DHCPOFFER
|     Server Identifier: 192.168.15.1
|     IP Address Lease Time: 1m00s
|     Renewal Time Value: 30s
|     Rebinding Time Value: 52s
|     Broadcast Address: 192.168.15.255
|     Subnet Mask: 255.255.255.0
|     Router: 192.168.15.1
|     Domain Name Server: 192.168.15.1
|_    NetBIOS Name Server: 192.168.15.1
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 10.11 seconds

I should mention that the IP ending in 42 above is not my own.

[x@archlinux ~]$ ./abbs net speed

1) Newark, USA                       4) Frankfurt, Deutschland           7) Sidney, Australia               10) मुंबई (Bombay), Bhārat (India)
2) Singapore                         5) Dallas, USA                      8) Atlanta, USA                    11) Fremont, USA
3) London, UK                        6) Toronto, Canada                  9) 東京都 (Tokyo), 日本国 (Japan)

Enter location: 5

1) 100MB
2) 1GB

Select size: 1

/dev/null                            100%[=====================================================================>] 100,00M   697KB/s    em 2m 24s  
2023-11-27 23:29:59 URL:http://speedtest.dallas.linode.com/100MB-dallas.bin [104857600/104857600] -> "/dev/null" [1]

I should also mention that none of those locations is even remotely close to me - I'm in Brazil. And since the script uses wget, the performance is abysmal as expected.
Retaking the test at

https://speedtest.sao-paulo.linode.com/

Without using wget or explicitly telling Firefox to download anything, the results are pretty much perfect: 20.20ms ping, 213.49Mbps download and 99.01Mbps upload - even higher than my nominal 200Mbps dl and 100Mbps ul.
But if I try to download anything anywhere, it alternates between 500KB/sec and up to 11.5MB/sec - ALWAYS, while curl and pacman get +20MB/sec consistently.

Also, anyone has any idea how can I test the performance inside the LAN to make sure this isn't the problem, since it only manifests itself with wget, firefox and browsers?

Finally, any idea as to what could be causing this apparent speed cap?
Thank you very much,
xenobro.

Last edited by xenobro (2023-11-28 02:49:11)

Head_on_a_Stick · 2023-11-28 08:32:40

Does disabling IPv6 make things better? https://stackoverflow.com/questions/940 … ttps-pages

xenobro · 2023-12-03 20:33:35

Hi, Head_on_a_Stick and thank you very much for your help!

Sorry for taking so long to reply, it was a long week... Anyway, you were right on the money:
Downloading the 1GB test files from this site, every single program - wget, firefox, curl, etc. - got 18~20MB/sec or more, but then trying to dl anything from any good mirror near me or even pretty much any file on a number of sites I've visited would get the usual results: 500~900KB/sec on wget and firefox, 20MB/sex or more on curl.
Then I followed your suggestion and after this:

# sysctl -w net.ipv6.conf.all.disable_ipv6=1

Downloads in wget and firefox started getting the same speeds as curl and pacman.
So it is really IPv6. What should I do then? Always turn down IPv6 for those sites?

rtorrent seems to have its own thing, with speeds rising rapdily to 18~20MB/sec until close to the end of the file, seemingly pausing for a few seconds, then coming crashing down to 5~7MB/sec where it stays until the end of the downloads.
No idea why.

Thank you very much,
xenobro.

seth · 2023-12-03 22:24:02

https://wiki.archlinux.org/title/IPv6#P … _over_IPv6
But indeed uncomment the entire precendence block above and alter the last line instead.
The https://man.archlinux.org/man/core/man- … precedence manpage is very explicit about this.

xenobro · 2023-12-08 21:14:58

Hi, seth, and thank you vvery much for your help.

I've changed /etc/gai.conf and rebooted as you've mentioned, and since everything seemed better, I was about to mark this thread as solved, when the download speeds started oscillating like this:

IPv6 ON:
------------
https://testfile.org/file-1GB
     firefox +20MB/s
     wget    +20MB/s
     curl     +20MB/s

https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst
     firefox 500KB/s
     wget    500KB/s
     curl      500KB/s-5MB/s
     pacman  5MB/s-10MB/s

IPv6 OFF (/etc/gai.conf):
------------------------
https://testfile.org/file-1GB
     firefox +20MB/s or oscillates between 02-10MB/s, averaging on 7MB/s
     wget    +20MB/s or oscillates between 01-05MB/s, averaging on 4MB/s
     curl    +20MB/s or oscillates between 01-12MB/s, averaging on 5MB/s

https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst
     firefox 07MB/s (Starts high at 17MB/s, falls to 5MB/s and then oscillates between 5-12MB/s, averaging on 07MB/s)
     wget    11MB/s (Starts high at 20MB/s, falls to 5MB/s and then oscillates between 5-20MB/s, averaging on 11MB/s)
     curl    11MB/s (Starts high at 16MB/s, falls to 5MB/s and then oscillates between 6-12MB/s, averaging on 11MB/s)
     pacman  11MB/s (Starts high at 20MB/s, falls to 5MB/s and then oscillates between 9-12MB/s, averaging on 11MB/s)

https://releases.ubuntu.com/22.04.3/ubuntu-22.04.3-desktop-amd64.iso
     firefox 3-9MB/s, average 5MB/s
     wget 800KB/s-13MB/s, no average - constantly going to the max speed, down to the minimum and up to the max again
     curl 800KB/s-10MB/s, no average - constantly going to the max speed, down to the minimum and up to the max again

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-12.2.0-amd64-kde.iso
     firefox 1-7MB/s, no average
     wget 1-7MB/s, no average
     curl 1-7MB/s, no average

And on Windows, testfile.org gets 20MB/s or more constantly, ubuntu and debian get 17MB/s or more constantly and the archlinux server gets 4-7MB/s for some reason.
I had to test an archlinux source since the oscillation is also happening with pacman.
Anyway, there's still a very clear improvement from a few KB/s with IPv6 on to MB/s with it off or with its precedence set to 100, but the oscillation is way too big to be normal.
These last few days I've been trying to figure out if the problem was with my ISP/connection or arch.

Anybody knows what's happening or what I should do here?
Thank you very much,
xenobro.

seth · 2023-12-08 21:34:26

Same oscillation when you completely disable IPv6?

Edit, sanity check:
Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Last edited by seth (2023-12-08 21:37:38)

xenobro · 2023-12-08 22:30:10

No, as I detailed in the last post, with IPv6 on it oscillated between 500KB/s and 5MB/s, many times staying with 500KB/s until the end. With IPv6 off it usually start high at 17MB/s, dips to 2MB/s and then goes up and down throughout, but it's usually averages between 5-7MB/s, so it's still much better than with IPv6 on, though Debian got around 1-5MB/s.

[x@archlinux ~]$ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
bluetooth.service                        | bluetooth.target.wants
ctrl-alt-del.target                      | system
dbus-org.bluez.service                   | system
dbus-org.freedesktop.resolve1.service    | system
dbus-org.freedesktop.timesync1.service   | system
getty@tty1.service                       | getty.target.wants
homefiller.service                       | multi-user.target.wants
lm_sensors.service                       | multi-user.target.wants
machines.target                          | multi-user.target.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire.socket                          | sockets.target.wants
pulseaudio.socket                        | sockets.target.wants
remote-cryptsetup.target                 | multi-user.target.wants
remote-fs.target                         | multi-user.target.wants
systemd-boot-update.service              | sysinit.target.wants
systemd-pstore.service                   | sysinit.target.wants
systemd-resolved.service                 | sysinit.target.wants
systemd-timesyncd.service                | sysinit.target.wants

seth · 2023-12-08 22:36:41

I meant sth. along https://wiki.archlinux.org/title/IPv6#Disable_IPv6 to completely disable the stack, not the gai precedence.

There's n network daemon in that list, how do you configure the network?
What if you disable resolved?

xenobro · 2023-12-12 02:05:14

Hi, setth, and thank you very much for your help.

I connect to the internet manually with wpa_supplicant and dhcpcd, or more especifically with this script in /usr/bin/:

[x@archlinux ~]$ cat /usr/bin/wifi
#!/bin/bash

if [ "$1" == "on" ]; then
         wpa_supplicant -B -iwlo1 -c/etc/wpa_supplicant/wpa_supplicant.conf
        dhcpcd wlo1
elif [ "$1" == "off" ]; then
         dhcpcd -k wlo1
        wpa_cli terminate wlo1
else
        echo "wifi: argument needs to be 'on' or 'off'."
fi

Or with "dhcpcd enp2s0" when using an ethernet cable. So I was very surprised to see that network daemon being activated on boot. In fact, I think something changed in the way systemd deals with boot, because there were a number of services in /etc/systemd/{system,user} that not only weren't there before, but there were some that I've disabled before and were now enabled again (ex. the lvm2 related files).
Anyway, I went ahead then and disabled systemd-resolved.service and systemd-timesyncd.service, changed /etc/gai.conf back to default, rebooted and first executed that command again:

[x@archlinux ~]$ find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
bluetooth.service                        | bluetooth.target.wants
ctrl-alt-del.target                      | system
dbus-org.bluez.service                   | system
getty@tty1.service                       | getty.target.wants
homefiller.service                       | multi-user.target.wants
lm_sensors.service                       | multi-user.target.wants
machines.target                          | multi-user.target.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire.socket                          | sockets.target.wants
pulseaudio.socket                        | sockets.target.wants
remote-cryptsetup.target                 | multi-user.target.wants
remote-fs.target                         | multi-user.target.wants
systemd-boot-update.service              | sysinit.target.wants
systemd-pstore.service                   | sysinit.target.wants

Subsequently I redid all tests, first with IPv6 on and then off (with "sysctl -w net.ipv6.conf.all.disable_ipv6=1"). These were the results:

IPv6 ON:
-------
https://testfile.org/file-1GB
        firefox +20MB/s
        wget    +20MB/s
        curl    +20MB/s

https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst
        firefox 500KB/s
        wget    500KB/s
        curl    500KB/s-16MB/s, but mostly between 7 and 11MB/s
        pacman  Starts high at up to 20MB/s, falls progressively as time goes on until 2-3MB/s, averaging on 7-10MB/s

https://releases.ubuntu.com/22.04.3/ubuntu-22.04.3-desktop-amd64.iso
        firefox oscillates between 0 and 7MB/s, download failed once
        wget    oscillates between 0 and 12MB/s, download failed once
        curl    oscillates between 0 and 12MB/s

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-12.2.0-amd64-kde.iso
        firefox oscillates between 0 and 4.5MB/s, averaging on 2MB/s
        wget    oscillates between 0 and 7MB/s, averaging on 2-3MB/s
        curl    oscillates between 0 and 7MB/s, averaging on 2-3MB/s

IPv6 OFF (sysctl -w net.ipv6.conf.all.disable_ipv6=1):
------------------------
https://testfile.org/file-1GB
        firefox +18MB/s (first time 3-7MB/s, once fell until 0KB/s and failed)
        wget    +18MB/s (mostly)
        curl    +18MB/s (but sometimes oscillates down to 11MB/s and back)

https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst
        firefox Starts high at 20MB/s then alternates between 7-10MB/s and +20MB/s
        wget    Starts high at 20MB/s, falls to 5MB/s and then oscillates between 6 and 12MB/s, ocasionally going higher and averaging on 8MB/s
        curl    Starts high at +16MB/s, falls to 6MB/s and then oscillates between 6 and 12MB/s
        pacman  11MB/s (Oscillates between 3 and 20MB/s

https://releases.ubuntu.com/22.04.3/ubuntu-22.04.3-desktop-amd64.iso
        firefox Starts at 11MB/s, falls to 5MB/s then oscillates between 1.5 and 7MB/s
        wget    Oscillates between 800KB/s and 11MB/s
        curl    Oscillates between 500KB/s and 6MB/s

https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-12.2.0-amd64-kde.iso
        firefox oscillates between 2.5 and 5MB/s
        wget    oscillates between 1.5 and 5MB/s
        curl    oscillates between 1.5 and 5MB/s

As you can see, the results were pretty much the same as before.
So what should I do next?

Thank you once again,
xenobro.

seth · 2023-12-12 07:18:04

Is this on the wired or wifi NIC?
If it's on wifi, try the wired connection…

xenobro · 2023-12-13 19:29:22

It happens on both wireless and ethernet, makes no difference.
Oh, and I also tested it again on Windows: The Debian iso gets +16MB/s, Ubuntu's get +14~15MB/s, testfile's get +18MB/s but the arch linux files get lower speeds, going from 500KB/s to 5-7MB/s.
Which is strange, since pacman and curl'ing the arch files used to get 20MB/s constantly.

Last edited by xenobro (2023-12-13 19:34:42)

seth · 2023-12-13 21:03:46

Just to be sure, you've disabled windows fast-start?
Also when downloading anything, make sure to direct it to /dev/null to avoid local IO issues being the bottleneck.

xenobro · 2023-12-14 01:27:00

Yes, I've disabled windows fast-start - in fact, I'm running my system from a pendrive and changed boot order in UEFI: it would be impossible to do that with fast-start active.
And I've run the tests again with IPv6 off, both directing to /dev/null and to my home folder, which is on RAM; the results are pretty much the same: downloading the arch linux files with firefox, wget or curl - these last 2 both to /home/x or /dev/null - it starts at 20MB/s, falls to ~7MB/s, goes up to 14MB/s, down to 10~11MB/s, up and down between 15MB/s and 7~9MB/s and finishes at 23MB/s or 11MB/s. testfile.org gets +20MB/s through and through.
In short, pretty much the same.

What should I do next then?

seth · 2023-12-14 09:20:01

I'm running my system from a pendrive

Big. Red. Flag.
I assume you're not booting windows "from a pendrive"?

Under normal circumstances the non-deterministic behavior (apparently it's *not* related to IPv6 or the specific implementation, curl, wget and FF are all affected the same) and fluctuating throughput would heavily suggest either HW issues (but different NICs are affected) or concurrent NM services knocking each other out (but w/ https://bbs.archlinux.org/viewtopic.php … 3#p2136803 that's not the case either)
There was a massive wifi bug in the 6.6.5 kernel, but the issue isn't specific to that kernel (or wifi) either.

Maybe post your complete system journal for a boot w/ IPv6 supposingly disabled and only using the wired NIC:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

to secure those assertions, but the next thing that is worrysome is blocking IO because of the pendrive.

Please conduct the download tests from a multi-user.target boot only to minize any system load and certainly see https://wiki.archlinux.org/title/Instal … isk_access (and the entire article)

xenobro · 2023-12-19 04:17:16

Hi, seth, and thank you very much once again for your help.

After running multiple tests these days, I may be wrong here, but I think there is more than one issue going on.
First, just to be clear, I already do pretty much everything here.
As you can see below, journal is off:

[x@archlinux ~]$ sudo debugfs -R "features" /dev/sda2
debugfs 1.47.0 (5-Feb-2023)
Filesystem features: ext_attr extent 64bit flex_bg sparse_super large_file huge_file dir_nlink extra_isize metadata_csum

Also my home folder is a tmpfs to which the skeleton files below are copied every boot:

/x/.mozilla/firefox/profiles.ini
/x/.mozilla/firefox/00000000.default/prefs.js
/x/.mozilla/firefox/00000000.default/bookmarkbackups/bookmarks-2023-03-16.json
/x/.mozilla/firefox/installs.ini
/x/.xbindkeysrc
/x/.bash_profile
/x/.bashrc
/x/.rtorrent.rc
/x/.xinitrc
/x/.Xresources

My total RAM is 8GB, and /dev/shm, /tmp and /home are all 3.8GB.

But anyway, when I first ran some preliminary tests a few days ago, I was suddenly getting +20MB/s on testfile and, strangely, 24.1MB/s through and through, nonstop, consistently with the arch zst file - even with wifi and my usual config.
On the following day, I did what you told me and, after changing the default to multi-user.target, including "ipv6.disable=1" in the kernel cmdline with grub-mkconfig and rebooting, I booted to console as usual and connected to net by cable and this is where I sent the journal. Next I executed the script below three times in a row, with a few minutes between each test:

[x@archlinux ~]$ cat dltst
#!/bin/bash

# sudo systemctl get-default
# sudo systemctl set-default multi-user.target
# sudo systemctl reboot

# sudo nano/etc/default/grub, GRUB_CMDLINE_LINUX='ipv6.disable=1', grub-mkconfig -o /boot/grub/grub.cfg
# OR sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1

# sudo journalctl -b | curl -F 'file=@-' 0x0.st
# http://0x0.st/HYua.txt

# https://testfile.org/
echo "*TESTFILE"
wget https://testfile.org/file-1GB -O /dev/null
curl -L https://testfile.org/file-1GB -o /dev/null
# curl -L to follow redirects
echo

# Arch Linux example file (0ad-data-a26)
echo "*ARCH"
wget https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst -O /dev/null
curl https://archlinux.c3sl.ufpr.br/extra/os/x86_64/0ad-data-a26-1-any.pkg.tar.zst -o /dev/null
echo

# Ubuntu 23.10.1 Desktop
echo "*UBUNTU"
wget https://releases.ubuntu.com/23.10.1/ubuntu-23.10.1-desktop-amd64.iso -O /dev/null
curl https://releases.ubuntu.com/23.10.1/ubuntu-23.10.1-desktop-amd64.iso -o /dev/null
echo

# Debian Live KDE
echo "*DEBIAN"
wget https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-12.4.0-amd64-kde.iso -O /dev/null
curl -L https://cdimage.debian.org/debian-cd/current-live/amd64/iso-hybrid/debian-live-12.4.0-amd64-kde.iso -o /dev/null
# "curl -L" to follow redirects
echo

Every single time I got precisely the same results, so I startx'ed into my wm (evilwm) and repeated the tests - with wget and curl as before, but now also including firefox, and once again, exactly the same results:

*testfile.org:18-21MB/s (avg. 20MB/s)
*arch zst ile: 24.1MB/s through and through
*ubuntu iso: 16-19MB/s (avg. 18MB/s)
*debian kde iso: 14-17MB/s (avg. 15, 16MB/s)

I then rebooted back into graphical.target, and stil got the same speeds.
The next day, still wired and using the same initial config you suggested, testfile and the arch zst still got the same speeds, but ubuntu and debian speed had fell a bit.
Disconnecting the cable and using wifi however, the difference in speeds was very clear: testfile didn't change with wget, but curl got 6-10MB/s; arch zst got 14-20MB/s, averaging on 14, both with wget and curl; ubuntu and debian oscillated between 0 and 10MB/s; firefox got about the same.
Downloading to /dev/null, /home or the pendrive had no clear difference in speed wired or wireless, except that, in both cases using firefox, "a few seconds remain" would crash from 18 seconds to a minute and a half when downloading to the pendrive.
Today all speeds were 100% again, wired and wireless (arch zst +20MB/s on wifi), except ubuntu that oscillated sometimes and was in general a bit slower.

So while these didn't solve entirely the issue, turning off ipv6 really raised the speeds.
But the most interesting thing is how consistent is the behaviour of the oscillation/low speeds:
1. testfile pretty much ALWAYS* has +20MB/s;
2. arch zst files now ALWAYS have 24.1MB/s through and through, thousands of times wired; but can oscillate between 14-22MB/s (or worse) wireless;
3, Ubuntu and Debian can oscillate even wired, but will ALWAYS be worse on wireless - usually a LOT worse;
*The very first time you run the test can sometimes oscillate/have low speeds, but then you can test hundreds of times under different circunstances and you'll get 100%. Also there's always a message like error 301 or 206 at th beginning of every test with wget, but that never has any effect.

So there seems to be the ipv6 issue, the wireless issue and finally, as you've mentioned before, possibly the pendrive IO issue.
I have a backup that hasn't been updated in the last month. Should I run the tests from it?
Or should I clone my system on a new pendrive and run the tests from it?
Or yet should I start from scratch in a new pendrive to run the tests there?

Also, what should I do about the random bad performance with wifi?
Once again thank you very much,
xenobro.

seth · 2023-12-19 15:13:56

testfile and the arch zst still got the same speeds, but ubuntu and debian speed had fell a bit

Unless there're meaningful differences in routing on your side (IPv6/4 or completely different route to the target IP) that's down to the targeted host, I suggest to focus on a single, speedtest oriented destination per IP stack (4/6)

Re-installing on the (same) pendrive is only gonna make matters worse as its performance will only degrade with use.

As for the wifi nic, please post your complete system journal for a boot after using it for some speedtests:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

xenobro · 2023-12-27 02:32:27

Hi, seth.

It took me a while to answer because I've had a lot of instability in my internet connection lately, which made it difficult to make accurate measurements and zero in on where the issues are, but eventually I managed to do it, I think.
To see if it was an issue with a faulty pendrive as you mentioned, I've repeated the tests on a backup of my system that hasn't been updated in a while and also with the latest arch linux iso - the former on an entirely different model and brand of pendrive (HP x795w) and the latter on another pendrive of the same model and brand (Sandisk Ultrafit), all USB 3,0 pendrives connected on USB 3.0 ports and confirmed to be fucntioning as USB 3.0 with "lsusb -t". The results were the same as on my system:
1. testfile.org gets +20MB/s almost* always, wired or over wifi, through and through;
2. arch zst gets 24.1MB/s all the way with no variation** wired and 20-24MB/s wireless;
Ubuntu and debian isos are WAY more affected by wireless and network issues:
3. Ubuntu's speeds are 16-18MB/s when it's good and 6-8MB/s when bad wired, and 10-12MB/s wireless when good but oscillating between 0 and 6-8MB/s wirelessly when bad constantly.
4. Debian's speeds are 14-16MB/s when it's good and 6-8MB/s when bad wired and 10-12MB/s wireless whan good but oscillating between 0 and 2-4MB/s wirelessly when bad constantly.
*The first time it usually gets 6-8MB/s, then it's 20MB/s all the way, except for the occasional drop to 12-14 for 3 or 4 seconds then back to 20MB/s;
**Arch zst will get 500KB/s-1.5MB/s randomly but very consistently, from beginning to end of the tests, but that is an issue with my ISP or the Arch mirror in question, affecting even Windows and Android devices when it happens;

This journal was created after testing got the following results:
Wireless: testfile 20MB/s, arch zst 0-1.5MB/s*, ubuntu 0-4.5MB/s, debian live kde 0-3.5MB/s
Wired: testfile 20MB/s, arch zst 0-1.5MB/s*, ubuntu 10MB/s, debian 7MB/s
*The server/ISP issue mentioned above;

I've also tested the speeds under Windows 11, in this same computer on SSD and with Firefox and got the following results:
1. testfile wired or not :20M-23MB/s;
2. arch zst wired or not: 100-300KB/s*
3.ubuntu iso: 10-12MB/s wired and 7-8MB/s wireless
But here is the strangest thing
4. debian iso: 20MB/s wired and 20MB/s** wireless
These while debian wireless on Linux it was 0-4MB/s;
*That arch mirror/ISP issue mentioned before;
**It oscillated between 0 and 20MB/s, but it was mainly 20MB/s;

In short, testfile and arch zsts don't seem to be really affected by the wireless issue, but ubuntu and debian isos very much so.
Oh, ipv6 off seems to help with the arch mirror on Linux, not the other downloads or Windows speeds.
Any ideas on what could be happening and how to proceed?

Also, I've asked in the previous message if it would be a good idea to test cloning my system to ANOTHER new USB3 pendrive, though it doesn't seem so right now.

Once again thank you very much,
xenobro.

seth · 2023-12-27 22:01:33

seth wrote:

I suggest to focus on a single, speedtest oriented destination per IP stack (4/6)

You already figures that there're issues w/ some OS/NIC unrelated arch mirror and the distro servers might produce random results because of random server loads.
Ideally you'd test inside your LAN using eg. iperf, but if that's not possible you want to pick a remote host that you can expect to be as reliable and consistent as possible.

testfile.org seems to produce consistent results across the board and it does have an IPv6 and an IPv4 (check the IPs resolved by curl or wget), so it's a good metric and afaict does not(?) seem to indicate any issues?

Arch Linux

#1 2023-11-28 02:46:48

Slow download speeds with wget, firefox and browsers

#2 2023-11-28 08:32:40

Re: Slow download speeds with wget, firefox and browsers

#3 2023-12-03 20:33:35

Re: Slow download speeds with wget, firefox and browsers

#4 2023-12-03 22:24:02

Re: Slow download speeds with wget, firefox and browsers

#5 2023-12-08 21:14:58

Re: Slow download speeds with wget, firefox and browsers

#6 2023-12-08 21:34:26

Re: Slow download speeds with wget, firefox and browsers

#7 2023-12-08 22:30:10

Re: Slow download speeds with wget, firefox and browsers

#8 2023-12-08 22:36:41

Re: Slow download speeds with wget, firefox and browsers

#9 2023-12-12 02:05:14

Re: Slow download speeds with wget, firefox and browsers

#10 2023-12-12 07:18:04

Re: Slow download speeds with wget, firefox and browsers

#11 2023-12-13 19:29:22

Re: Slow download speeds with wget, firefox and browsers

#12 2023-12-13 21:03:46

Re: Slow download speeds with wget, firefox and browsers

#13 2023-12-14 01:27:00

Re: Slow download speeds with wget, firefox and browsers

#14 2023-12-14 09:20:01

Re: Slow download speeds with wget, firefox and browsers

#15 2023-12-19 04:17:16

Re: Slow download speeds with wget, firefox and browsers

#16 2023-12-19 15:13:56

Re: Slow download speeds with wget, firefox and browsers

#17 2023-12-27 02:32:27

Re: Slow download speeds with wget, firefox and browsers

#18 2023-12-27 22:01:33

Re: Slow download speeds with wget, firefox and browsers

Board footer