[SOLVED] Question about reflector usage

Succulent of your garden · 2025-07-31 14:13:35

So I have been using reflector like this:

sudo reflector --age 12 --latest 20 --fastest 20 --threads 20 --sort rate --protocol https --save /etc/pacman.d/mirrorlist

to generate my mirror list, but I have another computer that have another mirror list created with the same command, but It seems that is way more faster than the last one that I had created in another computer.

So why could be this happening ? and also any help to improve my reflector command will be appreciated. I know the country flag, but in my case it seems to create a worse output now matter how good the internet of my country is

Last edited by Succulent of your garden (2025-08-03 14:52:22)

seth · 2025-07-31 15:25:04

I have another computer that have another mirror list created with the same command, but It seems that is way more faster than the last one that I had created in another computer.

I suppose a diff of the mirror lists would be useful - plus: you're comparing the update/download speeds on the same host?
You're not asking "why does my 64-core system w/ 1TB RAM and an nvme raid connected to DE-CIX via infiniband download stuff faster than my old single-core 512MB RAM notebook via 802.11a - the one where rust is falling out everytime the HDD makes weird noises"?

Succulent of your garden · 2025-08-01 13:35:24

Both computers are for this task almost the same. I mean i setup parallel downloads to 5 and the wifi card have the same speed, both are in the same room so I don't think about wifi delay issues. The list I think is very different in comparison with the older one. Today I made an update with the newer one in the computer and seems to work fine, but not sure if the older one is better, because while updating pacman said:

 cuda-12.9.1-2-x86_64                                                                                             659.2 MiB  5.71 MiB/s 01:56 [#######################################################################################] 100%
 Total (56/56)                                                                                                      2.8 GiB  6.78 MiB/s 07:00 [#######################################################################################] 100%
error: failed retrieving file 'vulkan-radeon-1:25.1.7-1-x86_64.pkg.tar.zst' from mirror.theash.xyz : The requested URL returned error: 404
error: failed retrieving file 'mesa-1:25.1.7-1-x86_64.pkg.tar.zst' from mirror.theash.xyz : The requested URL returned error: 404
error: failed retrieving file 'cuda-12.9.1-2-x86_64.pkg.tar.zst' from mirror.theash.xyz : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds
warning: too many errors from mirror.theash.xyz, skipping for the remainder of this transaction
(56/56) checking keys in keyring       
Blah blah blah then pacman does everything well and so on and so on :P

I don't get it why in the newer mirror list that I had made I'm having this kind of issues of retrieving packages, but with the older one everything seems to work fine. Or at least I can notice that difference between computers.

I will send the diff later, but in the meantime what do you think about htis ?

Last edited by Succulent of your garden (2025-08-01 13:38:53)

seth · 2025-08-01 13:51:05

wget -O /dev/null https://mirror.theash.xyz/arch/extra/os/x86_64/vulkan-radeon-1:25.1.7-1-x86_64.pkg.tar.zst

Works fine for me - 404 would indicate that the mirror had not properly synced at the time?

Succulent of your garden · 2025-08-01 20:55:05

seth wrote:

Works fine for me - 404 would indicate that the mirror had not properly synced at the time?

Maybe, since I just put age with 12 hours, I'm going to try but with less time. Do you recommend an specific input for the flag --age ?

By the way, the older computer with the older mirrorlist has just recently died , I'm going to see if I can fix it today, seems that can still kick in some time.

Last edited by Succulent of your garden (2025-08-01 20:55:41)

seth · 2025-08-01 21:22:51

Miss by an inch, miss by a mile - If you get the database from mirror A that has synced 5 minutes ago but try to get the package from mirror B that has synced 15 minutes ago and the package was updated inbetween, the gods hate you.
Also there might be some time passing between you updating the mirrorlist and actually updating.

12h is an upstream suggestion. It is for sure not the perfect™ duration, but I don't have any data to argue a better number

Succulent of your garden · 2025-08-01 22:10:07

Also there might be some time passing between you updating the mirrorlist and actually updating.

The last time was cuda update time, which take like 8 to 10 minutes just only with cuda, the rest of the haskell stuff among others was pretty quickly, like all in a minute to download. Do you think that could made the mirror desync ?

Last edited by Succulent of your garden (2025-08-01 22:10:57)

seth · 2025-08-02 06:31:49

error: failed retrieving file 'cuda-12.9.1-2-x86_64.pkg.tar.zst' from mirror.theash.xyz : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds

At this point you already had problems w/ the mirror.
You can run into random gaps any time. But if it's overly frequent or even reproducible you want to look into that specific situation.

Succulent of your garden · 2025-08-02 12:41:44

It is not frequent, it does happen but most of the time when I'm downloading heavy packages like cuda, an IDE, etc.

seth · 2025-08-02 12:59:32

Your NIC might collapse under load, this would have nothing to do w/ the mirror but the local network device - have you checked the system journal after such incidents?

Succulent of your garden · 2025-08-03 00:46:43

No I haven't. You mean using journalctl ? How can I check the nick using that cli in arch ? The other option could be wireshark i guess.

Last edited by Succulent of your garden (2025-08-03 00:47:08)

seth · 2025-08-03 07:28:09

You mean using journalctl ?

Yes.

The other option could be wireshark i guess.

No, you're not interested in the traffic but why it stops.

Succulent of your garden · 2025-08-03 14:51:48

Thanks Seth. Since I'm not having this issue currently now, i'm going to mark the thread as solved. But if suddenly happen again, I'll make another post with the journalctl logs to fix the issue.

Last edited by Succulent of your garden (2025-08-03 14:52:05)

cryptearth · 2025-08-03 18:44:35

btw: using multi-thread when comparing/sorting by speed is discouraged as multiple connection can impact eachother quite severely
or to put it this way: you try to rate 20 servers at the same time by the speed they give you - so they all have to share your bandwidth at the same time
this can give you an accurate result only if your bandwidth is more than all of the server's combined
so to get a more useful result remove the multi thread but test each server one at a time

Use n threads for rating mirrors. This option will
speed up the rating step but the results will be
inaccurate if the local bandwidth is saturated at any
point during the operation. If rating takes too long
without this option then you should probably apply
more filters to reduce the number of rated servers
before using this option.

Succulent of your garden · 2025-08-04 10:22:51

Thanks cryptearth, yesterday I made again another mirrorlist file without the thread flag, but yet there are no new packages to check. So I'm going to wait a little and test the results

Succulent of your garden · 2025-08-08 23:45:54

Thanks cryptearth for the info, the new mirrorlist without using the threads flags works far better than the initial command

Arch Linux

#1 2025-07-31 14:13:35

[SOLVED] Question about reflector usage

#2 2025-07-31 15:25:04

Re: [SOLVED] Question about reflector usage

#3 2025-08-01 13:35:24

Re: [SOLVED] Question about reflector usage

#4 2025-08-01 13:51:05

Re: [SOLVED] Question about reflector usage

#5 2025-08-01 20:55:05

Re: [SOLVED] Question about reflector usage

#6 2025-08-01 21:22:51

Re: [SOLVED] Question about reflector usage

#7 2025-08-01 22:10:07

Re: [SOLVED] Question about reflector usage

#8 2025-08-02 06:31:49

Re: [SOLVED] Question about reflector usage

#9 2025-08-02 12:41:44

Re: [SOLVED] Question about reflector usage

#10 2025-08-02 12:59:32

Re: [SOLVED] Question about reflector usage

#11 2025-08-03 00:46:43

Re: [SOLVED] Question about reflector usage

#12 2025-08-03 07:28:09

Re: [SOLVED] Question about reflector usage

#13 2025-08-03 14:51:48

Re: [SOLVED] Question about reflector usage

#14 2025-08-03 18:44:35

Re: [SOLVED] Question about reflector usage

#15 2025-08-04 10:22:51

Re: [SOLVED] Question about reflector usage

#16 2025-08-08 23:45:54

Re: [SOLVED] Question about reflector usage

Board footer