You are not logged in.

#1 2018-12-05 21:07:18

whitehorsesoft
Member
Registered: 2017-09-20
Posts: 10

pacman-mirrorlist package?

pacman -Syu just brought in one package, pacman-mirrorlist-20181205-1, that afaik is a mirrorlist file. Maybe I misunderstand the purpose, but seems like this one file should never be part of an upgrade (I can understand base or install .iso)? Or maybe someone can help me understand.

ref for package is at https://www.archlinux.org/packages/core … irrorlist/

Offline

#2 2018-12-05 21:10:35

WorMzy
Forum Moderator
From: Scotland
Registered: 2010-06-16
Posts: 8,404
Website

Re: pacman-mirrorlist package?

I take it you haven't configured your mirrorlist so far?

https://wiki.archlinux.org/index.php/In … he_mirrors

You can see what changed here: https://git.archlinux.org/svntogit/pack … c1b76d15fc


Sakura:-
Mobo: MSI X299 TOMAHAWK ARCTIC // Processor: Intel Core i7-7820X 3.6GHz // GFX: nVidia GeForce GTX 970 // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 5x 1TB HDD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#3 2018-12-05 21:10:47

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 26,203
Website

Re: pacman-mirrorlist package?

Mirrors change over time. The list reflects those changes.


Arch + dwm   •   Mercurial repos  •   Github

Registered Linux User #482438

Offline

#4 2018-12-05 21:22:00

whitehorsesoft
Member
Registered: 2017-09-20
Posts: 10

Re: pacman-mirrorlist package?

I'm afraid the question has been misunderstood. I regularly configure my mirrorlist with https://www.archlinux.org/mirrorlist and rankmirrors.

But my question is, why would this be overwritten by a package? My mirrorlist actually was updated just yesterday, which proves the point - now my updates have been overwritten (well, moved actually, there is helpful output for where the old went which i appreciate). But why? Makes no sense to me to have a mirrorlist as a package. Once it is initially installed with the defaults, I would expect it to be updated by the user, not by a package update - just like my connection settings, x server settings, and all other configuration files. But again, perhaps I'm misunderstanding something?

Offline

#5 2018-12-05 21:32:31

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 26,203
Website

Re: pacman-mirrorlist package?

It isn't overwritten. It creates a .pacnew which you can merge or delete.


Arch + dwm   •   Mercurial repos  •   Github

Registered Linux User #482438

Offline

#6 2018-12-06 03:56:07

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,533

Re: pacman-mirrorlist package?

The file is marked as a backup file, so it will *never* overwrite the existing file.

Many people merge the pacnew file and reconfigure their mirror based on this packaged mirrorlist.

They're silly and should do what I do... I have a custom "nomirrorlist" package which provides/conflicts "pacman-mirrorlist", depends on reflector, and packages a custom mirrorgen script that uses reflector to grab new mirrors and update the mirrorlist. I can run the script if and when I actually have problems with my mirror...


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#7 2018-12-06 05:18:19

mpan
Member
Registered: 2012-08-01
Posts: 460
Website

Re: pacman-mirrorlist package?

whitehorsesoft: note that the location of the mirror list used by pacman is not fixed in any way. It is added to the main “/etc/pacman.conf” by the Include directive, which simply adds the specified file into the current configuration. By default it leads to “/etc/pacman.d/mirrorlist”, which is a part of the pacman-mirrorlist package. It is managed by that package, so if you modify that mirrorlist, pacman will create a “.pacnew” file upon update. But there is nothing that stops you from including a different file, which is not managed by pacman-mirrorlist. As an example, you may ask reflector to generate a list of best mirrors for you and save it to another file, which is later included.

And yeah, I agree with eschwartz about the silly people. Do not try to guess which mirrors are best for you — most probably your choice will be based on wrong assumptions and will not be good. Just us a tool that does that for you. In particular forget about chosing servers based on country, unless you have some reasons that are political in nature or caused by some policies.

Last edited by mpan (2018-12-06 05:21:06)


Sometimes I’m a bit harsh — don’t get offended too easily!
PGP: 7C848198AE93D3BB | Coreutils SHA2 performance
Russian roulette: curl "https://ptpb.pw/$(cat /dev/urandom | tr -cd [:alnum:] | head -c 4)" | sudo bash

Offline

#8 2018-12-06 05:32:27

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 26,203
Website

Re: pacman-mirrorlist package?

mpan wrote:

I agree with eschwartz about the silly people. Do not try to guess which mirrors are best for you — most probably your choice will be based on wrong assumptions and will not be good.

There are some of us that don't have a choice...


Arch + dwm   •   Mercurial repos  •   Github

Registered Linux User #482438

Offline

#9 2018-12-06 05:42:34

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,533

Re: pacman-mirrorlist package?

mpan wrote:

And yeah, I agree with eschwartz about the silly people. Do not try to guess which mirrors are best for you — most probably your choice will be based on wrong assumptions and will not be good. Just us a tool that does that for you. In particular forget about chosing servers based on country, unless you have some reasons that are political in nature or caused by some policies.

Huh, what?

Servers based on country are one of your best metrics for detecting which servers are geographically close enough to have superior download speeds. Granted reflector has options that can sort all mirrors based on tested access times, which detects fast, close-by mirrors without needing to consult country tables.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#10 2018-12-06 06:11:59

fukawi2
Forum Moderator
From: .vic.au
Registered: 2007-09-28
Posts: 5,585
Website

Re: pacman-mirrorlist package?

jasonwryan wrote:

There are some of us that don't have a choice...

Or your ISP has an 'internal' mirror so you can download packages without wasting your download quota. Not relevant if you're on an unmetered connection, but the majority of Australia as one example have metered internet connectivity.

Offline

#11 2018-12-06 07:09:32

Allan
Member
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,779
Website

Re: pacman-mirrorlist package?

fukawi2 wrote:
jasonwryan wrote:

There are some of us that don't have a choice...

Or your ISP has an 'internal' mirror so you can download packages without wasting your download quota. Not relevant if you're on an unmetered connection, but the majority of Australia as one example have metered internet connectivity.

Really?   I though most non-Telstra plans were unlimited.

Offline

#12 2018-12-06 07:28:59

mpan
Member
Registered: 2012-08-01
Posts: 460
Website

Re: pacman-mirrorlist package?

jasonwryan wrote:

There are some of us that don't have a choice...

And it is not covered by the two exceptions I have listed?

eschwartz wrote:

Servers based on country are one of your best metrics for detecting which servers are geographically close enough to have superior download speeds.

Internet transfers data at the same rate from a server a kilometer away and from the other end of the world. What matters is the rate the server can deliver and this is independent of how far away you are from it. This is why I find reflector so useful: it meters the actual speeds and packages freshness. I may live in Poland, but my current first mirror is in Germany and in the past I was using US servers too.


Sometimes I’m a bit harsh — don’t get offended too easily!
PGP: 7C848198AE93D3BB | Coreutils SHA2 performance
Russian roulette: curl "https://ptpb.pw/$(cat /dev/urandom | tr -cd [:alnum:] | head -c 4)" | sudo bash

Offline

#13 2018-12-06 07:55:18

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 26,203
Website

Re: pacman-mirrorlist package?

Try moving somewhere like NZ where bandwidth is limited, international latency is atrocious and your maximum download speed if 5Mbp/s. Yeah, not so much choice.


Arch + dwm   •   Mercurial repos  •   Github

Registered Linux User #482438

Offline

#14 2018-12-06 07:59:56

mpan
Member
Registered: 2012-08-01
Posts: 460
Website

Re: pacman-mirrorlist package?

Then the measurement will clearly show that. No need for guessing.


Sometimes I’m a bit harsh — don’t get offended too easily!
PGP: 7C848198AE93D3BB | Coreutils SHA2 performance
Russian roulette: curl "https://ptpb.pw/$(cat /dev/urandom | tr -cd [:alnum:] | head -c 4)" | sudo bash

Offline

#15 2018-12-06 08:27:06

schard
Member
From: Hannover
Registered: 2016-05-06
Posts: 115

Re: pacman-mirrorlist package?

eschwartz wrote:

Many people merge the pacnew file and reconfigure their mirror based on this packaged mirrorlist.

They're silly and should do what I do... I have a custom "nomirrorlist" package which provides/conflicts "pacman-mirrorlist", depends on reflector, and packages a custom mirrorgen script that uses reflector to grab new mirrors and update the mirrorlist. I can run the script if and when I actually have problems with my mirror...

Calling users that do what the devs intended them to do silly, seems unreasonable to me. I take a similar approach as you do, though I do not replace the mirrorlist package with a custom built one, but rather just let reflector periodically refresh my mirrorlist using custom systemd units.
If the devs intended this kind of usage, they would not have given pacman a hard dependency on pacman-mirrorlist, but an optional one instead and / or maybe also would have given reflector a provides=(pacman-mirrorlist) entry.

Last edited by schard (2018-12-06 08:27:43)

Offline

#16 2018-12-06 08:55:02

jasonwryan
Anarchist
From: .nz
Registered: 2009-05-09
Posts: 26,203
Website

Re: pacman-mirrorlist package?

mpan wrote:

Then the measurement will clearly show that. No need for guessing.

You are missing the point: there is no need to measure anything. It is not "guessing" either. Believe it or not, there are circumstances where you don't need a tool to tell you what to do.


Arch + dwm   •   Mercurial repos  •   Github

Registered Linux User #482438

Offline

#17 2018-12-06 09:00:35

mpan
Member
Registered: 2012-08-01
Posts: 460
Website

Re: pacman-mirrorlist package?

schard: while using word “silly” my intention wasn’t to offsend users and I guess that it wasn’t eschartz’s intention either. The thing is that people sometimes do things that make little sense, despite the common sense may indicate otherwise. Denying obvious, hard data and relying on unfounded guessess is one of those things. As for devs choices: the people, who maintain Arch, might have different goals; they are not infallible too and some past choices may make little sense now.

Anyway, let’s settle this discussion elsewhere, as soon this thread will get TGN’d.


Sometimes I’m a bit harsh — don’t get offended too easily!
PGP: 7C848198AE93D3BB | Coreutils SHA2 performance
Russian roulette: curl "https://ptpb.pw/$(cat /dev/urandom | tr -cd [:alnum:] | head -c 4)" | sudo bash

Offline

#18 2018-12-06 15:13:34

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,533

Re: pacman-mirrorlist package?

schard wrote:

Calling users that do what the devs intended them to do silly, seems unreasonable to me. I take a similar approach as you do, though I do not replace the mirrorlist package with a custom built one, but rather just let reflector periodically refresh my mirrorlist using custom systemd units.
If the devs intended this kind of usage, they would not have given pacman a hard dependency on pacman-mirrorlist, but an optional one instead and / or maybe also would have given reflector a provides=(pacman-mirrorlist) entry.

Well, we do need to cater to people who don't understand the glory of reflector! The base install needs to work, that doesn't mean people shouldn't take care to optimize it. And it's pretty silly IMO to have the package reserve screen space in your updates if you literally never use it.

Anyway,

1) how do you bootstrap the mirrorlist without a mirrorlist to use to download reflector

2) how do we make sure people don't all use the handful of fast, high-bandwidth low-latency servers, except by assuming most of them will choose a mirror at random?


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#19 2018-12-06 15:23:36

Slithery
Forum Moderator
From: Norfolk, UK
Registered: 2013-12-01
Posts: 2,996

Re: pacman-mirrorlist package?

eschwartz wrote:

1) how do you bootstrap the mirrorlist without a mirrorlist to use to download reflector

curl -o /etc/pacman.d/mirrorlist https://www.archlinux.org/mirrorlist/all/

No, it didn't "fix" anything. It just shifted the brokeness one space to the right. - jasonwryan
Closing -- for deletion; Banning -- for muppetry. - jasonwryan

aur - dotfiles

Offline

#20 2018-12-06 15:42:02

eschwartz
Trusted User/Bug Wrangler
Registered: 2014-08-08
Posts: 2,533

Re: pacman-mirrorlist package?

So, you'd like to propose that all users know why and how to run this command in order to get a minimally functional install. Gotcha.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#21 2018-12-06 21:32:58

fukawi2
Forum Moderator
From: .vic.au
Registered: 2007-09-28
Posts: 5,585
Website

Re: pacman-mirrorlist package?

Allan wrote:

Really?   I though most non-Telstra plans were unlimited.

Unlimited plans are increasing with the lower-end providers (TPG, Dodo etc) IME, but metered connections are still the norm for home, especially with the better quality providers (Internode/iiNet, AussieBB).

Offline

#22 2018-12-08 19:30:49

mountaineerbr
Member
Registered: 2018-12-08
Posts: 1

Re: pacman-mirrorlist package?

Why bother use reflector often? Also, if you deleted the mirrorlist, it is not difficult to generate a new manually. All is needed is a new text file and *one* server listed.
I was trying to use solely my country servers for updates (because of download speed, I had reckoned) but all of a sudden,  every 7 or 8 of them had a synchronisation delay problemo for some weeks that took me  a while to realise, in which meantime I did not receive updates... Reflector is only needed for automation of that dull task, but you can also have a GUI-like manual mirror server OverView with lots and lots of Scores types for each mirror that you may prefer above others so that you can manually choose the *best* mirror server at https://www.archlinux.org/mirrors/status/

Since choosing and adding the best server, that was always synchronised and had 100% response to requests, to the top of my mirrorlist, I still have not worried further about that. In fact, I even forgot all options Reflector has got, though I am a sure a quick --help will do to set it to nice parametres.  I find that I usually have got reasonable download speeds with that afar server, not great speeds... But I am trying to not update every week anyways, and only when I have got some time... Most often I have got a wireless connection that limits the band more badly than any other reasonable factor... Also, https servers ought to be slower than http ones right, but it depends if you want to be safer or faster...

#Manually chosen 3 top rated servers at 27/10/18.
Server = http://arch.jensgutermuth.de/$repo/os/$arch
Server = http://arch.mirror.square-r00t.net/$repo/os/$arch
Server = http://mirrors.n-ix.net/archlinux/$repo/os/$arch

Last edited by mountaineerbr (2018-12-08 19:33:32)

Offline

#23 2018-12-08 20:43:06

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 20,435
Website

Re: pacman-mirrorlist package?

mountaineerbr wrote:

Also, https servers ought to be slower than http ones right

Right.  As long as by slower you mean the same as what I mean by longer when I say the distance from the sun to my front lawn is longer than the distance from the sun to my lawn mower.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

#24 2018-12-08 23:05:51

ratcheer
Member
Registered: 2011-10-09
Posts: 650

Re: pacman-mirrorlist package?

For years, I have ranked for the top "fastest" mirrors in the US, and made that my mirrorlist. A few weeks ago, I had the same problem as @mountaineerbr - fast mirrors being horribly unsynchronized, resulting in me getting no updates for days until it dawned on me what the problem was.

Now, I'm doing a more complex process, but with much better results. I'm testing (for speed) the US mirrors that are fully synchronized, and using only synchronized, fast mirrors in my mirrorlist. It's a pain in the butt, but it works well. I also note that this is basically what the Mirrors page on the wiki says to do.

Offline

#25 2018-12-08 23:50:18

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 20,435
Website

Re: pacman-mirrorlist package?

ratcheer wrote:

It's a pain in the butt

Why?  Reflector has flags to do precisely that.


"UNIX is simple and coherent..." - Dennis Ritchie, "GNU's Not UNIX" -  Richard Stallman

Online

Board footer

Powered by FluxBB