Why does Pacman seem to be signficant faster in releation to others?

mento · 2021-02-15 10:53:37

For several years I am asking myself why Pacman seems to be significant faster than other major packing managers like whatever Debian/Ubuntu/Opensuse are using.
Please note, I am using Archlinux since 2009 as my main driver and due to this short testing phase I am not very good at Archlinux (very good at pacman -Syu, though) or Linux or actually anything except eating Milk with cookies. I tried to do research via a modern internet search machine, but the answers are way too complicated and I am not an engineer. "Dependencies"?!, I am not very good with taxes.
Would you please explain in simple word using train and football analogues (mentioning Lionel Messi helps understanding) why Pacman seems to be significant faster than other major packing managers?

Thank you.

GaKu999 · 2021-02-15 14:58:54

My take is that it does less? The mirrors are good maybe? Maybe a combination of several factors?

Bleeding edge may be the thing, don’t update for a month and you get some more “slow” speed, tho I once waited a month and it got quite quick but my env is tiny...

Or eschwartz magic is the culprit, yeah, blame the wizard. That works.

schard · 2021-02-15 15:01:15

I don't see much difference between pacman and apt-get or yum anymore.
Pacman once was fast, because it was lightweight and simple.
It still somehow is, but since the introduction of hooks, I don't see any significant performance increase any more compared to other package managers, especially on large updates.
Don't get me wrong here: I love ALPM hooks and find them very useful. But everything has its price...

Last edited by schard (2021-02-15 15:02:22)

sabroad · 2021-02-15 15:35:10

mento wrote:

why Pacman seems to be significant faster than other major packing managers?

In Arch Linux, partial upgrades are unsupported and only a single version of each shared library [is] in the official repositories, in contrast to other (non-rolling-release) distros.

This simple rule avoids (for the most part) the intractable issue that dependency hell is NP-complete. Maintaining metadata for complex version resolution has the effect that Linux package managers are slow (2019).

Head_on_a_Stick · 2021-02-15 16:31:44

GaKu999 wrote:

My take is that it does less?

^ This.

APT has to deal with multiple releases, development branches and pinning, none of which apply to pacman; aptitude(8) even has a curses interface and uses an "advanced" dependency resolution algorithm which makes it even slower.

By contrast, Alpine's package manager (apk) does even less than pacman (it has simple triggers but no hooks, APKBUILDs don't offer optdepends and use POSIX sh instead of bash) and as a result it's even faster.

Alad · 2021-02-15 18:41:59

pacman doesn't run fsync() all the time, unlike apt-get. Disable it (e.g. through "eatmydata") and both are comparably fast.

Then there's dnf... which seems purposely designed to make package installations as long as installing an entire operating system. For that you get more-or-less useful features like transactions and rollbacks.

APKBUILDs don't offer optdepends and use POSIX sh instead of bash) and as a result it's even faster.

pacman doesn't do a whole lot with optdepends (apart from displaying them) so I doubt it plays a significant role.

Last edited by Alad (2021-02-15 18:46:00)

Head_on_a_Stick · 2021-02-15 19:45:35

Alad wrote:

pacman doesn't run fsync() all the time, unlike apt-get. Disable it (e.g. through "eatmydata") and both are comparably fast.

I take it you haven't used aptitude(8) much then? It's almost as slow as OpenBSD's package manager and that's written in perl.

Alad wrote:

APKBUILDs don't offer optdepends and use POSIX sh instead of bash) and as a result it's even faster.
pacman doesn't do a whole lot with optdepends (apart from displaying them) so I doubt it plays a significant role.

Probably not, no. I was just offering some examples of the reduced functionality offered by Alpine's package manager, which is the fastest around.

Anyway, this article may be of interest to the OP: https://michael.stapelberg.ch/posts/201 … -are-slow/

Allan · 2021-02-15 23:48:21

schard wrote:

It still somehow is, but since the introduction of hooks, I don't see any significant performance increase any more compared to other package managers, especially on large updates.

Strange... since hooks actually reduce the repetitive install scripts and make installing faster. e.g. every package installing a .desktop or .info file does not need to update the relevant database - it now happens once at the end.

But there are a lot more things that trigger mkinicpio these days...

Allan · 2021-02-15 23:49:52

GaKu999 wrote:

Or eschwartz magic is the culprit, yeah, blame the wizard. That works.

Has Eli submitted patches to pacman? makepkg yes, but pacman...

Xyne · 2021-02-16 01:20:15

Train and football analogies were specifically requested.

Packages: footballs
Mirrors: trains full of footballs
Pacman: Lionel Messi
Installing packages: Lionel Messi kicking footballs into the goal.

Faster mirrors -> faster trains, footballs get to Messi's feet faster (not sure that mirror speed matters, but it was mentioned in the thread so analogy, plus I don't see where else I'm going to fit in a train analogy here)

Hooks: Messi poses for the cameras once before kicking in the goals and once after, so faster than posing between each kick.

Single version of each package: all of the footballs are the same color. Messi doesn't need to pay attention to the color of the balls and carefully choose which ones he kicks in

No constant fsyncs: Messi doesn't have to wait for each goal to go in before kicking the next ball.

Cognitive bias: You're a Messi fan so you're biased and and won't admit that other players are just as good and some are even better.

eschwartz · 2021-02-16 01:33:50

Allan wrote:

Has Eli submitted patches to pacman? makepkg yes, but pacman...

It's not my forte, but I actually have!

There's the one-liner libarchive flag to enable xattrs I just contributed... https://git.archlinux.org/pacman.git/co … 7a105fe42b

My first ever patch to libalpm was fixing gcc8 warnings: https://git.archlinux.org/pacman.git/co … aeb65d9b7b

And I've even touched

vercmp:
https://git.archlinux.org/pacman.git/co … d3e6d60932
https://git.archlinux.org/pacman.git/co … 99f54b34fc

pacman:
https://git.archlinux.org/pacman.git/co … e5b6ea00c5
https://git.archlinux.org/pacman.git/co … 519f022386

pacman-conf:
https://git.archlinux.org/pacman.git/co … 1e5a56ba7f
https://git.archlinux.org/pacman.git/co … 6ed1d42a0a

Admittedly, I hardly ever touch pacman's code. But I do it often enough to pretend from a distance that I actually know C somewhat...

Last edited by eschwartz (2021-02-16 01:35:41)

Allan · 2021-02-16 01:35:40

Xyne wrote:

Cognitive bias: Your a Messi fan so you're biased and and won't admit that other players are just as good and some are even better.

I think we are all of the opinion that APK is super fast, and DNF is super slow.

I don't know round ball footballers to compare them too...

eschwartz · 2021-02-16 02:47:01

sabroad wrote:

mento wrote:
why Pacman seems to be significant faster than other major packing managers?
In Arch Linux, partial upgrades are unsupported and only a single version of each shared library [is] in the official repositories, in contrast to other (non-rolling-release) distros.
This simple rule avoids (for the most part) the intractable issue that dependency hell is NP-complete. Maintaining metadata for complex version resolution has the effect that Linux package managers are slow (2019).

Oh incidentally...

Note that some packagers today are quite anxious to modify packages, wherever possible, to depend on versioned provides, e.g. "libglib-2.0.so=0-64" rather than "glib2".

The goal is to have pacman refuse to update iff you try to do a partial update, there is zero intention to support complex version resolution in order to select the right package to install. Even so, I wonder what effect this will have on the pacman solver.

Allan · 2021-02-16 06:19:59

eschwartz wrote:

Even so, I wonder what effect this will have on the pacman solver.

Near zero surprisingly... At least according to the benchmarks I have seen. That still surprises me a lot.

Alad · 2021-02-16 09:35:19

Head_on_a_Stick wrote:

Alad wrote:
pacman doesn't run fsync() all the time, unlike apt-get. Disable it (e.g. through "eatmydata") and both are comparably fast.
I take it you haven't used aptitude(8) much then? It's almost as slow as OpenBSD's package manager and that's written in perl.

I stopped using it after it turned out it is not interchangeable with apt-get and broke my Debian sid system

OpenBSD's package manager is a whole different league. When I used it took three hours to install libreoffice and it was a binary package...

Xyne · 2021-02-16 16:19:59

Alad wrote:

OpenBSD's package manager is a whole different league. When I used it took three hours to install libreoffice and it was a binary package...

You have to use the "--disable-gentoo-emulation" flag to avoid that.

mhertz · 2021-03-09 16:22:32

Does anyone know why void's xbps package manager often is stated faster than pacman? Little broad slash stupid question probably, but just seems surpricing to me when considering seemingly has more advanced dependency resolution allowing for e.g partial upgrades. Just curious, and I guess on-topic.

Allan · 2021-03-09 23:18:16

mhertz wrote:

but just seems surpricing to me when considering seemingly has more advanced dependency resolution allowing for e.g partial upgrades.

Pacman can handle partial upgrades. Arch does not support that...

mhertz · 2021-03-10 17:18:28

Allan wrote:

Pacman can handle partial upgrades. Arch does not support that...

Thanks for correction, and reply. I didn't knew that, not surpricing lol, but I just read xbps touted as more advanced as tracking incompatible libs and support partial upgrades, but now that I think about it(well, after your reply rather), pacman does too with it's version operators or how called, so distro diffs as you stated. Thanks again.

Last edited by mhertz (2021-03-10 17:19:47)

eschwartz · 2021-03-11 01:34:19

IIRC xbps automatically tracks all shlibs, while pacman lets the PKGBUILD author opt in on an individual basis.

mhertz · 2021-03-12 13:18:29

I appreciate your reply eschwartz, thank you! Indeed I can see what you're saying when looking into it, and sorry for not using my good friend little better initially

Anyway, I guess also as the binaries are much splitted out in the xbps "suite", and maybe also smaller official repo's I dunno, might also have an effect(+ likely good c coding, like pacman i'd suspect). Never really used it myself, except a few mins in a VM several years ago.

I also read e.g. that what makes apk little faster than xbps(meaning faster than most package managers in general, i'd guess then, e.g. pacman also possibly), is tricks like e.g. it downloads, extracts and feeds hashing algo at same time. Don't know if pacman does same of-course, as this was xbps vs apk, as stated, but sounds nifty(easier said than done of-course ). Hmm, considering pacman's status messages on-screen during it's operation, then doesn't do that, unless i'm mistaken.

Last edited by mhertz (2021-03-12 13:54:04)

eschwartz · 2021-03-12 20:37:26

pacman does not extract packages in parallel, and it is not obvious that it can if you consider running install scripts per package.

pacman 6.0alpha1 does support parallel downloading.

My understanding is that apk has fewer features and thus does less administrative work, which helps it complete faster.

I believe it also opportunistically extracts packages to the filesystem while it is still downloading other packages or checking their integrity... this is in principle the absolutely worst thing you can ever do since you don't know whether the dependencies exist, or were downloaded securely/correctly, but... apk extracts them to temporary files, and once they're verified and the transaction is checked for validity, it renames all the files to replace the old versions of the files. If anything goes wrong, it simply deletes the temporary files.

This has an obvious downside: in the middle of the process, you're using twice as much space on disk as the size of the upgrade itself. Plus the size of the compressed package archives. And I assume they had to handle edge cases around permissions to avoid e.g. extracting malware SUID binaries, then later discovering it is an invalid package and aborting the transaction (it doesn't matter if you only extracted to a temp file, if attackers can run the temp file to get privesc).

mento · 2021-03-13 16:47:34

Xyne wrote:

…
.

Thank you for your multi-angle holistic explanation. This was helpful. I actually did not think about a cognitive bias. Looking at pacman operate is deeply ingrained into my brain, while other packing manager displaying it differently could induce negative emotions (it is scary and potentially demonically wrong), which is known to stretch time.
However, I think you should drop the train, football and Lionel Messi analogues. This is super strange.

(<3 Thank you. Your locale-en_xx is also extremely cool. So strange that here are only "hacks" for iso times, which are actually officially used in many countries.)

mhertz · 2021-03-13 17:37:11

Thanks alot many times eschwartz again, and I much appreciated your detailed posting! Anyway, i'm absolutely happy about pacman's speed, and everything else about it, almost runned it daily for over a decade by now Thanks for your makepkg/pacman work also btw + @Allan. Good weekend to all

Xyne · 2021-03-14 21:10:14

mento wrote:

(<3 Thank you. Your locale-en_xx is also extremely cool. So strange that here are only "hacks" for iso times, which are actually officially used in many countries.)

Arch Linux

#1 2021-02-15 10:53:37

Why does Pacman seem to be signficant faster in releation to others?

#2 2021-02-15 14:58:54

Re: Why does Pacman seem to be signficant faster in releation to others?

#3 2021-02-15 15:01:15

Re: Why does Pacman seem to be signficant faster in releation to others?

#4 2021-02-15 15:35:10

Re: Why does Pacman seem to be signficant faster in releation to others?

#5 2021-02-15 16:31:44

Re: Why does Pacman seem to be signficant faster in releation to others?

#6 2021-02-15 18:41:59

Re: Why does Pacman seem to be signficant faster in releation to others?

#7 2021-02-15 19:45:35

Re: Why does Pacman seem to be signficant faster in releation to others?

#8 2021-02-15 23:48:21

Re: Why does Pacman seem to be signficant faster in releation to others?

#9 2021-02-15 23:49:52

Re: Why does Pacman seem to be signficant faster in releation to others?

#10 2021-02-16 01:20:15

Re: Why does Pacman seem to be signficant faster in releation to others?

#11 2021-02-16 01:33:50

Re: Why does Pacman seem to be signficant faster in releation to others?

#12 2021-02-16 01:35:40

Re: Why does Pacman seem to be signficant faster in releation to others?

#13 2021-02-16 02:47:01

Re: Why does Pacman seem to be signficant faster in releation to others?

#14 2021-02-16 06:19:59

Re: Why does Pacman seem to be signficant faster in releation to others?

#15 2021-02-16 09:35:19

Re: Why does Pacman seem to be signficant faster in releation to others?

#16 2021-02-16 16:19:59

Re: Why does Pacman seem to be signficant faster in releation to others?

#17 2021-03-09 16:22:32

Re: Why does Pacman seem to be signficant faster in releation to others?

#18 2021-03-09 23:18:16

Re: Why does Pacman seem to be signficant faster in releation to others?

#19 2021-03-10 17:18:28

Re: Why does Pacman seem to be signficant faster in releation to others?

#20 2021-03-11 01:34:19

Re: Why does Pacman seem to be signficant faster in releation to others?

#21 2021-03-12 13:18:29

Re: Why does Pacman seem to be signficant faster in releation to others?

#22 2021-03-12 20:37:26

Re: Why does Pacman seem to be signficant faster in releation to others?

#23 2021-03-13 16:47:34

Re: Why does Pacman seem to be signficant faster in releation to others?

#24 2021-03-13 17:37:11

Re: Why does Pacman seem to be signficant faster in releation to others?

#25 2021-03-14 21:10:14

Re: Why does Pacman seem to be signficant faster in releation to others?

Board footer