pacman prioritize packages in system upgrades?

Tétrapyle · 2023-01-15 10:11:27

Hi,

I was wondering if it has already been considered to make pacman systematically upgrade some packages before others while upgrading the system.

I'm thinking of pacman-keyring. So as to avoid the need for this:

pacman -Sy archlinux-keyring && pacman -Syu

I think it would make sense, because pacman needs an updated keyring to work.

This kind of error has probably happend to people thousands of times, if not billions. I know that some people consider that it won't happen should you upgrade your system often enough. I respectfully disagree with this. I upgrade regularly and I often have to do it. Plus, when I upgrade my old backup laptop (it stays for months in the cupboard), I absolutely have to do it and it doesn't seem natural. It's not about being proficient or reading carefully the wiki or being used to Arch, it's about having things working in a, seems to me, logical way.

I suppose it might be useful for other packages. Maybe it's already implemented?

By the way, I don't know if there's a specific order in which the packages are upgraded. It doesn't seem to be alphabetical.

All the best,
Tétrapyle

ayekat · 2023-01-15 12:59:39

Upgrading archlinux-keyring before the rest requires more than simply changing the package order in one upgrade.
That's because checking (all) the packages is a step that happen before upgrading the packages, so a fully downloaded and upgraded archlinux-keyring is required before the rest of the downloaded packages are signature-checked.

About the order of installation/upgrade of packages in general, that actually is alphabetical. Pacman just additionally reorders packages to satisfy all dependencies.
So if you have these packages:
a, b, c, d
but b depends on d, you would ultimately get this installation/upgrade order:
a, d, b, c

You might have noticed that when there's a cyclical dependency, pacman gives you a warning like

warning: dependency cycle detected:
warning: mesa will be installed before its libglvnd dependency

yochananmarqos · 2023-01-15 19:43:36

Tétrapyle wrote:

Hi,
So as to avoid the need for this:
pacman -Sy archlinux-keyring && pacman -Syu

You can add this to /etc/pacman.conf instead of doing that with Manjaro's pacman patch:

SyncFirst = archlinux-keyring

Last edited by yochananmarqos (2023-01-15 19:46:55)

Scimmia · 2023-01-15 19:44:36

yochananmarqos wrote:

Tétrapyle wrote:
Hi,
So as to avoid the need for this:
pacman -Sy archlinux-keyring && pacman -Syu
Tétrapyle
You can add this to /etc/pacman.conf instead of doing that:
SyncFirst = archlinux-keyring

Only if you're on Manjaro. SyncFirst has been gone for a long time.

yochananmarqos · 2023-01-15 19:47:22

Scimmia wrote:

yochananmarqos wrote:
Tétrapyle wrote:
Hi,
So as to avoid the need for this:
pacman -Sy archlinux-keyring && pacman -Syu
Tétrapyle
You can add this to /etc/pacman.conf instead of doing that:
SyncFirst = archlinux-keyring
Only if you're on Manjaro. SyncFirst has been gone for a long time.

Oh, right. I edited my post above.

Tétrapyle · 2023-01-15 20:41:52

Thank you for your insight.
Is it possible with a pacman hook? I don't think so because while pacman is running, it would be locked.

mpan · 2023-01-15 20:46:49

Yes, there is an option for such specific scenarios. Making an alias or a function in shell, which will first reinstalls all keyrings you need and then do -Syu.

Prioritizing some packages was implemented in the past, with SyncFist. It was dropped in 2012, because “[it] has outlived its usefulness and cause[d] more problems than it solve[d].” The archlinux-keyring case has been taken into account in that decision.

Currently archlinux-keyring has archlinux-keyring-wkd-sync.{service,timer} rolled out. It is designed to address exactly the issue discussed here, by silently forcing keyring updates even on systems that remain not updated. The feature is experimental and so far seems to be somewhat buggy, but it should provide a clean way of doing that.
____
⁽¹⁾ From the perspective of a pure Arch archlinux-keyring looks special and somewhat “inherent” to pacman. But neither it is required by pacman, nor it is the only keyring possible. That’s because pacman is not used in Arch only, and the official repos are not the only repos. I am mentioning that, because you are not the first person who brings up this subject. In almost all previous instances the person had narrow view of the situation, leading to proposing solutions fitting just that very specific scenario.

Scimmia · 2023-01-16 00:38:38

archlinux-keyring-wkd-sync.{service,timer} will only fix it in certain specific circumstances. Mostly just when a key has been added to the keyring without the required signatures or expired keys.

Allan · 2023-01-16 00:43:24

Scimmia wrote:

archlinux-keyring-wkd-sync.{service,timer} will only fix it in certain specific circumstances. Mostly just when a key has been added to the keyring without the required signatures or expired keys.

That is the case that pacman does handle poorly. Importing new keys should work, particularly with Arch's WKD setup...

Awebb · 2023-01-16 09:14:56

1. pacman -Syu
2. something something keys... cancelled.
3. pacman -Sy archlinux-keyring && pacman -Syu
4. All done.

This has become second nature every time I upgrade a box that hasn't been touched in a couple of weeks. At this point, adding "update archlinux-keyring first and try again" to the error message might be something I could live with, too.

Allan · 2023-01-16 10:33:51

I upgrade every 4-6 weeks (it varies depending on how busy I am at work and the risk I want to take...) and according to my pacman log I have needed "pacman -S archlinux-keyring" 3 times since late 2020. I don't think I have improved anything in pacman-git that would fix this, so I get really confused why people need this so often.

Is it packages signed by a new key that you do not have on your system at all? Is pacman's WKD failing to import the key?
Is it keys that were in the keyring with not enough master key signatures?
Am I just really lucky with my package selection such that the developers packaging things I want have long established keys?

I guess I need to write a protocol for people to follow to send me information to improve this.

Lone_Wolf · 2023-01-16 11:34:02

allan wrote:

I get really confused why people need this so often.

Same here even though one of the systems I help maintain gets updates only every 3 -6 months .

Awebb · 2023-01-16 13:25:41

@Allan:

I've gone over all the machines.

One (my NAS) only had one instance of installing the keyring separately. The others didn't have archlinux-keyring-wkd-sync running for some reason. Most of them are ancient virtual machines.

seth · 2023-01-16 13:46:23

There was a phase during the last couple of years where this seemed to have happend *a lot*, going by people showing up on the forum w/ the problem.
But this has considerably calmed down - either because of the wkd service or because less packages on fresh keys ended up in the repos.

Lazy me, I just pre-updated the keyring between the updates that crossed the noisy phase - necessary or not.

The bigger issue seemed some occasions where the wkd interfered w/ the update, requiring a full keyring reset for some users and a there's also thread inquiring "what are the downside if i just turn off the wkd service" because of apparently a wonky uplink caused it to permanently fail, https://bbs.archlinux.org/viewtopic.php?id=279803

I'd go w/ mpan's suggestion (and while at it, block the god- and soulless "-Suy" heresy w/ a snarky comment

mpan · 2023-01-16 14:27:07

I skipped mentioning that in previous message as something not productive, but since it’s already in the thread, let me explain my position.

Last year there were frequent keyring updates, that caused some trouble even to frequently updating people. It didn’t affect me, but indeed there was such a situation. But that was exceptional.

Otherwise, my pacman log goes back to 2011. There is a single instance of an explicit archlinux-keyring update. That was after a successful -Syu less than 24h earlier, so that might’ve been an attempt to fix something unrelated to the keyring.

Both release history and ALA may be used to track changes. Not only they are rare, but even with updates done less than once a week there weren’t many candidates for causing breakage (with the exception of aforementioned 2022 turbulence).

Last edited by mpan (2023-01-16 14:27:59)

Trilby · 2023-01-16 14:54:32

On my primary system which is three years old which gets updates on average once per week (but this can vary), I've only needed pre-upgrade the keyring once according to my pacman log. I'm pretty sure it was in response to a front-page news item requiring it (ca-certificates were included with the keyring ... I don't think I would have done that without getting that command from the news).

On a ten year old install that was my old work computer and now has been a "media center" pc for the past 5-6 years and gets updates perhaps every couple months, I've only had to pre-upgrade the keyring twice ... in ten years.

So yeah, I'm not sure how others are hitting this so often. Perhaps my lack of issue with this is that I run pretty lean systems with relatively few packages resulting in fewer potential openings for the issue. And / or perhaps whatever is exactly behind this error is less likely to hit core (not repo, but fundamental) packages that tend to be maintained by devs who have been around a while; more 'exotic' packages may come and go and so too may their packagers (and their keys).

There was a recent uptick in threads related to this issue, but a large portion of those were due to it apparently being necessary with one of the monthly isos. As a result, I suspect many new users who used that iso "over-learned" from that experience, and when some other random issue popped up in an upgrade they wrongly assumed it was the same problem resulting in an over-reporting of the need for this intervention (hell, there's one active thread where the OP had an update fail for unclear reasons, only to come to the conclusion that they had to update the keyring 10 times in a loop and do a special rain dance for pacman to work).

EDIT: also may be worth noting, this thread is the first I've heard of this WKD service and I've never enabled / used it.

Last edited by Trilby (2023-01-17 00:27:53)

schard · 2023-01-16 20:43:00

I had to pre-update the keyring several times on our digital signage systems according to the logs.
However, I can also confirm that since the WKD service this issue has dramatically decreased.

icar · 2023-01-16 22:10:25

I have been using Arch for about 2 years now. I own two computers, one that gets updated once every few months and my daily used laptop which is updated almost daily. The former has required more than a couple of times to update the Arch's keyring before any other package, while the latter has only encountered this issue once.

But, since I have faced this several times, I have been updating my computer with a script that contains this line:

paru --show --news && sudo pacman -Sy archlinux-keyring --needed ; sudo pacman -Su ; paru -Syua

mpan · 2023-01-17 02:56:56

Trilby wrote:

EDIT: also may be worth noting, this thread is the first I've heard of this WKD service and I've never enabled / used it.

The timer is enabled by default:

 $ pacman -Ql archlinux-keyring | grep -F timers.target.wants/archlinux-keyring-wkd-sync.timer
archlinux-keyring /usr/lib/systemd/system/timers.target.wants/archlinux-keyring-wkd-sync.timer

Lone_Wolf · 2023-01-17 11:25:07

mpam wrote:

The timer is enabled by default:

fortunately it's not hard to disable timers permanently.

$ ls -l /etc/systemd/system/*wkd*
lrwxrwxrwx 1 root root 9 28 sep 12:52 /etc/systemd/system/archlinux-keyring-wkd-sync.timer -> /dev/null
$

Awebb · 2023-01-17 11:50:12

mpan wrote:

Trilby wrote:
EDIT: also may be worth noting, this thread is the first I've heard of this WKD service and I've never enabled / used it.
The timer is enabled by default:
 $ pacman -Ql archlinux-keyring | grep -F timers.target.wants/archlinux-keyring-wkd-sync.timer
archlinux-keyring /usr/lib/systemd/system/timers.target.wants/archlinux-keyring-wkd-sync.timer

It wasn't active on a couple of my systems, although I don't see how it couldn't have been without manually disabling the timer and I don't remember doing so.

Last edited by Awebb (2023-01-17 11:50:58)

Tétrapyle · 2023-04-21 18:03:02

Hi there. I had to do it today. On some usb stick I use sometimes.

j♭ sudo pacman -Syu
[...]
 Total (816/816)                                                                                                                          3.8 GiB  13.9 MiB/s 04:41 [######################################################################################################] 100%
(816/816) checking keys in keyring                                                                                                                                  [######################################################################################################] 100%
downloading required keys...
:: Import PGP key FDC3040B92ACA748, "Robin Candau <antiz@archlinux.org>"? [Y/n]
(816/816) checking package integrity                                                                                                                                [######################################################################################################] 100%
error: hidapi: signature from "Balló György <bgyorgy@archlinux.org>" is unknown trust
:: File /media/crucru/var/cache/pacman/pkg/hidapi-0.13.1-2-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] n
error: python-pydantic: signature from "Balló György <bgyorgy@archlinux.org>" is unknown trust
:: File /media/crucru/var/cache/pacman/pkg/python-pydantic-1.10.7-2-x86_64.pkg.tar.zst is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] n
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.
j♭ sudo pacman -S archlinux-keyring                                                                                                                                                                                                                                            ~
resolving dependencies...
looking for conflicting packages...

Packages (1) archlinux-keyring-20230320-1

Total Installed Size:  1.61 MiB
Net Upgrade Size:      0.02 MiB

:: Proceed with installation? [Y/n]
(1/1) checking keys in keyring                                                                                                                                      [######################################################################################################] 100%
(1/1) checking package integrity                                                                                                                                    [######################################################################################################] 100%
(1/1) loading package files                                                                                                                                         [######################################################################################################] 100%
(1/1) checking for file conflicts                                                                                                                                   [######################################################################################################] 100%
(1/1) checking available disk space                                                                                                                                 [######################################################################################################] 100%
:: Processing package changes...
(1/1) upgrading archlinux-keyring                                                                                                                                   [######################################################################################################] 100%
==> Appending keys from archlinux.gpg...
==> Disabling revoked keys in keyring...
  -> Disabled 3 keys.
==> Updating trust database...
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   6  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   6  signed:  95  trust: 1-, 0q, 0n, 5m, 0f, 0u
gpg: depth: 2  valid:  72  signed:  27  trust: 72-, 0q, 0n, 0m, 0f, 0u
gpg: next trustdb check due at 2023-07-12
==> Updating trust database...
gpg: next trustdb check due at 2023-07-12
:: Running post-transaction hooks...
(1/2) Reloading system manager configuration...
(2/2) Arming ConditionNeedsUpdate...
j♭ sudo pacman -Syu                                                                                                                                                                                                                                                            ~
:: Synchronizing package databases...
 core                                                                                                                                   153.3 KiB  2.49 MiB/s 00:00 [######################################################################################################] 100%
 extra                                                                                                                                 1773.5 KiB  23.7 MiB/s 00:00 [######################################################################################################] 100%
 community                                                                                                                                7.2 MiB  30.2 MiB/s 00:00 [######################################################################################################] 100%
 multilib is up to date
:: Starting full system upgrade...
:: Replace imagemagick-doc with extra/imagemagick? [Y/n]
:: Replace tbb with extra/onetbb? [Y/n]
resolving dependencies...
[...]

I've read some very interestings answers and facts, and thanks to them I do know more about how it works. I understand it's not that simple and it has been done as best could be or seemed to be. However, it still seems to me that something's not right in the message about an invalid or corrupted package.

Lone_Wolf · 2023-04-22 14:56:26

invalid or corrupted package

That's not the full message , (invalid or corrupted package (PGP signature)) is.

Tétrapyle · 2023-04-22 15:05:00

Yes, that's right. But still.

Tétrapyle · 2023-08-31 17:43:18

I updated my system 3 weeks ago.
Trying to update it today.

$ sudo pacman -Syu
[...]
Total (507/507)                                                                                                                          2.2 GiB  14.7 MiB/s 02:33 [######################################################################################################] 100%
(507/507) checking keys in keyring                                                                                                                                  [######################################################################################################] 100%
downloading required keys...
:: Import PGP key 9B7A287D9A2EC608, "David Runge <dvzrv@archlinux.org>"? [Y/n]
error: key "9B7A287D9A2EC608" could not be looked up remotely
error: required key missing from keyring
error: failed to commit transaction (unexpected error)
Errors occurred, no packages were upgraded.

$ sudo pacman -S archlinux-keyring
[...]
$ sudo pacman -Syu
[...]

Works.

I'm not sure how others are never hitting this.

Arch Linux

#1 2023-01-15 10:11:27

pacman prioritize packages in system upgrades?

#2 2023-01-15 12:59:39

Re: pacman prioritize packages in system upgrades?

#3 2023-01-15 19:43:36

Re: pacman prioritize packages in system upgrades?

#4 2023-01-15 19:44:36

Re: pacman prioritize packages in system upgrades?

#5 2023-01-15 19:47:22

Re: pacman prioritize packages in system upgrades?

#6 2023-01-15 20:41:52

Re: pacman prioritize packages in system upgrades?

#7 2023-01-15 20:46:49

Re: pacman prioritize packages in system upgrades?

#8 2023-01-16 00:38:38

Re: pacman prioritize packages in system upgrades?

#9 2023-01-16 00:43:24

Re: pacman prioritize packages in system upgrades?

#10 2023-01-16 09:14:56

Re: pacman prioritize packages in system upgrades?

#11 2023-01-16 10:33:51

Re: pacman prioritize packages in system upgrades?

#12 2023-01-16 11:34:02

Re: pacman prioritize packages in system upgrades?

#13 2023-01-16 13:25:41

Re: pacman prioritize packages in system upgrades?

#14 2023-01-16 13:46:23

Re: pacman prioritize packages in system upgrades?

#15 2023-01-16 14:27:07

Re: pacman prioritize packages in system upgrades?

#16 2023-01-16 14:54:32

Re: pacman prioritize packages in system upgrades?

#17 2023-01-16 20:43:00

Re: pacman prioritize packages in system upgrades?

#18 2023-01-16 22:10:25

Re: pacman prioritize packages in system upgrades?

#19 2023-01-17 02:56:56

Re: pacman prioritize packages in system upgrades?

#20 2023-01-17 11:25:07

Re: pacman prioritize packages in system upgrades?

#21 2023-01-17 11:50:12

Re: pacman prioritize packages in system upgrades?

#22 2023-04-21 18:03:02

Re: pacman prioritize packages in system upgrades?

#23 2023-04-22 14:56:26

Re: pacman prioritize packages in system upgrades?

#24 2023-04-22 15:05:00

Re: pacman prioritize packages in system upgrades?

#25 2023-08-31 17:43:18

Re: pacman prioritize packages in system upgrades?

Board footer