[SOLVED] Blocking package updates by version

ozwigh · 2019-03-12 18:02:14

Hi there.

Is there any simple way to block +gb8e5ebe82-type updates from official repos? (Why are dev versions there anyway? Shouldn't they be in the testing repo?) Too many troubles recently with broken dev packages (fontconfig, deluge etc).

Last edited by ozwigh (2019-03-13 02:42:03)

jasonwryan · 2019-03-12 18:04:16

man pacman.conf wrote:

IgnorePkg = package ...
Instructs pacman to ignore any upgrades for this package when performing a --sysupgrade. Shell-style glob patterns are allowed.

loqs · 2019-03-12 18:22:13

ozwigh wrote:

Is there any simple way to block +gb8e5ebe82-type updates from official repos? (Why are dev versions there anyway? Shouldn't they be in the testing repo?)

That naming convention does not indicate a dev package although the package could be using upstream head

Using fontconfig as an example
fontconfig 2:2.13.1+12+g5f5ec56-1
epoch 2 pkgver 2.13.1 plus 12 extra commits with the last commit having shortid 5f5ec56 release 1
Alternative naming stype using systemd as an example
systemd 241.7-2
pkgver 241 with seven commits from the stable branch release 2

If you want to hold packages back please be aware of the risks of partial updates.

V1del · 2019-03-12 18:27:24

Reads like a hard XY question. These kind of releases are usually done to fix issues noted in the latest "stable" releases, or provide compatibility with the rest of the system as libraries are being updated. You are much more likely to get into much worse trouble if you indiscriminately hold back packages just because they happen to be based on a git commit.

What's your actual issue?

ozwigh · 2019-03-12 18:33:11

jasonwryan wrote:

man pacman.conf wrote:
IgnorePkg = package ...
Instructs pacman to ignore any upgrades for this package when performing a --sysupgrade. Shell-style glob patterns are allowed.

Unfortunately, it doesn't allow blocking by combined name-version regexp, only by package name. So I need to manually detect all dev packages and blacklist them, then when a normal version is released, unblock. That's too much pain. What I want is blocking by regexp, like:

(?=[0-9a-fA-F]*)(?:.{8}|.{10})

Is it possible? I don't know pacman internals well, so that's why I'm asking.

ozwigh · 2019-03-12 18:38:28

V1del wrote:

Reads like a hard XY question. These kind of releases are usually done to fix issues noted in the latest "stable" releases, or provide compatibility with the rest of the system as libraries are being updated. You are much more likely to get into much worse trouble if you indiscriminately hold back packages just because they happen to be based on a git commit.
What's your actual issue?

Last "fix" for deluge, for example, broke new torrents downloading, they stuck at "checking 0.0%" and daemon itself can be restarted only by kill -9. I don't like that and prefer to wait for a stable release.

ozwigh · 2019-03-12 18:43:05

loqs wrote:

ozwigh wrote:
Is there any simple way to block +gb8e5ebe82-type updates from official repos? (Why are dev versions there anyway? Shouldn't they be in the testing repo?)
That naming convention does not indicate a dev package although the package could be using upstream head
Using fontconfig as an example
fontconfig 2:2.13.1+12+g5f5ec56-1
epoch 2 pkgver 2.13.1 plus 12 extra commits with the last commit having shortid 5f5ec56 release 1
Alternative naming stype using systemd as an example
systemd 241.7-2
pkgver 241 with seven commits from the stable branch release 2
If you want to hold packages back please be aware of the risks of partial updates.

I'm aware. But so far I had troubles mostly from dev packages. For example, fontconfig 2:2.13.1+12+g5f5ec56-1 somehow breaks subtitle support for all possible media players. Yep, it's most likely 'cause of some local font. But rolling back to 1:2.13.1-1 solves the problem.

Edit: systemd commits don't scare me that much, 'cause it's generally relatively well tested. Except for that pain with unlocking encrypted volumes recently.

Last edited by ozwigh (2019-03-12 18:46:57)

graysky · 2019-03-12 18:43:50

ozwigh wrote:

Last "fix" for deluge, for example, broke new torrents downloading, they stuck at "checking 0.0%" and daemon itself can be restarted only by kill -9. I don't like that and prefer to wait for a stable release.

You will be waiting a long time. The last stable release is from 2017. Just downgrade the last-known-good version for you and add 'deluge' to your Ignore array. Alternatively, try the git version from the AUR. Perhaps your issue if fixed upstream.

Last edited by graysky (2019-03-12 18:45:50)

ozwigh · 2019-03-12 18:53:38

graysky wrote:

ozwigh wrote:
Last "fix" for deluge, for example, broke new torrents downloading, they stuck at "checking 0.0%" and daemon itself can be restarted only by kill -9. I don't like that and prefer to wait for a stable release.
You will be waiting a long time. The last stable release is from 2017. Just downgrade the last-known-good version for you and add 'deluge' to your Ignore array. Alternatively, try the git version from the AUR. Perhaps your issue if fixed upstream.

What's wrong with 2 years old stable release if it's stable? I know how to ignore a package, but I want to avoid unnecessary manual labor: "Oh, my torrents are stuck. The hell? systemctl restart deluged. Fail. Wut? killall -9 deluged. downgrade deluged. reboot. add deluge to Ignore or remember to run syu --ignore=deluge every next upgrade." Why, if it can be avoided?

loqs · 2019-03-12 19:04:59

For the fontconfig I would suggest checking the git master or bisecting between 5f5ec5676c61b9773026a9335c9b0dfa73a73353 and 844d8709a1f3ecab45015b24b72dd775c13b2421 (2.31.1)

seth · 2019-03-12 20:18:07

You predominatly want the bugs to get fixed, because that will not happen by itself - notably not if it desn't get fixed asap w/ the next package release, because it likely means the problem doesn't exist outside your system.
So you want to file a bug against the package (to raise awareness and seek to change to a better commit) and upstream (to fix the issue or be told "not a a bug, fix your config")

The idea that single bad releases would justify widespread systematic use of partial updates, let alone ignoring pot. important CVE fixes in entirely different packages w/ a more reliable upstream, is … "questionable".
If you're looking for a more conservative distro with fewer updates and longer integration tests, debian is probably your choice.

ozwigh · 2019-03-13 00:28:15

seth wrote:

You predominatly want the bugs to get fixed, because that will not happen by itself - notably not if it desn't get fixed asap w/ the next package release, because it likely means the problem doesn't exist outside your system.

I don't think so. It worked flawlessly for years until a chain of dev updates (for a dead 1.x branch when they're focusing 2 years on 2.x) ruined everything. I really can't understand the logic behind decisions like that. Why pushing random commits, which obviously weren't tested enough (if at all), into the main repos? (deluge 1.3.15 was stable as a tank, working for months without reboot under the heavy load)

seth wrote:

So you want to file a bug against the package (to raise awareness and seek to change to a better commit) and upstream (to fix the issue or be told "not a a bug, fix your config")

I really prefer to avoid repeatable "dev update -> broken -> fix for dev update which introduces another bug -> repeat" cycle at all. Sometimes there is no time and/or desire to investigate the source of the bug, especially if it affects server software (even if it's a home server). If rollback solves the problem, it's a preferable solution. If numbered commit updates introduce problems, it's logical to eliminate the source of the problem, no?

seth wrote:

The idea that single bad releases would justify widespread systematic use of partial updates, let alone ignoring pot. important CVE fixes in entirely different packages w/ a more reliable upstream, is … "questionable".
If you're looking for a more conservative distro with fewer updates and longer integration tests, debian is probably your choice.

No, thanks. I switched from debian to Arch about 10 years ago and I don't want that dismal distro and its horrible package manager back. There were no problems until some time (year? two?) ago not tested dev build somehow found their way into official repos (I remember that I felt uneasy after seeing them for the first time, and for a reason). Why? No idea. Arch is a rolling distro, yes, but what's the point in turning it to alpha-testing distro? A rhetoric question, of course.

So, to summarize. Thank you, everyone, for lecturing, I really appreciate it. I wanted a simple answer to a simple question: yes/no and, if yes, how. So finally I answering myself after learning it the hard way: no, it's not possible 'cause it's alpm limitation (there is no hook for that). A wrapper script (or something that temporarily modifies pacman.conf) is the only way. Should've waste time writing it instead of asking questions. /sigh_smile

ozwigh · 2019-03-13 02:37:45

To productively close a trashtalk topic...

Pacman wrapper script (requires python) to block packages by version (or name/version), ppf(.py) [Pacman Package Filter]:

#!/usr/bin/python

import sys
import os
import subprocess
import collections
import re

VERSION = "R1"
CONFIG_FILENAME = '/etc/ppf.conf'

if os.geteuid() != 0:
    print("!! This script requires root priveleges.")
    sys.exit(1)

if len(sys.argv) < 2:
    print("?? usage: {} [pacman arguments]".format(os.path.basename(sys.argv[0])))
    sys.exit(1)

if not os.path.isfile(CONFIG_FILENAME):
    print("!! Config file `{}` not found.".format(CONFIG_FILENAME))
    sys.exit(1)
else:
    try:
        with open(CONFIG_FILENAME, 'r') as f:
            filters = f.readlines()
    except OSError as e:
        print("!! Error: {}".format(e))
        sys.exit(1)

Package = collections.namedtuple("Package", "name ver new ignored")

def normalize_packages(packages):
    result = []
    for p in packages:
        fields = p.split(' ')
        ignored = (len(fields) >= 5 and 'ignored' in fields[4])
        t = Package(fields[0], fields[1], fields[3], ignored)
        result.append(t)
    return result

def is_blacklisted(pkg, filters):
    for f in filters:
        if re.search(f, "{}-{}".format(pkg.name, pkg.new)):
            if not pkg.ignored:
                return True
    return False

blacklist = {}
filters = [x for x in [x.rstrip('\n').strip() for x in filters] if x]

if len(filters) == 0:
    print("?? No filters defined, passing through.")
else:
    print(">> {} filter(s) loaded.".format(len(filters)))
    packages = normalize_packages([x for x in subprocess.check_output(['pacman', '--color=never', '-Qu']).decode('utf-8').split("\n") if x])
    if len(packages) == 0:
        print(">> No packages to upgrade.")
        sys.exit(0)
    else:
        for pkg in packages:
            if is_blacklisted(pkg, filters):
                blacklist[pkg.name] = pkg.new

if len(blacklist) == 0:
    print(">> No packages blocked.")
    cmd = None
else:
    for b in blacklist:
        print(">> Blocked: `{}` to `{}`.".format(b, blacklist[b]))
        cmd = '--ignore={}'.format(",".join(list(blacklist.keys())))

if cmd:
    pacman_args = ['pacman', cmd] + sys.argv[1:]
else:
    pacman_args = ['pacman'] + sys.argv[1:] 
sys.exit(subprocess.call(pacman_args))

Config file, /etc/ppf.conf (one regexp per line), for example (to block dev commit packages):

\+[0-9a-hA-H]{8}-
\+[0-9a-hA-H]{10}-

Usage is simple, for example:

ppf -Syu

Sample output:

$ sudo ppf -Syu
>> 2 filter(s) loaded.
>> Blocked: `deluge` to `1.3.15+18+ge050905b2-1`.
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
 multilib is up to date
:: Starting full system upgrade...
warning: deluge: ignoring package upgrade (1.3.15-1 => 1.3.15+18+ge050905b2-1)
warning: fontconfig: ignoring package upgrade (1:2.13.1-1 => 2:2.13.1+12+g5f5ec56-1)
resolving dependencies...

Last edited by ozwigh (2019-03-13 02:40:06)

seth · 2019-03-13 09:19:20

You've been "lectured" that this is the wrong approach to begin with. I cannot speak to that torrent client, but the two most recent releases refer to upstream commits from November 2017 and January 2018 …
I assume the actual last commit is the culprit because it apparently changes a naming scheme for magnetic links what might break for running torrents (only)

Once again: the idea to block all patched releases (technically you're discriminating git commits, svn isn't covered) because a torrent client turned out to cause you some (likely transitional) issue is just setting you up for far bigger trouble down the road.
Even ignoring security concerns: This atm. covers several libraries and a missed ABI recompilation of any of those will easily render large parts of your system unusable.

Hold specific leaf packages if you don't trust upstream, even better communicate to the package maintainer, but your happy approach to systematically get your packages out of sync /will/ cause you major trouble at some point. Don't forget to mention that you intentionally break your system when you complain about that future incident here.

ozwigh · 2019-03-13 19:35:47

seth wrote:

You've been "lectured" that this is the wrong approach to begin with. I cannot speak to that torrent client, but the two most recent releases refer to upstream commits from November 2017 and January 2018 …
I assume the actual last commit is the culprit because it apparently changes a naming scheme for magnetic links what might break for running torrents (only)
Once again: the idea to block all patched releases (technically you're discriminating git commits, svn isn't covered) because a torrent client turned out to cause you some (likely transitional) issue is just setting you up for far bigger trouble down the road.

SVN isn't covered 'cause there were no problems with them (yet). But it's not hard to block them (as well as all "pre-1", "beta-2" etc). Torrent client is just the latest example. Recently I can remember problems with NetworkManager and modemmanager which were introduced by random commits. So far, commit updates never fixed serious bugs (unless the previous commit introduced them), but created a lot of troubles. I can understand when commit update is a critical/security fix, but they're mostly random untested changes (seriously, is it sane to change magnet naming scheme in the dead branch after two years of developing new major release?!).

seth wrote:

Even ignoring security concerns: This atm. covers several libraries and a missed ABI recompilation of any of those will easily render large parts of your system unusable.

It's not hard to recompile an old package, isn't it?

seth wrote:

Hold specific leaf packages if you don't trust upstream, even better communicate to the package maintainer, but your happy approach to systematically get your packages out of sync /will/ cause you major trouble at some point. Don't forget to mention that you intentionally break your system when you complain about that future incident here.

I said nowhere "block forever and forget". Block during a system update, to avoid momentary problems, and review manually later (when mood and time allow). Test in VM first, not on real PC. Is it a bad approach? But no, I'm "system breaker" now. Don't think I'll ever "complain" here again. Arch is great distro (EXCEPT the new policy of using not tested dev commits in main repos), but the community is... arrogant to say the least.

Last edited by ozwigh (2019-03-13 19:36:31)

seth · 2019-03-13 20:42:42

'cause there were no problems with them (yet)

Good thinking! Now take it through and understand that ignoring packages you do not trust because upstream makes wonky patches (btw, you do realize that there're other, maybe more reliable torrent clients?) and the maintainer does not necessarily catch them is a good idea.
Blindly blocking updates of things™ because other things crapped on you is not.

seriously, is it sane to change magnet naming scheme in the dead branch after two years of developing new major release?

From the commit message, this is a bugfix:

The bug is: when you add a new magnet link to download, even if you tick
the option *Copy of .torrent files to:*, deluge still fails to save a
copy of torrent. This patch adds extra code to do so.

I cannot say whether it's justified for that release cycle.

It's not hard to recompile an old package, isn't it?

No, but I don't think you understood the problem.
If something like fontconfig gets ABI incompatible for skipping an update, most of your GUI software will fail to start or crash on stack overflows. (and if this ever happened on libc or curses, you'd not even be able to boot the system)
What do you intend to do w/ such situation? Replace the distro package w/ your local compilation of the last package version you deemed good, taking it out of distro package control entirely?
Do you think that makes things easier for you?

Test in VM first

If integration tests on your site are an option to begin with, why don't you follow that in general to avoid issues on the production system?

I don't care what you think of "the community" or any of its members (me) but please be aware that facts are not "arrogant" - they tend to be stubborn, though.

ozwigh · 2019-03-14 03:05:46

seth wrote:

'cause there were no problems with them (yet)
Good thinking! Now take it through and understand that ignoring packages you do not trust because upstream makes wonky patches (btw, you do realize that there're other, maybe more reliable torrent clients?)

Yep. It's deluge without suspicious commits. Worked well for years under (extremely) heavy load. Also, there is no real alternative for a server-side headless torrent client (no, transmission won't do).

Also. It's not an upstream fault. They didn't do any releases, not even marked it as "pre-alpha-2" or so, it's just a commit. No reason to use in a major distro (unless it's sabotage). It's clearly packager's fault.

On the other hand, (for example) firefox 65 has a somewhat critical bug when it freezes on save dialog. It's already fixed in 66 branch, and most likely backported to 65. But. We don't have this critical dev commit for quite a popular app in Arch repos. But we have commit which breaks relatively unknown deluge. Logic? Where?

seth wrote:

Blindly blocking updates of things™ because other things crapped on you is not.

I said a few times - it's not "blindly". Again: it's a thoughtful decision. I prefer to risk some mythical problems (never happened) to avoid some real (I'd say - persistent) problems with commit updates.

seth wrote:

No, but I don't think you understood the problem.
If something like fontconfig gets ABI incompatible for skipping an update, most of your GUI software will fail to start or crash on stack overflows. (and if this ever happened on libc or curses, you'd not even be able to boot the system)
What do you intend to do w/ such situation? Replace the distro package w/ your local compilation of the last package version you deemed good, taking it out of distro package control entirely?
Do you think that makes things easier for you?

When that'll happen, I'll restore the system from a snapshot and will investigate a problem (when I have time), not earlier. "Blocking all updates until everything is sorted out" maybe not the noblest approach, but it's a good, solid approach.

seth wrote:

If integration tests on your site are an option to begin with, why don't you follow that in general to avoid issues on the production system?

I don't always have time to play with the package testing (again, non-releases should be in the testing repo) and certainly don't want to do it every -Syu. There was some (actually, huge) degree of trust about the quality of updates, but, sadly, it's changed since dev commits appear in non-testing repos.

seth wrote:

I don't care what you think of "the community" or any of its members (me) but please be aware that facts are not "arrogant" - they tend to be stubborn, though.

I didn't ask for a facts, but a simple yes/no question. No one even answered "No, it's not possible with pacman/libalpm" before starting to preach. Also, I marked the thread as [SOLVED] - it should be closed now, no? Let's do it.

Scimmia · 2019-03-14 03:14:21

What you're not getting is that doing this WILL break things. Not might, WILL. That's why partial updates are not supported. If you do this, you are completely on your own, you cannot get any support here, on the mailing lists, or on IRC. If you don't understand why this breaks things, you have little hope of making it.

V1del · 2019-03-14 13:03:44

This isn't going anywhere. As already mentioned, if you don't want to listen to well meant advice (I'm hardpressed to find anyone "trash talking" or "antagonizing" you here...) that this approach will certainly give you more trouble than the issue it fixes), then that's your call. The claim that inherently dev commits would decrease the quality or stability of the distribution as a whole simply doesn't hold for a large amount of them (FWIW I am somewhat willing to believe that the deluge issue is genuine (though, considering that deluge seems to be relatively popular here and I've yet to see anyone but you run into the issue you've detailed)) however applying that to every package is guaranteed to cause issues eventually.

In that case, please don't open issues or bbs posts caused by the fact that you are running partial updates.

Closing.

Last edited by V1del (2019-03-14 13:06:45)

Arch Linux

#1 2019-03-12 18:02:14

[SOLVED] Blocking package updates by version

#2 2019-03-12 18:04:16

Re: [SOLVED] Blocking package updates by version

#3 2019-03-12 18:22:13

Re: [SOLVED] Blocking package updates by version

#4 2019-03-12 18:27:24

Re: [SOLVED] Blocking package updates by version

#5 2019-03-12 18:33:11

Re: [SOLVED] Blocking package updates by version

#6 2019-03-12 18:38:28

Re: [SOLVED] Blocking package updates by version

#7 2019-03-12 18:43:05

Re: [SOLVED] Blocking package updates by version

#8 2019-03-12 18:43:50

Re: [SOLVED] Blocking package updates by version

#9 2019-03-12 18:53:38

Re: [SOLVED] Blocking package updates by version

#10 2019-03-12 19:04:59

Re: [SOLVED] Blocking package updates by version

#11 2019-03-12 20:18:07

Re: [SOLVED] Blocking package updates by version

#12 2019-03-13 00:28:15

Re: [SOLVED] Blocking package updates by version

#13 2019-03-13 02:37:45

Re: [SOLVED] Blocking package updates by version

#14 2019-03-13 09:19:20

Re: [SOLVED] Blocking package updates by version

#15 2019-03-13 19:35:47

Re: [SOLVED] Blocking package updates by version

#16 2019-03-13 20:42:42

Re: [SOLVED] Blocking package updates by version

#17 2019-03-14 03:05:46

Re: [SOLVED] Blocking package updates by version

#18 2019-03-14 03:14:21

Re: [SOLVED] Blocking package updates by version

#19 2019-03-14 13:03:44

Re: [SOLVED] Blocking package updates by version

Board footer