[solved] Python packages prevent pacman update

archelvetic · 2018-12-27 15:49:37

Hi

I've understood that I must not use pip without thoughts in future. Alas I did in past what probably causes the Pacman update problem below.

$ sudo pacman -Syyu

error: python-distlib: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-distlib-0.2.8-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: python-pytoml: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python-pytoml-0.1.20-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: python2-distlib: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python2-distlib-0.2.8-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: python2-pytoml: signature from "Eli Schwartz <eschwartz@archlinux.org>" is unknown trust
:: File /var/cache/pacman/pkg/python2-pytoml-0.1.20-1-any.pkg.tar.xz is corrupted (invalid or corrupted package (PGP signature)).
Do you want to delete it? [Y/n] y
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded

What did I try so far?

$ pip list

Package            Version  
------------------ ---------
anytree            2.4.3    
appdirs            1.4.3    
argh               0.26.2   
atomicwrites       1.2.0    
attrs              18.2.0   
beautifulsoup4     4.6.3    
black              18.6b4   
btrfsutil          1.0.0    
CacheControl       0.12.5   
certifi            2018.8.13
cffi               1.11.5   
chardet            3.0.4    
click              6.7      
colorama           0.3.9    
cssselect          1.0.3    
cycler             0.10.0   
Cython             0.28.5   
decorator          4.3.0    
defusedxml         0.5.0    
distlib            0.2.7    
distro             1.3.0    
docopt             0.6.2    
docutils           0.14     
html5lib           1.0.1    
idna               2.7      
img2pdf            0.3.1    
Jinja2             2.10     
keyutils           0.5      
kiwisolver         1.0.1    
lockfile           0.12.2   
louis              3.7.0    
lxml               4.2.5    
Markdown           2.6.11   
MarkupSafe         1.0      
matplotlib         2.2.3    
more-itertools     4.3.0    
msgpack            0.5.6    
natsort            5.4.1    
networkx           2.2      
nltk               3.3      
numpy              1.15.2   
ocrmypdf           7.0.3    
packaging          17.1     
pathtools          0.1.2    
pikepdf            0.3.2    
Pillow             5.2.0    
pip                18.0     
pipenv             2018.7.1 
pluggy             0.7.1    
progress           1.4      
py                 1.6.0    
PyAudio            0.2.11   
pycairo            1.17.0   
pycountry          18.5.26  
pycparser          2.18     
pyenchant          2.0.0    
Pygments           2.2.0    
PyGObject          3.30.1   
pyparsing          2.2.1    
PyPDF2             1.26.0   
pyPEG2             2.15.2   
PyQt5-sip          4.19.12  
pytest             3.7.3    
pytest-watch       4.2.0    
python-dateutil    2.7.3    
python-Levenshtein 0.12.0   
python-xmp-toolkit 2.0.1    
pytoml             0.1.19   
pytz               2018.5   
pywal              3.1.0    
pyxdg              0.26     
PyYAML             3.13     
qutebrowser        1.4.2    
ranger-fm          1.9.1    
reportlab          3.5.6    
requests           2.19.1   
retrying           1.3.3    
ruffus             2.7.0    
scikit-learn       0.19.2   
scipy              1.1.0    
Send2Trash         1.5.0    
setuptools         40.4.3   
sip                4.19.12  
six                1.11.0   
speedtest-cli      2.0.2    
sty                1.0.0b7  
termcolor          1.1.0    
toml               0.9.4    
trash-cli          0.17.1.14
udiskie            1.7.5    
urllib3            1.23     
virtualenv         16.0.0   
virtualenv-clone   0.3.0    
visidata           1.3.1    
Wand               0.4.5    
watchdog           0.9.0    
webencodings       0.5.1    
Whoosh             2.7.4

$ pip uninstall distlib

Cannot uninstall 'distlib'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

$ sudo pacman -Rns python-distlib

checking dependencies...
error: failed to prepare transaction (could not satisfy dependencies)
:: python-pip: removing python-distlib breaks dependency 'python-distlib'

Any ideas what else I could try?

I wish you all peaceful holidays

Last edited by archelvetic (2018-12-28 08:11:35)

eschwartz · 2018-12-27 16:05:13

1) Do not use -Syyu, since it will either waste bandwidth to no gain, or else cause a partial update. It will *never* be a better option than using one -y (as in -Syu), except in certain exotic fringe circumstances where you know exactly why the double -yy is beneficial (the only one I can think of, is recovering from db corruption due to captive portals).

2) The issue you are having is that you have not updated your system in two months. People who updated their systems 2 months ago received an "archlinux-keyring" package update containing a refreshed pacman-key keyring, which is required in order to update your system. There was a 45 day window to update, and now over the last 115 days, people who took a long time to update have hit the issue you just encountered.

Instructions for fixing this can be found here: https://wiki.archlinux.org/index.php/Pa … ion_failed

Do try to update more often, though.

...

Also see the following threads, which this is a duplicate of:

https://bbs.archlinux.org/viewtopic.php?id=242639
https://bbs.archlinux.org/viewtopic.php?id=242701
https://bbs.archlinux.org/viewtopic.php?id=242883

Last edited by eschwartz (2018-12-27 16:10:02)

headkase · 2018-12-27 16:23:10

eschwartz wrote:

1) Do not use -Syyu, since it will either waste bandwidth to no gain, or else cause a partial update. It will *never* be a better option than using one -y (as in -Syu), except in certain exotic fringe circumstances where you know exactly why the double -yy is beneficial (the only one I can think of, is recovering from db corruption due to captive portals).

A minor tangent, I use Syyu when I change my mirrors. That doesn't happen very often though, is that a danger for a partial update? I thought it was "do a full update from scratch" for Syyu..

Trilby · 2018-12-27 16:38:53

OP, your (current) problem has nothing to do with using pip. You just need to update your keyring. Note this exact error is covered in the wiki:
https://wiki.archlinux.org/index.php/Pa … ion_failed
Please check the wiki before posting.

Eschwartz, while I agree that the doubly 'y' is almost always useless and simply a (minor) drain on community resources, I can't see how it could possibly result in a partial upgrade. Can you elaborate on that?

Headkase, -Syyu is commonly recommended when one changes mirrors, but that really never made sense to me. If the new mirror has been updated more recently that the previous one, then the extra y is not needed as it will be synced anyways. If the new mirror has not been updated more recently than the previous one, then the extra y is not needed as there is no point in re-syncing. Different reasons, yes, but the result is the same: the extra y serves no purpose. If the old mirror was not just out of date but really broken and somehow misrepresenting the timestamps on the db, then the extra 'y' could be needed, but otherwise, not really. This exception is similar in flavor to eschwartz's: its not for when your dbs are simply stale or out of date, but when something has gone wrong and the dbs are "broken" in some way.

Last edited by Trilby (2018-12-27 16:43:44)

Scimmia · 2018-12-27 16:45:49

Trilby wrote:

Eschwartz, while I agree that the doubly 'y' is almost always useless and simply a (minor) drain on community resources, I can't see how it could possibly result in a partial upgrade. Can you elaborate on that?

Switch to an older mirror (your top one goes down, second in the list is used) and you end up with databases older than your current system.

eschwartz · 2018-12-27 16:46:41

headkase wrote:

eschwartz wrote:
1) Do not use -Syyu, since it will either waste bandwidth to no gain, or else cause a partial update. It will *never* be a better option than using one -y (as in -Syu), except in certain exotic fringe circumstances where you know exactly why the double -yy is beneficial (the only one I can think of, is recovering from db corruption due to captive portals).
A minor tangent, I use Syyu when I change my mirrors. That doesn't happen very often though, is that a danger for a partial update? I thought it was "do a full update from scratch" for Syyu..

Trilby wrote:

Eschwartz, while I agree that the doubly 'y' is almost always useless and simply a (minor) drain on community resources, I can't see how it could possibly result in a partial upgrade. Can you elaborate on that?

Because if you are switching mirrors, then using a single -y will do exactly what you want -- it will update your system to a newer state if the mirror is newer than your previous mirror, and it will leave you with the databases from the previous mirror if the new mirror is older. The worst that can happen is that you 404 on missing packages.

If you use -Syyu and the current mirror is equal to or newer than the previous mirror, the only harm done is a drain on community resources.

If you use -Syyu and the current mirror is older than the previous mirror, then you will get some warnings from pacman for local packages that are newer than the ones available in the repos (because everyone looks at warnings, right? ) and any new packages that you install with -S newpkg, will be older than the packages on your system. Partial upgrades of newer packages are just as much a partial upgrade as partial downgrades of older packages, except that the latter case is more famous (something I wish to rectify).

...

Since your intention in switching mirrors was *not* to perform an Arch Linux Archive rollback to a historic date, do not use -Syyu. In the exceedingly unlikely event that I am wrong and you would actually like to perform a historic rollback, 1) I encourage you to do so properly, using the ALA, 2) use -Syyuu in order to also downgrade packages as required.

Last edited by eschwartz (2018-12-27 16:49:05)

Trilby · 2018-12-27 17:54:30

Thanks. I hadn't thought of someone switching from a relatively up to date mirror to and older one. That seems to be an odd and foolish corner case. If someone is updating their mirror list, they shouldn't be chosing worse mirrors than what they already have. I suppose if they move and are updating their mirrorlist to be more local, they might coincidentally end up with mirrors a few hours behind what they had - but in such situations the travel to the new location likely took more than that few hours, so they couldn't really have any newer packages.

headkase · 2018-12-27 17:57:14

eschwartz wrote:

headkase wrote:
eschwartz wrote:
1) Do not use -Syyu, since it will either waste bandwidth to no gain, or else cause a partial update. It will *never* be a better option than using one -y (as in -Syu), except in certain exotic fringe circumstances where you know exactly why the double -yy is beneficial (the only one I can think of, is recovering from db corruption due to captive portals).
A minor tangent, I use Syyu when I change my mirrors. That doesn't happen very often though, is that a danger for a partial update? I thought it was "do a full update from scratch" for Syyu..
If you use -Syyu and the current mirror is older than the previous mirror, then you will get some warnings from pacman for local packages that are newer than the ones available in the repos (because everyone looks at warnings, right? ) and any new packages that you install with -S newpkg, will be older than the packages on your system. Partial upgrades of newer packages are just as much a partial upgrade as partial downgrades of older packages, except that the latter case is more famous (something I wish to rectify).

I have seen warnings like that very rarely over the years. Now I know of their actual significance, got it, never use "Syyu" always use "Syu". Thanks.

eschwartz · 2018-12-27 18:54:46

Trilby wrote:

Thanks. I hadn't thought of someone switching from a relatively up to date mirror to and older one. That seems to be an odd and foolish corner case. If someone is updating their mirror list, they shouldn't be chosing worse mirrors than what they already have. I suppose if they move and are updating their mirrorlist to be more local, they might coincidentally end up with mirrors a few hours behind what they had - but in such situations the travel to the new location likely took more than that few hours, so they couldn't really have any newer packages.

It doesn't even require switching mirrorlists, let's say an up to date mirror has temporary server problems and you drop back to a slower mirror which you -Syyu. Boom -- partial update situation.

Trilby · 2018-12-27 19:50:25

True. I still suspect that, while possible, that's a highly unlikely corner case in which one would have a mirror list including a top mirror with server problems and a second mirror that happends to be notably behind. So it's possibly but I think very highly unlikely for -Syyu to result in this sort of problem.

Although I may be underestimating how neglected some users' mirrorlists may be.

Last edited by Trilby (2018-12-27 19:51:11)

eschwartz · 2018-12-27 20:00:03

Trilby wrote:

Although I may be underestimating how neglected some users' mirrorlists may be.

It is indeed a corner case, and it is also equally the case that some users neglect their mirrorlists. It actually happens that people consider maintenance of their mirrorlist, to mean uncommenting all the mirrors because "more is better", and if they rank them at all it is only by adding the entirety of the country-specific mirrors to the top of the list (but having the ones from the other side of the world as backups). I don't have reliable data on whether these people trigger the edge cases I've been mentioning, but it's not implausible... then again these are not the people most likely to post about it here.

Anyway, I'm sure everyone realizes now that there's edge cases all over, when using options you shouldn't be using. To be fair, the fact that -yy exists at all is basically all about edge cases, so it doesn't surprise me one bit.

Arch Linux

#1 2018-12-27 15:49:37

[solved] Python packages prevent pacman update

#2 2018-12-27 16:05:13

Re: [solved] Python packages prevent pacman update

#3 2018-12-27 16:23:10

Re: [solved] Python packages prevent pacman update

#4 2018-12-27 16:38:53

Re: [solved] Python packages prevent pacman update

#5 2018-12-27 16:45:49

Re: [solved] Python packages prevent pacman update

#6 2018-12-27 16:46:41

Re: [solved] Python packages prevent pacman update

#7 2018-12-27 17:54:30

Re: [solved] Python packages prevent pacman update

#8 2018-12-27 17:57:14

Re: [solved] Python packages prevent pacman update

#9 2018-12-27 18:54:46

Re: [solved] Python packages prevent pacman update

#10 2018-12-27 19:50:25

Re: [solved] Python packages prevent pacman update

#11 2018-12-27 20:00:03

Re: [solved] Python packages prevent pacman update

Board footer