[SOLVED] ModuleNotFoundError in installing python packages

to_you_bannana · 2022-11-20 04:38:18

Recently I had decided to foolishly delete the python

site-packages

directory after multiple answers suggested it was safe to do so.

Many of my python applications stopped working as they are now missing modules. Each time I try to reinstall the said python applications, (gwe and reflector) they fail to build their python deps...
I wrote a script that tried to automatically install the python packages automatically but sometimes the python packages themselves required different python packages and it was a whole mess

Why won't my aur helper (paru) just install the packages as it did the first time?!?!
Following is one of the example logs when I try to install a broken package: http://sprunge.us/mp2adW

Last edited by to_you_bannana (2022-11-20 06:15:46)

Trilby · 2022-11-20 04:50:25

aarav2you wrote:

... after multiple answers suggested it was safe to do so.

Where were these answers? Stop going to wherever that was as removing that whole directory would break countless repo packages.

aarav2you wrote:

Why won't my aur helper (paru) just install the packages as it did the first time?!?!

Because this isn't a problem with installing AUR packages - you broke a potentially huge number of main repo packages on your system. And beyond that, if you see your AUR helper this way, it's not helping you; stop using it.

To fix your problem check with pacman to see which packages are broken (`pacman -Qk`) and then reinstall all of them.

EDIT: Actually as long as you haven't also done something to break your local package database, you could get a cleaner list of packages to reinstall with pacman -Qo:

pacman -Qo /usr/lib/python3.10/site-packages | pacman -S -

EDIT 2: The error output clearly shows that your site-packages directory exists and it still contains some content. So this just really doesn't add up. Did you selectively delete from that directory? What exactly did you do? Or was it all deleted and some of them restored by this script you mentioned? And what was in this script? Why does it sound like you were trying to manage dependencies manually? I take it you must have (again) circumvented pacman in doing this as pacman would have handled the dependencies for you (though I suppose it wouldn't reinstall a dependency that was already considered installed).

Last edited by Trilby (2022-11-20 04:58:20)

cfr · 2022-11-20 05:33:27

aarav2you wrote:

Recently I had decided to foolishly delete the python
site-packages
directory after multiple answers suggested it was safe to do so.

Er ... what?! Answers where and by whom? Not here and not the wiki, so presumably some random site somewhere.

To a close approximation, you should never remove files installed by pacman manually.

The files you deleted include files which are installed as part of the python package itself.

$ pacman -Ql python | grep site-packages
python /usr/lib/python3.10/site-packages/
python /usr/lib/python3.10/site-packages/README.txt

Admittedly, the README.txt won't be crucial, but likely you deleted the directory itself, too. ordered_set is provided by python-ordered-set:

$ pacman -Ql python-ordered-set | grep site-packages
python-ordered-set /usr/lib/python3.10/site-packages/
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set-4.1.0.dist-info/
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set-4.1.0.dist-info/METADATA
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set-4.1.0.dist-info/RECORD
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set-4.1.0.dist-info/WHEEL
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/__init__.py
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/__pycache__/
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/__pycache__/__init__.cpython-310.opt-1.pyc
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/__pycache__/__init__.cpython-310.pyc
python-ordered-set /usr/lib/python3.10/site-packages/ordered_set/py.typed

This is not in AUR, but the repos. You don't need paru and you don't need to build it.

aarav2you wrote:

Many of my python applications stopped working as they are now missing modules. Each time I try to reinstall the said python applications, (gwe and reflector) they fail to build their python deps...

reflector is not an AUR package. It is in the repos.

pacman -S reflector

aarav2you wrote:

Why won't my aur helper (paru) just install the packages as it did the first time?!?!

Because AUR helpers generally make it easy to make a mess, but leave you clueless how to recover from one. In this case, however, paru is only responsible for the second part. paru checks if necessary dependencies are satisfied and finds they are - because that's what your pacman database of installed packages claims. paru is hardly responsible if you've deleted a directory of crucial files. makepkg would be equally stymied.

pacman -Qkq --native

will generate a list of packages in the repos with files they are missing. These are files pacman thinks are installed, but they aren't.

pacman -Qkq --foreign

will generate a list of packages not in the repos and the files they are missing.

Start with the ones in the repos, which you can reinstall with pacman without building. Then figure out which of the other packages are in AUR (presumably most) and build them using makepkg. Or you might start by removing the foreign packages and then reinstall them, since you'll get dependency-checking then.

The first package to reinstall will almost certainly be python.

pacman -S python python-ordered-set

This will mess up the installation reasons in your database. If you don't want that, you can specify --asdeps as appropriate.

Trilby · 2022-11-20 05:45:31

cfr wrote:

This will mess up the installation reasons in your database...

No it wont. Reinstalling a package that is currently registered as installed (even if broken) will not change the installation reason (explicit vs dependency).

cfr · 2022-11-20 05:49:30

Trilby wrote:

No it wont. Reinstalling a package that is currently registered as installed (even if broken) will not change the installation reason (explicit vs dependency).

Sorry. I think I'm confusing it with replicating a set of packages on a new system.

to_you_bannana · 2022-11-20 06:14:57

Ah, thank you so much both of you, I have heeded your advice. I did not know before that the database was messed up which the aur helper relied upon. With both of your commands I used

 pacman -Qkq --native | cut -d ' ' -f1 | uniq | xargs pacman --asdeps --noconfirm -Sy

to fix the issue! I will have to research more before taking a suggestion from a random website. Works perfectly now!

to_you_bannana · 2022-11-20 06:22:20

Trilby wrote:

cfr wrote:
This will mess up the installation reasons in your database...
No it wont. Reinstalling a package that is currently registered as installed (even if broken) will not change the installation reason (explicit vs dependency).

I just realized that I had no reason of including asdeps in my command, furthermore I think I just ruined the installation reasons because it had some other packages which weren't dependencies. I did not see this before... nooo
Would it be a huge issue in the future?

Trilby · 2022-11-20 14:08:24

And how did you generate that pipleline command? There are several ill-advisable componenets there. You were told how to fix this, but instead you generated some rather convoluted command that went counter to all best practices and made the problem much worse. I'm not sure why we should give you advice on how to fix these problems if you'll just contort that advice into creating more problems.

First, pacman -Q would detect the packages with missing files, but it also detects all sorts of other issues - by using 'cut' you remove all that information and pass on every package that has any notification from any of the checks. You then use uniq and xargs which don't cause any problem here but are a bit silly - they serve absolutely no purpose. Then you use pacman with one set of flags that are well documented to break all sorts of things under the best of conditions, another that prevents you from having any opportunity to stop the impending carnage, and yet another that will whitewash installation reasons from your local database.

We just went from a very very simple fix that would have no side effects, to a complete mess from which there may be no simple recovery but will require a review of your pacman log to see what actually happened with that pipelined command, and a case-by-case sorting of packages into installation reasons of explicity / asdeps, then fixing each of those. Of course that's assuming the partial upgrade hasn't rendered the system unusuable ... I think I'd triage that first if I were handed your computer. But again, I'm not sure I want to give you the commands to do this if you'll just ignore them and do something else ridiculous instead.

Also note that while CFR recommended the --asdeps flag, it would have been pointless but harmless (other than creating a fair bit of extra commands) if used as instructed "as appropriate", not globally and without consideration.

Do you have backups from prior to this craziness? If so, restore, repent, and partial upgrade no-more. If not, do you have a Ubuntu installation disk? ...

Last edited by Trilby (2022-11-20 14:20:41)

Arch Linux

#1 2022-11-20 04:38:18

[SOLVED] ModuleNotFoundError in installing python packages

#2 2022-11-20 04:50:25

Re: [SOLVED] ModuleNotFoundError in installing python packages

#3 2022-11-20 05:33:27

Re: [SOLVED] ModuleNotFoundError in installing python packages

#4 2022-11-20 05:45:31

Re: [SOLVED] ModuleNotFoundError in installing python packages

#5 2022-11-20 05:49:30

Re: [SOLVED] ModuleNotFoundError in installing python packages

#6 2022-11-20 06:14:57

Re: [SOLVED] ModuleNotFoundError in installing python packages

#7 2022-11-20 06:22:20

Re: [SOLVED] ModuleNotFoundError in installing python packages

#8 2022-11-20 14:08:24

Re: [SOLVED] ModuleNotFoundError in installing python packages

Board footer