Nearly 70000 "lost files" ... how to manage?

amca01 · 2023-01-18 22:25:13

I don't consider myself a newbie; I've been using Arch happily for years, and Linux itself for years before that (way back to kernel 0.99). But I am very lazy, and of the "if it ain't broke, don't fix it" school. Anyway, every time I perform a system upgrade (which is far less often than I should), I run into the old "<file> exists in filesystem" error, which I usually manage by deleting them. (In this case almost all the errors are from files in the "/usr/lib/node_modules/npm/node_modules" directory.)

But this time I ran both the "lostfiles" script, and the "pacreport --unowned-files" command, in each case sending the output to a file, which I could examine at my leisure. Lostfiles returned 67597 files, and pacreport returned 3912 files.

In my hunting around forums and other postings, I find some people maintaining that a "<file> exists in filesystem" error is the result of an improperly maintained system. Other people maintain that most things should go in $HOME. I myself like to keep $HOME for personal files, and have apps installed globally.

I can fix all the individual errors myself, but I'm also curious. Over the years I've accumulated all sorts of apps and files; some installed using pacman, some from the AUR, some by hand. What, then, is a good strategy for maintaining a clean system, but allowing such flexibility?

cheers,
Al

mpan · 2023-01-19 02:08:52

This means you are using more than one program to manage your system and they will be in conflict. Except for package managers, which install exclusively to “/usr/local”, there is no way to prevent that kind of issues.⁽¹⁾ There are a few package-owned directories, which are meant to contain dynamic data, but those are specific exceptions.

In your particular case, looking at the name of the directory, you likely allow npm to control your system in parallel with pacman.

Simply use pacman only for global installation (other than “/usr/local”), and other package managers to install for your user only: npm, pip, cpan.

If you need to install a piece of software globally, make it into a pacman package: guides.
____
⁽¹⁾ Note that installing to “/usr/local” will only prevent conflicts with pacman-managed packages from official repos. It will not help with conflicts between other managers or breakage from incompatibilities.

amca01 · 2023-01-19 04:16:20

Thank you, mpan - that seems like very good advice. I have had the bad habit of installing everything globally - this is ingrained from the time when I used to have a small /home/ partition on my disk, which I needed to be careful not to overfill. (On my current Arch machine I have a 100G partition for /, currently 62% filled, and 120G for /home, which is 74% filled. I backup to a remote server, and also to local hard drives.)

But I do collect things from all over: pacman, AUR, specialized packages for systems like TeX or Python ...

Thanks again,
Al

Lone_Wolf · 2023-01-19 09:19:29

AUR packages should not be a problem in this regard, as they are managed by pacman.

I agree with mpan, especially "If you need to install a piece of software globally, make it into a pacman package" .

mpan · 2023-01-19 11:19:11

What takes 62G on your “/”? It is entirely possible to have that much, but perhaps there is some garbage there. For example old pacman cache, which grows quickly and many users are unaware. See paccache from community/pacman-contrib for cleaning it in a sane manner:

sudo paccache -rk3 # Removes cached files, except 3 last versions
sudo paccache -ruk0 # Removes all cached files for packages not installed

Also determine what uses disk space with ncdu (community/ncdu):

sudo ncdu -r /

Will take some time to collect data; -r for read-only to protect against accidental deletions from ncdu’s interface itself.

amca01 · 2023-01-19 13:00:34

Good question, mpan, and although I've used ncdu on my home directory, I've never done it on "/". I needed to include the "-x" switch as well. Anyway, some big space users are:

/usr/lib/ghc-9.0.2 (2.0Gb),
/usr/lib/python-3.10 (1.6Gb)
/usr/share/texmf-dist (2.9Gb)
/usr/share/doc (2.9Gb)
/var/cache/pacman (11.3Gb)

As to paccache, I used that not so long ago, so there's not a lot of old packages lying about.

Arch Linux

#1 2023-01-18 22:25:13

Nearly 70000 "lost files" ... how to manage?

#2 2023-01-19 02:08:52

Re: Nearly 70000 "lost files" ... how to manage?

#3 2023-01-19 04:16:20

Re: Nearly 70000 "lost files" ... how to manage?

#4 2023-01-19 09:19:29

Re: Nearly 70000 "lost files" ... how to manage?

#5 2023-01-19 11:19:11

Re: Nearly 70000 "lost files" ... how to manage?

#6 2023-01-19 13:00:34

Re: Nearly 70000 "lost files" ... how to manage?

Board footer