Statistics about used/unused packages

ttouch · 2013-11-17 11:50:39

Hello Arch forum,

I want a program that analyzes statistically the usage of the installed packages.
I don't want to measure the usage of plain files or libraries, so it will count packages installed manually (not dependencies) and that contain binary/executable files.
The reason I want to do that, is to be able to manage the monster I created. I use my desktop to test a lot of my insane ideas and the packages just sit there with the idea of "I'll finish this project later". And of course I forget about them...
I don't even know how to google that, so sorry if this is double post (please close the thread if this has been answered on another topic).

karol · 2013-11-17 11:57:58

I use vim.

I simply takes notes about the packages I install: when, why etc. When I'm done, it's 'pacman -Rns foo bar baz'.
If you have neglected the housekeeping, go through the list of installed packages and see if you know what each of them you need for.

graysky · 2013-11-17 12:00:28

Running `pacgraph -c` will give a nice list of installed packages listed by size... but that does not answer your original question about an automated method of keeping track

tomk · 2013-11-17 12:49:51

I've read your post a few times now, and I still have no idea what you're asking for.

First, can you define used/unused/usage? And what exactly does "packages installed manually" mean here? Are you installing stuff without pacman?

I also frequently forget about packages I have installed, but becuase I installed them with pacman, I know that I can tidy them up whenever I want.

Trilby · 2013-11-17 12:54:17

I read installed manually as installed explicitly (see the parenthetical note: not dependencies).

As far as what is used, do you not know what you use? I occasionally look at the output of `pacman -Qet` and if I see anything I don't recognize I check if it is in base or base-devel. If it isn't I remove it, if it is, I look closer (pacman -Ql <pkg>) to see if I can remove it.

EDIT: this will give all non base/base-devel packages that were explicitly installed and are not required by anything else:

for pkg in $(pacman -Qqet); do [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && echo $pkg; done

Last edited by Trilby (2013-11-17 13:01:50)

ttouch · 2013-11-17 13:39:57

@tomk: I meant explicitly as Tribly stated. No, I don't install things without pacman (ok, appart from dwm and dmenu, but they're not causing any problem at all)

@Tribly: Very useful script/command!
but, as stated I created a monster:

$ for pkg in $(pacman -Qqet); do [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && echo $pkg; done | wc -l
216

It's not that easy to look through 216 packages...

karol · 2013-11-17 13:46:13

Just curious, why do you install dwm & dmenu w/o pacman?

ttouch · 2013-11-17 13:52:05

It's far more simple (just make install)
I can debug (if make fails)
It has just 2 files (manual and binary) and I can't "forget" about them (It's my wm!)
I'm lazy (alter the md5s etc. etc. etc.)

Trilby · 2013-11-17 13:59:55

ttouch wrote:

It's not that easy to look through 216 packages...

Just one at a time. Get the list in one terminal, and in the other `pacman -Rsn ` then start appending any name from the first terminal that you don't know you need.

And FWIW, `makepkg -efi` seems just as easy as `make install` and checksums can be skipped either via the command line or with 'SKIP' in the PKGBUILD.

Last edited by Trilby (2013-11-17 14:01:30)

tomk · 2013-11-17 14:17:48

OK... I guess I'm just not getting what your concern is here.

You have installed various packages with pacman, you know you no longer need some of them, but you've forgotten what you installed, and even when presented with a list, you feel it's too much of an effort to go through it. Am I right so far?

What's the overall objective here, apart from vague notions like "to be able to manage the monster"? Is it simply the desire to recover disk space?

graysky · 2013-11-17 14:20:56

@tomk - I think the OP is wanting some metrics about file accesses and then mating that back to each package. How many times did I use the executable provided by foo? If 0 times in so many months, why do I have the package that provides foo installed. That is my take on the OP's post.

ttouch · 2013-11-17 14:24:46

@Tribly: it seems that there is no other way...
@tomk: I want a watchdog that will recommend me which packages I don't need, by analyzing the usage of each package (not every package, just those that are exclusively installed and contain executable files). I know I can do that manually, but it would save a great amount of time...
However I don't know how resource hungry would such a program be, but that's another topic...

karol · 2013-11-17 14:26:15

If access times are recorded, you can use them:

$ stat -c %x /usr/bin/yudit
2013-07-03 15:19:45.861327882 +0200
$ stat -c %x /usr/bin/pacman
2013-11-16 18:00:39.709435308 +0100

Seems I've been neglecting yudit :-)

ttouch · 2013-11-17 14:31:35

that's a good idea!
If I alter a bit Tribly's command, I think I'll get what I want!
Thank you guys!
I'll post the result...

karol · 2013-11-17 14:34:29

'stat' alone isn't that great since it will print binaries you haven't touched in a while, but you might be using other binaries from the same package every day.

ttouch · 2013-11-17 15:14:04

I know that, so I'll put a flag (ex. old =1) and even if one binary is used recently it will set the flag to 0 and break from the sub loop (it will loop through the binaries of each package), then I'll check the flag's status and either echo the package or not.
EDIT: found better way: find with -amin argument

Last edited by ttouch (2013-11-17 15:22:02)

Trilby · 2013-11-17 15:14:24

Also stat depends on the accesstimes - and depending on your mount options, they may not represent what you want them to.

The following command could be run periodically over a period of time to generate a list of processes that have run in that period of time, though it will miss any transient processes that are not "caught in the act":

sort -u <(ps -e -o comm) <~/binlist >~/binlist.tmp && mv binlist.tmp binlist

This can be run in a loop (even repeating every second) and it will use fewer system resources than any system monitor tool like conky or any status bar tools.

Last edited by Trilby (2013-11-17 15:22:45)

ttouch · 2013-11-17 15:25:22

I think that this will lose even more binaries, or it will blow your pc up by running once a second.

Trilby · 2013-11-17 15:25:58

ttouch wrote:

or it will blow your pc up by running once a second.

Eh, no - I have *very* old hardware, and this looping every second doesn't even register on system monitors for memory or cpu use.

Last edited by Trilby (2013-11-17 15:28:42)

ttouch · 2013-11-17 16:26:01

@Tribly: I don't think that state will be such a problem...
And as stated, I want suggestions, I don't mind if it's not accurate!
Here is the function:

oldpackages() {
    for pkg in $(pacman -Qqet); do
        [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && \
        [[ -n "$(find $(pacman -Ql $pkg | grep bin | awk '{print $2}' | grep -v '^/.*/$') -type f -executable -atime +$1)" ]] && \                                                                                             
        echo $pkg
    done
}

usage: oldpackages <more than n days old>

Thank you very much guys!

karol · 2013-11-17 16:36:38

You can tidy up the script: 'pacman -Qlq' will help with

pacman -Ql $pkg | grep bin | awk '{print $2}'

Why 'grep bin'? Are you sure you don't have anything in /opt? This kind of packages are easiest to overlook.

ttouch · 2013-11-17 17:08:56

you're right on both points.
EDIT: Also add 2>/dev/null at the end of the find, so it does not spit errors if files are missing (I use localepurge to clear unwanted locales)
EDIT 2: Also, if instead of pacman -Qqet, we use -Qet and grep it to remove description and "local/" we can take the group without -Qi the package. But too much effort for just a little performance bump.

Last edited by ttouch (2013-11-17 17:43:28)

Arch Linux

#1 2013-11-17 11:50:39

Statistics about used/unused packages

#2 2013-11-17 11:57:58

Re: Statistics about used/unused packages

#3 2013-11-17 12:00:28

Re: Statistics about used/unused packages

#4 2013-11-17 12:49:51

Re: Statistics about used/unused packages

#5 2013-11-17 12:54:17

Re: Statistics about used/unused packages

#6 2013-11-17 13:39:57

Re: Statistics about used/unused packages

#7 2013-11-17 13:46:13

Re: Statistics about used/unused packages

#8 2013-11-17 13:52:05

Re: Statistics about used/unused packages

#9 2013-11-17 13:59:55

Re: Statistics about used/unused packages

#10 2013-11-17 14:17:48

Re: Statistics about used/unused packages

#11 2013-11-17 14:20:56

Re: Statistics about used/unused packages

#12 2013-11-17 14:24:46

Re: Statistics about used/unused packages

#13 2013-11-17 14:26:15

Re: Statistics about used/unused packages

#14 2013-11-17 14:31:35

Re: Statistics about used/unused packages

#15 2013-11-17 14:34:29

Re: Statistics about used/unused packages

#16 2013-11-17 15:14:04

Re: Statistics about used/unused packages

#17 2013-11-17 15:14:24

Re: Statistics about used/unused packages

#18 2013-11-17 15:25:22

Re: Statistics about used/unused packages

#19 2013-11-17 15:25:58

Re: Statistics about used/unused packages

#20 2013-11-17 16:26:01

Re: Statistics about used/unused packages

#21 2013-11-17 16:36:38

Re: Statistics about used/unused packages

#22 2013-11-17 17:08:56

Re: Statistics about used/unused packages

Board footer