You are not logged in.

#1 2013-11-17 11:50:39

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Statistics about used/unused packages

Hello Arch forum,

I want a program that analyzes statistically the usage of the installed packages.
I don't want to measure the usage of plain files or libraries, so it will count packages installed manually (not dependencies) and that contain binary/executable files.
The reason I want to do that, is to be able to manage the monster I created. I use my desktop to test a lot of my insane ideas and the packages just sit there with the idea of "I'll finish this project later". And of course I forget about them...
I don't even know how to google that, so sorry if this is double post (please close the thread if this has been answered on another topic).

Offline

#2 2013-11-17 11:57:58

karol
Archivist
Registered: 2009-05-06
Posts: 25,434

Re: Statistics about used/unused packages

I use vim.

I simply takes notes about the packages I install: when, why etc. When I'm done, it's 'pacman -Rns foo bar baz'.
If you have neglected the housekeeping, go through the list of installed packages and see if you know what each of them you need for.

Offline

#3 2013-11-17 12:00:28

graysky
Member
From: The worse toilet in Scotland
Registered: 2008-12-01
Posts: 8,825
Website

Re: Statistics about used/unused packages

Running `pacgraph -c` will give a nice list of installed packages listed by size... but that does not answer your original question about an automated method of keeping track hmm


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#4 2013-11-17 12:49:51

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Statistics about used/unused packages

I've read your post a few times now, and I still have no idea what you're asking for.

First, can you define used/unused/usage? And what exactly does "packages installed manually" mean here? Are you installing stuff without pacman?

I also frequently forget about packages I have installed, but becuase I installed them with pacman, I know that I can tidy them up whenever I want.

Offline

#5 2013-11-17 12:54:17

Trilby
Forum Moderator
From: Massachusetts, USA
Registered: 2011-11-29
Posts: 14,817
Website

Re: Statistics about used/unused packages

I read installed manually as installed explicitly (see the parenthetical note: not dependencies).

As far as what is used, do you not know what you use?  I occasionally look at the output of `pacman -Qet` and if I see anything I don't recognize I check if it is in base or base-devel.  If it isn't I remove it, if it is, I look closer (pacman -Ql <pkg>) to see if I can remove it.

EDIT: this will give all non base/base-devel packages that were explicitly installed and are not required by anything else:

for pkg in $(pacman -Qqet); do [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && echo $pkg; done

Last edited by Trilby (2013-11-17 13:01:50)


InterrobangSlider
• How's my coding? See this page.
• How's my moderating? Feel free to email any concerns, complaints, or objections.

Offline

#6 2013-11-17 13:39:57

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

@tomk: I meant explicitly as Tribly stated. No, I don't install things without pacman (ok, appart from dwm and dmenu, but they're not causing any problem at all)

@Tribly: Very useful script/command!
but, as stated I created a monster:

$ for pkg in $(pacman -Qqet); do [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && echo $pkg; done | wc -l
216

It's not that easy to look through 216 packages... tongue

Offline

#7 2013-11-17 13:46:13

karol
Archivist
Registered: 2009-05-06
Posts: 25,434

Re: Statistics about used/unused packages

Just curious, why do you install dwm & dmenu w/o pacman?

Offline

#8 2013-11-17 13:52:05

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

It's far more simple (just make install)
I can debug (if make fails)
It has just 2 files (manual and binary) and I can't "forget" about them (It's my wm!)
I'm lazy (alter the md5s etc. etc. etc.)

Offline

#9 2013-11-17 13:59:55

Trilby
Forum Moderator
From: Massachusetts, USA
Registered: 2011-11-29
Posts: 14,817
Website

Re: Statistics about used/unused packages

ttouch wrote:

It's not that easy to look through 216 packages... tongue

Just one at a time.  Get the list in one terminal, and in the other `pacman -Rsn ` then start appending any name from the first terminal that you don't know you need.

And FWIW, `makepkg -efi` seems just as easy as `make install` and checksums can be skipped either via the command line or with 'SKIP' in the PKGBUILD.

Last edited by Trilby (2013-11-17 14:01:30)


InterrobangSlider
• How's my coding? See this page.
• How's my moderating? Feel free to email any concerns, complaints, or objections.

Offline

#10 2013-11-17 14:17:48

tomk
Forum Fellow
From: Ireland
Registered: 2004-07-21
Posts: 9,839

Re: Statistics about used/unused packages

OK... I guess I'm just not getting what your concern is here.

You have installed various packages with pacman, you know you no longer need some of them, but you've forgotten what you installed, and even when presented with a list, you feel it's too much of an effort to go through it. Am I right so far?

What's the overall objective here, apart from vague notions like "to be able to manage the monster"? Is it simply the desire to recover disk space?

Offline

#11 2013-11-17 14:20:56

graysky
Member
From: The worse toilet in Scotland
Registered: 2008-12-01
Posts: 8,825
Website

Re: Statistics about used/unused packages

@tomk - I think the OP is wanting some metrics about file accesses and then mating that back to each package.  How many times did I use the executable provided by foo?  If 0 times in so many months, why do I have the package that provides foo installed.  That is my take on the OP's post.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#12 2013-11-17 14:24:46

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

@Tribly: it seems that there is no other way...
@tomk: I want a watchdog that will recommend me which packages I don't need, by analyzing the usage of each package (not every package, just those that are exclusively installed and contain executable files). I know I can do that manually, but it would save a great amount of time...
However I don't know how resource hungry would such a program be, but that's another topic... tongue

Offline

#13 2013-11-17 14:26:15

karol
Archivist
Registered: 2009-05-06
Posts: 25,434

Re: Statistics about used/unused packages

If access times are recorded, you can use them:

$ stat -c %x /usr/bin/yudit
2013-07-03 15:19:45.861327882 +0200
$ stat -c %x /usr/bin/pacman
2013-11-16 18:00:39.709435308 +0100

Seems I've been neglecting yudit :-)

Offline

#14 2013-11-17 14:31:35

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

that's a good idea! big_smile
If I alter a bit Tribly's command, I think I'll get what I want!
Thank you guys!
I'll post the result...

Offline

#15 2013-11-17 14:34:29

karol
Archivist
Registered: 2009-05-06
Posts: 25,434

Re: Statistics about used/unused packages

'stat' alone isn't that great since it will print binaries you haven't touched in a while, but you might be using other binaries from the same package every day.

Offline

#16 2013-11-17 15:14:04

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

I know that, so I'll put a flag (ex. old =1) and even if one binary is used recently it will set the flag to 0 and break from the sub loop (it will loop through the binaries of each package), then I'll check the flag's status and either echo the package or not.
EDIT: found better way: find with -amin argument

Last edited by ttouch (2013-11-17 15:22:02)

Offline

#17 2013-11-17 15:14:24

Trilby
Forum Moderator
From: Massachusetts, USA
Registered: 2011-11-29
Posts: 14,817
Website

Re: Statistics about used/unused packages

Also stat depends on the accesstimes - and depending on your mount options, they may not represent what you want them to.

The following command could be run periodically over a period of time to generate a list of processes that have run in that period of time, though it will miss any transient processes that are not "caught in the act":

sort -u <(ps -e -o comm) <~/binlist >~/binlist.tmp && mv binlist.tmp binlist

This can be run in a loop (even repeating every second) and it will use fewer system resources than any system monitor tool like conky or any status bar tools.

Last edited by Trilby (2013-11-17 15:22:45)


InterrobangSlider
• How's my coding? See this page.
• How's my moderating? Feel free to email any concerns, complaints, or objections.

Offline

#18 2013-11-17 15:25:22

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

I think that this will lose even more binaries, or it will blow your pc up by running once a second.

Offline

#19 2013-11-17 15:25:58

Trilby
Forum Moderator
From: Massachusetts, USA
Registered: 2011-11-29
Posts: 14,817
Website

Re: Statistics about used/unused packages

ttouch wrote:

or it will blow your pc up by running once a second.

Eh, no - I have *very* old hardware, and this looping every second doesn't even register on system monitors for memory or cpu use.

Last edited by Trilby (2013-11-17 15:28:42)


InterrobangSlider
• How's my coding? See this page.
• How's my moderating? Feel free to email any concerns, complaints, or objections.

Offline

#20 2013-11-17 16:26:01

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

@Tribly: I don't think that state will be such a problem...
And as stated, I want suggestions, I don't mind if it's not accurate!
Here is the function:

oldpackages() {
    for pkg in $(pacman -Qqet); do
        [[ -z "$(pacman -Qi $pkg | grep "Groups *: base")" ]] && \
        [[ -n "$(find $(pacman -Ql $pkg | grep bin | awk '{print $2}' | grep -v '^/.*/$') -type f -executable -atime +$1)" ]] && \                                                                                             
        echo $pkg
    done
}

usage: oldpackages <more than n days old>

Thank you very much guys! big_smile

Offline

#21 2013-11-17 16:36:38

karol
Archivist
Registered: 2009-05-06
Posts: 25,434

Re: Statistics about used/unused packages

You can tidy up the script: 'pacman -Qlq' will help with

pacman -Ql $pkg | grep bin | awk '{print $2}' 

Why 'grep bin'? Are you sure you don't have anything in /opt? This kind of packages are easiest to overlook.

Offline

#22 2013-11-17 17:08:56

ttouch
Member
From: /dev/null
Registered: 2012-05-27
Posts: 130
Website

Re: Statistics about used/unused packages

you're right on both points.
EDIT: Also add 2>/dev/null at the end of the find, so it does not spit errors if files are missing (I use localepurge to clear unwanted locales)
EDIT 2: Also, if instead of pacman -Qqet, we use -Qet and grep it to remove description and "local/" we can take the group without -Qi the package. But too much effort for just a little performance bump.

Last edited by ttouch (2013-11-17 17:43:28)

Offline

Board footer

Powered by FluxBB