A utility for cleaning /var/cache/pacman/pkg

alterkacker · 2005-01-08 03:11:54

I've been using Arch for a few months now after years of dealing with RedHat bloat - what a pleasure! Here's my small contribution - a python script to clean the /var/cache/pacman/pkg directory but allow you to specify how many package versions should be retained. Hope it's useful.

#!/usr/bin/env python
"""pkg_clean - a simple python script to clean up the /var/cache/pacman/pkg directory.
More versatile than 'pacman -Sc' in that you can select how many old versions
to keep.
Usage: pkg_clean {-p} {-v} <# of copies to keep>
  # of copies to keep - (required) how many generations of each package to keep
  -p - (optional) preview what would be deleted; forces verbose (-v) mode.
  -v - (optional) show deleted packages."""
# Note that the determination of package age is done by simply looking at the date-time
# modified stamp on the file. There is just enough variation in the way package version
# is done that I thought this would be simpler & just as good.
# Also note that you must be root to run this script.

import getopt
import os
import re
import sys

# cleanup does the actual pkg file deletion
def cleanup(run_list):
  # strictly speaking only the first of these globals needs to be listed.
  global n_deleted, opt_verbose, opt_preview, n_to_keep
  # return if the run_list is too short
  if len(run_list) <= n_to_keep: return
  # Build list of tuples (date-time file modified, file name)
  dtm_list = [(os.stat(tfn)[8], tfn) for tfn in run_list]
  # Sort the list by date-time
  dtm_list.sort()
  # Build list of the filenames to delete (all but last n_to_keep).
  kill_list = [tfn[1] for tfn in dtm_list[:-n_to_keep]]
  if opt_verbose: print kill_list
  n_deleted += len(kill_list)
  if not opt_preview:
    for dfn in kill_list:
      os.unlink(dfn)

######################################################################
# mainline processing

# process command line options
try:
  opts, pargs = getopt.getopt(sys.argv[1:], 'vp')
  opt_dict = dict(opts)
  opt_preview = opt_dict.has_key('-p')
  opt_verbose = opt_dict.has_key('-v')
  if opt_preview: opt_verbose = True
  if len(pargs) == 1:
    n_to_keep = pargs[0]
  else:
    raise getopt.GetoptError("missing required argument.")
  try:
    n_to_keep = int(n_to_keep)
    if n_to_keep <= 0: raise ValueError
  except ValueError, e:
    raise getopt.GetoptError("# of copies to keep must be numeric > 0!")
except getopt.GetoptError, msg:
  print "Error:",msg,"n",__doc__
  sys.exit(1)

# change to the pkg directory & get a sorted list of its contents
os.chdir('/var/cache/pacman/pkg')
pkg_fns = os.listdir('.')
pkg_fns.sort()

# for pkg e.g. 'gnome-common-2.8.0-1.pkg.tar.gz' group(1) is 'gnome-common'.
bpat = re.compile(r'(.+)-[^-]+-.+.pkg.tar.gz$')
n_deleted = 0
pkg_base_nm = ''
# now look for "runs" of the same package differing only in version info.
for run_end in range(len(pkg_fns)):
  fn = pkg_fns[run_end]
  mo = bpat.match(fn) # test for a match of the package name pattern
  if mo:
    tbase = mo.group(1) # gets the 'base' package name
    # is it a new base name, i.e. the start of a new run?
    if tbase != pkg_base_nm: # if so then process the prior run
      if pkg_base_nm != '':
        cleanup(pkg_fns[run_start:run_end])
      pkg_base_nm = tbase # & setup for the new run
      run_start = run_end
  else:
    print >>sys.stderr, "File '"+fn+"' doesn't match package pattern!"

# catch the final run of the list
run_end += 1
cleanup(pkg_fns[run_start:run_end])

if opt_verbose:
  print n_deleted,"files deleted."

uberGeek · 2005-04-17 19:04:40

Thank you!

Stinky · 2005-04-17 19:46:15

excellent!! I would not use pacman to clean out old packages for the very reason I wanted to keep at least one old version just in case a new package is buggy. I've been cleaning it out by hand.
Thanks!

phrakture · 2005-04-28 17:17:15

Hmm, if you want to hack at it - I'd say patch pacman... this should be the default "-Sc" behavior... "-Scc" should remain as it is.
Perhaps the "number of versions to keep" can be stored in pacman.conf

dp · 2005-04-28 18:25:22

phrakture wrote:

Hmm, if you want to hack at it - I'd say patch pacman... this should be the default "-Sc" behavior... "-Scc" should remain as it is.
Perhaps the "number of versions to keep" can be stored in pacman.conf

or maybe providing it to the -Sc :

pacman -Sc 2

phrakture · 2005-04-28 18:48:38

with that in place, it wouldn't be hard to provide a "rollback" thing too.... would be useful as that's one of the biggest pacman gripes "how do I downgrade a package?"

pacman --rollback foo-package 2

thinking outloud... I'm hoping the pacman lib gets done soon, so I can start implementing things like this 8) - right now I don't want to make pacman changes because it's supposed to change

dhave · 2007-09-19 17:35:08

Thanks -- from over two years in the future. [Mysterious music plays]

mclabaut · 2008-07-29 12:33:23

Some packages were still left behind.
I changed the pattern to:

bpat = re.compile(r'(.+)-[^-]+-[0-9]+(-.+)?.pkg.tar.gz$')

Without this modification, xulrunner-1.8.1.14-2-i686.pkg.tar.gz and xulrunner-1.9-1-i686.pkg.tar.gz were left unerased in presence of xulrunner-1.9.0.1-1-i686.pkg.tar.gz.

Vintendo · 2008-07-29 13:40:26

Why is this not yet implented in pacman?

Daenyth · 2008-07-29 13:43:39

Because the forum is not the bug tracker.

mclabaut · 2008-07-29 14:17:56

Task 11049 http://bugs.archlinux.org/task/11049 added

shining · 2008-07-29 14:39:46

Also because we don't really want to add that kind of complexity to pacman.
It is probably fine as an external python script.
But well, if you prefer, you can always try to do it yourself, just for fun.

Though it is worth mentioning that we already increased a bit the complexity of -Sc, by proposing 2 different behaviors for pacman 3.2 :
http://projects.archlinux.org/?p=pacman … be93db0ca3
But well, the core part was like 5 lines of C code, so not that bad.

XazZ · 2009-12-05 23:44:18

evening

I just updated alterkacker's script in the wiki
(link to the wiki article: http://wiki.archlinux.org/index.php/CacheClean)

what I did:
-added -any architecture (so this script does not just support i686 and x86_64)
-fixed a bug where the script would fail if there were any directories in the cache

my /var/cache/pacman/pkg is a mounted container file, so I got a lost+found directory in there
previously the script simply failed - now it shows an info message on stdout + it continues

oh, I almost forgot to mention: maybe someone could take a quick look at my changes?
I don't know python - so I hope they are ok

Regards

Last edited by XazZ (2009-12-05 23:45:00)

7x1x · 2010-02-17 09:47:14

Great work, the script works well from what I've used it. However libjpeg doesn't fit the naming pattern.

File 'libjpeg-8-1-x86_64.pkg.tar.gz' doesn't match package pattern!
File 'libjpeg-8-2-x86_64.pkg.tar.gz' doesn't match package pattern!

Shouldn't it be libjpeg-8.2 not 8-2? I'm not sure as to how many packages deviate from the normal naming system, so it may not be worth changing.

jerryluc · 2010-02-17 12:00:57

I've written a python program that does this. It's not up-to-date, so it doesn't support the "any" architecture. But it shouldn't be much work. Maybe i'll do it soon, but anyone could do it. I'm not a good programmer so it good be buggy, it could be badly written or something else. It works for me, though.

pacleaner in AUR

edit: I have now fixed it so it works and so it handles "any"

Last edited by jerryluc (2010-02-17 23:12:41)

7x1x · 2010-03-23 21:57:46

Hi, to update XazZ's script from the wiki to handle .gz and the new .xz just up date the following line:

bpat = re.compile(r'^(.+)-\d[^-]+-.+?(-i686|-x86_64|-any)?\.pkg\.tar.+(gz|xz)?$')

I know nothing about python, but it works from what I've seen.

Last edited by 7x1x (2010-04-04 22:09:20)

graysky · 2010-05-28 23:11:01

Necrobump of the year but the op did a fantastic job with this script. I packaged the most recent incarnation of it into the cacheclean package in the AUR. 5+ years later and people keep finding value in the op's work

stqn · 2010-05-29 11:32:50

Thanks for the update... Documentation bug: The post install message tells us the command is named "cache_clean", but it is "cacheclean.py".

Edit: the script itself tells us it is "cache_clean".

Last edited by stqn (2010-05-29 11:33:48)

binskipy2u · 2010-11-11 05:26:31

does pacman -Sc when it asks you..
pacman -Sc
Cache directory: /var/cache/pacman/pkg/
Do you want to remove uninstalled packages from cache? [Y/n]

are these the programs "older" versions that are still there? it will keep the latest right?

the size of this folder is 2.7 gigs...im sure alot of that is older versions right?
I can't figure out or find out how to run the script seems like greek to me..
but wll the above command get rid of older versions? (which i assume is what the command output means by "uninstalled packages")?

and then after that, this output comes up..

/var/lib/pacman/
Do you want to remove unused repositories? [Y/n]

i use mirror status and reflector..so i'll proberly say "no" to this..
but saying yes to the first one shouldn't be a problem if as far as you know everything is working ok?
just wondering

Last edited by binskipy2u (2010-11-11 05:28:07)

stqn · 2010-11-17 16:08:41

binskipy2u wrote:

does pacman -Sc when it asks you..
pacman -Sc
Cache directory: /var/cache/pacman/pkg/
Do you want to remove uninstalled packages from cache? [Y/n]
are these the programs "older" versions that are still there? it will keep the latest right?

Yes, assuming the latest versions are what is installed. "pacman -Sc" will remove packages for anything that is not currently installed. That includes:

a) packages for programs that are not installed.
b) packages for programs that are installed, but in a different version.
For example if deadbeef 0.4.4 is installed, pacman will remove the package for deadbeef 0.4.1 but keep for one for deadbeef 0.4.4.
And if you once installed KDE just to try it and then uninstalled it, all KDE packages will be deleted.

Narga · 2012-09-29 09:34:59

How to make it work with alternative (absolute) cache directory PATH include subfolder?

Arch Linux

#1 2005-01-08 03:11:54

A utility for cleaning /var/cache/pacman/pkg

#2 2005-04-17 19:04:40

Re: A utility for cleaning /var/cache/pacman/pkg

#3 2005-04-17 19:46:15

Re: A utility for cleaning /var/cache/pacman/pkg

#4 2005-04-28 17:17:15

Re: A utility for cleaning /var/cache/pacman/pkg

#5 2005-04-28 18:25:22

Re: A utility for cleaning /var/cache/pacman/pkg

#6 2005-04-28 18:48:38

Re: A utility for cleaning /var/cache/pacman/pkg

#7 2007-09-19 17:35:08

Re: A utility for cleaning /var/cache/pacman/pkg

#8 2008-07-29 12:33:23

Re: A utility for cleaning /var/cache/pacman/pkg

#9 2008-07-29 13:40:26

Re: A utility for cleaning /var/cache/pacman/pkg

#10 2008-07-29 13:43:39

Re: A utility for cleaning /var/cache/pacman/pkg

#11 2008-07-29 14:17:56

Re: A utility for cleaning /var/cache/pacman/pkg

#12 2008-07-29 14:39:46

Re: A utility for cleaning /var/cache/pacman/pkg

#13 2009-12-05 23:44:18

Re: A utility for cleaning /var/cache/pacman/pkg

#14 2010-02-17 09:47:14

Re: A utility for cleaning /var/cache/pacman/pkg

#15 2010-02-17 12:00:57

Re: A utility for cleaning /var/cache/pacman/pkg

#16 2010-03-23 21:57:46

Re: A utility for cleaning /var/cache/pacman/pkg

#17 2010-05-28 23:11:01

Re: A utility for cleaning /var/cache/pacman/pkg

#18 2010-05-29 11:32:50

Re: A utility for cleaning /var/cache/pacman/pkg

#19 2010-11-11 05:26:31

Re: A utility for cleaning /var/cache/pacman/pkg

#20 2010-11-17 16:08:41

Re: A utility for cleaning /var/cache/pacman/pkg

#21 2012-09-29 09:34:59

Re: A utility for cleaning /var/cache/pacman/pkg

Board footer