You are not logged in.

#1 2005-01-08 03:11:54

alterkacker
Member
From: Peoples Republic of Boulder
Registered: 2005-01-08
Posts: 52

A utility for cleaning /var/cache/pacman/pkg

I've been using Arch for a few months now after years of dealing with RedHat bloat - what a pleasure! Here's my small contribution - a python script to clean the /var/cache/pacman/pkg directory but allow you to specify how many package versions should be retained. Hope it's useful.

#!/usr/bin/env python
"""pkg_clean - a simple python script to clean up the /var/cache/pacman/pkg directory.
More versatile than 'pacman -Sc' in that you can select how many old versions
to keep.
Usage: pkg_clean {-p} {-v} <# of copies to keep>
  # of copies to keep - (required) how many generations of each package to keep
  -p - (optional) preview what would be deleted; forces verbose (-v) mode.
  -v - (optional) show deleted packages."""
# Note that the determination of package age is done by simply looking at the date-time
# modified stamp on the file. There is just enough variation in the way package version
# is done that I thought this would be simpler & just as good.
# Also note that you must be root to run this script.

import getopt
import os
import re
import sys

# cleanup does the actual pkg file deletion
def cleanup(run_list):
  # strictly speaking only the first of these globals needs to be listed.
  global n_deleted, opt_verbose, opt_preview, n_to_keep
  # return if the run_list is too short
  if len(run_list) <= n_to_keep: return
  # Build list of tuples (date-time file modified, file name)
  dtm_list = [(os.stat(tfn)[8], tfn) for tfn in run_list]
  # Sort the list by date-time
  dtm_list.sort()
  # Build list of the filenames to delete (all but last n_to_keep).
  kill_list = [tfn[1] for tfn in dtm_list[:-n_to_keep]]
  if opt_verbose: print kill_list
  n_deleted += len(kill_list)
  if not opt_preview:
    for dfn in kill_list:
      os.unlink(dfn)

######################################################################
# mainline processing

# process command line options
try:
  opts, pargs = getopt.getopt(sys.argv[1:], 'vp')
  opt_dict = dict(opts)
  opt_preview = opt_dict.has_key('-p')
  opt_verbose = opt_dict.has_key('-v')
  if opt_preview: opt_verbose = True
  if len(pargs) == 1:
    n_to_keep = pargs[0]
  else:
    raise getopt.GetoptError("missing required argument.")
  try:
    n_to_keep = int(n_to_keep)
    if n_to_keep <= 0: raise ValueError
  except ValueError, e:
    raise getopt.GetoptError("# of copies to keep must be numeric > 0!")
except getopt.GetoptError, msg:
  print "Error:",msg,"n",__doc__
  sys.exit(1)

# change to the pkg directory & get a sorted list of its contents
os.chdir('/var/cache/pacman/pkg')
pkg_fns = os.listdir('.')
pkg_fns.sort()

# for pkg e.g. 'gnome-common-2.8.0-1.pkg.tar.gz' group(1) is 'gnome-common'.
bpat = re.compile(r'(.+)-[^-]+-.+.pkg.tar.gz$')
n_deleted = 0
pkg_base_nm = ''
# now look for "runs" of the same package differing only in version info.
for run_end in range(len(pkg_fns)):
  fn = pkg_fns[run_end]
  mo = bpat.match(fn) # test for a match of the package name pattern
  if mo:
    tbase = mo.group(1) # gets the 'base' package name
    # is it a new base name, i.e. the start of a new run?
    if tbase != pkg_base_nm: # if so then process the prior run
      if pkg_base_nm != '':
        cleanup(pkg_fns[run_start:run_end])
      pkg_base_nm = tbase # & setup for the new run
      run_start = run_end
  else:
    print >>sys.stderr, "File '"+fn+"' doesn't match package pattern!"

# catch the final run of the list
run_end += 1
cleanup(pkg_fns[run_start:run_end])

if opt_verbose:
  print n_deleted,"files deleted."

Offline

#2 2005-04-17 19:04:40

uberGeek
Member
From: Near Chicago
Registered: 2004-01-07
Posts: 65
Website

Re: A utility for cleaning /var/cache/pacman/pkg

Thank you!


carb $ now; uname -a; uptime
Sun 03Jul11 14:54:25 CDT -0500
Linux carb 2.6.17-ARCH #1 SMP PREEMPT Fri Jul 7 09:15:53 CEST 2006 i686 Pentium III (Coppermine) GenuineIntel GNU/Linux
14:54:25 up 1463 days,  9:24,  1 user,  load average: 1.00, 1.00, 1.00

Offline

#3 2005-04-17 19:46:15

Stinky
Member
From: The Colony, TX
Registered: 2004-05-28
Posts: 187

Re: A utility for cleaning /var/cache/pacman/pkg

excellent!!  I would not use pacman to clean out old packages for the very reason I wanted to keep at least one old version just in case a new package is buggy.  I've been cleaning it out by hand. 
Thanks!

Offline

#4 2005-04-28 17:17:15

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: A utility for cleaning /var/cache/pacman/pkg

Hmm, if you want to hack at it - I'd say patch pacman... this should be the default "-Sc" behavior... "-Scc" should remain as it is.
Perhaps the "number of versions to keep" can be stored in pacman.conf

Offline

#5 2005-04-28 18:25:22

dp
Member
From: Zürich, Switzerland
Registered: 2003-05-27
Posts: 3,363
Website

Re: A utility for cleaning /var/cache/pacman/pkg

phrakture wrote:

Hmm, if you want to hack at it - I'd say patch pacman... this should be the default "-Sc" behavior... "-Scc" should remain as it is.
Perhaps the "number of versions to keep" can be stored in pacman.conf

or maybe providing it to the -Sc :

pacman -Sc 2

The impossible missions are the only ones which succeed.

Offline

#6 2005-04-28 18:48:38

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: A utility for cleaning /var/cache/pacman/pkg

with that in place, it wouldn't be hard to provide a "rollback" thing too.... would be useful as that's one of the biggest pacman gripes "how do I downgrade a package?"

pacman --rollback foo-package 2

thinking outloud... I'm hoping the pacman lib gets done soon, so I can start implementing things like this 8) - right now I don't want to make pacman changes because it's supposed to change

Offline

#7 2007-09-19 17:35:08

dhave
Arch Linux f@h Team Member
From: Outside the matrix.
Registered: 2005-05-15
Posts: 1,110

Re: A utility for cleaning /var/cache/pacman/pkg

Thanks -- from over two years in the future. [Mysterious music plays]


Donate to Arch!

Tired? There's a nap for that. --anonymous

Offline

#8 2008-07-29 12:33:23

mclabaut
Member
Registered: 2007-06-18
Posts: 39

Re: A utility for cleaning /var/cache/pacman/pkg

Some packages were still left behind.
I changed the pattern to:

bpat = re.compile(r'(.+)-[^-]+-[0-9]+(-.+)?.pkg.tar.gz$')

Without this modification, xulrunner-1.8.1.14-2-i686.pkg.tar.gz and xulrunner-1.9-1-i686.pkg.tar.gz were left unerased in presence of xulrunner-1.9.0.1-1-i686.pkg.tar.gz.

Offline

#9 2008-07-29 13:40:26

Vintendo
Member
From: Netherlands
Registered: 2008-04-21
Posts: 375
Website

Re: A utility for cleaning /var/cache/pacman/pkg

Why is this not yet implented in pacman?

Offline

#10 2008-07-29 13:43:39

Daenyth
Forum Fellow
From: Boston, MA
Registered: 2008-02-24
Posts: 1,244

Re: A utility for cleaning /var/cache/pacman/pkg

Because the forum is not the bug tracker.

Offline

#11 2008-07-29 14:17:56

mclabaut
Member
Registered: 2007-06-18
Posts: 39

Re: A utility for cleaning /var/cache/pacman/pkg

Offline

#12 2008-07-29 14:39:46

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: A utility for cleaning /var/cache/pacman/pkg

Also because we don't really want to add that kind of complexity to pacman.
It is probably fine as an external python script.
But well, if you prefer, you can always try to do it yourself, just for fun.

Though it is worth mentioning that we already increased a bit the complexity of -Sc, by proposing 2 different behaviors for pacman 3.2 :
http://projects.archlinux.org/?p=pacman … be93db0ca3
But well, the core part was like 5 lines of C code, so not that bad.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#13 2009-12-05 23:44:18

XazZ
Member
From: Munich, Germany
Registered: 2006-12-30
Posts: 72

Re: A utility for cleaning /var/cache/pacman/pkg

evening

I just updated alterkacker's script in the wiki
(link to the wiki article: http://wiki.archlinux.org/index.php/CacheClean)

what I did:
-added -any architecture (so this script does not just support i686 and x86_64)
-fixed a bug where the script would fail if there were any directories in the cache

my /var/cache/pacman/pkg is a mounted container file, so I got a lost+found directory in there
previously the script simply failed - now it shows an info message on stdout + it continues

oh, I almost forgot to mention: maybe someone could take a quick look at my changes?
I don't know python - so I hope they are ok wink

Regards

Last edited by XazZ (2009-12-05 23:45:00)

Offline

#14 2010-02-17 09:47:14

7x1x
Member
From: New Zealand
Registered: 2009-10-16
Posts: 34

Re: A utility for cleaning /var/cache/pacman/pkg

Great work, the script works well from what I've used it. However libjpeg doesn't fit the naming pattern.

File 'libjpeg-8-1-x86_64.pkg.tar.gz' doesn't match package pattern!
File 'libjpeg-8-2-x86_64.pkg.tar.gz' doesn't match package pattern!

Shouldn't it be libjpeg-8.2 not 8-2? I'm not sure as to how many packages deviate from the normal naming system, so it may not be worth changing.

Offline

#15 2010-02-17 12:00:57

jerryluc
Member
From: Trondheim, Norway
Registered: 2008-05-20
Posts: 95

Re: A utility for cleaning /var/cache/pacman/pkg

I've written a python program that does this. It's not up-to-date, so it doesn't support the "any" architecture. But it shouldn't be much work. Maybe i'll do it soon, but anyone could do it. I'm not a good programmer so it good be buggy, it could be badly written or something else. It works for me, though.

pacleaner in AUR

edit: I have now fixed it so it works and so it handles "any"

Last edited by jerryluc (2010-02-17 23:12:41)

Offline

#16 2010-03-23 21:57:46

7x1x
Member
From: New Zealand
Registered: 2009-10-16
Posts: 34

Re: A utility for cleaning /var/cache/pacman/pkg

Hi, to update XazZ's script from the wiki to handle .gz and the new .xz just up date the following line:

bpat = re.compile(r'^(.+)-\d[^-]+-.+?(-i686|-x86_64|-any)?\.pkg\.tar.+(gz|xz)?$')

I know nothing about python, but it works from what I've seen.

Last edited by 7x1x (2010-04-04 22:09:20)

Offline

#17 2010-05-28 23:11:01

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,440
Website

Re: A utility for cleaning /var/cache/pacman/pkg

Necrobump of the year but the op did a fantastic job with this script.  I packaged the most recent incarnation of it into the cacheclean package in the AUR.  5+ years later and people keep finding value in the op's work smile


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#18 2010-05-29 11:32:50

stqn
Member
Registered: 2010-03-19
Posts: 1,189
Website

Re: A utility for cleaning /var/cache/pacman/pkg

Thanks for the update... Documentation bug: The post install message tells us the command is named "cache_clean", but it is "cacheclean.py".

Edit: the script itself tells us it is "cache_clean".

Last edited by stqn (2010-05-29 11:33:48)

Offline

#19 2010-11-11 05:26:31

binskipy2u
Banned
Registered: 2009-09-14
Posts: 212

Re: A utility for cleaning /var/cache/pacman/pkg

does pacman -Sc when it asks you..
pacman -Sc
Cache directory: /var/cache/pacman/pkg/
Do you want to remove uninstalled packages from cache? [Y/n]


are these the programs "older" versions that are still there? it will keep the latest right?

the size of this folder is 2.7 gigs...im sure alot of that is older versions right?
I can't figure out or find out how to run the script seems like greek to me..
but wll the above command get rid of older versions? (which i assume is what the command output means by "uninstalled packages")?

and then after that, this output comes up..

/var/lib/pacman/
Do you want to remove unused repositories? [Y/n]

i use mirror status and reflector..so i'll proberly say "no" to this..
but saying yes to the first one shouldn't be a problem if as far as you know everything is working ok?
just wondering

Last edited by binskipy2u (2010-11-11 05:28:07)


"Sometimes you comfort the afflicted, other times you AFFLICT the COMFORTABLE"

Offline

#20 2010-11-17 16:08:41

stqn
Member
Registered: 2010-03-19
Posts: 1,189
Website

Re: A utility for cleaning /var/cache/pacman/pkg

binskipy2u wrote:

does pacman -Sc when it asks you..
pacman -Sc
Cache directory: /var/cache/pacman/pkg/
Do you want to remove uninstalled packages from cache? [Y/n]

are these the programs "older" versions that are still there? it will keep the latest right?

Yes, assuming the latest versions are what is installed. "pacman -Sc" will remove packages for anything that is not currently installed. That includes:

a) packages for programs that are not installed.
b) packages for programs that are installed, but in a different version.
For example if deadbeef 0.4.4 is installed, pacman will remove the package for deadbeef 0.4.1 but keep for one for deadbeef 0.4.4.
And if you once installed KDE just to try it and then uninstalled it, all KDE packages will be deleted.

Offline

#21 2012-09-29 09:34:59

Narga
Member
From: Hải Phòng, Việt Nam
Registered: 2011-10-06
Posts: 56
Website

Re: A utility for cleaning /var/cache/pacman/pkg

How to make it work with alternative (absolute) cache directory PATH include subfolder?

Offline

Board footer

Powered by FluxBB