how about a script to clean up /var/cache/pacman/pkg

perbh · 2009-01-04 06:09:29

Granted the layout of the packages is
packagename-version-revision-architecture.pkg.tar.gz

I guess it's asking wayyyy too much to ask for a change, but
There _are_ packages that have a '-' other than as indicated above. This really plays havoc with any attempt to 'play around' with the filenames. I have been royally screwed by assuming that '-[0-9].+' is the start of 'version' (-75dpi, -100dpi as an example).
Personally I would like to see '~' or '@' as a seperator, but maybe the '-' could be changed to '_' like:
xorg_fonts_100dpi-1.0.1-1

Just a thought ... I do not expect any change as drastic as this, but maybe sometime in the future?

Allan · 2009-01-04 06:42:43

Or you can just remove sections from the back. Last is pkgrel, next is pkgver and all remaining is pkgname. It is easy to do this in bash.

jerryluc · 2009-01-04 11:21:49

Galdona wrote:

Haha I only just now saw the output of pacman -V. cute!
It would be nice to have the above-discussed functionality included in pacman3. eherm.

I thought pacman had nothing to do with the game

perbh · 2009-01-05 03:54:35

<code>
Allan said:
Or you can just remove sections from the back. Last is pkgrel, next is pkgver and all remaining is pkgname. It is easy to do this in bash.
</code>

Duh!! Sad state of affairs when fingers are faster than the brain ...
Thank you - I've always done the suckers from the front!

yitzle · 2009-06-03 04:34:00

Similar to #19, and will break horribly is a package has a newline in its name (#19's approach can potentially be fixed to deal with that):

ls -l /var/cache/pacman/pkg/ | sort | awk -F\- 'prevF==$1{print prevL} {prevL=$0; prevF=$1}' | while read i ; do rm "/var/cache/pacman/pkg/$i"; done

Hoping the locale's collate sorts properly, you can try:

cd /var/cache/pacman/pkg ; for i in su* ; do prevPkg="$pkg"; pkg=$(cut -d- -f1 <<< "$i"); [[ $pkg = $prevPkg ]] && echo "$prevFile"; prevFile="$i"; done

Change the 'echo' to 'rm' as desired.

If the package name contains a '-', you may experience unexpected behavior.

Allan · 2009-06-03 04:49:54

hmmm.... I don't think makepkg can actually put a newline in a packages name. But even if it did allow that, just don't do it..... ever!

jerryluc · 2009-06-03 15:56:12

i've been trying to figure out a script that deletes every packages except the two newest version of an installed packages. But i can't seem to get it right.... Any suggestions. What do you think of using the creation date of the file to sort the packages?

brisbin33 · 2009-06-03 16:19:32

this may be useful

> grep -A 13 pullout .bash_functions
pullout() {
  if [ $# -ne 2 ]; then
    echo "need proper arguments:"
    echo "pullout [file] [archive.tar.gz]"
    return 1
  fi
  case $2 in
    *.tar.gz|*.tgz)
    gunzip < $2 | tar -xf - $1
    ;;
    *)
    echo $2 is not a valid archive
    return 1
    ;;
  esac
  return 0
}

doesn't look like much, but this little function can pull the .PKGINFO file out of the .pkg.tar.gz file(s). apologies if it's been mentioned before but this'll let you sort your /var/cache/pacman/pkg directory w/o worrying about '-'s in pkgnames and w/o resorting to using creation dates.

could be time consuming, but i think it's the most accurate way to solve this problem (i may work on a script myself this weekend).

> pullout .PKGINFO /var/cache/pacman/pkg/xorg-server-1.6.1.901-1-x86_64.pkg.tar.gz
> egrep "(^pkgname|^pkgver)" .PKGINFO
pkgname = xorg-server
pkgver = 1.6.1.901-1

EDIT: got bored at work this afternoon:

> cat .bin/lite-clean
#!/bin/bash
#
# pbrisbin 2009
#
###

# number of versions to keep
NUM=2

# place packages reside
PD="/var/cache/pacman/pkg"

# place to do some work
WD="/tmp/lite-clean"

[ -d $WD ] && rm -r $WD
mkdir -p $WD/saveme

# rip out package names/versions from stored packages
# creates a *sorted* list of all pkgs in a parsable format
LC_ALL="C"
for package in $(find $PD -name *.pkg.tar.gz | sort -r); do
  pushd $WD &>/dev/null && gunzip < $package | tar -xf - .PKGINFO
  file="$package"
  name="$(awk '/^pkgname/ {print $3}' .PKGINFO)"
  vers="$(awk '/^pkgver/ {print $3}' .PKGINFO)"
  echo "$name $vers $file" >> $WD/all.lst
done
popd &>/dev/null

# create the save list
pacman -Qq | while read pack; do
  grep -m $NUM ^$pack\  $WD/all.lst >> $WD/keep.lst
done

# save the last $NUM versions
awk '{print $3}' $WD/keep.lst | while read pack; do
  cp $pack $WD/saveme/ && echo $pack saved
done

### This is the dangerous part, be careful
# check $WD/keep.lst and $WD/saveme/* before
# uncommenting this section
###

# remove all else
#rm $PD/*

# and put back the saved ones
#cp $WD/saveme/* $PD/

# [ $? -eq 0 ] && rm -r $WD

exit 0

> sizeof /var/cache/pacman/pkg/
2.4G    /var/cache/pacman/pkg/
> time lite-clean
...
real    2m4.190s
user    1m14.645s
sys     0m18.855s
> sizeof /tmp/lite-clean/saveme/
1.7G    /tmp/lite-clean/saveme/

Last edited by brisbin33 (2009-07-18 16:32:00)

jerryluc · 2009-07-09 19:18:23

I've now made a script that deletes the cache files for uninstalled packages and files older then the two newest versions of a package.

I'm a hobby programmer(and not a good one) so use with care. And it may be too complex for it's use, but I also made it because I want to learn programming.

It's called pacleaner and it's in AUR. I'm glad for any feedback -

toad · 2009-07-09 19:42:07

I've been looking for something that purges everything but the last two installed packages, but am no programmer. Anybody tried jerryluc's programme? brisbin33's code is too cryptic for me, I'm afraid

brisbin33 · 2009-07-09 23:01:38

toad wrote:

I've been looking for something that purges everything but the last two installed packages, but am no programmer. Anybody tried jerryluc's programme? brisbin33's code is too cryptic for me, I'm afraid

that's too bad b/c that's exactly what my script does .

edit: i took out my explanation here, and just commented the hell out of the script below. maybe it's a little less 'cryptic' now.

#!/bin/bash
#
# pbrisbin 2009
#
###

### config
N=3                             # how may versions to keep -- including installed
PD="/var/cache/pacman/pkg"      # cache location

### script
# define our 'working directory' as some folder in /tmp
WD="/tmp/pacclean"

# remove it in case it exists from some previous run
rm -rf $WD

# make it and a 'saveme' subdir at once
mkdir -p $WD/saveme

# incase this isn't set, set it.  w/o this, numbers are sorted
# as 1 10 11 2 23 3 4 5 and that's no good.
LC_ALL="C"

# pushd is just like `cd`, it moves us to $WD (working directory) 
# where we can work with files while contained in /tmp; and if we
# fail getting to /tmp we just exit
pushd $WD &>/dev/null || exit 1

# since we're in /tmp now, find returns the full path to all 
# packages in $PD (package directory, your cache)
# using that path, we tar out the .PKGINFO file and use
# awk to get the exact name and version of all your packages
# this is then echoed one by one into $WD/all.lst as
# [name] [version] [/path/to/package]
# it's reverse sorted by name, then version (this is key)
for package in $(find $PD -name *.pkg.tar.gz | sort -r); do
  gunzip < $package | tar -xf - .PKGINFO
  file="$package"
  name="$(awk '/^pkgname/ {print $3}' .PKGINFO)"
  vers="$(awk '/^pkgver/ {print $3}' .PKGINFO)"
  echo "$name $vers $file" >> $WD/all.lst
done

# popd just brings us back from wherever pushd put us.
popd &>/dev/null || exit 1

# now we ask pacman for what's installed by name and use 
# grep -m to get the top $N matches to that name from $WD/all.lst
# which happens to be the top $N versions of said program in your
# cache
# anything that matches, we use the third column (the path) to copy
# only those packages to $WD/saveme.
pacman -Qq | while read pack; do
  grep -m $N ^$pack\  $WD/all.lst | awk '{print $3}' | while read file; do
    cp -av $file $WD/saveme/
  done
done

# now that we've got N versions copied from $PD into
# $WD/saveme, we can clear the cache and put only
# the ones we want to keep back there.  if run as normal
# user, the script dies here.  but you've got the pkgversions
# you wanted sitting in $WD/saveme so you can check those
# out and manually clean your cache if you want
rm -v $PD/*
cp -av $WD/saveme/* $PD/

# only if the above cp succeeds do we remove $WD
[ $? -eq 0 ] && rm -r $WD

exit 0

Last edited by brisbin33 (2009-07-18 16:31:40)

jerryluc · 2009-07-10 19:07:54

it was too cryptic for me aswell!

i'll try it out. i'm still very happy to get my script(program? what's the difference?) working!

toad · 2009-07-10 19:17:41

Wicked stuff!!! Time to give it a name (cleanpac?) and put it in aur

brisbin33 · 2009-07-11 14:21:40

it's called pacclean on my system but cleanpac sounds good too. some day i'll learn to use the aur; but so far i've only written scripts so a quick wget and chmod is just as easy.

Gen2ly · 2009-07-18 07:31:00

No luck for me brisbin, deleted a good number of the pkg.tar.gz I had installed already. Could a read be put in the script before it actually does anything?

Last edited by Gen2ly (2009-07-18 07:41:19)

brisbin33 · 2009-07-18 15:51:59

that's really odd Gen2ly, i'm sorry about that. in the original script (post #33) the actual deletion of packages is commented and it's up to the user to check the contents of $WD/saveme before uncommenting this. in the newest version the safe approach would've been to run the script without root priveledges. this would've also put packages in $WD/saveme to be verified before running the script as root to do the actual cleaning.

the script wrote:

# now that we've got N versions copied from $PD into
# $WD/saveme, we can clear the cache and put only
# the ones we want to keep back there. if run as normal
# user, the script dies here. but you've got the pkgversions
# you wanted sitting in $WD/saveme so you can check those
# out and manually clean your cache if you want

very sorry about your cache; it worked perfectly on my system, not sure what could cause the descrepancy.

yitzle · 2009-10-06 04:30:45

bash 4.0 edition, using associative arrays:

#!/bin/bash

dir="$HOME/abs/packages"

cd "$dir"
shopt -s extglob

strip () {
        # Verbose. Can likely be shortened. But more flexible.
        n="$1"
        n="${n%.gz}"
        n="${n%.tar}"
        n="${n%.pkg}"
        n="${n%-i686}"
        n="${n%-any}"
        o="$n"
        n="${n%-+([0-9.])-+([0-9])}"
        v="${o#$n-}"
        echo "$v $n"
}

declare -A ver
declare -A file
for pkg in * ; do
        read v n <<< "$(strip "$pkg")"
        if [[ ${ver[$n]} ]] ; then
                vc="$( vercmp "${ver[$n]}" "$v" )"
                if (( vc < 0 )) ; then
                        rm "${file[$n]}"
                        ver[$n]="$v"
                        file[$n]="$pkg"
                elif (( vc == 0 )) ; then
                        echo "Same?"
                else
                        rm "$pkg"
                fi
        else
                ver[$n]="$v"
                file[$n]="$pkg"
        fi
done

Daemonjax · 2010-03-30 15:42:50

munkyeetr wrote:

Cerebral wrote:
There, this should be a 'fixed' version. As always, use at own risk.
#!/bin/bash

for pkgname in $(ls /var/cache/pacman/pkg/ | sed "s#$.*$-.*-.*#\1#g" | sort | uniq ); do
    pacman -Q $pkgname 2>/dev/null >/dev/null
    if [ $? -ne 0 ]; then
        echo "$pkgname in cache but not installed, removing."
        pkgs=$(ls /var/cache/pacman/pkg/$pkgname-* | grep -e "$pkgname-[0-9]")
        rm $pkgs
    fi
done
Sorry to ressurect such an old thread, but I just came across this and the script above worked great for me; cleared over 700MB from my .../pacman/pkg directory. I am curious though, why we want to keep the tar.gz's of installed programs in the cache. If the program is already installed, what purpose do they serve there? Just curious...

I just tried it.

It removed every package in my cache... including ones that were installed.

No big deal... just a heads up to others.

abauomy · 2010-09-08 03:35:11

i think we need ascript or function in pacman like "apt-get clean" in ubuntu it's only delete older ver from pkg you update
like:

netbeans-6.9.1-1-any.pkg.tar.xz
netbeans-6.9-2-any.pkg.tar.xz

it's only remove netbeans-6.9.1-1-any.pkg.tar.xz

fukawi2 · 2010-09-08 03:47:39

abauomy wrote:

netbeans-6.9.1-1-any.pkg.tar.xz
netbeans-6.9-2-any.pkg.tar.xz
it's only remove netbeans-6.9.1-1-any.pkg.tar.xz

That's what pacman -Sc does...

toad · 2010-09-08 06:25:35

And there is a number of scripts out there, even in the AUR, which "manage" your cache such as pacleaner, just search for them

Arch Linux

#26 2009-01-04 06:09:29

Re: how about a script to clean up /var/cache/pacman/pkg

#27 2009-01-04 06:42:43

Re: how about a script to clean up /var/cache/pacman/pkg

#28 2009-01-04 11:21:49

Re: how about a script to clean up /var/cache/pacman/pkg

#29 2009-01-05 03:54:35

Re: how about a script to clean up /var/cache/pacman/pkg

#30 2009-06-03 04:34:00

Re: how about a script to clean up /var/cache/pacman/pkg

#31 2009-06-03 04:49:54

Re: how about a script to clean up /var/cache/pacman/pkg

#32 2009-06-03 15:56:12

Re: how about a script to clean up /var/cache/pacman/pkg

#33 2009-06-03 16:19:32

Re: how about a script to clean up /var/cache/pacman/pkg

#34 2009-07-09 19:18:23

Re: how about a script to clean up /var/cache/pacman/pkg

#35 2009-07-09 19:42:07

Re: how about a script to clean up /var/cache/pacman/pkg

#36 2009-07-09 23:01:38

Re: how about a script to clean up /var/cache/pacman/pkg

#37 2009-07-10 19:07:54

Re: how about a script to clean up /var/cache/pacman/pkg

#38 2009-07-10 19:17:41

Re: how about a script to clean up /var/cache/pacman/pkg

#39 2009-07-11 14:21:40

Re: how about a script to clean up /var/cache/pacman/pkg

#40 2009-07-18 07:31:00

Re: how about a script to clean up /var/cache/pacman/pkg

#41 2009-07-18 15:51:59

Re: how about a script to clean up /var/cache/pacman/pkg

#42 2009-10-06 04:30:45

Re: how about a script to clean up /var/cache/pacman/pkg

#43 2010-03-30 15:42:50

Re: how about a script to clean up /var/cache/pacman/pkg

#44 2010-09-08 03:35:11

Re: how about a script to clean up /var/cache/pacman/pkg

#45 2010-09-08 03:47:39

Re: how about a script to clean up /var/cache/pacman/pkg

#46 2010-09-08 06:25:35

Re: how about a script to clean up /var/cache/pacman/pkg

Board footer