pacman -Sc chnged behavior, network-cached packages get deleted!

Harey · 2008-02-12 13:38:56

Hi everybody,

As of V 3.1 pacman -Sc changed behavior to delete all but the installed packages on one machine. I use a network shared pacman cache as described in this wiki entry:
http://wiki.archlinux.org/index.php/How … me_Network
This change killed the ability to only keep the newest packages in cache. If I do a

pacman -Sc

it deletes all packages that are not installed on this particular machine. If you have several machines with different setups this is not what is intended. Wouldn't it have been better to leave pacman -Sc as is and introduce a new option? Is there a way to work around that or do I have to keep track of whats new by myself?

Harvey

nj · 2008-02-12 14:11:10

This thread has some links to a few scripts which should do what you want. If you find one of the scripts useful, consider adding it to the wiki.

Cerebral · 2008-02-12 14:14:45

Yup, that behaviour changed because the previous way -Sc handled things was based on filename of the package in cache, which is poor behaviour.

http://projects.archlinux.org/git/?p=pa … 88c7a26325 <-- there's the commit that did it if you're interested.

I remember a big thread with some scripts to help you with what you want, but I've been searching the forums and can't find it.

-edit-
AHA, it was on the mailinglist - that's why I couldn't find it in the forums. Thanks nj

Last edited by Cerebral (2008-02-12 14:15:19)

shining · 2008-02-12 15:37:25

Harey wrote:

Hi everybody,
As of V 3.1 pacman -Sc changed behavior to delete all but the installed packages on one machine. I use a network shared pacman cache as described in this wiki entry:
http://wiki.archlinux.org/index.php/How … me_Network
This change killed the ability to only keep the newest packages in cache. If I do a
pacman -Sc
it deletes all packages that are not installed on this particular machine. If you have several machines with different setups this is not what is intended. Wouldn't it have been better to leave pacman -Sc as is and introduce a new option? Is there a way to work around that or do I have to keep track of whats new by myself?

You (or someone else) should probably add a note in that wiki page about this
It's probably better to let someone who actually uses a shared cache do it.

Harey · 2008-02-12 16:16:57

I worked through the mailing list entries but (nearly) all of them in a way tried to emulate the new behavior of pacman -Sc. However, there are two of them that look promising. I will try and report back later. Maybe I'll find a solution. Meanwhile I'll update the wiki page. Mmmmh, first time I do this...

Harvey

Last edited by Harey (2008-02-12 16:17:21)

lilsirecho · 2008-02-12 16:37:38

Perhaps using a flash drive as a cachedir with pacman will help.

Thus......pacman --cachedir /media/disk -Sw package(s).......

And.... pacman -Ud /media/disk/(package .pkg.tar.gz).... to install

Remove from flash drive in root......no pacman -Sc needed...........

Cerebral · 2008-02-12 16:42:42

lilsirecho wrote:

pacman --cachedir /media/disk -Sw package(s)
pacman -Ud /media/disk/(package .pkg.tar.gz) to install

Actually, you could even just do:

$ pacman --cachedir /media/disk -Syu

on every system. It'll download the files to cache as it upgrades, and it'll automatically check the cache for packages that might already be downloaded. Then pop it out, move it to the other PC, and do the same.

Harey · 2008-02-12 19:52:36

Cerebral, lilsirecho

this is not quite what I have in mind. Forgive me if I did not put this clear. The scenario is a small company with about 10-15 workstations which I administer mostly by telephone line via ssh. Alas, mostly all of them have slightly different setups. So running around with a flash drive is not an option
All workstations share a cache via nfs on one of the file servers, saving me update time and internet bandwidth. I used pacman -Sc simply to save space on the servers harddisk by deleting outdated packages I won't need anymore. All workstations get updated on a regular basis if the test system does not explode by using pacman -Syu

Harvey

Cerebral · 2008-02-12 20:04:45

Harey - I figured this was basically the setup you had; my main point was just to make lilsirecho's suggestion a bit simpler.

Any luck with the scripts on that email thread? I imagine this one is the one you'd want.

Harey · 2008-02-13 08:02:18

Cerbral,

yes, this is one of two I found promising. But it didn't work 'out of the box'. I'll have to take a closer look as soon as I have the spare time. I will report back and update the wiki ASAP.

Harvey

Harey · 2008-02-14 17:02:16

Ok, so I'm back with a few hairs lost

Here is my solution:

#!/bin/bash
#
# script cleans pacman cache by comparing existing version of a package
# to newest version of package available according to pacman-db
# WARNING! THIS SCRIPT WILL DELETE EVERY FILE IN /var/cache/pacman/pkg
# THAT IS NOT IN THE PACMAN-DB OR THAT DOES NOT FOLLOW THE CURRENT NAMING
# CONVENTIONS: <PACKAGENAME>.<VERSION>.<ARCHITECTURE>.pkg.tar.gz !!!

arc=$(uname -m)    
for fn in $(ls /var/cache/pacman/pkg);do 
    # strip off .pkg.tar.gz
    f=${fn/%\.pkg\.tar\.gz/}
    # strip off architecture if given in package name
    if echo $f | grep -q $arc; then
       f=${f/%$arc/};f=${f/%-/}
    fi
    # now divide string into version and package name as used by pacman -Si
    pk=$f;v=$f;pk=${pk%-*};pk=${pk%-*};v=${v/$pk};v=${v/-}
    # query pacman-database to get current version of package and delete cached version if it is older
    if ! pacman -Si $pk | grep -q $v ; then
       rm -v /var/cache/pacman/pkg/$fn
    fi
done
# end of file

Maybe not very short and pretty, but understandable and working

I also added this to the wiki.

Harvey

Last edited by Harey (2008-02-14 17:13:57)

shining · 2008-02-14 19:02:46

Harey wrote:

Ok, so I'm back with a few hairs lost
Here is my solution:
<snip>
Maybe not very short and pretty, but understandable and working

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?

Harey · 2008-02-14 21:04:25

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.

Close, but no cigar
Indeed, there are still problems remaining. If there are different repos enabled on different machines, the sync database of the actual machine where the script is started is used as reference. If others have more repos enabled, for example 'testing' these packages will be deleted, even if newer. Don't know how to work around this one by now. But my machines all have the same repos enabled, so it won't hurt me much.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?

Well you changed behavior before
But actually I wouldn't change it back either, because the majority of users will need the current implementation. I would tend to make it an extra option, if possible.

Harvey

shining · 2008-02-14 21:21:09

Harey wrote:

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.
Close, but no cigar
Indeed, there are still problems remaining. If there are different repos enabled on different machines, the sync database of the actual machine where the script is started is used as reference. If others have more repos enabled, for example 'testing' these packages will be deleted, even if newer. Don't know how to work around this one by now. But my machines all have the same repos enabled, so it won't hurt me much.

Otherwise, if you run it on the box which has the most repos enabled, you should be fine.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?
Well you changed behavior before
But actually I wouldn't change it back either, because the majority of users will need the current implementation. I would tend to make it an extra option, if possible.

Sorry if it wasn't clear, that's what I was asking, which extra option it could be?
Because we would need a clear name and/or shortname to avoid confusing users with two different -Sc behaviors possible.

Harey · 2008-02-15 07:24:28

Otherwise, if you run it on the box which has the most repos enabled, you should be fine.

Not really. Think of a box, that has 'testing' enabled. It will delete the package from the other repo, even though it is still current to normal systems.

Sorry if it wasn't clear, that's what I was asking, which extra option it could be?
Because we would need a clear name and/or shortname to avoid confusing users with two different -Sc behaviors possible.

If you just ask me for a name, how about pacman -Scn (clear network) or -Sco (clear outdated), as it belongs to -Sc in context and the usage of -Scc is known from older versions? Naming it like this would not break naming conventions, I guess.

Harvey

N.B. I like the -Sco one - will they sue us for that?

Last edited by Harey (2008-02-15 07:33:06)

shining · 2008-02-15 07:33:11

Harey wrote:

If you just ask me for a name, how about pacman -Scn (clear network), as it belongs to -Sc in context and the usage of -Scc is known from older versions? Naming it like this would not break naming conventions, I guess.

I was originally thinking about something like that, but then I asked in pacman-dev ML also, and Romashka suggested using only a long option, in pacman.conf :
http://www.archlinux.org/pipermail/pacm … 11141.html
It's actually the kind of behavior that you select once, and not at each run. So an option in pacman.conf makes sense.

Well, we still need to find a good name. And then I would need to convince Dan, since he is not a fan of turning pacman into a cache cleaner tool. But this feature looks totally non-intrusive to me.

Harey · 2008-02-15 11:31:03

I was originally thinking about something like that, but then I asked in pacman-dev ML also, and Romashka suggested using only a long option, in pacman.conf :
http://www.archlinux.org/pipermail/pacm … 11141.html
It's actually the kind of behavior that you select once, and not at each run. So an option in pacman.conf makes sense.

I agree, pacman.conf would be the better place for that kind of change. Good luck with convincing Dan. I personally think that this is not really a new function but only a twist of existing functionality. This might help.

Harvey

Arch Linux

#1 2008-02-12 13:38:56

pacman -Sc chnged behavior, network-cached packages get deleted!

#2 2008-02-12 14:11:10

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#3 2008-02-12 14:14:45

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#4 2008-02-12 15:37:25

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#5 2008-02-12 16:16:57

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#6 2008-02-12 16:37:38

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#7 2008-02-12 16:42:42

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#8 2008-02-12 19:52:36

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#9 2008-02-12 20:04:45

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#10 2008-02-13 08:02:18

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#11 2008-02-14 17:02:16

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#12 2008-02-14 19:02:46

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#13 2008-02-14 21:04:25

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#14 2008-02-14 21:21:09

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#15 2008-02-15 07:24:28

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#16 2008-02-15 07:33:11

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

#17 2008-02-15 11:31:03

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Board footer