You are not logged in.

#1 2008-02-12 13:38:56

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

pacman -Sc chnged behavior, network-cached packages get deleted!

Hi everybody,

As of V 3.1 pacman -Sc changed behavior to delete all but the installed packages on one machine. I use a network shared pacman cache as described in this wiki entry:
http://wiki.archlinux.org/index.php/How … me_Network
This change killed the ability to only keep the newest packages in cache. If I do a

pacman -Sc

it deletes all packages that are not installed on this particular machine. If you have several machines with different setups this is not what is intended. Wouldn't it have been better to leave pacman -Sc as is and introduce a new option? Is there a way to work around that or do I have to keep track of whats new by myself? yikes

Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#2 2008-02-12 14:11:10

nj
Member
Registered: 2007-04-06
Posts: 93

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

This thread has some links to a few scripts which should do what you want. If you find one of the scripts useful, consider adding it to the wiki.

Offline

#3 2008-02-12 14:14:45

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Yup, that behaviour changed because the previous way -Sc handled things was based on filename of the package in cache, which is poor behaviour.

http://projects.archlinux.org/git/?p=pa … 88c7a26325 <-- there's the commit that did it if you're interested.

I remember a big thread with some scripts to help you with what you want, but I've been searching the forums and can't find it.  hmm

-edit-
AHA, it was on the mailinglist - that's why I couldn't find it in the forums. tongue  Thanks nj

Last edited by Cerebral (2008-02-12 14:15:19)

Offline

#4 2008-02-12 15:37:25

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Harey wrote:

Hi everybody,

As of V 3.1 pacman -Sc changed behavior to delete all but the installed packages on one machine. I use a network shared pacman cache as described in this wiki entry:
http://wiki.archlinux.org/index.php/How … me_Network
This change killed the ability to only keep the newest packages in cache. If I do a

pacman -Sc

it deletes all packages that are not installed on this particular machine. If you have several machines with different setups this is not what is intended. Wouldn't it have been better to leave pacman -Sc as is and introduce a new option? Is there a way to work around that or do I have to keep track of whats new by myself? yikes

You (or someone else) should probably add a note in that wiki page about this smile
It's probably better to let someone who actually uses a shared cache do it.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#5 2008-02-12 16:16:57

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

I worked through the mailing list entries but (nearly) all of them in a way tried to emulate the new behavior of pacman -Sc. However, there are two of them that look promising. I will try and report back later. Maybe I'll find a solution. Meanwhile I'll update the wiki page. Mmmmh, first time I do this...

Harvey

Last edited by Harey (2008-02-12 16:17:21)


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#6 2008-02-12 16:37:38

lilsirecho
Veteran
Registered: 2003-10-24
Posts: 5,000

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Perhaps using a flash drive as a cachedir with pacman will help.

Thus......pacman --cachedir /media/disk -Sw package(s).......

And....    pacman -Ud /media/disk/(package .pkg.tar.gz).... to install

Remove from flash drive in root......no pacman -Sc needed...........


Prediction...This year will be a very odd year!
Hard work does not kill people but why risk it: Charlie Mccarthy
A man is not complete until he is married..then..he is finished.
When ALL is lost, what can be found? Even bytes get lonely for a little bit!     X-ray confirms Iam spineless!

Offline

#7 2008-02-12 16:42:42

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

lilsirecho wrote:

pacman --cachedir /media/disk -Sw package(s)
pacman -Ud /media/disk/(package .pkg.tar.gz)  to install

Actually, you could even just do:

$ pacman --cachedir /media/disk -Syu

on every system.  It'll download the files to cache as it upgrades, and it'll automatically check the cache for packages that might already be downloaded.  Then pop it out, move it to the other PC, and do the same.

Offline

#8 2008-02-12 19:52:36

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Cerebral, lilsirecho

this is not quite what I have in mind. Forgive me if I did not put this clear. The scenario is a small company with about 10-15 workstations which I administer mostly by telephone line via ssh. Alas, mostly all of them have slightly different setups. So running around with a flash drive is not an option wink
All workstations share a cache via nfs on one of the file servers, saving me update time and internet bandwidth. I used pacman -Sc simply to save space on the servers harddisk by deleting outdated packages I won't need anymore. All workstations get updated on a regular basis if the test system does not explode by using pacman -Syu big_smile

Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#9 2008-02-12 20:04:45

Cerebral
Forum Fellow
From: Waterloo, ON, CA
Registered: 2005-04-08
Posts: 3,108
Website

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Harey - I figured this was basically the setup you had; my main point was just to make lilsirecho's suggestion a bit simpler.

Any luck with the scripts on that email thread?  I imagine this one is the one you'd want.

Offline

#10 2008-02-13 08:02:18

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Cerbral,

yes, this is one of two I found promising. But it didn't work 'out of the box'. I'll have to take a closer look as soon as I have the spare time. I will report back and update the wiki ASAP.

Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#11 2008-02-14 17:02:16

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Ok, so I'm back with a few hairs lost smile

Here is my solution:

#!/bin/bash
#
# script cleans pacman cache by comparing existing version of a package
# to newest version of package available according to pacman-db
# WARNING! THIS SCRIPT WILL DELETE EVERY FILE IN /var/cache/pacman/pkg
# THAT IS NOT IN THE PACMAN-DB OR THAT DOES NOT FOLLOW THE CURRENT NAMING
# CONVENTIONS: <PACKAGENAME>.<VERSION>.<ARCHITECTURE>.pkg.tar.gz !!!

arc=$(uname -m)    
for fn in $(ls /var/cache/pacman/pkg);do 
    # strip off .pkg.tar.gz
    f=${fn/%\.pkg\.tar\.gz/}
    # strip off architecture if given in package name
    if echo $f | grep -q $arc; then
       f=${f/%$arc/};f=${f/%-/}
    fi
    # now divide string into version and package name as used by pacman -Si
    pk=$f;v=$f;pk=${pk%-*};pk=${pk%-*};v=${v/$pk};v=${v/-}
    # query pacman-database to get current version of package and delete cached version if it is older
    if ! pacman -Si $pk | grep -q $v ; then
       rm -v /var/cache/pacman/pkg/$fn
    fi
done
# end of file

Maybe not very short and pretty, but understandable and working cool

I also added this to the wiki.

Harvey

Last edited by Harey (2008-02-14 17:13:57)


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#12 2008-02-14 19:02:46

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Harey wrote:

Ok, so I'm back with a few hairs lost smile

Here is my solution:
<snip>
Maybe not very short and pretty, but understandable and working cool

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#13 2008-02-14 21:04:25

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.

Close, but no cigar lol
Indeed, there are still problems remaining. If there are different repos enabled on different machines, the sync database of the actual machine where the script is started is used as reference. If others have more repos enabled, for example 'testing' these packages will be deleted, even if newer. Don't know how to work around this one by now. But my machines all have the same repos enabled, so it won't hurt me much.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?

Well you changed behavior before wink
But actually I wouldn't change it back either, because the majority of users will need the current implementation. I would tend to make it an extra option, if possible.

Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#14 2008-02-14 21:21:09

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Harey wrote:

Hm, looking at this solution, it's actually pretty close to the current algorithm pacman uses.
But instead of checking if the package is in the local database, you check if it's the sync database.

Close, but no cigar lol
Indeed, there are still problems remaining. If there are different repos enabled on different machines, the sync database of the actual machine where the script is started is used as reference. If others have more repos enabled, for example 'testing' these packages will be deleted, even if newer. Don't know how to work around this one by now. But my machines all have the same repos enabled, so it won't hurt me much.

Otherwise, if you run it on the box which has the most repos enabled, you should be fine.

So it is very easy to implement this in pacman. The problem is that the current behavior is better
for caches that are not network shared, so we cannot simply replace it.
And if we add this other -Sc behavior, we then need a way to choose between the two behaviors.
Any suggestions?

Well you changed behavior before wink
But actually I wouldn't change it back either, because the majority of users will need the current implementation. I would tend to make it an extra option, if possible.

Sorry if it wasn't clear, that's what I was asking, which extra option it could be?
Because we would need a clear name and/or shortname to avoid confusing users with two different -Sc behaviors possible.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#15 2008-02-15 07:24:28

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Otherwise, if you run it on the box which has the most repos enabled, you should be fine.

Not really. Think of a box, that has 'testing' enabled. It will delete the package from the other repo, even though it is still current to normal systems.

Sorry if it wasn't clear, that's what I was asking, which extra option it could be?
Because we would need a clear name and/or shortname to avoid confusing users with two different -Sc behaviors possible.

If you just ask me for a name, how about pacman -Scn (clear network) or -Sco (clear outdated), as it belongs to -Sc in context and the usage of -Scc is known from older versions? Naming it like this would not break naming conventions, I guess.

Harvey

N.B. I like the -Sco one - will they sue us for that? lol

Last edited by Harey (2008-02-15 07:33:06)


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

#16 2008-02-15 07:33:11

shining
Pacman Developer
Registered: 2006-05-10
Posts: 2,043

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

Harey wrote:

If you just ask me for a name, how about pacman -Scn (clear network), as it belongs to -Sc in context and the usage of -Scc is known from older versions? Naming it like this would not break naming conventions, I guess.

I was originally thinking about something like that, but then I asked in pacman-dev ML also, and Romashka suggested using only a long option, in pacman.conf :
http://www.archlinux.org/pipermail/pacm … 11141.html
It's actually the kind of behavior that you select once, and not at each run. So an option in pacman.conf makes sense.

Well, we still need to find a good name. And then I would need to convince Dan, since he is not a fan of turning pacman into a cache cleaner tool. But this feature looks totally non-intrusive to me.


pacman roulette : pacman -S $(pacman -Slq | LANG=C sort -R | head -n $((RANDOM % 10)))

Offline

#17 2008-02-15 11:31:03

Harey
Member
From: Bavaria, Germany
Registered: 2007-03-24
Posts: 359

Re: pacman -Sc chnged behavior, network-cached packages get deleted!

I was originally thinking about something like that, but then I asked in pacman-dev ML also, and Romashka suggested using only a long option, in pacman.conf :
http://www.archlinux.org/pipermail/pacm … 11141.html
It's actually the kind of behavior that you select once, and not at each run. So an option in pacman.conf makes sense.

I agree, pacman.conf would be the better place for that kind of change. Good luck with convincing Dan. I personally think that this is not really a new function but only a twist of existing functionality. This might help.

Harvey


Linux is like a wigwam: No Gates, no Windows and an Apache inside

Offline

Board footer

Powered by FluxBB