outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

michaelks · 2009-02-17 18:34:03

This is pretty involved, bear with me. I have found (don't ask me how) that when there are outdated entries for a package in the local database apart from the current one, pacman -Syu will compare the sync entry for the package with the old local entry instead of the new one, and will then download the package again. When after the download it attempts to install it, the error 'file already exists' will appear. If the old entry(ies) is removed from local, the problem disappears, of course. Here for example are two entries for alsa-lib in
/var/lib/pacman/local :

alsa-lib-1.0.18-1/ 4096 090116 21:44:52 rwxr-xr-x root/root
alsa-lib-1.0.19-1/ 4096 090216 00:23:09 rwxr-xr-x root/root

This is the entry in /var/lib/pacman/sync/extra :

alsa-lib-1.0.19-1/ 4096 090217 13:29:48 rwxr-xr-x root/root

Now pacman -Syu :

root@michael /home/michael #pacman -Syu
:: Synchronising package databases...
core is up to date
extra is up to date
community is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Targets (140): alsa-lib-1.0.19-1 ncurses-5.7-2 dialog-1.1_20080819-1
pciutils-3.1.2-1 alsa-utils-1.0.19-1 libmad-0.15.1b-3
libvorbis-1.2.1rc1-2 audacious-plugins-1.5.1-2
readline-5.2.013-2 bash-3.2.048-3 texinfo-4.13a-3
........[huge list]

Now pacman -S alsa-lib ...
root@michael /home/michael #pacman -S alsa-lib
resolving dependencies...
looking for inter-conflicts...

Targets (1): alsa-lib-1.0.19-1

Total Download Size: 0.51 MB
Total Installed Size: 1.92 MB

Proceed with installation? [Y/n]
:: Retrieving packages from extra...
error: failed retrieving file 'alsa-lib-1.0.19-1-x86_64.pkg.tar.gz' from ftp.archlinux.org : Protocol error
alsa-lib-1.0.19-1-x... 517.8K 90.8K/s 00:00:06 [#############################] 100%
checking package integrity...
(1/1) checking for file conflicts [#############################] 100%
error: could not create directory /var/lib/pacman/local/alsa-lib-1.0.19-1/: File exists

Now, however alsa-lib-1.0.18-1/ has been removed from local automatically by this first attempt, and we can install it :

root@michael /home/michael #pacman -S alsa-lib
warning: alsa-lib-1.0.19-1 is up to date -- reinstalling
resolving dependencies...
looking for inter-conflicts...

Targets (1): alsa-lib-1.0.19-1

Total Download Size: 0.00 MB
Total Installed Size: 1.92 MB

Proceed with installation? [Y/n]
checking package integrity...
(1/1) checking for file conflicts [#############################] 100%
(1/1) upgrading alsa-lib [#############################] 100%
Optional dependencies for alsa-lib
python

Wanna test this ? just put in a dummy directory into local, e.g. if you have
abiword-2.6.6-1 , mkdir /var/lib/pacman/local/abiword-2.6.6-0 ,then run pacman -Syu and sure enough, abiword will pop up for upgrade (along with complaints about the empty dir).

I'm eagerly awaiting enlightenment about this.

Last edited by michaelks (2009-02-22 03:19:33)

phrakture · 2009-02-17 22:51:10

This should never happen and pacman shouldn't ever have two directories under normal functionality. You either hit a one-time bug, or were using one of those "pacman filesystem" scripts at one point.

Just remove the old entries if you come across them unless you can somehow force pacman to recreate it everytime - then report it as a bug

michaelks · 2009-02-18 00:21:12

This happened to me twice already, i'm paranoid about hard disk failures, so i have two backups, a removable disk and another machine, that i backup regularly into using rsync . After each of these backups i had this pacman problem, these huge upgrade lists, an hour to download and then the 'file already present' message. Only now was i able to see the cause, but I'm still baffled as to where those double (sometimes even triple) entries are coming from. It cannot be rsync's fault, after all it must have copied the original local database (that has no double entries on the mother ship, pacman works flawlessly there) as it stood, so why does pacman somehow merge the incoming and resident files to produce these double entries ? I'll wait a few days until the local db has changed enough and try simply copying it into the backup (no rsync) to see if the same thing happens.

ataraxia · 2009-02-18 00:26:53

It's not clear to me whether you're restoring (or otherwise running) from these backups, or you're getting problems with the original /var/lib/pacman/local after copying it somewhere else. If the former, make sure you have "--delete" in your rsync flags, as it doesn't do it by default. I'd suggest rsync flags of something like "-aiPAX --delete" overall, YMMV.

michaelks · 2009-02-18 02:39:08

I use them for occasional restore and also recovery of deleted stuff. Sometimes they've been lifesavers. I always do rsync -av / backup:/ --exclude-from=file --delete . I tried renaming a few dirs upwards on /var/lib/pacman/local , then ran rsync on it, and as expected it dutifully deleted these files on the backup replacing them with the renamed ones. So it's not an rsync issue. I'll just wait for updates and try reproducing the whole procedure.

michaelks · 2009-02-18 17:51:19

Same thing all over again...Four upgrades came up today, so i do

pacman -Syu

here's the log...

[2009-02-17 23:34] synchronizing package lists
[2009-02-17 23:34] starting full system upgrade
[2009-02-18 13:43] synchronizing package lists
[2009-02-18 13:44] starting full system upgrade
[2009-02-18 13:54] upgraded man-pages (3.17-1 -> 3.18-1)
[2009-02-18 13:54] upgraded tzdata (2009a-1 -> 2009b-1)
[2009-02-18 13:54] Building VirtualBox modules...
[2009-02-18 13:54]
[2009-02-18 13:54] ---------------------------------------------------------------------------
[2009-02-18 13:54] IMPORTANT NOTES:
[2009-02-18 13:54]
[2009-02-18 13:54] - Run "vbox_build_module" as root every time your kernel is upgraded, to
[2009-02-18 13:54] compile the module for the new kernel version.
[2009-02-18 13:54] - Add your user to the vboxusers group:
[2009-02-18 13:54] gpasswd -a USERNAME vboxusers
[2009-02-18 13:54] - Add "vboxdrv" to the MODULES array in your "/etc/rc.conf"
[2009-02-18 13:54] - Add "vboxnetflt" to MODULES if you want Host Interface networking.
[2009-02-18 13:54] - Add the following line to "/etc/fstab" if you want the vboxusers to have
[2009-02-18 13:54] permission to emulate USB devices:
[2009-02-18 13:54] none /proc/bus/usb usbfs auto,busgid=108,busmode=0775,devgid=108,devmode=0664 0 0
[2009-02-18 13:54] ------
[[2009-02-18 13:54] upgraded virtualbox_bin (2.1.2-1 -> 2.1.4-1)
[2009-02-18 13:54] upgraded xfsprogs (2.10.2-1 -> 3.0.0-1)

Here are the new entries in /var/lib/pacman/local...

man-1.6f-2/ 4096 090117 01:22:34 rwxr-xr-x root/root
man-pages-3.18-1/ 4096 090218 13:54:22 rwxr-xr-x root/root
mcpp-2.7.2-1/ 4096 090116 22:18:42 rwxr-xr-x root/root

ttf-dejavu-2.28-1/ 4096 090116 23:50:13 rwxr-xr-x root/root
tunepimp-0.5.3-5/ 4096 090117 15:56:53 rwxr-xr-x root/root
tzdata-2009b-1/ 4096 090218 13:54:22 rwxr-xr-x root/root
udev-135-1/ 4096 090119 03:11:09 rwxr-xr-x root/root

videoproto-2.2.2-1/ 4096 090116 22:18:40 rwxr-xr-x root/root
virtualbox_bin-2.1.4-1/ 4096 090218 13:54:23 rwxr-xr-x root/root
vlc-0.9.8a-5/ 4096 090203 00:32:52 rwxr-xr-x root/root

xfprint-4.4.3-2/ 4096 090204 18:49:33 rwxr-xr-x root/root
xfsprogs-3.0.0-1/ 4096 090218 13:54:53 rwxr-xr-x root/root
xfwm4-4.4.3-1/ 4096 090204 18:49:33 rwxr-xr-x root/root

No doubles here, as it should be ; so now i want to update my backup disk as well :

root@michael /home/michael #rsync -av / /backup --exclude-from=excludes --delete

and here is how these things appear in /backup/var/lib/pacman/local...

man-1.6f-2/ 4096 090117 01:22:34 rwxr-xr-x root/root
man-pages-3.17-1/ 4096 090128 15:28:01 rwxr-xr-x root/root
man-pages-3.18-1/ 4096 090218 13:54:22 rwxr-xr-x root/root

tunepimp-0.5.3-5/ 4096 090117 15:56:53 rwxr-xr-x root/root
tzdata-2009a-1/ 4096 090207 19:39:28 rwxr-xr-x root/root
tzdata-2009b-1/ 4096 090218 13:54:22 rwxr-xr-x root/root

virtualbox_bin-2.1.2-1/ 4096 090210 00:12:23 rwxr-xr-x root/root
virtualbox_bin-2.1.4-1/ 4096 090218 13:54:23 rwxr-xr-x root/root
vlc-0.9.8a-5/ 4096 090203 00:32:52 rwxr-xr-x root/root
vpnc-0.5.3-1/ 4096 090119 03:11:22 rwxr-xr-x root/root

xfsprogs-2.10.2-1/ 4096 090119 03:11:23 rwxr-xr-x root/root
xfsprogs-3.0.0-1/ 4096 090218 13:54:53 rwxr-xr-x root/root
xfwm4-4.4.3-1/ 4096 090204 18:49:33 rwxr-xr-x root/root

To make matters worse still, the same happens in /backup/var/lib/pacman/sync/, (i hadn't noticed that before) for example

xfsprogs-2.10.2-1/ 4096 090215 19:21:25 rwxr-xr-x root/root
xfsprogs-3.0.0-1/ 4096 090218 13:44:01 rwxr-xr-x root/root
xinetd-2.3.14-3/ 4096 090218 13:44:01 rwxr-xr-x root/root

So now of course if i reboot into the backup disk and run pacman -Syu , it'll tell me the four updates are pending, download them again, then say it can't proceed because the entries already exist, as before. Of course if i do this twice, the old entries get erased and the problem goes away, but that's hardly practical, especially if it's been some time since the last backup. The only things i can think of are :
1-there's a permission problem that doesn't let rsync delete anything in local and sync (it can write though, since the new entries are not pacman-generated)
2-could it be that installing something without pacman (e. g. via rsync) also creates an entry for the app in local and sync ? That would explain the double entries, but it's weird.

michaelks · 2009-02-19 00:54:03

ndiswrapper-utils-1.54-1 ndiswrapper-1.54-1 came up to upgrade, so i ran pacman and then rsync on the backup disk - but this time excluding /var/lib/pacman. Then, after the sync, i copied -R /var/lib/pacman to /backup/var/lib, and this is what i found there :
ndiswrapper-1.53-4/ 4096 090119 03:11:11 rwxr-xr-x root/root
ndiswrapper-1.54-1/ 4096 090218 21:41:22 rwxr-xr-x root/root
ndiswrapper-utils-1.53-1/ 4096 090119 03:11:11 rwxr-xr-x root/root
ndiswrapper-utils-1.54-1/ 4096 090218 21:41:15 rwxr-xr-x root/root

Very oddly, the previous double entries from the previous updates also refused to be overwritten, so i still have

xfsprogs-2.10.2-1/ 4096 090119 03:11:23 rwxr-xr-x root/root
xfsprogs-3.0.0-1/ 4096 090218 13:54:53 rwxr-xr-x root/root

So the bottom line seems to be : only pacman can delete anything in its /var/lib/pacman directories. I've copied and rsynced updated versions of them, the new entries get added, but the old ones remain. So if you are making a backup of your system, get a script that will erase the older version of any package file that has one, or you'll run into trouble with pacman. That's it, i guess.

Last edited by michaelks (2009-02-19 04:27:23)

michaelks · 2009-02-20 16:33:24

I've royally been made a fool of by rsync . It suddenly struck me that when running rsync -av / /backup --exclude-from=file --delete ,
as i've been doing for years, the "deleting" messages weren't flying by anymore as they used to do - well, there wasn't any deletion happening ! When i tried
rsync -av /var/ /backup/var/ --delete ,
sure enough "deleting" was all over the screen and all the old stuff was removed. So now i have to run rsync per directory instead of globally. Sorry about all this.

Last edited by michaelks (2009-02-21 05:08:29)

Arch Linux

#1 2009-02-17 18:34:03

outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#2 2009-02-17 22:51:10

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#3 2009-02-18 00:21:12

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#4 2009-02-18 00:26:53

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#5 2009-02-18 02:39:08

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#6 2009-02-18 17:51:19

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#7 2009-02-19 00:54:03

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

#8 2009-02-20 16:33:24

Re: outdated /var/lib/pacman/local entries causing pacman ..[SOLVED]

Board footer