what happens when a mirror doesn't have the packages

graysky · 2009-05-09 13:12:16

I went to update my system this morning and found that the mirror didn't have half of the updates, but according to the pacman output, it did indeed download something for the package. Further, when I ran pacman again, it told me my system was indeed up to date. What's going on?

# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra                    377.7K  826.8K/s 00:00:00 [##############################################] 100%
 community                365.2K  304.2K/s 00:00:01 [##############################################] 100%
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Targets (4): gparted-0.4.5-1  lsof-4.82-1  neon-0.28.4-1  python-nose-0.11.0-1  

Total Download Size:    1.68 MB
Total Installed Size:   5.44 MB

Proceed with installation? [Y/n] y
:: Retrieving packages from extra...
 gparted-0.4.5-1-x86_64  1129.8K  936.5K/s 00:00:01 [##############################################] 100%
error: failed retrieving file 'lsof-4.82-1-x86_64.pkg.tar.gz' from mirrors.gigenet.com : Not Found
 lsof-4.82-1-x86_64        96.7K  267.0K/s 00:00:00 [##############################################] 100%
 neon-0.28.4-1-x86_64     231.6K  413.9K/s 00:00:01 [##############################################] 100%
error: failed retrieving file 'python-nose-0.11.0-1-x86_64.pkg.tar.gz' from mirrors.gigenet.com : Not Found
 python-nose-0.11.0-...   259.7K  407.4K/s 00:00:01 [##############################################] 100%
checking package integrity...
(4/4) checking for file conflicts                   [##############################################] 100%
(1/4) upgrading gparted                             [##############################################] 100%
Optional dependencies for gparted
    e2fsprogs: for ext2/ext3 partitions
    dosfstools: for fat16/32 partitions
    jfsutils: for jfs partitions
    ntfsprogs: for ntfs partitions
    reiserfsprogs: for reiser partitions
    reiser4progs: for reiser partitions
    xfsprogs: for xfs partitions
    gksu: to run gparted directly from menu
(2/4) upgrading lsof                                [##############################################] 100%
(3/4) upgrading neon                                [##############################################] 100%
(4/4) upgrading python-nose                         [##############################################] 100%

Now running it again:

# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 local database is up to date

Allan · 2009-05-09 13:16:13

It downloads (or attempts to download) the package from the next mirror in your /etc/pacman.d/mirrorlist file. e.g from your output, lsof was not on your first mirror but was on the next.

graysky · 2009-05-09 13:22:12

EDIT -- WHOOA, I just commented out the gigenet mirror (the next one on my list is the rit.edu mirror) and did:

# pacman -Syy

to update and then repeated the update which found a BUNCH of packages it wanted to bring down. Is this normal?

# pacman -Syy
:: Synchronizing package databases...
 core                      33.5K  164.7K/s 00:00:00 [##############################################] 100%
 extra                    377.9K  540.3K/s 00:00:01 [##############################################] 100%
 community                365.3K  441.6K/s 00:00:01 [##############################################] 100%
[root@novelty pacman.d]# pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Targets (10): policykit-0.9-8  consolekit-0.3.0-4  file-5.03-1  gconf-2.26.1-1  policykit-gnome-0.9.2-4  
              gnome-power-manager-2.26.1-2  libgsf-1.14.13-1  libtheora-1.0-1.1  rsync-3.0.6-1  
              vinagre-2.26.1-2  

Total Download Size:    9.13 MB
Total Installed Size:   32.34 MB

Allan · 2009-05-09 13:24:11

Yes - these are all fairly recent updates (in the last day) so that mirror is just more up to date.

graysky · 2009-05-09 13:34:51

Thanks for the quick replies! So do all mirrors get the 'list' of packages before they get the actual package itself?

Allan · 2009-05-09 13:40:59

The mirrors just do an rsync from the official location. If more packages get added while the syncing is happening, the mirror can become slightly out of sync for a while. Not sure if there is a good way to deal with that...

ozar · 2009-05-10 18:33:22

Hi Allan

I get the same "failed retrieving file" problems as posted by the OP quite often, lately, and it doesn't seem to matter what mirror is used in the mirrorlist. Is there some way that this can be corrected at the server level so that it doesn't happen?

Unfortunately, I'm not familiar enough with pacman or programming/coding to suggest a fix, but just curious if this is a bug that should be treated as such, or would it be considered a feature request that should be made elsewhere on the Arch website?

Thank you for your help.

kumyco · 2009-05-12 21:42:40

try your updates at a different time, maybe a couple hours different than when you normally do
----------
i actually worked out a simple way to resolve that issue a few days ago, i'll comment here because i'm lazy to file a bug report
it requires some backend work though so i don't know how many mirrors would be able to support it
basically, set rsync to ignore the non .pkg files, sync normally then manually get the other files
you might even do a check before moving the new files in-place
maybe something like

for pkg in $(tar -tzf [repo].db.tar.gz | grep /$ | sed 's|/|.pkg.tar.gz|'); do
    if [ ! -f [base]/$pkg ]; then
        return 1
    fi
done

ozar · 2009-05-12 22:07:27

Thanks for the reply and the information, kumyco.

It would seem to me that the problem is not when I'm trying to upgrade, but more a problem with the mirror itself, that problem being either the packages list is wrong, or the packages in the repo are not current. Surely there is some way for a mirror to test itself as needed to either get the correct list, or to fetch any packages it needs to match the list. If that is in fact the way it's supposed to work, perhaps there is some way of letting the user know that the mirror is off-line until everything is updated and correct.

For whatever reason, this issue seems to be happening more all the time. The other day when it happened, I waited a while and tried again, and a new package list was downloaded with more packages only for the entire problem to repeat itself. Again, it doesn't matter where in the world the mirrors are, it just keeps happening.

In the end, it seems more like a bug that should be filed rather than a feature request, but I'm not even sure.

Anyway, thanks again!

kumyco · 2009-05-12 23:46:16

i can't comment on how other mirrors work, by my ARM mirror (currently) simply rsyncs with iblibio (and hope it's synced)
it does it once a day, midnight one of the usa timezones which is prolly not the wisest idea, because it increases the risk of db being out of sync but it only happens this once and at the same time very night so you can avoid it.
i experience this problem occasionally, well every time i sync in the early hours maybe 02:00-04:00 here in the uk
but by 08:00 the mirrors i used are all synced up. you can just change mirrors, some of them go dead after a while anyway

DanuX · 2009-05-15 16:52:44

I'm posting my question here because i didn't want to make a new topic for this simple question.....

Anyway, I'm trying to download pigdin MSN client via pacman, but I don't know how.
I tryed to use command pacman -Ss pigdin but it only thinks for a while and then it stops and nothing happens.
Where can I check is it on the database?

Thanks for help

DanuX

Edit: Problem solved. I spelled it incorrectly, its suppose to be Pidgin not Pigdin....

Last edited by DanuX (2009-05-15 17:41:03)

kumyco · 2009-05-15 19:52:19

the package name is pidgin, not d before g

Arch Linux

#1 2009-05-09 13:12:16

what happens when a mirror doesn't have the packages

#2 2009-05-09 13:16:13

Re: what happens when a mirror doesn't have the packages

#3 2009-05-09 13:22:12

Re: what happens when a mirror doesn't have the packages

#4 2009-05-09 13:24:11

Re: what happens when a mirror doesn't have the packages

#5 2009-05-09 13:34:51

Re: what happens when a mirror doesn't have the packages

#6 2009-05-09 13:40:59

Re: what happens when a mirror doesn't have the packages

#7 2009-05-10 18:33:22

Re: what happens when a mirror doesn't have the packages

#8 2009-05-12 21:42:40

Re: what happens when a mirror doesn't have the packages

#9 2009-05-12 22:07:27

Re: what happens when a mirror doesn't have the packages

#10 2009-05-12 23:46:16

Re: what happens when a mirror doesn't have the packages

#11 2009-05-15 16:52:44

Re: what happens when a mirror doesn't have the packages

#12 2009-05-15 19:52:19

Re: what happens when a mirror doesn't have the packages

Board footer