Synchronizing package database get's close but can't complete

cmtm · 2014-01-22 19:30:30

I've been trying to do a regular pacman sync and upgrade since moving my virtual arch linux guest to a new host.
The issue is that it can never get past the "Synchronizing package databases" step. This is the output of a simple pacman command.

chris@aKeeper:~/ott-18$ sudo pacman -Syu
:: Synchronizing package databases...
 core                                                                                     106.5 KiB   155K/s 00:01 [####################################################################] 100%
error: failed retrieving file 'extra.db' from mirror.de.leaseweb.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds################################--------]  89%
error: failed retrieving file 'extra.db' from archlinux.c3sl.ufpr.br : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds################################--------]  89%
error: failed retrieving file 'extra.db' from mirror.us.leaseweb.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds##############################----------]  86%
error: failed retrieving file 'extra.db' from ftp.nluug.nl : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds#######################################-----------]  84%
^Cxtra                                                                                1375.6 KiB  7.00B/s 06:12:34 [############################################################--------]  89%
Interrupt signal received

It makes it to about 89% very fast and then slows down to a stop.
pacman -Syy does something similar

chris@aKeeper:~/ott-18$ sudo pacman -Syy
:: Synchronizing package databases...
 core                                                                                     106.5 KiB  62.9K/s 00:02 [####################################################################] 100%
error: failed retrieving file 'extra.db' from mirror.de.leaseweb.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds----------------------------------------]   0%
error: failed retrieving file 'extra.db' from archlinux.c3sl.ufpr.br : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds----------------------------------------]   0%
error: failed retrieving file 'extra.db' from mirror.us.leaseweb.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds----------------------------------------]   0%
error: failed retrieving file 'extra.db' from ftp.nluug.nl : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds--------------------------------------------------]   0%
error: failed retrieving file 'extra.db' from archlinux.mirror.kangaroot.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds--------------------------------]   0%
error: failed retrieving file 'extra.db' from archlinux.c3sl.ufpr.br : Protocol rsync not supported or disabled in libcurl
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from mirror.de.leaseweb.net : Operation too slow. Less than 1 bytes/sec transferred the last 10 seconds##############################------]  92%
^Community                                                                               1914.5 KiB   866B/s 03:31 [#############################################################-------]  91%
Interrupt signal received

I suspected it might be a problem with mirrorlist, so I updated that using reflector, but the problem still persists.
I manually downloaded the repos and it worked, but I'd obviously like a more permanent fix so that I don't have to keep doing this.

Here is the description of the environment to the best of my knowledge.
Arch64 guest on Windows 7 64-bit host. VirtualBox hypervisor.
IPv6 network, this is the network at my work place.
I noticed someone mentioning network issues here: https://www.virtualbox.org/ticket/9380, but in my case, all other network traffic runs at regular speeds.

Lone_Wolf · 2014-01-23 11:18:16

Check /etc/pacman.conf , are you using any XferCommand or the default one ?
Eitherway try switching to another XferCommand.

cmtm · 2014-01-23 15:17:24

I was using the default XferCommand. After uncomenting the 'wget' one, it worked, though my transfer speeds were low.

XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u

It's still seems to have intermittent trouble connecting though

:: Synchronizing package databases...
--2014-01-23 10:14:04--  http://mirror.de.leaseweb.net/archlinux/core/os/x86_64/core.db
Resolving mirror.de.leaseweb.net (mirror.de.leaseweb.net)... 46.165.198.1, 2a00:c98:2010:1:1:face:d06:f00d
Connecting to mirror.de.leaseweb.net (mirror.de.leaseweb.net)|46.165.198.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 109046 (106K) [application/octet-stream]
Saving to: ‘/var/lib/pacman/sync/core.db.part’

100%[====================================================>] 109,046     55.0KB/s   in 1.9s   

Last-modified header invalid -- time-stamp ignored.
2014-01-23 10:14:07 (55.0 KB/s) - ‘/var/lib/pacman/sync/core.db.part’ saved [109046/109046]

--2014-01-23 10:14:07--  http://mirror.de.leaseweb.net/archlinux/core/os/x86_64/core.db.sig
Resolving mirror.de.leaseweb.net (mirror.de.leaseweb.net)... 46.165.198.1, 2a00:c98:2010:1:1:face:d06:f00d
Connecting to mirror.de.leaseweb.net (mirror.de.leaseweb.net)|46.165.198.1|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-01-23 10:14:08 ERROR 404: Not Found.

--2014-01-23 10:14:10--  http://mirror.de.leaseweb.net/archlinux/extra/os/x86_64/extra.db
Resolving mirror.de.leaseweb.net (mirror.de.leaseweb.net)... 46.165.198.1, 2a00:c98:2010:1:1:face:d06:f00d
Connecting to mirror.de.leaseweb.net (mirror.de.leaseweb.net)|46.165.198.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1566417 (1.5M) [application/octet-stream]
Saving to: ‘/var/lib/pacman/sync/extra.db.part’

100%[====================================================>] 1,566,417   80.5KB/s   in 38s    

Last-modified header invalid -- time-stamp ignored.
2014-01-23 10:14:47 (40.8 KB/s) - ‘/var/lib/pacman/sync/extra.db.part’ saved [1566417/1566417]

--2014-01-23 10:14:47--  http://mirror.de.leaseweb.net/archlinux/extra/os/x86_64/extra.db.sig
Resolving mirror.de.leaseweb.net (mirror.de.leaseweb.net)... 46.165.198.1, 2a00:c98:2010:1:1:face:d06:f00d
Connecting to mirror.de.leaseweb.net (mirror.de.leaseweb.net)|46.165.198.1|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2014-01-23 10:14:48 ERROR 404: Not Found.

--2014-01-23 10:14:50--  http://mirror.de.leaseweb.net/archlinux/community/os/x86_64/community.db
Resolving mirror.de.leaseweb.net (mirror.de.leaseweb.net)... 46.165.198.1, 2a00:c98:2010:1:1:face:d06:f00d
Connecting to mirror.de.leaseweb.net (mirror.de.leaseweb.net)|46.165.198.1|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2144895 (2.0M) [application/octet-stream]
Saving to: ‘/var/lib/pacman/sync/community.db.part’

cmtm · 2014-01-23 15:28:57

I just realized that when messing around with the mirrorlist to fix this problem, I ended up with a bad mirror list. I'm rating all of them right now and will update this thread once it's done.

cmtm · 2014-01-23 16:14:22

no dice. it still doesn't working after updating the mirrorlist.

Lone_Wolf · 2014-01-25 23:00:02

Ok, wget output does show your VM is able to resolve both ipv4 and ipv6 addresses .
wget uses ipv4 by default, unless it's configured specifically to prefer ipv6.

I don't know how the default pacman transfer command (I THINK it uses curl) deals with ipv6 / ipv4 , but since you mentioned the company network used for the VM is an ipv6 network, this does suggest the cause might be related to issues with ipv4 traffic over your companies ipv6 network.

check /etc/wgetrc , near the bottom you'll find a line like this :

# To try ipv6 addresses first:
#prefer-family = IPv6

I suggest you uncomment the #prefer-family = IPv6 line, then check if that works better.

Arch Linux

#1 2014-01-22 19:30:30

Synchronizing package database get's close but can't complete

#2 2014-01-23 11:18:16

Re: Synchronizing package database get's close but can't complete

#3 2014-01-23 15:17:24

Re: Synchronizing package database get's close but can't complete

#4 2014-01-23 15:28:57

Re: Synchronizing package database get's close but can't complete

#5 2014-01-23 16:14:22

Re: Synchronizing package database get's close but can't complete

#6 2014-01-25 23:00:02

Re: Synchronizing package database get's close but can't complete

Board footer