[SOLVED] Pacman: failed to synchronize all databases

spoonman · 2024-08-15 23:50:58

Hi,

I'm having a problem with pacman for some weeks and i can't figure it out. Here is the result of "sudo pacman -Syyu":

:: Synchronizing package databases...
core.db failed to download
extra.db failed to download
error: failed retrieving file 'core.db' from linorg.usp.br : Connection time-out
error: failed retrieving file 'extra.db' from linorg.usp.br : Connection time-out
error: failed to synchronize all databases (download library error)

When i run checkupdates i always get: ERROR: Cannot fetch updates.

I already tried to update the mirrors from the arch mirrors page and tried with a few different mirrors from some countries and it always gives me the above output.

Last edited by spoonman (2024-08-17 18:29:29)

seth · 2024-08-16 01:28:03

linorg.usp.br doesn't respond for me => https://archlinux.org/packages/extra/any/reflector/

Then please check the pacman manpage, understand what the second "y" there does, why it's a really bad idea to unconditionally put i there and then stop doing that.

Edit: ok, the server doesn't respond to ICMP requests, but actually on tcp/80+443
Is there a wider issue w/ your network stability?

Sanity check: please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Last edited by seth (2024-08-16 01:29:38)

spoonman · 2024-08-16 01:40:03

linorg.usp.br doesn't respond for me => https://archlinux.org/packages/extra/any/reflector/

I can't install any packages.

seth wrote:

Sanity check: please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

Here is the output of the previous command:

cronie.service                           | multi-user.target.wants
dbus-org.freedesktop.nm-dispatcher.service | system
dbus-org.freedesktop.timesync1.service   | system
fstrim.timer                             | timers.target.wants
gcr-ssh-agent.socket                     | sockets.target.wants
getty@tty1.service                       | getty.target.wants
NetworkManager.service                   | multi-user.target.wants
NetworkManager-wait-online.service       | network-online.target.wants
p11-kit-server.socket                    | sockets.target.wants
pipewire.socket                          | sockets.target.wants
pulseaudio.socket                        | sockets.target.wants
remote-fs.target                         | multi-user.target.wants
systemd-timesyncd.service                | sysinit.target.wants

dakota · 2024-08-16 02:46:50

spoonman wrote:

linorg.usp.br doesn't respond for me => https://archlinux.org/packages/extra/any/reflector/
I can't install any packages.

Have you tried to update your mirrorlist with a known good mirror?

seth · 2024-08-16 07:43:08

linorg.usp.br is 100% synced, @spoonman, can you download https://archlinux.org/packages/extra/an … /download/ ?
There're no concurrent services

ping -c3 archlinux.thaller.ws
ping -3 linorg.usp.br # for a basic DNS test this is gonna suffice

Edit: missing blank

Last edited by seth (2024-08-16 20:40:38)

spoonman · 2024-08-16 20:39:12

seth wrote:

linorg.usp.br is 100% synced, @spoonman, can you download https://archlinux.org/packages/extra/an … /download/ ?

I downloaded and installed the reflector package and then i did this:

reflector --latest 10 --sort rate --protocol https --save /etc/pacman.d/mirrorlist
pacman -Syy

Here is the output:

:: Synchronizing package databases...
 core.db failed to download
 extra.db failed to download
error: failed retrieving file 'extra.db' from mirror.ufscar.br : Connection time-out
error: failed retrieving file 'core.db' from mirror.ufscar.br : Connection time-out
error: failed retrieving file 'extra.db' from america.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'core.db' from america.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'core.db' from europe.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'extra.db' from europe.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'extra.db' from mirror.ubrco.de : Connection time-out
error: failed retrieving file 'core.db' from mirror.ubrco.de : Connection time-out
error: failed retrieving file 'extra.db' from mirror.theo546.fr : Connection time-out
error: failed retrieving file 'core.db' from mirror.theo546.fr : Connection time-out
error: failed retrieving file 'extra.db' from mirror.moson.org : Connection time-out
error: failed retrieving file 'core.db' from mirror.moson.org : Connection time-out
error: failed retrieving file 'extra.db' from md.mirrors.hacktegic.com : Connection time-out
error: failed retrieving file 'core.db' from md.mirrors.hacktegic.com : Connection time-out
error: failed retrieving file 'extra.db' from archlinux.thaller.ws : Connection time-out
error: failed retrieving file 'core.db' from archlinux.thaller.ws : Connection time-out
error: failed retrieving file 'core.db' from asia.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'extra.db' from asia.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'extra.db' from mirrors.neusoft.edu.cn : Connection time-out
error: failed retrieving file 'core.db' from mirrors.neusoft.edu.cn : Connection time-out
error: failed to synchronize all databases (download library error)

So the problem persists...

seth · 2024-08-16 20:43:22

Have you tried to "ping -c3 archlinux.thaller.ws"?
Also

curl -Lv 'http://archlinux.thaller.ws/core/os/x86_64/core.db' >/dev/null
wget -vO /dev/null 'http://archlinux.thaller.ws/core/os/x86_64/core.db'

spoonman · 2024-08-16 20:52:21

seth wrote:

Have you tried to "ping -c3 archlinux.thaller.ws"?

PING archlinux.thaller.ws (162.55.244.107) 56(84) bytes of data.
64 bytes from archlinux.thaller.ws (162.55.244.107): icmp_seq=1 ttl=50 time=241 ms
64 bytes from archlinux.thaller.ws (162.55.244.107): icmp_seq=2 ttl=50 time=239 ms
64 bytes from archlinux.thaller.ws (162.55.244.107): icmp_seq=3 ttl=50 time=238 ms

--- archlinux.thaller.ws ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 238.207/239.457/240.940/1.127 ms

seth wrote:

curl -Lv 'http://archlinux.thaller.ws/core/os/x86_64/core.db' >/dev/null

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:14 --:--:--     0* Host archlinux.thaller.ws:80 was resolved.
* IPv6: (none)
* IPv4: 162.55.244.107
*   Trying 162.55.244.107:80...
* Connected to archlinux.thaller.ws (162.55.244.107) port 80
> GET /core/os/x86_64/core.db HTTP/1.1
> Host: archlinux.thaller.ws
> User-Agent: curl/8.9.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< server: nginx
< date: Fri, 16 Aug 2024 20:47:46 GMT
< content-type: application/octet-stream
< content-length: 121491
< last-modified: Fri, 16 Aug 2024 11:24:10 GMT
< vary: Accept-Encoding
< etag: "66bf36da-1da93"
< accept-ranges: bytes
<
{ [2630 bytes data]
100  118k  100  118k    0     0   7510      0  0:00:16  0:00:16 --:--:-- 27712
* Connection #0 to host archlinux.thaller.ws left intact

seth wrote:

wget -vO /dev/null 'http://archlinux.thaller.ws/core/os/x86_64/core.db'

wget is not installed in this computer.

seth · 2024-08-16 20:55:57

You've high latency (to that host at least) but no connection or download issues.

pacman-conf

spoonman · 2024-08-16 21:00:48

seth wrote:

You've high latency (to that host at least) but no connection or download issues.
pacman-conf

[options]
RootDir = /
DBPath = /var/lib/pacman/
CacheDir = /var/cache/pacman/pkg/
HookDir = /etc/pacman.d/hooks/
GPGDir = /etc/pacman.d/gnupg/
LogFile = /var/log/pacman.log
HoldPkg = pacman
HoldPkg = glibc
IgnorePkg = networkmanager-openconnect
Architecture = x86_64
Color
CheckSpace
ParallelDownloads = 5
CleanMethod = KeepInstalled
SigLevel = PackageRequired
SigLevel = PackageTrustedOnly
SigLevel = DatabaseOptional
SigLevel = DatabaseTrustedOnly
LocalFileSigLevel = PackageOptional
LocalFileSigLevel = PackageTrustedOnly
[core]
Usage = All
Server = https://mirror.ufscar.br/archlinux/core/os/x86_64
Server = https://america.mirror.pkgbuild.com/core/os/x86_64
Server = https://europe.mirror.pkgbuild.com/core/os/x86_64
Server = https://mirror.ubrco.de/archlinux/core/os/x86_64
Server = https://mirror.theo546.fr/archlinux/core/os/x86_64
Server = https://mirror.moson.org/arch/core/os/x86_64
Server = https://md.mirrors.hacktegic.com/archlinux/core/os/x86_64
Server = https://archlinux.thaller.ws/core/os/x86_64
Server = https://asia.mirror.pkgbuild.com/core/os/x86_64
Server = https://mirrors.neusoft.edu.cn/archlinux/core/os/x86_64
[extra]
Usage = All
Server = https://mirror.ufscar.br/archlinux/extra/os/x86_64
Server = https://america.mirror.pkgbuild.com/extra/os/x86_64
Server = https://europe.mirror.pkgbuild.com/extra/os/x86_64
Server = https://mirror.ubrco.de/archlinux/extra/os/x86_64
Server = https://mirror.theo546.fr/archlinux/extra/os/x86_64
Server = https://mirror.moson.org/arch/extra/os/x86_64
Server = https://md.mirrors.hacktegic.com/archlinux/extra/os/x86_64
Server = https://archlinux.thaller.ws/extra/os/x86_64
Server = https://asia.mirror.pkgbuild.com/extra/os/x86_64
Server = https://mirrors.neusoft.edu.cn/archlinux/extra/os/x86_64

seth · 2024-08-16 21:02:40

Tried to disable parallel downloads?

spoonman · 2024-08-16 21:09:08

seth wrote:

Tried to disable parallel downloads?

disabling parallelDownloads and running pacman -Syy

core.db failed to download
error: failed retrieving file 'core.db' from mirror.ufscar.br : Connection time-out
error: failed retrieving file 'core.db' from america.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'core.db' from europe.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'core.db' from mirror.ubrco.de : Connection time-out
error: failed retrieving file 'core.db' from mirror.theo546.fr : Connection time-out
error: failed retrieving file 'core.db' from mirror.moson.org : Connection time-out
error: failed retrieving file 'core.db' from md.mirrors.hacktegic.com : Connection time-out
error: failed retrieving file 'core.db' from archlinux.thaller.ws : Connection time-out
error: failed retrieving file 'core.db' from asia.mirror.pkgbuild.com : Connection time-out
error: failed retrieving file 'core.db' from mirrors.neusoft.edu.cn : Connection time-out
error: failed to synchronize all databases (download library error)

seth · 2024-08-16 21:25:16

Does that actually take like 100s or do they run through in the wink of an eye?
Does

curl -Lv 'https://archlinux.thaller.ws/core/os/x86_64/core.db' >/dev/null

timeout (nb. the "httpS")?
You're ignorign NM-openconnect, do you use some VPN?

spoonman · 2024-08-16 21:35:56

seth wrote:

Does that actually take like 100s or do they run through in the wink of an eye?

pacman -Syy is running slow. I have a good internet connection, just pacman that is slow.

seth wrote:

Does

curl -Lv 'https://archlinux.thaller.ws/core/os/x86_64/core.db' >/dev/null

timeout (nb. the "httpS")?

No, it took 16s right now.

seth wrote:

You're ignorign NM-openconnect, do you use some VPN?

I'm ignoring because i was having some problems with the newer versions when i was using a VPN,
but right now i'm not using any VPN.

seth · 2024-08-16 22:17:51

I have a good internet connection … it took 16s right now.

The file is 120kB - that's 60 kBit/s, ie. v90/v92 or single channel ISDN - that's "good" by mid 90ies standards…

wget -nv --show-progress -O /dev/null "http://speedtest.frankfurt.linode.com/100MB-frankfurt.bin"

https://archlinux.org/packages/extra/x8 … /download/

spoonman · 2024-08-16 22:23:35

seth wrote:

I have a good internet connection … it took 16s right now.
The file is 120kB - that's 60 kBit/s, ie. v90/v92 or single channel ISDN - that's "good" by mid 90ies standards…
wget -nv --show-progress -O /dev/null "http://speedtest.frankfurt.linode.com/100MB-frankfurt.bin"
https://archlinux.org/packages/extra/x8 … /download/

/dev/null                                         100%[============================================================================================================>] 100.00M  7.47MB/s    in 15s
2024-08-16 19:22:57 URL:http://speedtest.frankfurt.linode.com/100MB-frankfurt.bin [104857600/104857600] -> "/dev/null" [1]

seth · 2024-08-16 22:25:17

Ok, there's obviously no wget + linode issue
Since you now have wget:

wget -vO /dev/null 'http://archlinux.thaller.ws/core/os/x86_64/core.db'

to see whether the problem is w/ curl

spoonman · 2024-08-16 22:29:56

seth wrote:

Ok, there's obviously no wget + linode issue
Since you now have wget:
wget -vO /dev/null 'http://archlinux.thaller.ws/core/os/x86_64/core.db'
to see whether the problem is w/ curl

--2024-08-16 19:27:35--  http://archlinux.thaller.ws/core/os/x86_64/core.db
Resolving archlinux.thaller.ws (archlinux.thaller.ws)... 162.55.244.107
Connecting to archlinux.thaller.ws (archlinux.thaller.ws)|162.55.244.107|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 121491 (119K) [application/octet-stream]
Saving to: ‘/dev/null’

/dev/null                                         100%[============================================================================================================>] 118.64K   168KB/s    in 0.7s

2024-08-16 19:27:51 (168 KB/s) - ‘/dev/null’ saved [121491/121491]

Hey, although we didn't figure out the problem yet, i really want to thank you for your attention until now!

seth · 2024-08-16 22:41:58

wget took less than a second, was that significantly faster than curl?
Does https://wiki.archlinux.org/title/Pacman … ricks#wget allow you to run updates?

spoonman · 2024-08-16 22:52:40

seth wrote:

wget took less than a second, was that significantly faster than curl?
Does https://wiki.archlinux.org/title/Pacman … ricks#wget allow you to run updates?

pacman -Syy output:

:: Synchronizing package databases...
--2024-08-16 19:45:41--  http://arch.mirror.constant.com/core/os/x86_64/core.db
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... 108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 121491 (119K) [application/octet-stream]
Saving to: ‘/var/lib/pacman/sync/core.db.part’

/var/lib/pacman/sync/core.db.part                 100%[============================================================================================================>] 118.64K   208KB/s    in 0.6s

2024-08-16 19:45:57 (208 KB/s) - ‘/var/lib/pacman/sync/core.db.part’ saved [121491/121491]

--2024-08-16 19:45:57--  https://arch.mirror.constant.com/core/os/x86_64/core.db.sig
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... 108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-08-16 19:46:14 ERROR 404: Not Found.

--2024-08-16 19:46:16--  http://arch.mirror.constant.com/extra/os/x86_64/extra.db
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... 108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-08-16 19:46:31 ERROR 404: Not Found.

--2024-08-16 19:46:33--  https://arch.mirror.constant.com/extra/os/x86_64/extra.db
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... 108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-08-16 19:46:49 ERROR 404: Not Found.

--2024-08-16 19:46:51--  https://mirror.moson.org/arch/extra/os/x86_64/extra.db
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving mirror.moson.org (mirror.moson.org)... 202.61.244.119
Connecting to mirror.moson.org (mirror.moson.org)|202.61.244.119|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7622180 (7.3M) [application/octet-stream]
Saving to: ‘/var/lib/pacman/sync/extra.db.part’

/var/lib/pacman/sync/extra.db.part                100%[============================================================================================================>]   7.27M  3.47MB/s    in 2.1s

2024-08-16 19:47:09 (3.47 MB/s) - ‘/var/lib/pacman/sync/extra.db.part’ saved [7622180/7622180]

--2024-08-16 19:47:09--  https://mirror.23m.com/archlinux/extra/os/x86_64/extra.db.sig
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving mirror.23m.com (mirror.23m.com)... 212.83.32.30
Connecting to mirror.23m.com (mirror.23m.com)|212.83.32.30|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-08-16 19:47:25 ERROR 404: Not Found.

pacman -Su output:

Total Download Size:    891.37 MiB
Total Installed Size:  1898.81 MiB
Net Upgrade Size:         6.24 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages...
--2024-08-16 19:47:40--  http://arch.mirror.constant.com/core/os/x86_64/sqlite-3.46.1-1-x86_64.pkg.tar.zst
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... 108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1774095 (1.7M) [application/octet-stream]
Saving to: ‘/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg.tar.zst.part’

/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg. 100%[============================================================================================================>]   1.69M  1.60MB/s    in 1.1s

2024-08-16 19:47:57 (1.60 MB/s) - ‘/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg.tar.zst.part’ saved [1774095/1774095]

--2024-08-16 19:47:57--  https://arch.mirror.constant.com/core/os/x86_64/sqlite-3.46.1-1-x86_64.pkg.tar.zst.sig
Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'
Resolving arch.mirror.constant.com (arch.mirror.constant.com)... ^[[B^[[B^[[B^[[B^[[B^[[B^[[B108.61.5.83
Connecting to arch.mirror.constant.com (arch.mirror.constant.com)|108.61.5.83|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 310 [application/octet-stream]
Saving to: ‘/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg.tar.zst.sig.part’

/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg. 100%[============================================================================================================>]     310  --.-KB/s    in 0s

2024-08-16 19:48:12 (3.81 MB/s) - ‘/var/cache/pacman/pkg/sqlite-3.46.1-1-x86_64.pkg.tar.zst.sig.part’ saved [310/310]
...

It's downloading the packages now, although very slow compared to when pacman was working normally...

seth · 2024-08-16 23:01:51

https://arch.mirror.constant.com/extra/ … 4/extra.db is indeed 404 although the mirror is reported ok.
Edit the mirrorlist and limit it to archlinux.thaller.ws, try again.

Then, if you managed to perform a full update, see whether the issue remains.
(There might be/have been an issue between pacman and libcurl?)

spoonman · 2024-08-17 00:19:13

The update finished, and i commented the XferCommand line again in pacman-conf, but it kept with problems.
Then i had the idea to check what nameserver i was using because while it was updating i noted that resolving the ip
was very slow. Turned out i was using the nameserver from my internet provider and i decided to change to using the
google nameserver and that apparently solved the problem. I will do some more tests this weekend to confirm that.

Thank you.

seth · 2024-08-17 07:03:33

Magenta-colored ISP?
https://bbs.archlinux.org/viewtopic.php?id=298370

spoonman · 2024-08-17 18:28:38

seth wrote:

Magenta-colored ISP?
https://bbs.archlinux.org/viewtopic.php?id=298370

No, i'm using a different ISP.
So far, so good. I believe the problem is solved.

Thank you!

Last edited by spoonman (2024-08-17 18:29:49)

Arch Linux

#1 2024-08-15 23:50:58

[SOLVED] Pacman: failed to synchronize all databases

#2 2024-08-16 01:28:03

Re: [SOLVED] Pacman: failed to synchronize all databases

#3 2024-08-16 01:40:03

Re: [SOLVED] Pacman: failed to synchronize all databases

#4 2024-08-16 02:46:50

Re: [SOLVED] Pacman: failed to synchronize all databases

#5 2024-08-16 07:43:08

Re: [SOLVED] Pacman: failed to synchronize all databases

#6 2024-08-16 20:39:12

Re: [SOLVED] Pacman: failed to synchronize all databases

#7 2024-08-16 20:43:22

Re: [SOLVED] Pacman: failed to synchronize all databases

#8 2024-08-16 20:52:21

Re: [SOLVED] Pacman: failed to synchronize all databases

#9 2024-08-16 20:55:57

Re: [SOLVED] Pacman: failed to synchronize all databases

#10 2024-08-16 21:00:48

Re: [SOLVED] Pacman: failed to synchronize all databases

#11 2024-08-16 21:02:40

Re: [SOLVED] Pacman: failed to synchronize all databases

#12 2024-08-16 21:09:08

Re: [SOLVED] Pacman: failed to synchronize all databases

#13 2024-08-16 21:25:16

Re: [SOLVED] Pacman: failed to synchronize all databases

#14 2024-08-16 21:35:56

Re: [SOLVED] Pacman: failed to synchronize all databases

#15 2024-08-16 22:17:51

Re: [SOLVED] Pacman: failed to synchronize all databases

#16 2024-08-16 22:23:35

Re: [SOLVED] Pacman: failed to synchronize all databases

#17 2024-08-16 22:25:17

Re: [SOLVED] Pacman: failed to synchronize all databases

#18 2024-08-16 22:29:56

Re: [SOLVED] Pacman: failed to synchronize all databases

#19 2024-08-16 22:41:58

Re: [SOLVED] Pacman: failed to synchronize all databases

#20 2024-08-16 22:52:40

Re: [SOLVED] Pacman: failed to synchronize all databases

#21 2024-08-16 23:01:51

Re: [SOLVED] Pacman: failed to synchronize all databases

#22 2024-08-17 00:19:13

Re: [SOLVED] Pacman: failed to synchronize all databases

#23 2024-08-17 07:03:33

Re: [SOLVED] Pacman: failed to synchronize all databases

#24 2024-08-17 18:28:38

Re: [SOLVED] Pacman: failed to synchronize all databases

Board footer