[SOLVED] error failed retrieving file 'core.db' from mirror

racer · 2024-08-01 19:06:52

During my attempts to install linux for the past few weeks I have been running into this error message when running pacstrap:

error: failed retrieving file 'core.db' from mirror: Connection time-out
error: failed retrieving file 'extra.db' from mirror: Connection time-out

This issue also occurs when i try to use pacman -Syyy or the usual pacman commands. I have been having this issue on my main system and an old laptop of mine from 2014.

What I have tried:
Changing the dns in resolv.conf to 8.8.8.8 and 8.8.4.4 (this helped me get a response from pacman at least as with my default nameserver it hangs on core)
Generating the latest mirrors with reflector ( A majority of the mirrors generated have a rate of 0kbs I even manually compiled mirrors that did return a rate into a mirrorlist but the issue persisted)
Switched from wifi to ethernet (slightly improved pacman speeds but issue persisted)
Verify the local time in timedatectl

Any help on how alleviate this issue would be greatly appreciated.

Last edited by racer (2024-08-06 21:03:12)

seth · 2024-08-01 20:39:48

A majority of the mirrors generated have a rate of 0kbs

What reflector command did you use? Don't apply national filters or stuff like that.

How's the network speed generally in this context?
ABBS has a speedtest using wget and some linode servers.

Sidebar: don't use more than one "y" unless you actually mean to (ie. almost never)

cryptearth · 2024-08-01 20:41:54

If you have access to reflector try something like this:

reflector --save /etc/pacman.d/mirrorlist --country <your Country> --protocol https --fastest 20 --verbose --age 12

this should give you at least one working mirror last synced within the past 12 hours
if this doesn't result in something usefull let out the country parameter and maybe also the protocol parameter

/// edit

seth wrote:

Don't apply national filters or stuff like that.

well - without any filters pacman currently lists me 875 registered mirrors
with only limiting to https protocol it's still 316
adding --age 24 only gets down to 293 and with --age 12 to 287

tldr: I guess adding a country filter seems fair to get that down to a more reasonable number - although yes, I do know that a server physical located near you don't mean it's logical near you and that others could be faster anyway

Last edited by cryptearth (2024-08-02 20:57:07)

racer · 2024-08-03 20:01:22

seth wrote:

A majority of the mirrors generated have a rate of 0kbs
What reflector command did you use? Don't apply national filters or stuff like that.
How's the network speed generally in this context?
ABBS has a speedtest using wget and some linode servers.
Sidebar: don't use more than one "y" unless you actually mean to (ie. almost never)

For my mirrors I used:
reflector --verbose --sort rate -l 40 --save /etc/pacman.d/mirrorlist

Also I followed the instructions to install ABBS but for some reason the curl wasnt providing me with all the text needed for it to run. I was able use the internet through lynx with both my original nameserver and 8.8.8.8. However I was unsure of how to copy the text on the page needed for ABBS to run if you could provide guidance on how to do so or any alternative speedtests that would be helpful.

racer · 2024-08-03 20:10:05

cryptearth wrote:

If you have access to reflector try something like this:
reflector --save /etc/pacman.d/mirrorlist --country <your Country> --protocol https --fastest 20 --verbose --age 12
this should give you at least one working mirror last synced within the past 12 hours
if this doesn't result in something usefull let out the country parameter and maybe also the protocol parameter
/// edit
seth wrote:
Don't apply national filters or stuff like that.
well - without any filters pacman currently lists me 875 registered mirrors
with only limiting to https protocol it's still 316
adding --age 24 only gets down to 293 and with --age 12 to 287
tldr: I guess adding a country filter seems fair to get that down to a more reasonable number - although yes, I do know that a server physical located near you don't mean it's logical near you and that others could be faster anyway

I used to the reflector command provided but I ran into issues when using the country filter as I believe mine isnt listed there so I removed it and got around 286 mirrors to test. I was able to get far more mirrors that actually displayed a rate but for some reason the error persists:

error: failed retrieving file from 'core.db' from mirror: Connection time-out

Also before running the reflector command I set my nameserver to 8.8.8.8 and 8.8.4.4 as I usually have to do that to even get a response from these things but after creating the mirrorlist it was reverted. It would also be reverted to the original nameserver after I ran a test trying to install something with pacman -Sy.

seth · 2024-08-03 20:54:53

for some reason the curl wasnt providing me with all the text needed for it to run

"ix.io is taking a break"… I'll have to upload it to 0x0.st (gists are impractical because of the epically long hashes)

wget -nv --show-progress -O /dev/null "http://speedtest.${server}.linode.com/1GB-${server}.bin"

server can be any of ("newark" "singapore" "london" "frankfurt" "dallas" "toronto1" "syd1" "atlanta" "tokyo2" "mumbai1" "fremont")

Also, sanity check: Please post the output of

find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f

@cryptearth, as the OP suggests and your own comment predicted: not everyone lives in a country w/ plenty of servers, if you've only three and they're all lousy, you'd probably be much better off looking across the border.

macromal · 2024-08-03 20:58:34

Alternatively, if you have another machine with Arch Linux available, you can create a custom repository and install the packages in a few seconds.

mkdir repo && cd repo || exit 1
cp /var/cache/pacman/pkg/* .
repo-add ./custom.db.tar.gz ./*[!sig]

Transfer this folder to a external medium and uncomment the following settings in /etc/pacman.conf:

# An example of a custom package repository.  See the pacman manpage for
# tips on creating your own repositories.
[custom]
SigLevel = Optional TrustAll
Server = file:///repo

You have to comment the core and extra repositories configuration too.

racer · 2024-08-05 14:42:10

seth wrote:

for some reason the curl wasnt providing me with all the text needed for it to run
"ix.io is taking a break"… I'll have to upload it to 0x0.st (gists are impractical because of the epically long hashes)
wget -nv --show-progress -O /dev/null "http://speedtest.${server}.linode.com/1GB-${server}.bin"
server can be any of ("newark" "singapore" "london" "frankfurt" "dallas" "toronto1" "syd1" "atlanta" "tokyo2" "mumbai1" "fremont")
Also, sanity check: Please post the output of
find /etc/systemd -type l -exec test -f {} \; -print | awk -F'/' '{ printf ("%-40s | %s\n", $(NF-0), $(NF-1)) }' | sort -f
@cryptearth, as the OP suggests and your own comment predicted: not everyone lives in a country w/ plenty of servers, if you've only three and they're all lousy, you'd probably be much better off looking across the border.

I had to use curl for the speedtests since I couldnt get wget to work I hope that wouldnt impact the testing. The speedtests were completed successfully below is a quick summary of the results:

 

Newark:
Average download:1153k
Time total: 14:48

Singapore:
Average download:1235k
Time total: 13:48

London:
Average download:1169k
Time total: 14:35

Frankfurt:
Average download: 1312k
Time total: 12:59

Dallas:
Average download: 1256k
Time total: 13:34

toronto1:
Average download: 1430k
Time total: 11:55

Syd1:
Average download: 1615k
Total time: 10:33

Atlanta:
Average download: 1527k
Time total: 11:10

Tokyo2:
Average download: 1175k
Time total: 14:31

Mumbai1:
Average download: 1462k
Time total: 11:40

Fremont:
Average download: 1209k
Time total: 14:06

Also here is the output requested:

choose-mirror.service                    | multi-user.target.wants
cloud-config.service                     | cloud-init.target.wants
cloud-final.service                      | cloud-init.target.wants
cloud-init-local.service                 | cloud-init.target.wants
cloud-init.service                       | cloud-init.target.wants
ctrl-alt-del.target                      | system
dbus-org.freedesktop.home1.service       | system
dbus-org.freedesktop.ModemManager1.service | system
dbus-org.freedesktop.network1.service    | system
dbus-org.freedesktop.resolve1.service    | system
dbus-org.freedesktop.timesync1.service   | system
getty@tty1.service                       | getty.target.wants
hv_fcopy_daemon.service                  | multi-user.target.wants
hv_kvp_daemon.service                    | multi-user.target.wants
hv_vss_daemon.service                    | multi-user.target.wants
iwd.service                              | multi-user.target.wants
livecd-alsa-unmuter.service              | sound.target.wants
livecd-talk.service                      | multi-user.target.wants
machines.target                          | multi-user.target.wants
ModemManager.service                     | multi-user.target.wants
p11-kit-server.socket                    | sockets.target.wants
pacman-init.service                      | multi-user.target.wants
pcscd.socket                             | sockets.target.wants
reflector.service                        | multi-user.target.wants
remote-cryptsetup.target                 | multi-user.target.wants
remote-fs.target                         | multi-user.target.wants
sshd.service                             | multi-user.target.wants
systemd-boot-update.service              | sysinit.target.wants
systemd-confext.service                  | sysinit.target.wants
systemd-homed-activate.service           | systemd-homed.service.wants
systemd-homed.service                    | multi-user.target.wants
systemd-journald-audit.socket            | sockets.target.wants
systemd-journald-audit.socket            | systemd-journald.service.wants
systemd-mountfsd.socket                  | sockets.target.wants
systemd-network-generator.service        | sysinit.target.wants
systemd-networkd-wait-online.service     | network-online.target.wants
systemd-networkd.service                 | multi-user.target.wants
systemd-networkd.socket                  | sockets.target.wants
systemd-nsresourced.socket               | sockets.target.wants
systemd-pstore.service                   | sysinit.target.wants
systemd-resolved.service                 | multi-user.target.wants
systemd-resolved.service                 | sysinit.target.wants
systemd-sysext.service                   | sysinit.target.wants
systemd-time-wait-sync.service           | sysinit.target.wants
systemd-timesyncd.service                | sysinit.target.wants
systemd-userdbd.socket                   | sockets.target.wants
vboxservice.service                      | multi-user.target.wants
vmtoolsd.service                         | multi-user.target.wants
vmware-vmblock-fuse.service              | multi-user.target.wants

Last edited by racer (2024-08-06 20:56:22)

racer · 2024-08-05 15:15:14

macromal wrote:

Alternatively, if you have another machine with Arch Linux available, you can create a custom repository and install the packages in a few seconds.
mkdir repo && cd repo || exit 1
cp /var/cache/pacman/pkg/* .
repo-add ./custom.db.tar.gz ./*[!sig]
Transfer this folder to a external medium and uncomment the following settings in /etc/pacman.conf:
# An example of a custom package repository.  See the pacman manpage for
# tips on creating your own repositories.
[custom]
SigLevel = Optional TrustAll
Server = file:///repo
You have to comment the core and extra repositories configuration too.

Unfortunately I dont have an arch system set up but I do know someone that does would it still work if I had them do it for me?

seth · 2024-08-05 15:35:31

Please use [code][/code] tags, not "quote" tags. Edit your post in this regard.

There seem no conflicting services (this is all still from the install iso anyway?) and you've like 10Mbit/s downstream (which isn't very fast but certainly sufficient)

Wrt

Also before running the reflector command I set my nameserver to 8.8.8.8 and 8.8.4.4 as I usually have to do that to even get a response from these things but

Changing the dns in resolv.conf to 8.8.8.8 and 8.8.4.4

if this is down to some DNS issue, don't edit resolv.conf but https://wiki.archlinux.org/title/System … d#Manually though the reset sounds like the conneciton would periodically break down.

If you can, please post your complete system journal for the boot:

sudo journalctl -b | curl -F 'file=@-' 0x0.st

cryptearth · 2024-08-05 17:10:39

what type of connection is it?
mobile?
dial-up?
dsl?
cable?
as you seem to have good peering (as in same speed to any mirror) the other stuff sounds like a dns issue not able to correctly resolve the hostnames - could be the isp at fault

racer · 2024-08-06 21:00:35

seth wrote:

Please use [code][/code] tags, not "quote" tags. Edit your post in this regard.
There seem no conflicting services (this is all still from the install iso anyway?) and you've like 10Mbit/s downstream (which isn't very fast but certainly sufficient)
Wrt
Also before running the reflector command I set my nameserver to 8.8.8.8 and 8.8.4.4 as I usually have to do that to even get a response from these things but
Changing the dns in resolv.conf to 8.8.8.8 and 8.8.4.4
if this is down to some DNS issue, don't edit resolv.conf but https://wiki.archlinux.org/title/System … d#Manually though the reset sounds like the conneciton would periodically break down.
If you can, please post your complete system journal for the boot:
sudo journalctl -b | curl -F 'file=@-' 0x0.st

My apologies I edited my post. Editing the dns in /etc/systemd/resolved.conf.d/dns_servers.conf solved the issue and I was able to use pacman smoothly along with running the pacstrap command. Honestly thank you for all your help I am curious as to why I would need to edit my dns for this to work as I encountered this issue 2 years prior where I was forced to edit resolv.conf for it to work.

seth · 2024-08-06 21:15:10

What's the default DNS?
(Most likely your ISP or an ISP issued modemrouterswitch combo that forwards some ISP DNS? In the latter case you'll have to inspect the router configuration)

The default DNS provided by your ISP might be underdimensioned and not be able to respond

Arch Linux

#1 2024-08-01 19:06:52

[SOLVED] error failed retrieving file 'core.db' from mirror

#2 2024-08-01 20:39:48

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#3 2024-08-01 20:41:54

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#4 2024-08-03 20:01:22

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#5 2024-08-03 20:10:05

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#6 2024-08-03 20:54:53

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#7 2024-08-03 20:58:34

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#8 2024-08-05 14:42:10

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#9 2024-08-05 15:15:14

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#10 2024-08-05 15:35:31

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#11 2024-08-05 17:10:39

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#12 2024-08-06 21:00:35

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

#13 2024-08-06 21:15:10

Re: [SOLVED] error failed retrieving file 'core.db' from mirror

Board footer