[SOLVED] Mirrorlist updater webpage

justasug · 2017-03-19 12:56:26

I've been using the Mirrorlist updater webpage for years now to generate new mirror lists without problems. Recently, I don't know exactly when, I noticed that the output changed.

I am getting "weird" mirrors on top and the score next to each mirror's location in the comment is gone. I am still using the exact settings when I click the "Generate List" button. Ticked are: HTTP, HTTPS, IPv4 and Use mirror status and "Country" is set to "All". Click for the URL it gives me.

I have compared it to the output on the Mirror Status page and it's not sorting them like on that page. It also gives me completely different results whenever I refresh the page.

Mirrorlist updater result:

##
## Arch Linux repository mirrorlist
## Filtered by mirror score from mirror status page
## Generated on 2017-03-19
##

## Philippines
#Server = http://mirror.rise.ph/archlinux/$repo/os/$arch
## Germany
#Server = http://mirror.loli.forsale/arch/$repo/os/$arch
## Czech Republic
#Server = https://mirror.dkm.cz/archlinux/$repo/os/$arch
## Germany
#Server = http://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch
## Netherlands
#Server = http://mirror.i3d.net/pub/archlinux/$repo/os/$arch
## Germany
#Server = https://mirror.bethselamin.de/$repo/os/$arch
## Germany
#Server = http://archlinux.honkgong.info/$repo/os/$arch
## United States
#Server = http://mirror.vtti.vt.edu/archlinux/$repo/os/$arch
## Ukraine
#Server = http://archlinux.ip-connect.vn.ua/$repo/os/$arch
## Germany
#Server = https://mirror.f4st.host/archlinux/$repo/os/$arch
## France
#Server = http://mirror.archlinux.ikoula.com/archlinux/$repo/os/$arch
## Ecuador
#Server = http://mirror.uta.edu.ec/archlinux/$repo/os/$arch
...

The first result is a Filipino server, that according to the Mirror Status page has a score of 4.2 and is a Tier 2 mirror. That doesn't sound quite right. Did something change or is this a bug?

Last edited by justasug (2017-05-04 15:41:29)

onslow77 · 2017-03-19 13:31:12

Hello!

@justasug - you are not alone, I also get the odd result that you describe when I use Pacman Mirrorlist Generator.

Regards
Martin

nixIT · 2017-03-22 13:13:25

+1 on odd mirrorlist behavior.

I noticed that after my last update on 3/16/2017, when mirrorlist.pacnew downloaded, I merged the new mirrorlist, and since then I have not had updates. I tried the mirrorlist generated this morning and it was flagging many updates to download.

Not which is accurate, the mirrorlist.pacnew that came down during last weeks update, or this weeks mirrorlist generator.

--nixIT

Xyne · 2017-03-22 14:15:57

Open a ticket on the bug tracker to report the issue. Some of the devs do not browse the forum.

Incidentally, if you want more control over the generated mirrorlist (e.g. sort order), use reflector

hashhar · 2017-04-14 23:40:59

In the meantime use the following script to generate a mirrorlist sorted by score. It only includes the mirrors with completion %.

If you only want mirrors of a particular type, replace the curl line with

curl https://www.archlinux.org/mirrors/status/json/ | jq '.urls | sort_by(.score) | .[] | select(.score != null) | select(.completion_pct >= 0.9) | select(.protocol == "http") | { protocol: .protocol, url: .url, country: .country, score: .score }' > "$TEMP_FILE"

You can change the part that says select(.protocol == "http") to either https or rsync.

You'll need to install

jq

though. It's in the repos and quite a small package.

TEMP_FILE='/tmp/mirrors.json'
# Non-null score, completion>90%, sorted by score
curl https://www.archlinux.org/mirrors/status/json/ | jq '.urls | sort_by(.score) | .[] | select(.score != null) | select(.completion_pct >= 0.9) | { protocol: .protocol, url: .url, country: .country, score: .score }' > "$TEMP_FILE"

urls=( $(jq -r '.url' "$TEMP_FILE") )
scores=( $(jq '.score' "$TEMP_FILE") )
# Bash >= 4.0
if (( BASH_VERSINFO >= 4 )); then
    readarray -t countries <<<"$(jq '.country' "$TEMP_FILE")"
else
    IFS=$'\n' read -rd '' -a countries <<<"$(jq '.country' "$TEMP_FILE")"
fi
count=${#urls[@]}

for (( i=0; i < count; ++i )); do
    printf '## %s, Score = %s\nServer = %s$repo/os/$arch\n' "${countries[$i]}" "${scores[$i]}" "${urls[$i]}"
done

# vim: et sw=4

EDIT: I was actually waiting to make this script power a webpage where you can interactively add options like only a certain protocol, or only certain countries or a completion % or delay or score or ANY COMBINATION.

If anyone's interested I will continue but otherwise this script as is fullfills my needs.

Last edited by hashhar (2017-04-14 23:43:48)

loqs · 2017-04-14 23:57:23

justasug wrote:

That doesn't sound quite right. Did something change or is this a bug?

https://lists.archlinux.org/pipermail/a … 28681.html?

hashhar · 2017-04-15 00:55:19

loqs wrote:

justasug wrote:
That doesn't sound quite right. Did something change or is this a bug?
https://lists.archlinux.org/pipermail/a … 28681.html?

Damn! And here I was creating a script.

Arch Linux

#1 2017-03-19 12:56:26

[SOLVED] Mirrorlist updater webpage

#2 2017-03-19 13:31:12

Re: [SOLVED] Mirrorlist updater webpage

#3 2017-03-22 13:13:25

Re: [SOLVED] Mirrorlist updater webpage

#4 2017-03-22 14:15:57

Re: [SOLVED] Mirrorlist updater webpage

#5 2017-04-14 23:40:59

Re: [SOLVED] Mirrorlist updater webpage

#6 2017-04-14 23:57:23

Re: [SOLVED] Mirrorlist updater webpage

#7 2017-04-15 00:55:19

Re: [SOLVED] Mirrorlist updater webpage

Board footer