Use Arch Mirrorcheck !

Raffles10 · 2009-12-22 21:26:25

Just a quick heads up for potential Arch ftp installers. Check Arch Mirrorcheck before installing via ftp.

You may think that by doing an ftp install you're getting all the latest that Arch has to offer, a quick look at Mirrorcheck will ensure that you do. After spending some time playing with another distro' I re-installed Arch + KDE two days ago choosing ftp://mirrors.uk2.net as the mirror for installation. Two days have gone by with no updates so I checked Mirrorcheck. It shows, unmissably emblazoned in red, that ftp://mirrors.uk2.net hasn't synced for 33 days ! So switching to another mirror I now have 650mb of updates to install, it looks like the entire KDE has updated in the last 33 days, just my luck.

Kinda wish I'd checked it before installing.

some-guy94 · 2009-12-22 21:45:27

archlinux.unixheads.org is also outdated, I went nearly 30 days without updating.

Meyithi · 2009-12-22 21:57:31

Must admit same thing happened to me

Is nothing on the beginners guide to warn about it, but afterwards it dawns on you at some stage that the reason you aren't getting updates is because the mirror is dormant.

Was no harm done though and it was months ago.

ctarwater · 2009-12-22 22:02:44

I've been in the process of organizing all of my install/post-install notes and just today organized my mirrorlist crapola to make sure you have the fastest most up to date mirrors (for your area).

1) install reflector (AUR)
2) run reflector 'reflector -l 10 -r -o /etc/pacman.d/mirrorlist' will scan mirrorcheck for the most up to date mirrors and write the 10 most recent to your mirrorlist file.
3) backup your mirrorlist file 'cp mirrorlist mirrorlist.backup'
4) run rankmirrors to choose your fastest connection from the list reflector just provided 'rankmirrors -n 6 mirrorlist.backup > mirrorlist'

Most of this is in the wiki but not in such plain language.

majiq · 2009-12-22 22:03:03

I wrote a little script to check the uncommented mirrors from pacman.d for me from the command line. Maybe someone will find it useful. It's written using python3, which I'm hardly competent with, so suggestions are appreciated. It's probably also a little convoluted; I couldn't find a DOM XML parser for python. It's good for when you see that you have no updates, you can just run mirrorcheck.py and it'll tell you whether you're being an Arch update-addict or using bad mirrors.

#!/usr/bin/python3

import subprocess
import http.client
from html.parser import HTMLParser

servers_to_check = []

#-------------------------------------------------------------------------------
#Define the parser
class server_checker (HTMLParser):
    check_servers = []
    current_repo = ''
    inA = 0
    inCaption = 0
    foundServer = 0
    printMode = 0
    blockMode = 0

    def handle_starttag (self, tag, attrs):
        if tag == 'caption':
            self.inCaption = 1
        if tag == 'a':
            self.inA = 1
    
    def handle_endtag (self, tag):
        if tag == 'caption':
            self.inCaption = 0
        if tag == 'a':
            self.inA = 0
        if tag == 'td':
            if self.blockMode > 0:
                self.blockMode -= 1

    def handle_data (self, text):
        if self.inA == 1:
            self.printMode = -1
            for i in range(0,len(self.check_servers)):
                if text.find(self.check_servers[i].split(' ')[0]) != -1:
                    self.printMode = 1
                    print("Server: ", self.check_servers[i])
                    self.current_repo = self.check_servers[i].split(' ')[1]
                    break
            if self.printMode == -1:
                self.printMode = 0
        elif self.printMode == 1:
            if text.find(self.current_repo) > 0 and self.blockMode == 0:
                self.blockMode = 4
            if len(text.strip('\n ')) > 0 and self.blockMode > 0:
                if self.blockMode == 4:
                    print('\t' , text.strip('\n '), end = '\t')
                elif self.blockMode == 2:
                    print(text.strip('\n '))
                else:
                    print(text.strip('\n '), end='\t')

#-------------------------------------------------------------------------------
#Set up the parser

myServerChecker = server_checker()

i = 1
for line in subprocess.getoutput("grep -- '^[^#]' /etc/pacman.d/mirrorlist | sed -e 's|^.*://||;s|/.*/| |'").split('\n'):
    myServerChecker.check_servers.append(line)
    print ("Server",i,": ",line.split(' ')[0])
    i+=1

connection = http.client.HTTPSConnection('users.archlinux.de')
connection.request('GET','/~gerbra/mirrorcheck.html')
response = connection.getresponse()

print ("Connection response is" , response.status)

if response.status != 200:
    print ("Connection error")
    exit

#-------------------------------------------------------------------------------
#Run the parser
myServerChecker.feed(response.read().decode().strip('\n'))
myServerChecker.close()

Edit: Well...aren't I shamefaced for having reinvented the wheel. Oh well, it was a fun distraction. :-)

Last edited by majiq (2009-12-22 22:04:03)

Xyne · 2009-12-22 22:28:08

ctarwater wrote:

I've been in the process of organizing all of my install/post-install notes and just today organized my mirrorlist crapola to make sure you have the fastest most up to date mirrors (for your area).
1) install reflector (AUR)
2) run reflector 'reflector -l 10 -r -o /etc/pacman.d/mirrorlist' will scan mirrorcheck for the most up to date mirrors and write the 10 most recent to your mirrorlist file.
3) backup your mirrorlist file 'cp mirrorlist mirrorlist.backup'
4) run rankmirrors to choose your fastest connection from the list reflector just provided 'rankmirrors -n 6 mirrorlist.backup > mirrorlist'
Most of this is in the wiki but not in such plain language.

"reflector -r" already runs the list through rankmirrors, so there's no point in running rankmirrors again on the list in step 4.

ctarwater · 2009-12-22 22:35:04

really? sweet, I didn't know that. Thanks Xyne.

Raffles10 · 2009-12-22 23:27:25

Updates complete, no errors. I'll try reflector, sounds very useful.

lolilolicon · 2009-12-23 02:35:33

I find the mirror status chart here better formatted than gerbra's:

https://www.archlinux.de/?page=MirrorSt … ync;sort=1

...and there're missing mirrors at gerbra's page too(e.g. ftp://mirrors.kernel.org/archlinux/)

Xyne · 2010-01-07 00:26:24

lolilolicon wrote:

I find the mirror status chart here better formatted than gerbra's:
https://www.archlinux.de/?page=MirrorSt … ync;sort=1
...and there're missing mirrors at gerbra's page too(e.g. ftp://mirrors.kernel.org/archlinux/)

Ah, this is where I saw that. I've switched Reflector over to a customize MirrorStatus page on archlinux.de (thanks Pierre).
It's available on my site for now... I'll push it to community once I feel the latest perl-xyne-* packages have stabilized again.

lolilolicon · 2010-01-07 12:26:31

That's great Xyne.

Arch Linux

#1 2009-12-22 21:26:25

Use Arch Mirrorcheck !

#2 2009-12-22 21:45:27

Re: Use Arch Mirrorcheck !

#3 2009-12-22 21:57:31

Re: Use Arch Mirrorcheck !

#4 2009-12-22 22:02:44

Re: Use Arch Mirrorcheck !

#5 2009-12-22 22:03:03

Re: Use Arch Mirrorcheck !

#6 2009-12-22 22:28:08

Re: Use Arch Mirrorcheck !

#7 2009-12-22 22:35:04

Re: Use Arch Mirrorcheck !

#8 2009-12-22 23:27:25

Re: Use Arch Mirrorcheck !

#9 2009-12-23 02:35:33

Re: Use Arch Mirrorcheck !

#10 2010-01-07 00:26:24

Re: Use Arch Mirrorcheck !

#11 2010-01-07 12:26:31

Re: Use Arch Mirrorcheck !

Board footer