Mlopt, a script to keep your servers updated

stealthy · 2011-05-13 16:34:53

Hi everybody this is my first post.

I was tired of having to check and see if the mirrors are %100 complete, so I wrote this.

#!/usr/bin/env python2


#Sort by speed
#Then sort by score
#It will never append incomplete servers
#Will exit at the first sign of no internet, (saves CPU)


#sudo crontab -eu root 
#@hourly /usr/bin/mlopt

import urllib2
import json
import os
from time import gmtime, strftime

server_info = {}
curr_servers = []
arch = "$repo/os/x86_64"

os.system("rankmirrors -n 5 /etc/pacman.d/mirrorlist.mlchk > /etc/pacman.d/mirrorlist.fast")

try:
    mirrorlist = open("/etc/pacman.d/mirrorlist.fast").read().split("\n")
except:
    exit(1)

for line in mirrorlist:
    if line.startswith("#") or line == "":
        continue
    try:
        t = line.split("Server = ")[-1]
        curr_servers.append(t.split("/")[0] + "//" + t.split("/")[1] + t.split("/")[2])
    except:
        os.system('notify-send "Mirror-Update" "There is a error in your mirrorlist"')
        exit(1)
        
try:
    page = json.loads(urllib2.urlopen("http://www.archlinux.org/mirrors/status/json/").read())
except:
    exit(1)


for segment in page['urls']:
    for server in curr_servers:
        if segment['url'].startswith(server): #Hax
            if segment['completion_pct'] != 1.0: #Dont even append if its not %100
                continue
            else:
                server_info["%s%s" % (segment['url'], arch)] = segment['score']



    
try:
    new_list = open("/etc/pacman.d/mirrorlist", "w")
    new_list.write("#"+strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())+"\n")
    new_list.write("#Generated by mirror-list\n")

except:
    exit(1)

for key, value in sorted(server_info.iteritems(), key=lambda (k,v): (v,k)): #http://www.saltycrane.com/blog/2007/09/how-to-sort-python-dictionary-by-keys/        
    new_list.write("Server = %s\n" % (key))
    
os.system('notify-send "Mirror-Update" "Mirrorlist updated"')

You will have to make a "/etc/pacman.d/mirrorlist.mlchk" which it will read from and generate a new list called "/etc/pacman.d/mirrorlist.fast".

Just run "sudo cp /etc/pacman.d/mirrorlist /etc/pacman.d/mirrorlist.mlchk"

It will sort them by score and create your new "/etc/pacman.d/mirrorlist". (It will not append the server if it not %100 complete)

I have mine under /usr/bin/mlopt

You will need the rankmirrors bash script and Python2.6.

Last edited by stealthy (2011-05-13 17:18:04)

demian · 2011-05-13 16:54:43

An empty mirrorlist.mlchk will produce an empty mirrorlist (which will leave you unable to get new packages including pacman-mirrorlist).
Also, your script explicitly names the architecture in the mirrorlist file. Afaik it should just be $arch.
But it works and it's very handy, so thanks!

Regards,
mikar

Last edited by demian (2011-05-13 17:32:39)

Arch Linux

#1 2011-05-13 16:34:53

Mlopt, a script to keep your servers updated

#2 2011-05-13 16:54:43

Re: Mlopt, a script to keep your servers updated

Board footer