Mirrorlist Updating by way of cronie

James_Epp · 2014-09-19 22:34:01

Goal

I have a pfSense router with a squid cache. This caches archlinux packages. I have already wrote some (noobish/poor/mediocre) scripts on my home server to download all the packages I have. So when the script runs, it downloads the packages to the /tmp dir, in the process updating any contents of the router's cache. However, I have many arch systems in this network, and sometimes their mirrorlists differ, so we must keep those consistent.

Therefor, the end goal of this post is to update the mirrorlist periodically based on a set of criteria. The criteria is basically that it pulls a new mirrorlist from https://www.archlinux.org/mirrorlist/?c … _status=on . This is preferably in a script, placed in /etc/cron.hourly to be ran by the cronie daemon.

What I have

On the server I have a script with the executable bit as follows:

/etc/cron.hourly/archlinux_packages

#!/bin/sh
rm -R /tmp/pacman
list=$(</srv/data/isos/linux/archlinux/scripts/archlinux_packages)
mkdir -p /tmp/pacman/{i686,x86_64}/{cachedir,dbpath}
wget "https://projects.archlinux.org/svntogit/packages.git/plain/trunk/pacman.conf.i686?h=packages/pacman" -O "/tmp/pacman/i686.conf"
wget "https://projects.archlinux.org/svntogit/packages.git/plain/trunk/pacman.conf.x86_64?h=packages/pacman" -O "/tmp/pacman/x86_64.conf"
pacman -Syw --noconfirm --config="/tmp/pacman/i686.conf" --cachedir="/tmp/pacman/i686/cachedir" --dbpath="/tmp/pacman/i686/dbpath" --arch=i686 $list
pacman -Syw --noconfirm --config="/tmp/pacman/x86_64.conf" --cachedir="/tmp/pacman/x86_64/cachedir" --dbpath='/tmp/pacman/x86_64/dbpath' --arch=x86_64 $list
rm -R /tmp/pacman

The archlinux_packages file in line two is a plain text, space separated file of all packages to be downloaded. Following is an excerpt:

base base-devel audacity geany gparted k3b

On the server and every client machine I have a script with the executable bit as follows:

/etc/cron.hourly/mirrorlist_update

#!/bin/sh
wget "https://www.archlinux.org/mirrorlist/?country=CA&protocol=http&ip_version=4&use_mirror_status=on" -O /etc/pacman.d/mirrorlist
sed -i 's/^#Server/Server/g' "/etc/pacman.d/mirrorlist"

Simple stuff. I stole line three from someone. I wish I could give credit.

Yes, I have enabled/started the cronie daemon/service on all systems.

Problem

Scripts don't appear to be running. I have added logging to the mirrorlist_update script (>> /tmp/mirrorlist_update) and the file is not created.

Scripts do run as root, right?

Here is some possibly interesting output:

# run-parts --test /etc/cron.hourly
/etc/cron.hourly/0anacron
/etc/cron.hourly/archlinux_packages
/etc/cron.hourly/mirrorlist_update

Also, the contents of all the /etc/pacman.d/mirrorlist files should have today's date (2014-09-19) in them, but instead they have 2014-09-18. This is from yesterday when I had tested them manually.

I'm kinda stuck at this point. I can't prove it, but at one point during testing yesterday I was SSH'd into my server, and the time stamps on the /etc/pacman.d/mirrorlist file were accurate, and I could see the file structure in /tmp being made and populated. Is it a Schrodinger problem where I need to keep an eye on it for it to run?

Any help is greatly appreciated. User error is great.

I probably didn't RTFM enough like usual, right?

progandy · 2014-09-19 22:56:17

I'd suggest you do not use the squid cache. Simply run pacserve on all arch machines to share packages, let your home server download the packages regularily while it runs pacserve constantly. Or maybe even build a full local mirror and use that on all machines.
http://xyne.archlinux.ca/projects/pacserve/

Last edited by progandy (2014-09-19 22:58:44)

Arch Linux

#1 2014-09-19 22:34:01