Arch Linux

bananagranola

Member

From: US

Registered: 2011-08-07

Posts: 88

Website

Hacking pacget script to use axel downloader

I've been using the pacget script as my pacman XferCommand for awhile now, and just for a (very newbie) scripting learning experience, I tried hacking it to use axel instead of aria2c. It allows segmented downloading from multiple mirrors.

#!/bin/bash
# hacked pacget script to use axel
# place XferCommand = /usr/bin/pacaxel %u %o in /etc/pacman.conf

# change the number of connections axel makes
# I use the number of mirrors in /etc/pacman.d/mirrorlist, from reflector script
CONNECTIONS=5

msg() {
    echo ""
    echo -e "   \033[1;34m->\033[1;0m \033[1;1m${1}\033[1;0m" >&2
}

error() {
     echo -e "\033[1;31m==> ERROR:\033[1;0m \033[1;1m$1\033[1;0m" >&2
}


# sanity check: is axel installed?
AXEL=$(which axel 2> /dev/null)
if [ ! -x "$AXEL" ]; then
    error "axel was not found or isn't executable."
    exit 1
fi

# sanity check: are there enough arguments for a url and an output location?
if [ $# -le 1 ]; then
    error "incorrect number of arguments."
    exit 1
fi

filename=$(basename $1)
server=${1%/$filename}
arch=$(grep ^Architecture /etc/pacman.conf | cut -d '=' -f2 | sed 's/ //g')
if [[ $arch = "auto" ]]; then
  arch=$(uname -m)
fi
# Determine which repo is being used
repo=$(awk -F'/' '$(NF-2)~/^(community|core|extra|testing|comunity-testing|multilib)$/{print $(NF-2)}' <<< $server)
[ -z $repo ] && repo="custom"

# for db files, or when using a custom repo (no mirrors),
# use only the URL passed by pacman
# and limit connections to 1
# otherwise, extract the list of servers to download from
url="$1"
if ! [[ $filename = *.db || $repo = "custom" ]]; then
    mirrorlist=$(awk -F' *= *' '$0~"^\\["r"\\]",/Include *= */{l=$2} END{print l}' r=$repo /etc/pacman.conf)
    if [ -n mirrorlist ]; then
        url=$(sed -r '/^Server *= */!d; s/Server *= *//; s/\$repo'"/$repo/"'; s/\$arch'"/$arch/; s/$/\/$filename/" $mirrorlist | head -n $CONNECTIONS | tr '\n' ' ')
    fi
else
    CONNECTIONS=1
fi

# axel creates a file named as if it is complete
# this is a problem when the process is killed with ^C mid-download
# when pacman tries to install the "complete" file from /var/cache/pacman/pkg
# it is incomplete, and pacman thinks it is "corrupt"
# so download it to a temporary location, then move it to /var/cache/pacman/pkg
tmpDir="${TMPDIR:-/tmp}/pacaxeltmp-$USER"
[[ -d "$tmpDir" ]] && sudo rm -rf "$tmpDir" &>/dev/null
mkdir -p "$tmpDir"

# download!
msg "DOWNLOADING $filename FROM $url"
$AXEL -n $CONNECTIONS -v -a -o "$tmpDir/$filename" $url

# for some reason, axel ^C returns exit code 0, not 1 or 2
# so if the partial .st remains, the download was incomplete
# if complete download, move to proper output location
if [ ! -e "$tmpDir/$filename.st" ]; then
    mv "$tmpDir/$filename" "$BASH_ARGV"
fi

# exit with last exit value
exit $?

When I use it, axel (whether I use it in the script or by itself) only names a single mirror:

Initializing download: http://mirror.ece.vt.edu/archlinux/extra/os/x86_64/chromium-18.0.1025.142-1-x86_64.pkg.tar.xz
File size: 26121808 bytes

and not the list of mirrors from /etc/pacman.d/mirrorlist like I was expecting. Is it that axel only displays the first mirror when given a list of mirrors? I googled it, and the only example I found at http://markusthielmann.com/blog/axel_do … le_sources gave of using axel is the way I did: by listing one or more urls to the same file separated by spaces.

The reason I'm worried about #2 is that if I was only downloading from a single mirror, I'd be making 5 different connections to that single mirror.  That seems rather badly mannered, resource-usage wise.

Thanks!

Last edited by bananagranola (2012-04-05 21:33:18)