Script to grab packages from AUR

Dusty · 2005-06-04 20:37:04

The following python script can be used to semi-automate the install process for packages in AUR. It could use a little work, especially the part that grabs the version.

To use it, first check in AUR to ensure the package exists, and that the PKGBUILD is not malicious or something you think will screw your system. AUR packages in unsupported CANNOT BE TRUSTED indiscriminantly.

Then run:

aurbuild packagename

If you are root, this will download, build, and install the package. If you are a user, it will only be downloaded and built. The script currently doesn't remove the build files and directory, which are created in the current directory.

I've only tested this with one package, so its probably buggy.

#!/usr/bin/env python
import sys, os, urllib, tarfile, re

def getsubstring(url, reg, prefix, suffix):
    page=urllib.urlopen(url)
    for line in page.readlines():
            if  re.search(reg, line):
                break
    p1=line.find(prefix) + len(prefix)
    p2=line.find(suffix, p1)
    return line[p1:p2]

if len(sys.argv) != 2:
    print "Usage: aurbuild.py <package>ntWhere <package> is the name of a valid package in AUR"
    sys.exit(1)
name=sys.argv[1]

page="http://aur.archlinux.org/packages.php?K=" + name
reg="'/packages.php.*?><span class='black'>" + name
ID=getsubstring(page, reg, "ID=", "&")

page="http://aur.archlinux.org/packages.php?do_Details=1&ID=" + ID
reg="<a href='.*?>Tarball</a> ::"
tar="http://aur.archlinux.org" + getsubstring(page, reg, "<a href='", "'>")
        
urllib.urlretrieve(tar, name + ".tar.gz")
tar=tarfile.open(name + ".tar.gz", "r:gz")
for member in tar.getmembers():
    tar.extract(member)
    
os.chdir(name)
os.spawnvp(os.P_WAIT, "makepkg", ["makepkg"])

files = os.listdir(".")
for name in files:
    if name.find(".pkg.tar.gz") >-1:
        pkgname=name
        break

if os.getuid() == 0:
    os.spawnvp(os.P_WAIT, "pacman",  ["pacman", "-U",  pkgname])
else:
    print "You must be root to install the package. cd into the " + name
    print " directory and run pacman -U  " + pkgname + "."

i3839 · 2005-06-04 20:48:08

Browsing through http://aur.archlinux.org/packages/ I'd say there is no version info anywhere at all.

Simply downloading all files in

"http://aur.archlinux.org/packages/" + name + "/" + name

should be enough. The tar.gz can contain the binary too, and you don't want that.

(Python seems interesting, maybe I should learn it...)

Dusty · 2005-06-04 20:50:22

I don't think it can contain the binary, I believe AUR rebuilds the tar.gz files after they are uploaded, only including files that are in the PKGBUILD. I may be wrong about that.

I did find version info, as shown in the script, I just thought there might be a better way. It takes 7 lines, which in Python is a very messy hack!

Dusty

i3839 · 2005-06-04 20:55:12

That's strange, the only package I happened to have installed from AUR is the only one I can found which has a precompiled binary included. Which is also placed in the name/name dir, so skip the idea above. Perhaps downloading everything not ending in pkg.tar.gz works.

Oh now I get why you need the version info. Isn't it better to extract it from the downloaded PKGBUILD instead? Scanning for pkgver and pkgrel should do it.

Edit: Argh, brwosing through that site it looksl ike there's no standard at all, some have only a PKGBUILD in the topdir, others have a subdir and and a tar.gz, but some with and others without version info, what's going on?

Dusty · 2005-06-04 20:59:42

I need the version to know what the url for the tar.gz is.

Note, this is in the new scripts section of the wiki now:
http://wiki2.archlinux.org/index.php/aurbuild

Feel free to improve.

Dusty

i3839 · 2005-06-04 21:17:38

I thought you needed it for something else (to know which version you just made, but that info comes from the pkgbuild of course).

But as I said before, most packages don't have a file with a version number in their name, but some have. E.g. look at the following urls:

http://aur.archlinux.org/packages/ppracer/
With version, and only build files in the ppraces subdir.

http://aur.archlinux.org/packages/pptpclient/
No version info, like most packages.

http://aur.archlinux.org/packages/xerces-c/
http://aur.archlinux.org/packages/acidrip/
Version info, and the binary included in the tar.gz and the subdir.

So it looks like there was a change in AUR at some point but existing packages weren't converted or something.

xerxes2 · 2005-06-04 22:15:50

this is a wonderful idea, i've been thinking of writing it myself but now i don't have to,

maybe you smart guys(dusty, i3839 among others) could submit patches or suggestions to the maintainers of aur(or join the core team),

cactus · 2005-06-04 22:33:05

Well, assuming the user has found the package on the aur and verified it..he could just provide the pkgid of the package he wants to build.

for example http://aur.archlinux.org/packages.php?d … s=1&ID=287
that is the pptpclient url. With the ID as 287.
Have the python script harvest that website, looking for the "Files" link, and fetch all the files from there. or just the "tarball link" and download and extract and enter the resulting dir.

That way you are less dependent upon how the user uploads the tarball (name) compared to the name of the package.

Would require a tiny bit of regexp to ferret out the links from the html for the package page, but it should be relatively trivial.

aurbuild 287

just the command and the pkgid.

i3839 · 2005-06-04 22:37:37

After some fishing I found xentac.net/svn/aur/. I may add some features like voting for package safety, if no one minds, but not this month, as I'll be busy then. Where or how does the AUR development and discussion happen? Any mailinglist or something?

phrakture · 2005-06-04 22:53:51

holyomgwtf
dusty has had 2 great ideas today

haven't tested it, but I plan on it 8)

Dusty · 2005-06-05 17:57:05

Ok, I edited to grab tarballs via links. It still uses name rather than ID (name is cooler, ID is grabbed internally). There's still one problem, if tarballs are not properly formatted (the PKGBUILD isn't inside a directory by the name of the package), it will choke. I've submitted a bug report requesting more control over tarball structure:

http://bugs.archlinux.org/index.php?do=details&id=2803

Dusty

T-Dawg · 2005-06-11 00:04:46

Nice Dusty.
How about adding -b to makepkg to build the dependencies? I had a couple build attempts crap out on me because they were missing.

Dusty · 2005-06-11 00:26:12

Penguin wrote:

How about adding -b to makepkg to build the dependencies? I had a couple build attempts crap out on me because they were missing.

IMHO this is not a good idea because building missing deps from source without checking the integrity of the PKGBUILD is not a good idea. Having it automatically pacman in dependencies that are in a verified repo wouldn't be a bad idea, except I think it would make the script a lot more complex.

Of course, feel free to edit it. :-)

Dusty

cactus · 2005-06-11 00:33:25

dusty, xerxes has the python libpypac too. You might look at using some of that functionaly.
I believe he has a replacement, in pure python, to work with pkgbuild files as well.

T-Dawg · 2005-06-11 00:51:28

ah, ok. I didn't realize makepkg was sourcing PKGBUILDS only in aur.....
I was thinking locally in abs (arch PKGBUILDS) as it normally does.

miracle · 2005-06-13 09:07:45

great !
And can write a script to search the PKGBUILDs in AUR

Arch Linux

#1 2005-06-04 20:37:04

Script to grab packages from AUR

#2 2005-06-04 20:48:08

Re: Script to grab packages from AUR

#3 2005-06-04 20:50:22

Re: Script to grab packages from AUR

#4 2005-06-04 20:55:12

Re: Script to grab packages from AUR

#5 2005-06-04 20:59:42

Re: Script to grab packages from AUR

#6 2005-06-04 21:17:38

Re: Script to grab packages from AUR

#7 2005-06-04 22:15:50

Re: Script to grab packages from AUR

#8 2005-06-04 22:33:05

Re: Script to grab packages from AUR

#9 2005-06-04 22:37:37

Re: Script to grab packages from AUR

#10 2005-06-04 22:53:51

Re: Script to grab packages from AUR

#11 2005-06-05 17:57:05

Re: Script to grab packages from AUR

#12 2005-06-11 00:04:46

Re: Script to grab packages from AUR

#13 2005-06-11 00:26:12

Re: Script to grab packages from AUR

#14 2005-06-11 00:33:25

Re: Script to grab packages from AUR

#15 2005-06-11 00:51:28

Re: Script to grab packages from AUR

#16 2005-06-13 09:07:45

Re: Script to grab packages from AUR

Board footer