You are not logged in.

#1 2005-06-04 20:37:04

Dusty
Schwag Merchant
From: Medicine Hat, Alberta, Canada
Registered: 2004-01-18
Posts: 5,986
Website

Script to grab packages from AUR

The following python script can be used to semi-automate the install process for packages in AUR. It could use a little work, especially the part that grabs the version.

To use it, first check in AUR to ensure the package exists, and that the PKGBUILD is not malicious or something you think will screw your system. AUR packages in unsupported CANNOT BE TRUSTED indiscriminantly.

Then run:

aurbuild packagename

If you are root, this will download, build, and install the package. If you are a user, it will only be downloaded and built.  The script currently doesn't remove the build files and directory, which are created in the current directory.

I've only tested this with one package, so its probably buggy.

#!/usr/bin/env python
import sys, os, urllib, tarfile, re

def getsubstring(url, reg, prefix, suffix):
    page=urllib.urlopen(url)
    for line in page.readlines():
            if  re.search(reg, line):
                break
    p1=line.find(prefix) + len(prefix)
    p2=line.find(suffix, p1)
    return line[p1:p2]

if len(sys.argv) != 2:
    print "Usage: aurbuild.py <package>ntWhere <package> is the name of a valid package in AUR"
    sys.exit(1)
name=sys.argv[1]

page="http://aur.archlinux.org/packages.php?K=" + name
reg="'/packages.php.*?><span class='black'>" + name
ID=getsubstring(page, reg, "ID=", "&")

page="http://aur.archlinux.org/packages.php?do_Details=1&ID=" + ID
reg="<a href='.*?>Tarball</a> ::"
tar="http://aur.archlinux.org" + getsubstring(page, reg, "<a href='", "'>")
        
urllib.urlretrieve(tar, name + ".tar.gz")
tar=tarfile.open(name + ".tar.gz", "r:gz")
for member in tar.getmembers():
    tar.extract(member)
    
os.chdir(name)
os.spawnvp(os.P_WAIT, "makepkg", ["makepkg"])

files = os.listdir(".")
for name in files:
    if name.find(".pkg.tar.gz") >-1:
        pkgname=name
        break

if os.getuid() == 0:
    os.spawnvp(os.P_WAIT, "pacman",  ["pacman", "-U",  pkgname])
else:
    print "You must be root to install the package. cd into the " + name
    print " directory and run pacman -U  " + pkgname + "."

Offline

#2 2005-06-04 20:48:08

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: Script to grab packages from AUR

Browsing through http://aur.archlinux.org/packages/ I'd say there is no version info anywhere at all.

Simply downloading all files in

"http://aur.archlinux.org/packages/" + name + "/" + name

should be enough. The tar.gz can contain the binary too, and you don't want that.

(Python seems interesting, maybe I should learn it...)

Offline

#3 2005-06-04 20:50:22

Dusty
Schwag Merchant
From: Medicine Hat, Alberta, Canada
Registered: 2004-01-18
Posts: 5,986
Website

Re: Script to grab packages from AUR

I don't think it can contain the binary, I believe AUR rebuilds the tar.gz files after they are uploaded, only including files that are in the PKGBUILD. I may be wrong about that.

I did find version info, as shown in the script, I just thought there might be a better way. wink It takes 7 lines, which in Python is a very messy hack!

Dusty

Offline

#4 2005-06-04 20:55:12

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: Script to grab packages from AUR

That's strange, the only package I happened to have installed from AUR is the only one I can found which has a precompiled binary included. Which is also placed in the name/name dir, so skip the idea above. Perhaps downloading everything not ending in pkg.tar.gz works.

Oh now I get why you need the version info. Isn't it better to extract it from the downloaded PKGBUILD instead? Scanning for pkgver and pkgrel should do it.


Edit: Argh, brwosing through that site it looksl ike there's no standard at all, some have only a PKGBUILD in the topdir, others have a subdir and and a tar.gz, but some with and others without version info, what's going on?

Offline

#5 2005-06-04 20:59:42

Dusty
Schwag Merchant
From: Medicine Hat, Alberta, Canada
Registered: 2004-01-18
Posts: 5,986
Website

Re: Script to grab packages from AUR

I need the version to know what the url for the tar.gz is.

Note, this is in the new scripts section of the wiki now:
http://wiki2.archlinux.org/index.php/aurbuild

Feel free to improve.

Dusty

Offline

#6 2005-06-04 21:17:38

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: Script to grab packages from AUR

I thought you needed it for something else (to know which version you just made, but that info comes from the pkgbuild of course).

But as I said before, most packages don't have a file with a version number in their name, but some have. E.g. look at the following urls:

http://aur.archlinux.org/packages/ppracer/
With version, and only build files in the ppraces subdir.

http://aur.archlinux.org/packages/pptpclient/
No version info, like most packages.

http://aur.archlinux.org/packages/xerces-c/
http://aur.archlinux.org/packages/acidrip/
Version info, and the binary included in the tar.gz and the subdir.

So it looks like there was a change in AUR at some point but existing packages weren't converted or something.

Offline

#7 2005-06-04 22:15:50

xerxes2
Member
From: Malmoe, Sweden
Registered: 2004-04-23
Posts: 1,249
Website

Re: Script to grab packages from AUR

this is a wonderful idea, i've been thinking of writing it myself but now i don't have to,

maybe you smart guys(dusty, i3839 among others) could submit patches or suggestions to the maintainers of aur(or join the core team),


arch + gentoo + initng + python = enlisy

Offline

#8 2005-06-04 22:33:05

cactus
Taco Eater
From: t͈̫̹ͨa͖͕͎̱͈ͨ͆ć̥̖̝o̫̫̼s͈̭̱̞͍̃!̰
Registered: 2004-05-25
Posts: 4,615
Website

Re: Script to grab packages from AUR

Well, assuming the user has found the package on the aur and verified it..he could just provide the pkgid of the package he wants to build.

for example http://aur.archlinux.org/packages.php?d … s=1&ID=287
that is the pptpclient url. With the ID as 287.
Have the python script harvest that website, looking for the "Files" link, and fetch all the files from there. or just the "tarball link" and download and extract and enter the resulting dir.

That way you are less dependent upon how the user uploads the tarball (name) compared to the name of the package.

Would require a tiny bit of regexp to ferret out the links from the html for the package page, but it should be relatively trivial.

aurbuild 287

just the command and the pkgid.


"Be conservative in what you send; be liberal in what you accept." -- Postel's Law
"tacos" -- Cactus' Law
"t̥͍͎̪̪͗a̴̻̩͈͚ͨc̠o̩̙͈ͫͅs͙͎̙͊ ͔͇̫̜t͎̳̀a̜̞̗ͩc̗͍͚o̲̯̿s̖̣̤̙͌ ̖̜̈ț̰̫͓ạ̪͖̳c̲͎͕̰̯̃̈o͉ͅs̪ͪ ̜̻̖̜͕" -- -̖͚̫̙̓-̺̠͇ͤ̃ ̜̪̜ͯZ͔̗̭̞ͪA̝͈̙͖̩L͉̠̺͓G̙̞̦͖O̳̗͍

Offline

#9 2005-06-04 22:37:37

i3839
Member
Registered: 2004-02-04
Posts: 1,185

Re: Script to grab packages from AUR

After some fishing I found xentac.net/svn/aur/. I may add some features like voting for package safety, if no one minds, but not this month, as I'll be busy then. Where or how does the AUR development and discussion happen? Any mailinglist or something?

Offline

#10 2005-06-04 22:53:51

phrakture
Arch Overlord
From: behind you
Registered: 2003-10-29
Posts: 7,879
Website

Re: Script to grab packages from AUR

holyomgwtf
dusty has had 2 great ideas today

haven't tested it, but I plan on it 8)

Offline

#11 2005-06-05 17:57:05

Dusty
Schwag Merchant
From: Medicine Hat, Alberta, Canada
Registered: 2004-01-18
Posts: 5,986
Website

Re: Script to grab packages from AUR

Ok, I edited to grab tarballs via links. It still uses name rather than ID (name is cooler, ID is grabbed internally). There's still one problem, if tarballs are not properly formatted (the PKGBUILD isn't inside a directory by the name of the package), it will choke. I've submitted a bug report requesting more control over tarball structure:

http://bugs.archlinux.org/index.php?do=details&id=2803

Dusty

Offline

#12 2005-06-11 00:04:46

T-Dawg
Forum Fellow
From: Wilmington, NC
Registered: 2005-01-29
Posts: 2,734

Re: Script to grab packages from AUR

Nice Dusty.  big_smile
How about adding -b to makepkg to build the dependencies? I had a couple build attempts crap out on me because they were missing.

Offline

#13 2005-06-11 00:26:12

Dusty
Schwag Merchant
From: Medicine Hat, Alberta, Canada
Registered: 2004-01-18
Posts: 5,986
Website

Re: Script to grab packages from AUR

Penguin wrote:

How about adding -b to makepkg to build the dependencies? I had a couple build attempts crap out on me because they were missing.

IMHO this is not a good idea because building missing deps from source without checking the integrity of the PKGBUILD is not a good idea. Having it automatically pacman in dependencies that are in a verified repo wouldn't be a bad idea, except I think it would make the script a lot more complex.

Of course, feel free to edit it. :-)

Dusty

Offline

#14 2005-06-11 00:33:25

cactus
Taco Eater
From: t͈̫̹ͨa͖͕͎̱͈ͨ͆ć̥̖̝o̫̫̼s͈̭̱̞͍̃!̰
Registered: 2004-05-25
Posts: 4,615
Website

Re: Script to grab packages from AUR

dusty, xerxes has the python libpypac too. You might look at using some of that functionaly.
I believe he has a replacement, in pure python, to work with pkgbuild files as well.


"Be conservative in what you send; be liberal in what you accept." -- Postel's Law
"tacos" -- Cactus' Law
"t̥͍͎̪̪͗a̴̻̩͈͚ͨc̠o̩̙͈ͫͅs͙͎̙͊ ͔͇̫̜t͎̳̀a̜̞̗ͩc̗͍͚o̲̯̿s̖̣̤̙͌ ̖̜̈ț̰̫͓ạ̪͖̳c̲͎͕̰̯̃̈o͉ͅs̪ͪ ̜̻̖̜͕" -- -̖͚̫̙̓-̺̠͇ͤ̃ ̜̪̜ͯZ͔̗̭̞ͪA̝͈̙͖̩L͉̠̺͓G̙̞̦͖O̳̗͍

Offline

#15 2005-06-11 00:51:28

T-Dawg
Forum Fellow
From: Wilmington, NC
Registered: 2005-01-29
Posts: 2,734

Re: Script to grab packages from AUR

ah, ok. I didn't realize makepkg was sourcing PKGBUILDS only in aur.....
I was thinking locally in abs (arch PKGBUILDS) as it normally does.

Offline

#16 2005-06-13 09:07:45

miracle
Member
Registered: 2005-02-23
Posts: 40

Re: Script to grab packages from AUR

great !
And can write a script to search the  PKGBUILDs in AUR  lol

Offline

Board footer

Powered by FluxBB