PKGBUILD parser in Python.

Husio · 2008-07-30 11:37:58

I've wrote PKGBUILD parser in Python and thought someone else might also want to use it. Here's a little example of how it works

# python
Python 2.5.2 (r252:60911, Jul  8 2008, 21:21:10) 
[GCC 4.3.1 20080626 (prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pkgbuild
>>> p = pkgbuild.PKGBUILD('/home/piotrek/aurshell/xfburn-svn/PKGBUILD')
>>> p.parse_pkgname()
['xfburn-svn']
>>> p.parse_makedepends()
['gcc', 'xfce4-dev-tools', 'pkgconfig', 'subversion']
>>> p.parseall()
{'replaces': [], 'pkgname': ['xfburn-svn'], 'maintrainer': [], 'license': ['GPL'], 'source': [], 'makedepends': ['gcc', 'xfce4-dev-tools', 'pkgconfig', 'subversion'], 'provides': ['xfburn'], 'contributor': [" Jakub 'Icar' Jar\xc3\xb3\xc5\xbcek <machine.head.fan@gmail.com>"], 'depends': ['libburn', 'libisofs', 'libxfcegui4', 'thunar', 'gcc', 'xfce4-dev-tools', 'pkgconfig', 'subversion'], 'md5sums': [], 'flags': ['--prefix=/usr', '--sysconfdir=/etc'], 'build': ['build() {\n\ncd $startdir/src\n  msg "Connecting to svn.xfce.org svn server...."\n  svn co $_svntrunk $_svnmod\n\n  msg "SVN checkout done or server timeout"\n  msg "Starting make..."\n\n  cp -R $startdir/src/$_svnmod $startdir/src/$_svnmod-build\n  cd $startdir/src/$_svnmod-build\n\n  ./autogen.sh --prefix=/usr --sysconfdir=/etc\n  make\n  make DESTDIR=$startdir/pkg install\n\n}'], 'install': ['xfburn.install'], 'conflicts': ['xfburn'], 'pkgver': ['5141'], 'arch': ['i686', 'x86_64'], 'pkgdesc': ['Xfburn is a simple CD/DVD burning tool based on libburnia libraries. It can blank CD-RWs, burn and create iso images, as well as burn personal compositions of data to either CD or DVD.'], 'pkgrel': ['1']}
>>> from pprint import pprint
>>> pprint(p.parseall())
{'arch': ['i686', 'x86_64'],
 'build': ['build() {\n\ncd $startdir/src\n  msg "Connecting to svn.xfce.org svn server...."\n  svn co $_svntrunk $_svnmod\n\n  msg "SVN checkout done or server timeout"\n  msg "Starting make..."\n\n  cp -R $startdir/src/$_svnmod $startdir/src/$_svnmod-build\n  cd $startdir/src/$_svnmod-build\n\n  ./autogen.sh --prefix=/usr --sysconfdir=/etc\n  make\n  make DESTDIR=$startdir/pkg install\n\n}'],
 'conflicts': ['xfburn'],
 'contributor': [" Jakub 'Icar' Jar\xc3\xb3\xc5\xbcek <machine.head.fan@gmail.com>"],
 'depends': ['libburn',
             'libisofs',
             'libxfcegui4',
             'thunar',
             'gcc',
             'xfce4-dev-tools',
             'pkgconfig',
             'subversion'],
 'flags': ['--prefix=/usr', '--sysconfdir=/etc'],
 'install': ['xfburn.install'],
 'license': ['GPL'],
 'maintrainer': [],
 'makedepends': ['gcc', 'xfce4-dev-tools', 'pkgconfig', 'subversion'],
 'md5sums': [],
 'pkgdesc': ['Xfburn is a simple CD/DVD burning tool based on libburnia libraries. It can blank CD-RWs, burn and create iso images, as well as burn personal compositions of data to either CD or DVD.'],
 'pkgname': ['xfburn-svn'],
 'pkgrel': ['1'],
 'pkgver': ['5141'],
 'provides': ['xfburn'],
 'replaces': [],
 'source': []}

Daenyth · 2008-07-30 13:24:55

Neat, but I'm kind of wondering what is the point of making a python script to parse a bash script which will be sourced by another bash script is. Also, there's no way you can easily support all bash syntax.

Husio · 2008-07-30 13:30:27

It doesn't support bash syntax. It can only parse _correct_ PKGBUILDs. Nothing more.

I was updating my script to search/download/install from AUR and I needed something to check the dependencies. Then I've wrote the whole parser. No idea if someone will ever use it.

Daenyth · 2008-07-30 14:23:20

A PKGBUILD is a bash script. Period. Anything that works in a bash script works in a PKGBUILD, and is correct.

Husio · 2008-07-30 16:17:30

Ok, that's true (or even sh script). But I don't need bash to parse it. You have all PKGBUILD variables using only Python with standard library. I'm not running os.popen('. PKGBUILD && echo pkgname') to get pkgname variable, instead I'm reading the file as a plain text.

Don't need it? Just don't use it.

Last edited by Husio (2008-07-30 16:18:10)

Xilon · 2008-12-21 04:31:46

This is actually quite nice. When I was developing AUR2 a problem with using plain bash to parse the PKGBUILDs was found - it's insecure and could potentially freeze the machine[1]. This would be a very bad thing to happen to a server running AUR. We were thinking of actually writing a parser for bash to support minimal functionality, since some PKGBUILDs do actually use bash constructs. Something like this would be a nice start, although I don't see any support for variable substitution?

[1] http://archlinux.org/pipermail/aur-dev/ … 00373.html

Husio · 2008-12-21 12:03:23

Basic support for simple bash variables usage isn't that hard to add. But it's working only for $<word> syntax:

 macbook # python pkgbuild.py /home/piotrek/aurshell/python-markdown/PKGBUILD 
{'arch': ['x86_64', 'i686'],
 'build': ['build() {\n  cd ${startdir}/src/markdown-$_pkgver\n  python setup.py install --root=$startdir/pkg\n}'],
 'conflicts': [],
 'contributor': ' Lauri J\xc3\xa4ntti <lauri.jantti@cs.joensuu.fi>',
 'depends': ['python'],
 'flags': [],
 'install': [],
 'license': [],
 'maintrainer': [],
 'makedepends': [],
 'md5sums': ['99b1721d723e8c64381fc023d23b202a'],
 'pkgdesc': 'A Python implementation of John Gruber',
 'pkgname': 'python-markdown',
 'pkgrel': '1',
 'pkgver': '1.7',
 'provides': [],
 'replaces': [],
 'source': ['http://dfn.dl.sourceforge.net/sourceforge/python-markdown/markdown-1.7.zip']}

 macbook # cat /home/piotrek/aurshell/python-markdown/PKGBUILD 
# Contributor: Lauri Jäntti <lauri.jantti@cs.joensuu.fi>
pkgname=python-markdown
pkgver=1.7
_pkgver=1.7
pkgrel=1
pkgdesc="A Python implementation of John Gruber's Markdown. current version implements all Markdown syntax features and fully passes Markdown Test Suite 1.0."
url="http://www.freewisdom.org/projects/python-markdown/"
license="custom"
arch=(x86_64 i686)
depends=('python')
source=(http://dfn.dl.sourceforge.net/sourceforge/python-markdown/markdown-$pkgver.zip)
md5sums=('99b1721d723e8c64381fc023d23b202a')

build() {
  cd ${startdir}/src/markdown-$_pkgver
  python setup.py install --root=$startdir/pkg
}

Allan · 2008-12-21 12:20:04

FYI, namcap "parses" PKGBUILDs using bash into a form that is the read by python. Maybe you will find something useful there.

Xilon · 2008-12-22 12:24:16

Allan wrote:

FYI, namcap "parses" PKGBUILDs using bash into a form that is the read by python. Maybe you will find something useful there.

As mentioned in my previous post, doing so is still harmful to the host OS. AUR2 uses the same method as namcap (slightly modified). It works quite well but the problem is mostly with making the host system hang - not good for a server

Arch Linux

#1 2008-07-30 11:37:58

PKGBUILD parser in Python.

#2 2008-07-30 13:24:55

Re: PKGBUILD parser in Python.

#3 2008-07-30 13:30:27

Re: PKGBUILD parser in Python.

#4 2008-07-30 14:23:20

Re: PKGBUILD parser in Python.

#5 2008-07-30 16:17:30

Re: PKGBUILD parser in Python.

#6 2008-12-21 04:31:46

Re: PKGBUILD parser in Python.

#7 2008-12-21 12:03:23

Re: PKGBUILD parser in Python.

#8 2008-12-21 12:20:04

Re: PKGBUILD parser in Python.

#9 2008-12-22 12:24:16

Re: PKGBUILD parser in Python.

Board footer