You are not logged in.

#1 2010-01-10 03:27:12

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

A method for safe PKGBUILD metadata extraction... help me improve it.

I've already shared my thoughts on using PKGBUILDs written in Bash and the problems such an approach entails. If you are unaware of the issues with current PKGBUILDs, you can probably just click back right now as I doubt this will be very interesting for you.


For everyone else, it's story time...

Background
A few days ago, on a computer far, far away, I was writing Bauerbilland hit a wall when I came across the kernel26 PKGBUILD, which is a split package. Up until then, I had been using the basic "source the PKGBUILD and echo the variables" approach (following user inspection, of course). This seemed reasonable because the PKGBUILD would have to be sourced when building the package and the user would be required to inspect it anyway.

The problem with split packages is that they contain variables within package_<pkgname> functions and these variables are not interpretted when the PKGBUILD is sourced. When I realized this, I actually thought that all my work on Bauerbill might have been for absolutely nothing, as it has to be able to extract the depends and makedepends arrays to be able to build packages. After a mental rant about how PKGBUILDs and makepkg is going in the wrong direction with split PKGBUILDs, I gave up and went to bed, pissed off at having wasted time on what seemed to be a futile effort.

The next day though, a few ideas popped into my head and after testing a few things, I realized the following:
* code inside a function is not executed yet it is still parsed when the file is sourced
* parsed functions are included in the output of "set"
* the output of "set"is canonical, i.e. completely regular and easily parsed
* the actual subset of Bash syntax required to interpret variable interpolation is actually quite limited

I thus came up with this method to safely parse PKGBUILDs for metadata:

The method
* Check the PKGBUILD with "/bin/bash -n" to make sure that the syntax is valid.
* Wrap the PKGBUILD in a function, e.g.

pkgbuild() {
<PKGBUILD>
}

This PKGBUILD cannot escape the function with extra trailing braces ("}") because of the first step, which has already validated the syntax.
* Capture the output of "set", which contains all environment variables and the entire code of the PKGBUILD, wrapped in the "pkgbuild" function. This output is in a standard format, e.g. variable assignments are all on single lines, if/then/else blocks follow a specific format.
* Parse the "set" output line by line, keeping track of all variables and their scope. In particular, walk the "pkgbuild" function line by line, updating and interpolating variables as they are encountered.
* Use the parsed data for your target application (e.g. determining makedepends and depends in Bauerbill, providing JSON data for the AUR, etc).

The advantages of this method
No code is actually executed. You can use it to blindly parse PKGBUILDs from any source (unless I've missed something, but I think I'm right).
You only have to deal with a limited subset of Bash syntax.
You can implement this without dependencies on anything but Bash and the language that the parse is written in, e.g. you don't need something like Yacc/Lex.


The implementation
Obviously the parsing step is the most difficult as that is where Bash syntax is required. My current parser is contained in the latest perl-xyne-arch package and is (clearly) written in Perl (the module is "Xyne::Arch::PKGBUILD"). It currently handles interpolation of simple variables, i.e. "$foo" and "${foo}" (taking context into consideration) and all types of string substitution, i.e. ${foo/bar/baz}, ${foo//bar/baz}, ${foo/#bar/baz} and ${foo/%bar/baz} (including cases with no specified replacement).

I intend to add other cases as they pop up, including variables set using conditions (e.g. [ "$ARCH" == "x86_64" ] && depends=...), and possible allow thecommand output from a whitelist of harmless commands.

This is where the "help me improve it" part comes in. Here's a simple script based using my module to parse a PKGBUILD and spit out a representation of the metadata in a derivative of JSON (grab the latest version of perl-xyne-arch from the repo on my site to run this):

#!/usr/bin/perl
use strict;
use warnings;

use Xyne::Arch::PKGBUILD qw/parse_pkgbuild/;

use Xyne::Common::Functions qw/str/;
#use Xyne::Common::JSON qw/obj_to_json/;

foreach my $pkgbuild (@ARGV)
{
  foreach my $o (&parse_pkgbuild($pkgbuild))
  {
    next if not $o;
    print &str($o,1);
    #print &obj_to_json($o,1);
    print "\n\n";
  }
}
#print &obj_to_json(&parse_pkgbuild(@ARGV),1);

You can move around the comments to make it spit out actual JSON. Remove the ",1" to "unprettify" the output.

Here's the output from parsing the current kernel26 PKGBUILD, which is a split package:

{
  "INSTALL_MOD_PATH":"${pkgdir}",
  "KARCH":"x86",
  "_basekernel":"2.6.32",
  "_kernelname":"${pkgname#kernel26}",
  "_kernver":"$(make kernelrelease)",
  "_patchname":"patch-2.6.32.3-1-ARCH",
  "arch":[
    "i686",
    "x86_64"
  ],
  "backup":[
    "etc/mkinitcpio.d/kernel26.preset"
  ],
  "depends":[
    "coreutils",
    "kernel26-firmware>=2.6.32",
    "module-init-tools",
    "mkinitcpio>=0.5.20"
  ],
  "groups":[
    "base"
  ],
  "install":"kernel26.install",
  "license":[
    "GPL2"
  ],
  "md5sums":[
    "260551284ac224c3a43c4adac7df4879",
    "7fba47d9b1d87de87a612a265fbfb4b1",
    "c114c5d89622a93165bb948d62d966b5",
    "5c91374d56f115ba4324978d5b002711",
    "25584700a0a679542929c4bed31433b6"
  ],
  "optdepends":[
    "crda: to set the correct wireless channels of your country"
  ],
  "pkgbase":"kernel26",
  "pkgdesc":"The Linux Kernel and modules",
  "pkgname":"kernel26",
  "pkgrel":"1",
  "pkgver":"2.6.32.3",
  "replaces":[
    "kernel24",
    "kernel24-scsi",
    "kernel26-scsi",
    "alsa-driver",
    "ieee80211",
    "hostap-driver26",
    "pwc",
    "nforce",
    "squashfs",
    "unionfs",
    "ivtv",
    "zd1211",
    "kvm-modules",
    "iwlwifi",
    "rt2x00-cvs",
    "gspcav1",
    "atl2",
    "wlan-ng26",
    "rt2500"
  ],
  "source":[
    "ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.tar.bz2",
    "ftp://ftp.archlinux.org/other/kernel26/patch-2.6.32.3-1-ARCH.bz2",
    "config",
    "config.x86_64",
    "kernel26.preset"
  ],
  "url":"http://www.kernel.org"
}

{
  "INSTALL_MOD_PATH":"${pkgdir}",
  "_basekernel":"2.6.32",
  "_kernelname":"${pkgname#kernel26}",
  "_patchname":"patch-2.6.32.3-1-ARCH",
  "arch":[
    "i686",
    "x86_64"
  ],
  "groups":[
    "base"
  ],
  "license":[
    "GPL2"
  ],
  "md5sums":[
    "260551284ac224c3a43c4adac7df4879",
    "7fba47d9b1d87de87a612a265fbfb4b1",
    "c114c5d89622a93165bb948d62d966b5",
    "5c91374d56f115ba4324978d5b002711",
    "25584700a0a679542929c4bed31433b6"
  ],
  "pkgbase":"kernel26",
  "pkgdesc":"The included firmware files of the Linux Kernel",
  "pkgname":"kernel26-firmware",
  "pkgrel":"1",
  "pkgver":"2.6.32.3",
  "source":[
    "ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.tar.bz2",
    "ftp://ftp.archlinux.org/other/kernel26/patch-2.6.32.3-1-ARCH.bz2",
    "config",
    "config.x86_64",
    "kernel26.preset"
  ],
  "url":"http://www.kernel.org"
}

{
  "_basekernel":"2.6.32",
  "_kernelname":"${pkgname#kernel26}",
  "_patchname":"patch-2.6.32.3-1-ARCH",
  "arch":[
    "i686",
    "x86_64"
  ],
  "license":[
    "GPL2"
  ],
  "md5sums":[
    "260551284ac224c3a43c4adac7df4879",
    "7fba47d9b1d87de87a612a265fbfb4b1",
    "c114c5d89622a93165bb948d62d966b5",
    "5c91374d56f115ba4324978d5b002711",
    "25584700a0a679542929c4bed31433b6"
  ],
  "pkgbase":"kernel26",
  "pkgdesc":"Header files and scripts for building modules for kernel26",
  "pkgname":"kernel26-headers",
  "pkgrel":"1",
  "pkgver":"2.6.32.3",
  "source":[
    "ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.tar.bz2",
    "ftp://ftp.archlinux.org/other/kernel26/patch-2.6.32.3-1-ARCH.bz2",
    "config",
    "config.x86_64",
    "kernel26.preset"
  ],
  "url":"http://www.kernel.org"
}

As you can see, it manages to extract everything including the variables nested in the packaging functions. Note that "$pkgdir" is not set by the PKGBUILD, so it should remain uninterpolated.

What I ask is that you test this on different PKGBUILDs and report cases of "normal" syntax which somehow slip past the parser. By "normal", I mean syntax that can reasonable be expected in a PKGBUILD, not completely obfuscated shite from some "clever" packager who's trying to show off his leet Bash-fu.

Of course, if you want to contribute code as well, I will gladly consider it.

Last edited by Xyne (2010-01-10 03:49:07)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#2 2010-01-10 03:37:58

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,385
Website

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

Xyne wrote:

What I ask is that you test this on different PKGBUILDs and report cases of "normal" syntax which somehow slip past the parser. By "normal", I mean syntax that can reasonable be expected in a PKGBUILD, not completely obfuscated shite from some "clever" packager who's trying to show off his leet Bash-fu.

Is the sunbird-i18n PKGBUILD in [community] to "leet Bash-fu"?  I am lazy so I have not actually tested it but...

_languages=ca,cs,da,de,es-AR,es-ES,eu,fr,ga-IE,hu,is,it,ja,ka,ko,lt,nb-NO,nl,nn-NO,pl,pt-BR,pt-PT,ro,ru,sk,sl,sv-SE,uk,zh-CN,zh-TW
eval source=(ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/${pkgver}/langpacks/{${_languages}}.xpi)

"Sensible" tricks like this is why I am a fan of bash PKGBUILDs tongue

Offline

#3 2010-01-10 03:55:34

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

That's a good example, actually. I agree that it is a sensible trick. The difficulty is that the parser has to catch the "eval" step, which adds further bash syntax to the mix.

Here's the current output:

{
  "_languages":"ca,cs,da,de,es-AR,es-ES,eu,fr,ga-IE,hu,is,it,ja,ka,ko,lt,nb-NO,nl,nn-NO,pl,pt-BR,pt-PT,ro,ru,sk,sl,sv-SE,uk,zh-CN,zh-TW",
  "arch":[
    "i686",
    "x86_64"
  ],
  "depends":[
    "sunbird=0.9"
  ],
  "license":[
    "MPL"
  ],
  "makedepends":[
    "unzip"
  ],
  "md5sums":[
    "d3de5b43b25f0d8b68e515dbb06052e7",
    "25152b290f87bfc02445f8297a403644",
    "aa0e4022bd538cd033d94c0571a59b2c",
    "3f549ba15e76abbd153e0b0342f5ebdb",
    "7b8163f0a458ae980fa9d37454289d73",
    "74a9be1292d50c645ff33e1cc2c378ff",
    "7e54e3a2bcf842d14803bda2862b4db2",
    "1b4d394049c17175e27d502d657b5938",
    "21993a02c28c4ee2b9cb8eab483d5198",
    "78c92149bb63a35be40f4dffbd639638",
    "161b183b3723454823d1b5d8d38e46e8",
    "5500c10aa729ccc6002b6a541f7dc9f6",
    "b0207b572fa9775f2088743e1f7dc0aa",
    "9e85fd0d3fa4cd030b255d4c64b88d80",
    "dbbbdc6cf6db7378e05d2ab50f403ea2",
    "94359e18b7271b0dd6b37ded878a4659",
    "15b3a56ca4107da58c2cfc53d88051f5",
    "3ae36d5a1cd2cea41f28bf40bec99f56",
    "d5ee25458f7517f6a801d9f96f12392b",
    "d923f4a955a241c0863ea28a4b794d59",
    "a4d64b260795c9f16bca3a2cc4739990",
    "dda6b745526e85126012e374476c57cc",
    "d871b87338082e8b9ecc5c96ae508d11",
    "961e62525a09271add4d0d9cfe071402",
    "3d891899835d73506ce9c793acccac2b",
    "f05e4a3e93394693be844c90e2ca0f7c",
    "27d1175daf451e4ecaabe648fe76640b",
    "a0a35823fdf3a1fa3f217eebe447b902",
    "ef85d04ebcb24b864fc0357568efbfe8",
    "516dae7254589e17c0fc3bba48ec68c4"
  ],
  "pkgdesc":"Language packs for Sunbird",
  "pkgname":"sunbird-i18n",
  "pkgrel":"1",
  "pkgver":"0.9",
  "source":[
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/{ca,cs,da,de,es-AR,es-ES,eu,fr,ga-IE,hu,is,it,ja,ka,ko,lt,nb-NO,nl,nn-NO,pl,pt-BR,pt-PT,ro,ru,sk,sl,sv-SE,uk,zh-CN,zh-TW}.xpi"
  ],
  "url":"http://www.mozilla.com/"
}

I'll look into catching the eval statement and checking it for simple interpolations. In this case it should be trivial, but it could easily be make difficult.


*edit*

I've added a "safe_eval" function which currently only interpolate multiple arguments specified as "foo{this,that}bar". including nested cases. The output is now:

{
  "_languages":"ca,cs,da,de,es-AR,es-ES,eu,fr,ga-IE,hu,is,it,ja,ka,ko,lt,nb-NO,nl,nn-NO,pl,pt-BR,pt-PT,ro,ru,sk,sl,sv-SE,uk,zh-CN,zh-TW",
  "arch":[
    "i686",
    "x86_64"
  ],
  "depends":[
    "sunbird=0.9"
  ],
  "license":[
    "MPL"
  ],
  "makedepends":[
    "unzip"
  ],
  "md5sums":[
    "d3de5b43b25f0d8b68e515dbb06052e7",
    "25152b290f87bfc02445f8297a403644",
    "aa0e4022bd538cd033d94c0571a59b2c",
    "3f549ba15e76abbd153e0b0342f5ebdb",
    "7b8163f0a458ae980fa9d37454289d73",
    "74a9be1292d50c645ff33e1cc2c378ff",
    "7e54e3a2bcf842d14803bda2862b4db2",
    "1b4d394049c17175e27d502d657b5938",
    "21993a02c28c4ee2b9cb8eab483d5198",
    "78c92149bb63a35be40f4dffbd639638",
    "161b183b3723454823d1b5d8d38e46e8",
    "5500c10aa729ccc6002b6a541f7dc9f6",
    "b0207b572fa9775f2088743e1f7dc0aa",
    "9e85fd0d3fa4cd030b255d4c64b88d80",
    "dbbbdc6cf6db7378e05d2ab50f403ea2",
    "94359e18b7271b0dd6b37ded878a4659",
    "15b3a56ca4107da58c2cfc53d88051f5",
    "3ae36d5a1cd2cea41f28bf40bec99f56",
    "d5ee25458f7517f6a801d9f96f12392b",
    "d923f4a955a241c0863ea28a4b794d59",
    "a4d64b260795c9f16bca3a2cc4739990",
    "dda6b745526e85126012e374476c57cc",
    "d871b87338082e8b9ecc5c96ae508d11",
    "961e62525a09271add4d0d9cfe071402",
    "3d891899835d73506ce9c793acccac2b",
    "f05e4a3e93394693be844c90e2ca0f7c",
    "27d1175daf451e4ecaabe648fe76640b",
    "a0a35823fdf3a1fa3f217eebe447b902",
    "ef85d04ebcb24b864fc0357568efbfe8",
    "516dae7254589e17c0fc3bba48ec68c4"
  ],
  "pkgdesc":"Language packs for Sunbird",
  "pkgname":"sunbird-i18n",
  "pkgrel":"1",
  "pkgver":"0.9",
  "source":[
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ca.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/cs.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/da.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/de.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/es-AR.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/es-ES.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/eu.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/fr.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ga-IE.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/hu.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/is.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/it.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ja.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ka.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ko.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/lt.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/nb-NO.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/nl.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/nn-NO.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/pl.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/pt-BR.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/pt-PT.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ro.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/ru.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/sk.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/sl.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/sv-SE.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/uk.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/zh-CN.xpi",
    "ftp://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/releases/0.9/langpacks/zh-TW.xpi"
  ],
  "url":"http://www.mozilla.com/"
}

Also, if it came down to it, I would argue that it's more reasonable to ask the packager to insert the output of the eval statement than to expect the parser to handle exotic cases.

Last edited by Xyne (2010-01-10 04:54:40)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#4 2010-01-10 19:26:24

thoughtcrime
Member
Registered: 2008-09-27
Posts: 156
Website

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

There's a package I made where I included a variable in the pkgname, and because of that, I was kind of suprised that the AUR could parse it right.

http://aur.archlinux.org/packages.php?ID=31796
PKGBUILD: http://aur.archlinux.org/packages/gtkra … n/PKGBUILD


Your parser works just fine with it big_smile

thoughtcrime ~/builds/gtkradiant/svn/gtkradiant-gamepack-urt-svn  $  ~/bash/pkgbuildparser PKGBUILD 
{
  "_gamepackname":"UrTPack",
  "_gamepackshort":"urt",
  "_svnmod":"UrTPack",
  "_svntrunk":"https://zerowing.idsoftware.com/svn/radiant.gamepacks/UrTPack",
  "arch":[
    "x86_64",
    "i686"
  ],
  "backup":[
    
  ],
  "conflicts":[
    "gtkradiant-gamepack-urt"
  ],
  "depends":[
    "gtkradiant-svn"
  ],
  "groups":[
    
  ],
  "license":[
    "unknown"
  ],
  "makedepends":[
    "subversion"
  ],
  "md5sums":[
    
  ],
  "noextract":[
    
  ],
  "options":[
    
  ],
  "pkgdesc":"Urban Terror gamepack for gtkradiant-svn",
  "pkgname":"gtkradiant-gamepack-urt-svn",
  "pkgrel":"3",
  "pkgver":"42",
  "provides":[
    "gtkradiant-gamepack-urt"
  ],
  "replaces":[
    
  ],
  "source":[
    
  ],
  "url":"http://www.qeradiant.com/cgi-bin/trac.cgi/wiki/ZeroRadiant"
}

- blog (about arch and other stuff): http://thoughtyblog.wordpress.com/
- x86_64 user

Offline

#5 2010-01-11 06:06:17

Profjim
Member
From: NYC
Registered: 2008-03-24
Posts: 658

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

Hi Xyne, does your not-a-parser parse both of these constructs:

[[ $ARCH = "x86_64" ]] && depends=( ${depends[@]} newdepends)
[[ $ARCH = "x86_64" ]] && depends+=(newdepends)

Offline

#6 2010-01-11 14:31:14

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

Profjim wrote:

Hi Xyne, does your not-a-parser parse both of these constructs:

[[ $ARCH = "x86_64" ]] && depends=( ${depends[@]} newdepends)
[[ $ARCH = "x86_64" ]] && depends+=(newdepends)
Xyne wrote:

intend to add other cases as they pop up, including variables set using conditions (e.g. [ "$ARCH" == "x86_64" ] && depends=...), and possible allow the command output from a whitelist of harmless commands.

I know there's a lot of text in the OP compared to a regular forum post, but please read it before posting questions to check if they've already been answered. *edit* You could have checked it too, btw.

Post an example of such a PKGBUILD if you have one in mind and I"ll try to get it working. That's what I meant by "as they pop up"... instead of going after every possible case that I can think of, I'll wait until I come across each one and deal with it accordingly.

Last edited by Xyne (2010-01-11 14:31:40)


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#7 2010-01-11 14:51:21

Profjim
Member
From: NYC
Registered: 2008-03-24
Posts: 658

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

Xyne wrote:

intend to add other cases as they pop up, including variables set using conditions (e.g. [ "$ARCH" == "x86_64" ] && depends=...), and possible allow the command output from a whitelist of harmless commands.

I know there's a lot of text in the OP compared to a regular forum post, but please read it before posting questions to check if they've already been answered. *edit* You could have checked it too, btw

Post an example of such a PKGBUILD if you have one in mind and I"ll try to get it working. That's what I meant by "as they pop up"... instead of going after every possible case that I can think of, I'll wait until I come across each one and deal with it accordingly.

Sorry you're right I should have re-read the thread. I had read it twice earlier and when I came back, I remembered that you mentioned that constuction, but had obviously misremembered why you mentioned it. And I didn't then re-read the thread. What prompted me to post was actually the bit after the test.

I was working with a PKGBUILD last night (no longer remember which atm) that used the first technique, had the thought, oh they should have done the second it's cleaner, and had the thought, oh I'll pass those on to Xyne as constructions sometimes appearing in PKGBUILDs.

Sorry I forget which PKGBUILD it is. I'm not working with your package yet myself, and if it had to wait for me to test it it wouldn't get done for a while if at all. I thought though you wanted to use this thread to collect mentions of techniques people are actually using in PKGBUILDs. The first at least is a technique I have seen several times. And in fact:

fgrep -Rl CARCH /var/abs | wc -l

returns 163. Don't know how often it comes up in AUR.

I agree with you it's probably not worth trying to make your non-parser handle every possible case. For that, you'd need a real parser anyway (like the one at http://github.com/sebnow/).

EDIT: as to the VAR+=(...) construction, I just grepped ABS and it doesn't seem to be used in practice. But it may in the future, since it's the preferred way to append to arrays. The VAR=(${VAR[@]} ...) technique is used in at least bash, readline, blender, and cegui.

Last edited by Profjim (2010-01-11 15:06:32)

Offline

#8 2010-01-11 15:03:59

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: A method for safe PKGBUILD metadata extraction... help me improve it.

Thanks, Profjim. I'll take a look at the specific PKGBUILDs that you've mentioned (not today though, too busy).


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

Board footer

Powered by FluxBB