Aur and package scraper

cactus · 2005-06-16 17:01:29

I talked a bit with neotuli about integrating a package scraper into the aur. Here is how I envision it working in an integrated fashion.

The aur would include either an xml-rpc page, or a soap page. I would lean more towards soap with a wsdl definition, so that clients are even simpler. Dead simple.

Here is an example client written in ruby, that would be functional for searching for app names:

#!env ruby
require 'soap/wsdlDriver'

searchstring=ARGV.join(" ")
WSDL_URL="http://aur.archlinux.org/soapy.php?wsdl"
soap=SOAP::WSDLDriverFactory.new(WSDL_URL).createDriver
result=soap.search(searchstring)
puts result

It would be used like this:

./scraper toon
unsupported-kde
  gartoon 1.3-1   Gartoon is a fun looking icon pack for KDE
  Maintainer: Pointer
  Url: http://aur.archlinux.org/packages.php?do_Details=1&ID=1199

Or something similar.

If anyone is interested, I have a soap web service setup on a little web app. If you want to see some code, or test it out, shoot me a pm.

phrakture · 2005-06-16 19:18:10

brilliant idea cactus... AUR style emulation of "pacman -Ss"

The only thing I fear is that it will export the "id" which some people don't like (getting the PKGBUILD and automating it after that point is trivial)

In order to solve that, I'd simply put a "Unsafe: Yes" flag in the output, to determine if it's been verified...
which goes to the next step - allowing a TU to verify a PKGBUILD as safe for consumption.

Hmmm... I'm rambling, but to me it sounds good. Here's the steps:

Add package "Safe = yes|no" flag. On upload/upgrade, package is always set to unsafe... TUs allow to toggle flag.

With packages exposed through soap, combine with Dusty's aurbuild script, and viola... we now have a "aurpac" which will only allow a user to auto-build validated PKGBUILDs

phrakture · 2005-06-16 19:19:45

or, even better, "srcpac --aur package" to indicate it needs to use to web service

neotuli · 2005-06-16 23:54:13

Yes indeed, we did talk about this.
The first version of this system would certainly only be a pacman -Ss equivalent, no PKGBUILDs transfered.
I'm not sure yet, but we're thinking about going ahead and letting people use cvsup on unsupported... but then we'd also have to add code so that those directories actually get deleted when the package does....

Right, well anyhow...a big warning in RED would probably be good if we ever allow direct download of PKGBUILDs through the same interface as this.

I should also say that this interface would only be for unsupported, [community] is already taken care of because it works no differently than an official repo.

dtw · 2005-08-08 13:58:08

So - now we have flag safe - does that help?

Arch Linux

#1 2005-06-16 17:01:29

Aur and package scraper

#2 2005-06-16 19:18:10

Re: Aur and package scraper

#3 2005-06-16 19:19:45

Re: Aur and package scraper

#4 2005-06-16 23:54:13

Re: Aur and package scraper

#5 2005-08-08 13:58:08

Re: Aur and package scraper

Board footer