Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

allive · 2010-01-25 15:04:09

Hey, thanks for your great app.
I've noticed a strange behaviour of bauerbill, when it builds from AUR
If PKGBUILD has such lines:

pkgname=pymacs
pkgver=0.24_beta1
source=("http://pymacs.progiciels-bpi.ca/archives/P${pkgname:1}-${pkgver/_/-}.tar.gz")

Bauerbill suggests to download the following:
http://pymacs.progiciels-bpi.ca/archives/P${pkgname:1}-0.24-beta1.tar.gz
It, for some reason, doesn't understand such modifier.

Xyne · 2010-01-25 17:41:11

@xilef
I've added the MakepkgArgs option, which lets you pass additional arguments to makepkg. If you pass it "-f" or "--force", bauerbill will detect it and run makepkg even if the package already exists.

The CacheBuilt option works too, although makepkg's PKGDEST takes precedence.

@gonX
I've added an option which you can use in conjunction with "--noconfirm" to bypass all PKGBUILD inspection prompts. It begins with "b" and you can find it in the help text and the man page.

@lucke
Bauerbill and Powerpill now catch the signal from ctrl-c and remove the database lock if they have created one. Really though, I do not understand why you kill it often enough during a database update for this to be have been an issue.

I'm not opposed to including the zsh completion file for both powerpill and bauerbill (powerpill has a bash completion file already), but at this point there are so many powerpill/bauerbill-specific options that the file should really include these as well.

It shouldn't be difficult to generate these automatically using the OptionHandler module that I wrote, but I don't really have enough of an interest in learning the ins and outs of both bash and zsh completion to be willing to spend my time on it. If someone writes some code to do this, I will consider its inclusion.

@allive
The PKGBUILD parser now supports substrings, which it didn't before. I've tested it with the pymacs package and it correctly interpolates the url now.

I was actually surprised that the url was case-sensitive.

eyolf · 2010-01-25 17:59:40

Nice work!
Just a question, though, out of curiosity -- and this goes to the whole thread, not just to xyne: what's so bad about yaourt?! It's great when someone makes something that's even better than what we had before, but many of the comments puzzle me -- the ones that go along the lines of "Yess! Now I can finally remove yaourt!" I must say, yaourt has served me excellently for a long time now, and except for the minor glitches after major pacman updates, I haven't had any problems with it.
So why this animosity against a great piece of software?

Another question: is there a zsh-completion package in the making...?

peti · 2010-01-25 18:55:09

It appears that bauerbill has a dependency parsing bug. I just ran:

bauerbill --abs -S haskell-quickcheck

Then I edited the PKGBUILD and added the dependency on "haskell-mtl" which is missing. However, bauerbill didn't recognize that the list of dependencies had changed, i.e. it did not try to build haskell-mtl. Instead, it tried to build haskell-quickcheck and failed with some weird error message.

Is this a known problem?

gonX · 2010-01-25 19:34:55

Aw, looks like I'll have to make an alias. Thanks

gonX · 2010-01-25 19:37:00

Ugh, new feature request: support -LATEST PKGBUILDs, such as chromium-browser-bin. It does not seem to update my version of it.

Xyne · 2010-01-25 21:34:16

@peti
I was "cleverly" using the sync database to get the list of dependencies for ABS packages because it should always be accurate, unlike the PKGBUILD parser, which may fail in some cases. I hadn't even considered changing dependencies at the prompt tbh.

It now parses the dependencies in the PKGBUILD.

gonX wrote:

Aw, looks like I'll have to make an alias. Thanks

hehe

I can't find any pacman/makepkg documentation which mentions setting pkgver to "LATEST" in the PKGBUILD. Show me some official documentation or describe how you think bauerbill should handle "LATEST".

Last edited by Xyne (2010-01-25 21:35:21)

brando56894 · 2010-01-26 05:07:36

It doesn't want to work for me, no matter what I do it always says "switching to pacman: /usr/bin/pacman" then will give me an error about the options.

heres an example:

[bran@ra ~]$ bauerbill --build-deps --aur arch-service
--> Switching to pacman: /usr/bin/pacman arch-service

jowilkin · 2010-01-26 05:22:40

brando - you need to use the same style as pacman. So if you are trying to install a program called arch-service, you need the -S flag. i.e.

bauerbill --build-deps --aur -S arch-service

gonX · 2010-01-26 15:16:25

I don't see anything about setting the pkgver to LATEST either. If you take a look at the chromium-browser-bin package, you can see what it does though:

# trick re-determining the build revision and, or entertain the hidden
# makepkg option --forcever, huh
if [ -z "$FORCE_VER" ]; then
    msg "Determining latest build revision..."
    newpkgver="$(wget -qO - ${_bldroot}/LATEST)"
    if [ -n "$newpkgver" -a "$newpkgver" != "$pkgver" ]; then
    msg2 "Version found: %s" "$newpkgver"
    FORCE_VER="$newpkgver"
    else
    FORCE_VER="$pkgver"
    fi
fi
if [ "$FORCE_VER" = "LATEST" -a "$SOURCEONLY" -eq 0 ]; then
    error "Plz omit '--forcever LATEST'; I'll auto-detect the LATEST ;)"
    exit 1
elif [ "$FORCE_VER" != "$pkgver" ]; then
    rm -f "$SRCDEST/${_bldname}"
    devel_update
fi

Also the option you gave me (that extremely long string) does not seem to work. It still asks me to source the packages.

Xyne · 2010-01-26 16:11:06

Bauerbill won't be able to use logic inside of a package to determine if the package should be downloaded, because it would have to download the package first anyway. You're better off just specifying it on the command line when you want to check for upgrades. It should then build the latest version on its own using the logic in the package.

I'll eventually get to adding options to handle development packages differently.

Are you using the "--noconfirm" option? That seemed to be what you wanted it to work with.

peti · 2010-01-26 19:19:32

Xyne, thank you very much for the quick update!

I have one more question. Yaourt had a nice command, -G,that could be used to download an AUR package into the current directory. Does bauerbill have something similar? I realize that I can run "bauerbill --aur -S <pkg>" and retrieve the taurball from /tmp/bauerbill/taurballs, but that's slightly inconvenient comparing to having to run "bauerbill -G <pkg>" and be done with it. Does that feature exist somewhere?

Primoz · 2010-01-26 20:27:08

Can bauerbill use aurvote for voting packages from AUR to community?
Yaourt has this option. It would be cool if bauerbill would have it to.

(Sorry if someone already asked this question)

Xyne · 2010-01-26 20:31:41

@peti
That's on my todo list. In the meantime, take a look at pbget on my site/in the AUR.

@Primoz
No. I might write a stand-alone app to do that, but I don't think that voting for packages is part of package management.

ataraxia · 2010-01-27 20:47:37

Any interest in supporting builds from SVN rather than ABS, the way that pbget can do it? The ABS tree isn't so useful as SVN since the ABS tarballs are only updated once a day, while repos usually update much more often than that. I end up having to wait a day before I can update things I build from source, or just go straight to SVN myself.

Xyne · 2010-01-27 22:35:07

The problem is that there is no readily accessible interface for using SVN. Scraping the html the way pbget does is not efficient, and may require updates every time the web page layout changes. I have requested an RPC interface to the ABS tree to match the AUR RPC interface, but that's not on any of the web dev's priority list.

I also want to be able to support unofficial repos and it would get really messy if I have to add several blocks of code to determine how to access PKGBUILDs for a given repo (core and extra use different paths than community, others don't have a web interface, etc).

I don't know how the $repo.abs.tar.gz files are created, but they're probably cron'd on the main server (I suspect that recreating them for every submit would require too much overhead). You can always set your main mirror to archlinux.org which will let bauerbill use it for database updates and taurball retrievals. As long as you have other mirrors in your list and/or use the Reflect option, downloads will still go through other servers so the bandwidth cap isn't really an issue.

Ideally there would be individual taurballs for each package, which would always be in sync with the binary package, but I don't think the devs would seriously consider that unless there were many users who voted for it. My past experience with feature requests on the bug tracker has left me a bit too cynical of the process to open a ticket for this (not in the mood to be told to submit a patch), but if someone else does I'll vote for it and comment when appropriate.

jankowiak · 2010-01-27 22:48:17

To build on what ataraxia said and I know it was mentioned earlier in the thread, would it be possible that when upgradedable packages are found it shows something like this:

:: The following packages will be downloaded:
   [core]
     kernel26 current_version > new_version > pacman_db_version
     kernel26-firmware current_version > new_version > pacman_db_version
     kernel26-headers current_version > new_version > pacman_db_version
     zlib current_version > new_version > pacman_db_version
   [extra]
     libxml2 current_version > new_version  > pacman_db_version

This would make it much easier to determine updates when the issue of abs tarballs being out of date (it should check the PKGBUILD instead of the current check in the pacman db) compared to the binary repos. I would also be nice if we could get a similar display to even the standard view when using the --abs flag instead of i

Also does the repo used with the abs command also have the same one day delay as the tarballs? I was under the impression that it doesn't as there are several PKGBUILDs that I have found in an updated abs tree but that aren't in the tarballs. If this is the case it might be more beneficial to use the abs tree instead of the tarballs.

edit::
I have archlinux.org set as my repo just for the tarballs and I still have a delay (since yesterday I can upgrade kernel26 without the abs flag but with it downloads the same version already installed)

Last edited by jankowiak (2010-01-27 22:51:43)

res · 2010-01-28 00:03:35

bauerbill doesn't interpret pkg/srcdest correctly when using ~/.makepkg.conf and it's not specified in bauerbill.conf. Also, it does not find the tarball, or at least when it's not placed in it's default location (./). Maybe less wrapage and just makepkg -i ?

When it is defined in bauerbill.conf, it doesn't like $EMAIL in $PACKAGER, nor does it like paths starting with $HOME or ~/. When I used EMAIL, checking the resulting package with pacman -Qi showed 'res <>' in the packager field, and when trying the latter bauerbill just made directory /root/$HOME/...

I guess this is a limitation of not being able to source bash files directly? In any case, I suggest sticking to pkg/srcdest to default instead of taurballdir and whatever is the equivalent of pkgdest.

Also, having -c as a default makepkgarg makes sense given that people that are drawn to these tools usually don't want to fiddle with manual aur procedures, let alone debug packages by looking at the build leftovers.

Last edited by res (2010-01-28 00:04:54)

Xyne · 2010-01-28 00:30:45

@jankowiak
As I've said before, I intend to try to make the output of different commands customizable but that will not be trivial to do. I do not want to hard-code anything which deviates significantly from standard pacman out. I'm working on other things right now though so I don't know how long it will take me to get to that.

The abs archive in the repo is not directly connected to the abs tree in the abs package. I really have no idea how they generate the $repo.abs.tar.gz files or why they're sometimes incomplete or delayed.

Instead of asking me to limit ABS functionality to the official repos, it would be nice if bauerbill users would instead try to get the repo managers to improve their scripts in order to keep the archives in sync with the binary packages. It would also be nice if users expressed a desire to the managers of other unofficial repos to include this archive to make it possible to build packages in those repos from source as well.

res wrote:

bauerbill doesn't interpret pkg/srcdest correctly when using ~/.makepkg.conf and it's not specified in bauerbill.conf

Well, if you don't tell bauerbill to use it, how is it supposed to know? Adding extra checks for such files just isn't reasonable, especially when you consider the complexity of cases such as when it's run as root... should it check the BuildAs user's home dir, or root's, or just use the default? The option is easy enough to set in the config, and there is a command line option as well.

res wrote:

Maybe less wrapage and just makepkg -i ?

"makepkg -i" won't work for two reasons:
1) It requires sudo to install packages because it's not run as root, and that would make "sudo" a dependency.
2) It would install packages explicitly. Bauerbill keeps track of explicitly and implicitly installed packages and preserves this status when building target packages and their dependencies.

res wrote:

When it is defined in bauerbill.conf, it doesn't like $EMAIL in $PACKAGER, nor does it like paths starting with $HOME or ~/. When I used EMAIL, checking the resulting package with pacman -Qi showed 'res <>' in the packager field, and when trying the latter bauerbill just made directory /root/$HOME/...

Are you running bauerbill as root or as a normal user? Bauerbill doesn't load those variables or affect them in any way. The only thing it pulls from makepkg.conf are SRCDEST and PKGDEST. I suspect that you're running into the same issues that others have had with the $EDITOR variable only being set for the normal user and not root.

res wrote:

Also, having -c as a default makepkgarg makes sense given that people that are drawn to these tools usually don't want to fiddle with manual aur procedures, let alone debug packages by looking at the build leftovers.

Just set the MakepkgArgs option in the config if you want to always pass it "-c".

Last edited by Xyne (2010-01-28 00:32:48)

res · 2010-01-28 00:52:09

Xyne wrote:

Well, if you don't tell bauerbill to use it, how is it supposed to know?

Considering that one of the goals was to remain as 1:1 with the apps behavior as realistically possible (or at least I got that impression), bauerbill should wrap less, not more, when it has the opportunity. I think this is no longer (never was?) a concern?.

Xyne wrote:

"makepkg -i" won't work for two reasons:
1) It requires sudo to install packages because it's not run as root, and that would make "sudo" a dependency.

But makepkg also uses sudo as a dependency for the same purpose.

Xyne wrote:

2) It would install packages explicitly. Bauerbill keeps track of explicitly and implicitly installed packages and preserves this status when building target packages and their dependencies.

Since this is the only legitimate reason behind wrapping makepkg so much, just edit the database manually after it's done. Seems hackish, yet less hackish than wrapping a bash script, imo. Not to mention that it restores makepkg's ability of being the only one that has to deal with its own config...

Xyne wrote:

Are you running bauerbill as root or as a normal user? Bauerbill doesn't load those variables or affect them in any way. The only thing it pulls from makepkg.conf are SRCDEST and PKGDEST. I suspect that you're running into the same issues that others have had with the $EDITOR variable only being set for the normal user and not root.

It is ran as root on both occations, dropping privileges to my user. The problem is not that bauerbill is touching EMAIL... it just doesn't like that var. PACKAGER was set to "$USER <${EMAL/@/ }>"; bauerbill picked up on USER but disregarded EMAIL. Also, how would it being unset explain not picking up on $HOME for pkgdest?

Xyne wrote:

Just set the MakepkgArgs option in the config if you want to always pass it "-c".

bauerbill is mirroing a lot of things makepkg already does, uneedingly imo. At the very least, it shouldn't grow TMPDIR for no reason, or at least not by default.

Xyne · 2010-01-28 01:38:53

res wrote:

Xyne wrote:
"makepkg -i" won't work for two reasons:
1) It requires sudo to install packages because it's not run as root, and that would make "sudo" a dependency.
But makepkg also uses sudo as a dependency for the same purpose.

You misread my reply. I do not want to make "sudo" a dependency of bauerbill, which using "makepkg -i" would.

res wrote:

Xyne wrote:
2) It would install packages explicitly. Bauerbill keeps track of explicitly and implicitly installed packages and preserves this status when building target packages and their dependencies.
Since this is the only legitimate reason behind wrapping makepkg so much, just edit the database manually after it's done. Seems hackish, yet less hackish than wrapping a bash script, imo. Not to mention that it restores makepkg's ability of being the only one that has to deal with its own config...

This point is moot in light of the one above.

res wrote:

It is ran as root on both occations, dropping privileges to my user. The problem is not that bauerbill is touching EMAIL... it just doesn't like that var. PACKAGER was set to "$USER <${EMAL/@/ }>"; bauerbill picked up on USER but disregarded EMAIL. Also, how would it being unset explain not picking up on $HOME for pkgdest?

What do you mean "bauerbill picked up on USER"? How does bauerbill use the PACKAGER variable? This is what I don't understand.
$HOME in pkgdest is probably due to a lack of parsing. I think the makepkg.conf parser is just a couple of naïve regexes, although I can switch this to use the PKGBUILD parser.

res wrote:

Xyne wrote:
Just set the MakepkgArgs option in the config if you want to always pass it "-c".
bauerbill is mirroing a lot of things makepkg already does, uneedingly imo. At the very least, it shouldn't grow TMPDIR for no reason, or at least not by default.

Huh? What other functionality is it duplicating? Where do you want it to dump files?

Also, this is Arch. Users can be expected to edit a config file if they don't like the defaults.

res · 2010-01-28 02:44:09

Xyne wrote:

Also, this is Arch. Users can be expected to edit a config file if they don't like the defaults.

Let's just say bauerbill users aren't slurpy users, and that bauerbill doesn't follow Arch's principles to begin with.

aria2c is also overkill for what amounts to a couple of kb...

Sigh, I think it's best if I don't recommend and roll my own instead.

Xyne · 2010-01-28 06:21:57

res wrote:

Xyne wrote:
Also, this is Arch. Users can be expected to edit a config file if they don't like the defaults.
Let's just say bauerbill users aren't slurpy users, and that bauerbill doesn't follow Arch's principles to begin with.
aria2c is also overkill for what amounts to a couple of kb...

Sigh, I think it's best if I don't recommend and roll my own instead.

A couple of kb?

That line along with several other remarks clearly show that you don't understand what bauerbill actually does. Your attitude has been unpleasant prior to the last post but coming back to roll your eyes, claim that bauerbill doesn't follow Arch's prinicples, then sigh and head off to supposedly roll your own is just rude. All this seemingly because editing a file and adding "-c" is a big deal for you and you expect others to have to edit their files instead.

If you look through this or any other of my contribution's threads I think you'll see that I'm generally open to suggestions and feature requests. However, when someone basically says "this sucks, you should do it this way, even though I don't understand why you did it that way" and doesn't even bother to try to understand why something's been done the way it has, I'm not exactly motivated to put time into helping that person.

I will change the makepkg.conf parser to interpolate variables in PKGDEST and SRCDEST.

I was still trying to figure out what you meant by bauerbill "not liking" the EMAIL var, because bauerbill doesn't use the PACKAGER var for anything whatsoever. I suspect that you have misunderstood something about environment variable scoping.

Despite your attitude, I was still willing to let you explain what you meant by "bauerbill is mirroing a lot of things makepkg already does" and "it shouldn't grow TMPDIR for no reason" and I would have considered any changes that I felt would be an improvement.

res · 2010-01-28 07:32:47

I brought up how it handles makepkg and how it shouldn't wrap it, sidestepped by -c.

Funny how it's costumer service, too. Sorry if you interpret these as requests. I just noticed bauerbill's 3000++ sloc...

Please don't troll me by saying these are pleads...

e: lol it breaks because it drops privileges but no login shell... maybe it bauerbill.pl should have a bash parser to for ~/.profile

Last edited by res (2010-01-28 09:57:51)

Xyne · 2010-01-28 07:55:01

res wrote:

Please don't troll me by saying these are pleads...

kthxbai

Arch Linux

#251 2010-01-25 15:04:09

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#252 2010-01-25 17:41:11

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#253 2010-01-25 17:59:40

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#254 2010-01-25 18:55:09

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#255 2010-01-25 19:34:55

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#256 2010-01-25 19:37:00

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#257 2010-01-25 21:34:16

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#258 2010-01-26 05:07:36

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#259 2010-01-26 05:22:40

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#260 2010-01-26 15:16:25

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#261 2010-01-26 16:11:06

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#262 2010-01-26 19:19:32

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#263 2010-01-26 20:27:08

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#264 2010-01-26 20:31:41

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#265 2010-01-27 20:47:37

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#266 2010-01-27 22:35:07

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#267 2010-01-27 22:48:17

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#268 2010-01-28 00:03:35

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#269 2010-01-28 00:30:45

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#270 2010-01-28 00:52:09

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#271 2010-01-28 01:38:53

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#272 2010-01-28 02:44:09

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#273 2010-01-28 06:21:57

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#274 2010-01-28 07:32:47

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#275 2010-01-28 07:55:01

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Board footer