packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

extofme · 2010-01-08 09:05:41

bruenig wrote:

I will probably add an --auronly flag or something soon for that. Obviously if one wants to disable aur stuff for a particular action, they should just use pacman. If someone is running packer as root, and they have an opportunity to edit the PKGBUILD, what can you do really? Should I hold their hand? I guess I could prompt one more time in that case and ask them to please consider dropping to an unprivileged user or to give me a username to do so or something. I don't want to be too much of a hand holder is all. I agree with you though on the makepkg as root stuff, just realize that it is their system and such.

for my purposes, i need a permanent pacman replacement. i have other prototypes that i've been toying with like proxying, dynamic "composite" repositories, and a git-based D[istributed]AUR of sorts if you will, and will be possibly trying to tie them in nicely. also i want to integrate the BTRFS stuff i talked about earlier, and i would expect it to work for any kind of upgrade, so using vanilla pacman would not be an option really.

from a usability perspective, when one runs pacman, they are expected to run it as root. i should not need to exit the app and manually drop privileges. app scales up using sudo, needs to scale down also when necessary. i think purpose of the app is for a seamless hybrid of AUR+official? should feel that way too. if user wants permanent AUR enabled/disabled, they can make an "alias bruenig="bruenig --aur"" (or vice versa) in their login scripts

Xyne wrote:

Sorry, I really don't want to promote a "competitor" in this thread, but I have to mention that Bauerbill does exactly that. If it detects that it's running as root, it will change to a specified user before building packages (see the BuildAs/--build-as option... it prompts for a user if that isn't set though). It also only enables AUR support when the "--aur" option is passed. Building repo packages from source can be enabled separately.

this looks really good, albeit i haven't looked at any source yet here, but the feature list looks impressive and professional. thanks for the tip, im just looking for whatever suits my needs ... but at the same time, i'm skilled bash/python/php/etc... never written any perl

drcouzelis · 2010-01-08 12:04:54

bruenig wrote:

Ok I figured it out. I was grepping simply for 'Version' which was actually in the description too which meant the grep had two line hits when it should have just had one and that created problems obviously. I changed it to grep for '"Version":"' now which is more specific and how I parse it in other parts of the script. It should not ever capture anything other than the Version line in the file I parse. Sorry about that. The git is fixed. Should be able to update it.

Thanks, it works!

mezkov · 2010-01-09 18:58:42

There is a bug - it tries to build dependencies in wrong order, that doesn't satisfy dependencies.

f.e. if you want to build package A, which depends on B and B depends on C, it will try to build it in order BCA. The right order is CBA.

this can be solved by replacing line 121 (function finddeps)

aurdeps+=("$dep")

with something like that:

aurdeps=( "$dep" "${aurdeps[@]}" )

mezkov · 2010-01-09 21:47:25

but there is bigger problem - that script instals AUR packages by command "pacman -U *.pkg.tar.gz", which cause problems on 64bit system when installing lib32_* or bin32_* package. It is because lib32_package is built from i686 package, so after build there are two files with .pkg.tar.gz extension but only one should be installed.

bruenig · 2010-01-10 00:06:20

mezkov, I pushed changes to both parts of the code. I do not know about this lib32 stuff as I have not dealt with it, so it still may very well not work. What I have done is set pacman to install "$pkgname-$pkgver-$pkgrel-$CARCH.pkg.tar.gz" where $CARCH comes from makepkg.conf. If this is still not correct, please tell me what I do need to put to install the correct thing.

mezkov · 2010-01-10 10:34:44

thx a lot, it works fine now.

na12 · 2010-01-10 11:42:04

I think you have to set packer to continue upgrade if some package in AUR is unavailable,for example:

 aur                                         56  56 [###########] 100%
:: Starting full aur upgrade...
resolving dependencies...

Aur Targets    (5): hwinfo libx86emu nerolinux pysvn tint2-svn

Proceed with installation? [Y/n] 
--2010-01-10 12:35:55--  http://aur.archlinux.org/packages/hwinfo/hwinfo.tar.gz
Resolving aur.archlinux.org... 208.92.232.29
Connecting to aur.archlinux.org|208.92.232.29|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 758 [application/x-tgz]
Saving to: "hwinfo.tar.gz"

100%[============================>] 758         --.-K/s   in 0s      

2010-01-10 12:36:00 (24.0 MB/s) - "hwinfo.tar.gz" saved [758/758]

Edit hwinfo PKGBUILD with $EDITOR? [Y/n] n
==> Making package: hwinfo 16.10-12 x86_64 (Sun Jan 10 12:36:08 CET 2010)
==> Checking Runtime Dependencies...
==> Checking Buildtime Dependencies...
==> Retrieving Sources...
  -> Downloading hwinfo-16.10-3.18.src.rpm...
--2010-01-10 12:36:08--  http://download.opensuse.org/source/factory/repo/oss/suse/src/hwinfo-16.10-3.18.src.rpm
Resolving download.opensuse.org... 195.135.221.130
Connecting to download.opensuse.org|195.135.221.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2010-01-10 12:36:09 ERROR 404: Not Found.

==> ERROR: Failure while downloading hwinfo-16.10-3.18.src.rpm
    Aborting...
The build failed.
┌─[ dany @ laptop ] - [ 12:36:09 Sun Jan 10 ] - [ ~ ]
└─[$]>

brisbin33 · 2010-01-10 17:25:36

na12 wrote:

I think you have to set packer to continue upgrade if some package in AUR is unavailable,for example:

 aur                                         56  56 [###########] 100%
:: Starting full aur upgrade...
resolving dependencies...

Aur Targets    (5): hwinfo libx86emu nerolinux pysvn tint2-svn

Proceed with installation? [Y/n] 
--2010-01-10 12:35:55--  http://aur.archlinux.org/packages/hwinfo/hwinfo.tar.gz
Resolving aur.archlinux.org... 208.92.232.29
Connecting to aur.archlinux.org|208.92.232.29|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 758 [application/x-tgz]
Saving to: "hwinfo.tar.gz"

100%[============================>] 758         --.-K/s   in 0s      

2010-01-10 12:36:00 (24.0 MB/s) - "hwinfo.tar.gz" saved [758/758]

Edit hwinfo PKGBUILD with $EDITOR? [Y/n] n
==> Making package: hwinfo 16.10-12 x86_64 (Sun Jan 10 12:36:08 CET 2010)
==> Checking Runtime Dependencies...
==> Checking Buildtime Dependencies...
==> Retrieving Sources...
  -> Downloading hwinfo-16.10-3.18.src.rpm...
--2010-01-10 12:36:08--  http://download.opensuse.org/source/factory/repo/oss/suse/src/hwinfo-16.10-3.18.src.rpm
Resolving download.opensuse.org... 195.135.221.130
Connecting to download.opensuse.org|195.135.221.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2010-01-10 12:36:09 ERROR 404: Not Found.

==> ERROR: Failure while downloading hwinfo-16.10-3.18.src.rpm
    Aborting...
The build failed.
┌─[ dany @ laptop ] - [ 12:36:09 Sun Jan 10 ] - [ ~ ]
└─[$]>

I ran into this as well. it was simply fixed with

[ -z "$versN" ] && continue

inside the for (or while) loop of foreign packages.

basically, if you don't find any new version info, it's a foreign package (as reported by -Qqm) but it's no longer (or never was) available on AUR. therefore, it should be ignored in the upgrade.

edit:

i also ran into the lib32 issue, so far it's sufficient to match on $PACK*.pkg.tar.gz as one's lib32something and the other is just something.

Last edited by brisbin33 (2010-01-10 17:27:33)

Dead Code · 2010-01-10 19:05:21

i would found bug where in while installation packer is trying to source the revision (in case of SVN) mentioned in the PKGBUILD rather than the actual revision

==> Finished making: bespin-svn 964-1 i686 (Mon Jan 11 00:31:17 IST 2010)
Password:
loading package data...
error: 'bespin-svn-956-1-i686.pkg.tar.gz': cannot open package file

[ /tmp/packer-1000/bespin-svn/bespin-svn ]$> ls
bespin-svn-964-1-i686.pkg.tar.gz  pkg  PKGBUILD  src

Last edited by Dead Code (2010-01-10 19:06:58)

bruenig · 2010-01-11 01:22:22

Dead Code wrote:

i would found bug where in while installation packer is trying to source the revision (in case of SVN) mentioned in the PKGBUILD rather than the actual revision
==> Finished making: bespin-svn 964-1 i686 (Mon Jan 11 00:31:17 IST 2010)
Password:
loading package data...
error: 'bespin-svn-956-1-i686.pkg.tar.gz': cannot open package file

[ /tmp/packer-1000/bespin-svn/bespin-svn ]$> ls
bespin-svn-964-1-i686.pkg.tar.gz  pkg  PKGBUILD  src

This was introduced when I tried to fix the mezkov bug. I have changed the install line again to just use "$pkgname-*-$CARCH.pkg.tar.gz" so it should work.

bruenig · 2010-01-11 01:28:49

na12 wrote:

I think you have to set packer to continue upgrade if some package in AUR is unavailable,for example:

 aur                                         56  56 [###########] 100%
:: Starting full aur upgrade...
resolving dependencies...

Aur Targets    (5): hwinfo libx86emu nerolinux pysvn tint2-svn

Proceed with installation? [Y/n] 
--2010-01-10 12:35:55--  http://aur.archlinux.org/packages/hwinfo/hwinfo.tar.gz
Resolving aur.archlinux.org... 208.92.232.29
Connecting to aur.archlinux.org|208.92.232.29|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 758 [application/x-tgz]
Saving to: "hwinfo.tar.gz"

100%[============================>] 758         --.-K/s   in 0s      

2010-01-10 12:36:00 (24.0 MB/s) - "hwinfo.tar.gz" saved [758/758]

Edit hwinfo PKGBUILD with $EDITOR? [Y/n] n
==> Making package: hwinfo 16.10-12 x86_64 (Sun Jan 10 12:36:08 CET 2010)
==> Checking Runtime Dependencies...
==> Checking Buildtime Dependencies...
==> Retrieving Sources...
  -> Downloading hwinfo-16.10-3.18.src.rpm...
--2010-01-10 12:36:08--  http://download.opensuse.org/source/factory/repo/oss/suse/src/hwinfo-16.10-3.18.src.rpm
Resolving download.opensuse.org... 195.135.221.130
Connecting to download.opensuse.org|195.135.221.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2010-01-10 12:36:09 ERROR 404: Not Found.

==> ERROR: Failure while downloading hwinfo-16.10-3.18.src.rpm
    Aborting...
The build failed.
┌─[ dany @ laptop ] - [ 12:36:09 Sun Jan 10 ] - [ ~ ]
└─[$]>

So what is really happening here is that the pkgbuild in the updated aur package is broken and so the build fails. Right now I have it set up such that if a build fails, the program just exists. I have changed the behavior to say that if the build fails, to simply quit trying to install that particular package. This means that if there are any other packages to be installed, it will move on to those. Please update and tell me if it works.

r6 · 2010-01-11 06:43:40

Works great for me so far

Please reconsider adding the --noconfirm option

bruenig · 2010-01-15 07:11:02

r6 wrote:

Works great for me so far
Please reconsider adding the --noconfirm option

I have added the --noconfirm option. I never intended to not have this option, I just wanted to make sure everything else worked first.

na12 · 2010-01-15 11:08:26

It would be nice if you would add option to avoid particular package to upgrade, when making upgrade from AUR. In yaourt it is A-aborting option.

bruenig · 2010-01-16 08:00:16

na12 wrote:

It would be nice if you would add option to avoid particular package to upgrade, when making upgrade from AUR. In yaourt it is A-aborting option.

I have added IgnorePkg support. All packages that one doesn't wish to update should be put in the IgnorePkg = part of ones /etc/pacman.conf.

Some limitations:
1) If are trying to install (-S) 2 aur packages and one of them is in IgnorePkg, it will prompt you about whether you wish to go forward with it like pacman does. If you say no, it installs the other package, but will also install any dependencies for the package that you ignored. Have to deal with this for now at least since I am not sure how yet to efficiently delineate between which packages have which dependencies after I have done all of that dependency finding.

2) If you have a dependency of a aur target you are trying to install (-S) in IgnorePkg for some reason, it will prompt you about whether you want to install that dependency. If you say no, it will not install any of the aur targets. Like (1) says, it is not able to know which of all of the aur dependencies for all of the aur packages belong to which specific aur packages. So if you say no to one of the IgnorePkg'd dependencies, it simply will not install any of the aur packages. (This is an extremely unlikely case that you would have a package IgnorePkg'd that you didn't have installed and you happened to hit it as an aur dependency, so I don't think it is a big deal).

3) If for some reason an installed aur package needs to update (-Su), and the updated version of that package has a dependency that you have in IgnorePkg, it does not currently respect that. But this again is just completely unlikely to ever happen. Why you would have a random aur package in IgnorePkg that accidentally got pulled in on an aur upgrade is beyond me.

So as you can see, very obscure cases all of them. But that is how it should behave and so yeah there it is.

Last edited by bruenig (2010-01-16 08:02:32)

drcouzelis · 2010-01-16 14:45:55

Ok, I think I found a bug.

I installed "vultureseye" (nethack) from the AUR. It compiled correctly, but installing the package failed. It said:

error: 'vultureseye-*-x86_64.pkg.tar.gz': cannot open package file

I went to "/tmp/packer-1000/vultureseye/vultureseye" and used "sudo pacman -U vultureseye-2.2.100-2-any.pkg.tar.gz" to finish installing the package.

I had the same problem with "openbox-dyne-theme".

error: 'openbox-dyne-theme-*-x86_64.pkg.tar.gz': cannot open package file

and the file is "/tmp/packer-1000/openbox-dyne-theme/openbox-dyne-theme/openbox-dyne-theme-20090406-1-any.pkg.tar.gz".

Thank you for your work! I only use packer now.

brisbin33 · 2010-01-16 15:53:02

bruenig, you should stop matching on $CARCH in your pacman -U as the 'any' package fail. $pkgname-*.pkg.tar.gz is enough to get around the lib32 issue that prompted the change.

bruenig · 2010-01-16 18:57:42

drcouzelis wrote:

Ok, I think I found a bug.
I installed "vultureseye" (nethack) from the AUR. It compiled correctly, but installing the package failed. It said:
error: 'vultureseye-*-x86_64.pkg.tar.gz': cannot open package file
I went to "/tmp/packer-1000/vultureseye/vultureseye" and used "sudo pacman -U vultureseye-2.2.100-2-any.pkg.tar.gz" to finish installing the package.
I had the same problem with "openbox-dyne-theme".
error: 'openbox-dyne-theme-*-x86_64.pkg.tar.gz': cannot open package file
and the file is "/tmp/packer-1000/openbox-dyne-theme/openbox-dyne-theme/openbox-dyne-theme-20090406-1-any.pkg.tar.gz".
Thank you for your work! I only use packer now.

You have an old version of packer that erroneously put that glob in quotes. You will probably need to manually build the newest packer. It uses brisbin's solution also. packer -S packer, then install it from /tmp.

drcouzelis · 2010-01-16 19:10:00

bruenig wrote:

You have an old version of packer that erroneously put that glob in quotes. You will probably need to manually build the newest packer. It uses brisbin's solution also. packer -S packer, then install it from /tmp.

Oops! I thought I had the most recent version.

It works great!

Octoploid · 2010-01-16 19:24:21

It would be great if you could implement the --ignore option.

 ~ % packer -Syu --ignore evince
Option `--ignore' is not valid.

Update: Just read the script and setting IgnorePkg=evince in /etc/pacman.conf
works fine and is sufficient for me.

Last edited by Octoploid (2010-01-16 19:35:39)

hatten · 2010-01-16 20:32:47

I've seen it said in several threads, what is it that's so awful about yaourt?

res · 2010-01-16 21:17:34

He, and several others, have already posted performance comparisons.

Daenyth · 2010-01-16 21:21:04

$ head -n1 /usr/bin/yaourt ; wc -l /usr/bin/yaourt 
#!/bin/bash
1137 /usr/bin/yaourt

That's what.

bruenig · 2010-01-17 04:52:26

hatten wrote:

I've seen it said in several threads, what is it that's so awful about yaourt?

One aspect of it is code quality. The code is really really poor. For instance,

 631     if echo $(cat $file) | grep -q "%PROVIDES%.*$1"; then return 0; else continue;fi

The proper way to do this is grep $file. I have seen some very new bash coders use cat file | grep, but I have never seen someone use something as astonishingly redundant and slow as echo $(cat $file) | grep.

There are dozens of other examples like this. But the real issue is the performance such shoddy coding creates.

The command yaourt -Ss python takes 56.280 seconds to execute.

The command packer -Ss python takes 4.146 seconds to execute.

Yaourt is 13.5 times slower than packer. That is pretty serious.

Other examples abound, but that gives you a sense. Yaourt is just sloooooow.

r6 · 2010-01-17 08:14:17

Bruenig, one more thing, could you make -Ss the default option? This is the behaviour I'm used to with yaourt. (eg. you can type yaourt python to search for python).

Arch Linux

#51 2010-01-08 09:05:41

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#52 2010-01-08 12:04:54

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#53 2010-01-09 18:58:42

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#54 2010-01-09 21:47:25

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#55 2010-01-10 00:06:20

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#56 2010-01-10 10:34:44

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#57 2010-01-10 11:42:04

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#58 2010-01-10 17:25:36

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#59 2010-01-10 19:05:21

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#60 2010-01-11 01:22:22

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#61 2010-01-11 01:28:49

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#62 2010-01-11 06:43:40

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#63 2010-01-15 07:11:02

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#64 2010-01-15 11:08:26

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#65 2010-01-16 08:00:16

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#66 2010-01-16 14:45:55

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#67 2010-01-16 15:53:02

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#68 2010-01-16 18:57:42

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#69 2010-01-16 19:10:00

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#70 2010-01-16 19:24:21

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#71 2010-01-16 20:32:47

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#72 2010-01-16 21:17:34

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#73 2010-01-16 21:21:04

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#74 2010-01-17 04:52:26

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#75 2010-01-17 08:14:17

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Board footer