packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

drcouzelis · 2010-01-07 16:01:35

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.

I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?

Consider using backslashes:

aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"

...or something.

Last edited by drcouzelis (2010-01-07 16:02:52)

Dead Code · 2010-01-07 16:21:25

nice. i am gonna give packer a try.. need a speedier alternative to yaourt!

Dead Code · 2010-01-07 16:30:18

quite cool.. speedy! kudos

Gallaecio · 2010-01-07 17:20:16

O.O It's reaaaaaaaaaaaaaaaaaaaaaaaally fast. For those who does like the yaourt stuff, hope yaourt developers (who has done a really great job, btw) take note of whatever makes this app so fast. Thanks!

bruenig · 2010-01-07 20:02:14

Keenard. I have just pushed an update that is intended to fix the root/sudo/su problems. packer -S packer, or you can visit the gibhub link in OP to see what it looks like. The function is runasroot(). It is used now to wrap all commands that need root privileges. On line 37.

Here is the diff on github

Last edited by bruenig (2010-01-07 20:05:25)

bruenig · 2010-01-07 20:20:06

Mikko777 wrote:

Does it do/ can you add these features:
1: yaourt -Syu --aur --devel --noconfirm (--devel is nice)
2: on first install edit/ show pkgbuild
3: yaourt -C <-- list and edit .pacnew files
4: vote for packages
Those are the ones i like yaourt for.
Performance is like the lowest priority I can think off, who watches updates anyways

1. I will probably end up implementing a --devel and --noconfirm at some point (both are pretty trivial), it already includes --aur for any -Syu though
2. It always prompts you to edit PKGBUILDs
3. Another application should be made for .pacnew files in my opinion
4. aurvote already exists, in fact yaourt simply wraps aurvote. So if you like an application, after install, just aurvote -v $package.

Last edited by bruenig (2010-01-07 20:23:22)

bruenig · 2010-01-07 20:26:27

drcouzelis wrote:

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.
I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?
Consider using backslashes:
aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"
...or something.

The quotes inside the quotes are not a problem because the parameter expansion is in a subshell. Anyhow, could you tell me what the local installed version is and the aur version is for that package. Right now, I do a simple string comparison when comparing versions, but there is a chance this might not work. Also, make sure you have the latest packer (packer -S packer) as I fiddled around with that code a day or so ago because it wasn't updating at all! (found out I had mistyped a variable name).

Last edited by bruenig (2010-01-07 20:27:02)

brisbin33 · 2010-01-07 21:29:31

bruenig, i notice you use simple bash string comps for version checking. i found this didn't work well depending on some versioning schemes. i find vercmp to be much more reliable and it's provided by pacman so you can assume the users's got it installed. here's the snippet from aurget:

check=$(vercmp "$vers" "$versN")
[ $check -gt 0 ] && warn "$name: local ($vers) is newer than aur ($versN)"
[ $check -lt 0 ] && echo $name >> $WD/upgrades.lst

where $vers is the local version and $versN the aur version reported by JSON.

kylepotts · 2010-01-07 22:07:30

this looks really promising. Here are some test

yaourt -Ss firefox
real 0m8.516s
user 0m4.516s
sys 0m3.150s

packer -Ss firefox
real 0m0.735s
user 0m0.313s
sys 0m0.220s

brisbin33 · 2010-01-07 22:26:34

hmmm.

time (pacman -Ss firefox; aurget -s firefox)
real    0m0.669s
user    0m0.200s
sys     0m0.123s

wonder how Xyne's bauerbill stacks up.

Inxsible · 2010-01-07 22:38:27

So aurget is even faster.

Does aurget handle installing all packages (core,extra repos) or only the ones from aur? What about packer?

Last edited by Inxsible (2010-01-07 22:39:06)

brisbin33 · 2010-01-07 22:41:12

Inxsible wrote:

So aurget is even faster.

Does aurget handle installing all packages (core,extra repos) or only the ones from aur? What about packer?

packer does both as yaourt does. i can see the benefit but personally, i like to keep the operations separate so aurget only handles aur (hence why i timed the two searches together to be fair). all depends on what you're looking for.

Inxsible · 2010-01-07 22:49:01

I have aliases for yaourt and pacman, but I hardly ever use the yaourt ones. Mostly all my work is done with pacman. So I guess keeping them separate is not a big issue with me.

And would you believe it, I have 2 packages installed from aur. Of those, I am the maintainer for 1 (musca) -- so I always know if there is a newer version available

and the other is yaourt, which obviously will be removed, if I get something else.

Mind you, this is a 10 yr old dino laptop, so I can't go trigger happy and install anything and everything from aur either. Maybe my newer machines (if I ever get to buying one eventually) will probably have a lot more software for trying/testing/using/or just plain there for show.

I will definitely be trying out packer, aurget and bauerbill tonight and see which one fits the bill for my usage.

Last edited by Inxsible (2010-01-07 22:51:30)

keenerd · 2010-01-07 23:03:28

Works like a charm. Thanks!

Xyne · 2010-01-07 23:19:25

You need to add "git" to the PKGBUILD makedepends array.

bruenig · 2010-01-08 01:46:13

brisbin33 wrote:

hmmm.
time (pacman -Ss firefox; aurget -s firefox)
real    0m0.669s
user    0m0.200s
sys     0m0.123s
wonder how Xyne's bauerbill stacks up.

Minor variations of 0.6 seconds are more likely accounted for by bandwidth differences

  # thanks to bruenig for this approach

especially since you have switched the script to an approach very similar to mine! .

However, I should say that aurget is very good and had I known about it before this, I would have probably forked that instead of writing it all myself.

Last edited by bruenig (2010-01-08 01:48:24)

bruenig · 2010-01-08 01:57:40

drcouzelis wrote:

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.
I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?
Consider using backslashes:
aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"
...or something.

I used brisbin's solution from above to use pacman's vercmp (did not know that existed). Should work now, pushed changes.

some-guy94 · 2010-01-08 02:42:43

bruenig wrote:

some-guy94 wrote:
I don't know where else to post this, so here's a patch that remove the need for the tmpfile, and stores everything in variables
http://pastie.org/769777
Myself and a few others spent about 2 hours in ##off-archlinux on freenode going through a whole series of different combinations of external apps (awk, sed, grep, cut), tmpfiles, variables, etc. And you may be surprised, but it was found to be significantly faster to use tmpfiles than variables on searches. It is a pain to use them, but I'd rather deal with that than slower speeds. Wait until you see the mess of a tmpfile clusterfuck on the new -Ss function now (which includes support for multiple search terms).

In that case, you might want to look at mktemp.

bruenig · 2010-01-08 02:51:47

frigaut,

I pushed provides checking. It is a tricky situation in that pacman does not provide a way to simply see if X is provided for by anything that is installed. Yaourt uses a really awful grepping strategy on the pacman db that is hideously slow and even poorly written for what it is (it literally at one point does echo $(cat $file) | grep). Anyways, what happens now is: it checks whether a dependency is installed locally, if it is not, then it uses a combination of pacman -Qqs and pacman -Qi to see whether if it is provided locally. If it is either of those, it simply considers the dependency satisfied and does not include it in anything.

Could you 1) please test and 2) tell me whether this enough for provides support or if there is something else I should need to do.

drcouzelis · 2010-01-08 03:37:17

I am using the latest packer. Here is my bug:

Packer thinks "snes9x-gtk" in the AUR is always out of date, and always wants to "upgrade" it. On my computer it is version 1.51.78-1. It is the same version on the AUR.

I looked at the packer function "aurversionisnewer". I added the "echo" command to see the two version numbers for snes9x-gtk. This is what it says (my modified packer output):

Package "snes9x-gtk" is out of date.
Local version is "1.51.78-1".
AUR version is "1.51.78-1 GTK+ Version of snes9x, a portable Emulator for the Super Nintento Entertainment System".

It looks like packer is parsing the AUR version number incorrectly.

Here is the snes9x-gtk AUR link: http://aur.archlinux.org/packages.php?ID=15207

Thanks!

====

Just in case you were wondering, here is my modified "aurversionisnewer" function:

aurversionisnewer() {
  getjson "http://aur.archlinux.org/rpc.php?type=info&arg=$1"
  unset aurversion
  if ! grep -Fq ':"No result found"' "$tmpfile"; then
    aurversion="$(grep -F Version "$tmpfile" | cut -d '"' -f 4)"
    if [[ "$(vercmp "$aurversion" "$2")" -eq 1  ]]; then
      echo Package \"$1\" is out of date.
      echo Local version is \"$2\".
      echo AUR version is \"$aurversion\".
      return 0
    else
      return 1
    fi
  else
    return 1
  fi
}

Last edited by drcouzelis (2010-01-08 03:37:37)

frigaut · 2010-01-08 05:21:46

bruenig wrote:

frigaut,
I pushed provides checking. It is a tricky situation in that pacman does not provide a way to simply see if X is provided for by anything that is installed. Yaourt uses a really awful grepping strategy on the pacman db that is hideously slow and even poorly written for what it is (it literally at one point does echo $(cat $file) | grep). Anyways, what happens now is: it checks whether a dependency is installed locally, if it is not, then it uses a combination of pacman -Qqs and pacman -Qi to see whether if it is provided locally. If it is either of those, it simply considers the dependency satisfied and does not include it in anything.
Could you 1) please test and 2) tell me whether this enough for provides support or if there is something else I should need to do.

1) I have tested in a couple of different situations. It works (i.e. honor "provides") on what I tested it with. Thanks for such a quick turn around!
2) right now I can't think of anything else with respect to "provides", but I'll be back if I think of something or hit on another limitation.

bruenig · 2010-01-08 05:23:55

drcouzelis wrote:

I am using the latest packer. Here is my bug:
Packer thinks "snes9x-gtk" in the AUR is always out of date, and always wants to "upgrade" it. On my computer it is version 1.51.78-1. It is the same version on the AUR.
I looked at the packer function "aurversionisnewer". I added the "echo" command to see the two version numbers for snes9x-gtk. This is what it says (my modified packer output):
Package "snes9x-gtk" is out of date.
Local version is "1.51.78-1".
AUR version is "1.51.78-1 GTK+ Version of snes9x, a portable Emulator for the Super Nintento Entertainment System".
It looks like packer is parsing the AUR version number incorrectly.
Here is the snes9x-gtk AUR link: http://aur.archlinux.org/packages.php?ID=15207
Thanks!
====
Just in case you were wondering, here is my modified "aurversionisnewer" function:
aurversionisnewer() {
  getjson "http://aur.archlinux.org/rpc.php?type=info&arg=$1"
  unset aurversion
  if ! grep -Fq ':"No result found"' "$tmpfile"; then
    aurversion="$(grep -F Version "$tmpfile" | cut -d '"' -f 4)"
    if [[ "$(vercmp "$aurversion" "$2")" -eq 1  ]]; then
      echo Package \"$1\" is out of date.
      echo Local version is \"$2\".
      echo AUR version is \"$aurversion\".
      return 0
    else
      return 1
    fi
  else
    return 1
  fi
}

Ok I figured it out. I was grepping simply for 'Version' which was actually in the description too which meant the grep had two line hits when it should have just had one and that created problems obviously. I changed it to grep for '"Version":"' now which is more specific and how I parse it in other parts of the script. It should not ever capture anything other than the Version line in the file I parse. Sorry about that. The git is fixed. Should be able to update it.

Last edited by bruenig (2010-01-08 05:25:09)

extofme · 2010-01-08 05:27:32

this is interesting. because pacman doesn't support hooks of any kind, i was considering writing a wrapper for pacman but this script should work fine.

i have created a BTRFS mkinitcpio hook here:

http://bbs.archlinux.org/viewtopic.php?id=88195

i wish to link it to pacman so that everytime i do a major upgrade/install/whatever pacman... or packer... automatically "snapshots" my / so that i can rollback my entire system should anything go wrong.

it would be similar to what fedora is attempting:

https://fedoraproject.org/wiki/Features … kWithBtrfs

with this ability, one could simply append their kernel boot line with "rootflags=subvol=rollback" or similar, and they could boot into their / BEFORE the upgrade was performed. with this, anyone could safely update their system with very little risk; the only exception being kernel updates because as of right now /boot MUST be on a separate, non-btrfs partition. however, packer could simply backup vmlinuz/kernel26.img in /boot and be on its merry way.

i will keep an eye on this and probably send a patch soon.

i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot". i dont trust people from the aur, or myself. i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar. --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.

also a way to disable AUR or official repos completely would be good as others has semi-mentioned. i like to keep the two separate

bruenig · 2010-01-08 05:51:59

extofme wrote:

this is interesting. because pacman doesn't support hooks of any kind, i was considering writing a wrapper for pacman but this script should work fine.
i have created a BTRFS mkinitcpio hook here:
http://bbs.archlinux.org/viewtopic.php?id=88195
i wish to link it to pacman so that everytime i do a major upgrade/install/whatever pacman... or packer... automatically "snapshots" my / so that i can rollback my entire system should anything go wrong.
it would be similar to what fedora is attempting:
https://fedoraproject.org/wiki/Features … kWithBtrfs
with this ability, one could simply append their kernel boot line with "rootflags=subvol=rollback" or similar, and they could boot into their / BEFORE the upgrade was performed. with this, anyone could safely update their system with very little risk; the only exception being kernel updates because as of right now /boot MUST be on a separate, non-btrfs partition. however, packer could simply backup vmlinuz/kernel26.img in /boot and be on its merry way.
i will keep an eye on this and probably send a patch soon.
i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot". i dont trust people from the aur, or myself. i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar. --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.
also a way to disable AUR or official repos completely would be good as others has semi-mentioned. i like to keep the two separate

I will probably add an --auronly flag or something soon for that. Obviously if one wants to disable aur stuff for a particular action, they should just use pacman. If someone is running packer as root, and they have an opportunity to edit the PKGBUILD, what can you do really? Should I hold their hand? I guess I could prompt one more time in that case and ask them to please consider dropping to an unprivileged user or to give me a username to do so or something. I don't want to be too much of a hand holder is all. I agree with you though on the makepkg as root stuff, just realize that it is their system and such.

Last edited by bruenig (2010-01-08 05:53:10)

Xyne · 2010-01-08 05:59:26

extofme wrote:

i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot". i dont trust people from the aur, or myself. i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar. --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.
also a way to disable AUR or official repos completely would be good as others has semi-mentioned. i like to keep the two separate

Sorry, I really don't want to promote a "competitor" in this thread, but I have to mention that Bauerbill does exactly that. If it detects that it's running as root, it will change to a specified user before building packages (see the BuildAs/--build-as option... it prompts for a user if that isn't set though). It also only enables AUR support when the "--aur" option is passed. Building repo packages from source can be enabled separately.

Arch Linux

#26 2010-01-07 16:01:35

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#27 2010-01-07 16:21:25

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#28 2010-01-07 16:30:18

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#29 2010-01-07 17:20:16

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#30 2010-01-07 20:02:14

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#31 2010-01-07 20:20:06

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#32 2010-01-07 20:26:27

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#33 2010-01-07 21:29:31

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#34 2010-01-07 22:07:30

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#35 2010-01-07 22:26:34

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#36 2010-01-07 22:38:27

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#37 2010-01-07 22:41:12

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#38 2010-01-07 22:49:01

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#39 2010-01-07 23:03:28

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#40 2010-01-07 23:19:25

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#41 2010-01-08 01:46:13

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#42 2010-01-08 01:57:40

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#43 2010-01-08 02:42:43

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#44 2010-01-08 02:51:47

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#45 2010-01-08 03:37:17

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#46 2010-01-08 05:21:46

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#47 2010-01-08 05:23:55

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#48 2010-01-08 05:27:32

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#49 2010-01-08 05:51:59

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

#50 2010-01-08 05:59:26

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Board footer