You are not logged in.

#26 2010-01-07 16:01:35

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.

I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?

Consider using backslashes:

aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"

...or something. tongue

Last edited by drcouzelis (2010-01-07 16:02:52)

Offline

#27 2010-01-07 16:21:25

Dead Code
Member
Registered: 2009-01-18
Posts: 227

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

nice. i am gonna give packer a try.. need a speedier alternative to yaourt!

Offline

#28 2010-01-07 16:30:18

Dead Code
Member
Registered: 2009-01-18
Posts: 227

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

quite cool.. speedy! kudos

Offline

#29 2010-01-07 17:20:16

Gallaecio
Member
From: Vigo (Spain)
Registered: 2009-04-10
Posts: 50
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

O.O It's reaaaaaaaaaaaaaaaaaaaaaaaally fast. For those who does like the yaourt stuff, hope yaourt developers (who has done a really great job, btw) take note of whatever makes this app so fast. Thanks!

Offline

#30 2010-01-07 20:02:14

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Keenard. I have just pushed an update that is intended to fix the root/sudo/su problems. packer -S packer, or you can visit the gibhub link in OP to see what it looks like. The function is runasroot(). It is used now to wrap all commands that need root privileges. On line 37.

Here is the diff on github

Last edited by bruenig (2010-01-07 20:05:25)

Offline

#31 2010-01-07 20:20:06

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Mikko777 wrote:

Does it do/ can you add these features:

1: yaourt -Syu --aur --devel --noconfirm    (--devel is nice)

2: on first install edit/ show pkgbuild

3: yaourt -C  <-- list and edit .pacnew files

4: vote for packages

Those are the ones i like yaourt for.
Performance is like the lowest priority I can think off, who watches updates anyways roll

1. I will probably end up implementing a --devel and --noconfirm at some point (both are pretty trivial), it already includes --aur for any -Syu though
2. It always prompts you to edit PKGBUILDs
3. Another application should be made for .pacnew files in my opinion
4. aurvote already exists, in fact yaourt simply wraps aurvote. So if you like an application, after install, just aurvote -v $package.

Last edited by bruenig (2010-01-07 20:23:22)

Offline

#32 2010-01-07 20:26:27

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

drcouzelis wrote:

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.

I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?

Consider using backslashes:

aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"

...or something. tongue

The quotes inside the quotes are not a problem because the parameter expansion is in a subshell. Anyhow, could you tell me what the local installed version is and the aur version is for that package. Right now, I do a simple string comparison when comparing versions, but there is a chance this might not work. Also, make sure you have the latest packer (packer -S packer) as I fiddled around with that code a day or so ago because it wasn't updating at all! (found out I had mistyped a variable name).

Last edited by bruenig (2010-01-07 20:27:02)

Offline

#33 2010-01-07 21:29:31

brisbin33
Member
From: boston, ma
Registered: 2008-07-24
Posts: 1,796
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

bruenig, i notice you use simple bash string comps for version checking.  i found this didn't work well depending on some versioning schemes.  i find vercmp to be much more reliable and it's provided by pacman so you can assume the users's got it installed.  here's the snippet from aurget:

check=$(vercmp "$vers" "$versN")
[ $check -gt 0 ] && warn "$name: local ($vers) is newer than aur ($versN)"
[ $check -lt 0 ] && echo $name >> $WD/upgrades.lst

where $vers is the local version and $versN the aur version reported by JSON.

Offline

#34 2010-01-07 22:07:30

kylepotts
Member
Registered: 2009-06-04
Posts: 38
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

this looks really promising. Here are some test

yaourt -Ss firefox
real    0m8.516s
user    0m4.516s
sys    0m3.150s

packer -Ss firefox
real    0m0.735s
user    0m0.313s
sys    0m0.220s

Offline

#35 2010-01-07 22:26:34

brisbin33
Member
From: boston, ma
Registered: 2008-07-24
Posts: 1,796
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

hmmm.

time (pacman -Ss firefox; aurget -s firefox)
real    0m0.669s
user    0m0.200s
sys     0m0.123s

wonder how Xyne's bauerbill stacks up.

Offline

#36 2010-01-07 22:38:27

Inxsible
Forum Fellow
From: Chicago
Registered: 2008-06-09
Posts: 9,183

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

So aurget is even faster. wink


Does aurget handle installing all packages (core,extra repos) or only the ones from aur?  What about packer?

Last edited by Inxsible (2010-01-07 22:39:06)


Forum Rules

There's no such thing as a stupid question, but there sure are a lot of inquisitive idiots !

Offline

#37 2010-01-07 22:41:12

brisbin33
Member
From: boston, ma
Registered: 2008-07-24
Posts: 1,796
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Inxsible wrote:

So aurget is even faster. wink


Does aurget handle installing all packages (core,extra repos) or only the ones from aur?  What about packer?

packer does both as yaourt does.  i can see the benefit but personally, i like to keep the operations separate so aurget only handles aur (hence why i timed the two searches together to be fair).  all depends on what you're looking for.

Offline

#38 2010-01-07 22:49:01

Inxsible
Forum Fellow
From: Chicago
Registered: 2008-06-09
Posts: 9,183

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

I have aliases for yaourt and pacman, but I hardly ever use the yaourt ones. Mostly all my work is done with pacman. So I guess keeping them separate is not a big issue with me.


And would you believe it, I have 2 packages installed from aur. Of those, I am the maintainer for 1 (musca) -- so I always know if there is a newer version available wink

and the other is yaourt, which obviously will be removed, if I get something else.

Mind you, this is a 10 yr old dino laptop, so I can't go trigger happy and install anything and everything from aur either. Maybe my newer machines (if I ever get to buying one eventually) will probably have a lot more software for trying/testing/using/or just plain there for show.


I will definitely be trying out packer, aurget and bauerbill tonight and see which one fits the bill for my usage.

Last edited by Inxsible (2010-01-07 22:51:30)


Forum Rules

There's no such thing as a stupid question, but there sure are a lot of inquisitive idiots !

Offline

#39 2010-01-07 23:03:28

keenerd
Package Maintainer (PM)
Registered: 2007-02-22
Posts: 647
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

Works like a charm.  Thanks!

Offline

#40 2010-01-07 23:19:25

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

You need to add "git" to the PKGBUILD makedepends array.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#41 2010-01-08 01:46:13

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

brisbin33 wrote:

hmmm.

time (pacman -Ss firefox; aurget -s firefox)
real    0m0.669s
user    0m0.200s
sys     0m0.123s

wonder how Xyne's bauerbill stacks up.

Minor variations of 0.6 seconds are more likely accounted for by bandwidth differences

  # thanks to bruenig for this approach

especially since you have switched the script to an approach very similar to mine! smile.

However, I should say that aurget is very good and had I known about it before this, I would have probably forked that instead of writing it all myself.

Last edited by bruenig (2010-01-08 01:48:24)

Offline

#42 2010-01-08 01:57:40

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

drcouzelis wrote:

I think I found a bug. I think AUR updates are a little confused. For example, "snes9x-gtk" was updated in Nov 2009, but "packer -Syu" always says there is a new version.

I looked at the packer source code. Is there a problem in the "aurversionisnewer" function? Especially on line 149 (the grep command). Can you have parenthesis inside parenthesis?

Consider using backslashes:

aurversion="$(grep -F Version \"$tmpfile\" | cut -d \" -f 4)"

...or something. tongue

I used brisbin's solution from above to use pacman's vercmp (did not know that existed). Should work now, pushed changes.

Offline

#43 2010-01-08 02:42:43

some-guy94
Member
Registered: 2009-08-15
Posts: 360

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

bruenig wrote:
some-guy94 wrote:

I don't know where else to post this, so here's a patch that remove the need for the tmpfile, and stores everything in variables
http://pastie.org/769777

Myself and a few others spent about 2 hours in ##off-archlinux on freenode going through a whole series of different combinations of external apps (awk, sed, grep, cut), tmpfiles, variables, etc. And you may be surprised, but it was found to be significantly faster to use tmpfiles than variables on searches. It is a pain to use them, but I'd rather deal with that than slower speeds. Wait until you see the mess of a tmpfile clusterfuck on the new -Ss function now (which includes support for multiple search terms).

In that case, you might want to look at mktemp.

Offline

#44 2010-01-08 02:51:47

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

frigaut,

I pushed provides checking. It is a tricky situation in that pacman does not provide a way to simply see if X is provided for by anything that is installed. Yaourt uses a really awful grepping strategy on the pacman db that is hideously slow and even poorly written for what it is (it literally at one point does echo $(cat $file) | grep). Anyways, what happens now is: it checks whether a dependency is installed locally, if it is not, then it uses a combination of pacman -Qqs and pacman -Qi to see whether if it is provided locally. If it is either of those, it simply considers the dependency satisfied and does not include it in anything.

Could you 1) please test and 2) tell me whether this enough for provides support or if there is something else I should need to do.

Offline

#45 2010-01-08 03:37:17

drcouzelis
Member
From: Connecticut, USA
Registered: 2009-11-09
Posts: 4,092
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

I am using the latest packer. Here is my bug:

Packer thinks "snes9x-gtk" in the AUR is always out of date, and always wants to "upgrade" it. On my computer it is version 1.51.78-1. It is the same version on the AUR.

I looked at the packer function "aurversionisnewer". I added the "echo" command to see the two version numbers for snes9x-gtk. This is what it says (my modified packer output):

Package "snes9x-gtk" is out of date.
Local version is "1.51.78-1".
AUR version is "1.51.78-1 GTK+ Version of snes9x, a portable Emulator for the Super Nintento Entertainment System".

It looks like packer is parsing the AUR version number incorrectly.

Here is the snes9x-gtk AUR link: http://aur.archlinux.org/packages.php?ID=15207

Thanks!

====

Just in case you were wondering, here is my modified "aurversionisnewer" function:

aurversionisnewer() {
  getjson "http://aur.archlinux.org/rpc.php?type=info&arg=$1"
  unset aurversion
  if ! grep -Fq ':"No result found"' "$tmpfile"; then
    aurversion="$(grep -F Version "$tmpfile" | cut -d '"' -f 4)"
    if [[ "$(vercmp "$aurversion" "$2")" -eq 1  ]]; then
      echo Package \"$1\" is out of date.
      echo Local version is \"$2\".
      echo AUR version is \"$aurversion\".
      return 0
    else
      return 1
    fi
  else
    return 1
  fi
}

Last edited by drcouzelis (2010-01-08 03:37:37)

Offline

#46 2010-01-08 05:21:46

frigaut
Member
From: Canberra, Australia
Registered: 2009-05-10
Posts: 215
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

bruenig wrote:

frigaut,

I pushed provides checking. It is a tricky situation in that pacman does not provide a way to simply see if X is provided for by anything that is installed. Yaourt uses a really awful grepping strategy on the pacman db that is hideously slow and even poorly written for what it is (it literally at one point does echo $(cat $file) | grep). Anyways, what happens now is: it checks whether a dependency is installed locally, if it is not, then it uses a combination of pacman -Qqs and pacman -Qi to see whether if it is provided locally. If it is either of those, it simply considers the dependency satisfied and does not include it in anything.

Could you 1) please test and 2) tell me whether this enough for provides support or if there is something else I should need to do.

1) I have tested in a couple of different situations. It works (i.e. honor "provides") on what I tested it with. Thanks for such a quick turn around!
2) right now I can't think of anything else with respect to "provides", but I'll be back if I think of something or hit on another limitation.


Archer since 03/2009 - AUR packages

Offline

#47 2010-01-08 05:23:55

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

drcouzelis wrote:

I am using the latest packer. Here is my bug:

Packer thinks "snes9x-gtk" in the AUR is always out of date, and always wants to "upgrade" it. On my computer it is version 1.51.78-1. It is the same version on the AUR.

I looked at the packer function "aurversionisnewer". I added the "echo" command to see the two version numbers for snes9x-gtk. This is what it says (my modified packer output):

Package "snes9x-gtk" is out of date.
Local version is "1.51.78-1".
AUR version is "1.51.78-1 GTK+ Version of snes9x, a portable Emulator for the Super Nintento Entertainment System".

It looks like packer is parsing the AUR version number incorrectly.

Here is the snes9x-gtk AUR link: http://aur.archlinux.org/packages.php?ID=15207

Thanks!

====

Just in case you were wondering, here is my modified "aurversionisnewer" function:

aurversionisnewer() {
  getjson "http://aur.archlinux.org/rpc.php?type=info&arg=$1"
  unset aurversion
  if ! grep -Fq ':"No result found"' "$tmpfile"; then
    aurversion="$(grep -F Version "$tmpfile" | cut -d '"' -f 4)"
    if [[ "$(vercmp "$aurversion" "$2")" -eq 1  ]]; then
      echo Package \"$1\" is out of date.
      echo Local version is \"$2\".
      echo AUR version is \"$aurversion\".
      return 0
    else
      return 1
    fi
  else
    return 1
  fi
}

Ok I figured it out. I was grepping simply for 'Version' which was actually in the description too which meant the grep had two line hits when it should have just had one and that created problems obviously. I changed it to grep for '"Version":"' now which is more specific and how I parse it in other parts of the script. It should not ever capture anything other than the Version line in the file I parse. Sorry about that. The git is fixed. Should be able to update it.

Last edited by bruenig (2010-01-08 05:25:09)

Offline

#48 2010-01-08 05:27:32

extofme
Member
From: here + now
Registered: 2009-10-10
Posts: 174
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

this is interesting.  because pacman doesn't support hooks of any kind, i was considering writing a wrapper for pacman but this script should work fine.

i have created a BTRFS mkinitcpio hook here:

http://bbs.archlinux.org/viewtopic.php?id=88195

i wish to link it to pacman so that everytime i do a major upgrade/install/whatever pacman... or packer... automatically "snapshots" my / so that i can rollback my entire system should anything go wrong.

it would be similar to what fedora is attempting:

https://fedoraproject.org/wiki/Features … kWithBtrfs

with this ability, one could simply append their kernel boot line with "rootflags=subvol=rollback" or similar, and they could boot into their / BEFORE the upgrade was performed.  with this, anyone could safely update their system with very little risk; the only exception being kernel updates because as of right now /boot MUST be on a separate, non-btrfs partition. however, packer could simply backup vmlinuz/kernel26.img in /boot and be on its merry way.

i will keep an eye on this and probably send a patch soon.

i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot".  i dont trust people from the aur, or myself.  i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar.  --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.

also a way to disable AUR or official repos completely would be good as others has semi-mentioned.  i like to keep the two separate


what am i but an extension of you?

Offline

#49 2010-01-08 05:51:59

bruenig
Member
Registered: 2007-05-20
Posts: 175

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

extofme wrote:

this is interesting.  because pacman doesn't support hooks of any kind, i was considering writing a wrapper for pacman but this script should work fine.

i have created a BTRFS mkinitcpio hook here:

http://bbs.archlinux.org/viewtopic.php?id=88195

i wish to link it to pacman so that everytime i do a major upgrade/install/whatever pacman... or packer... automatically "snapshots" my / so that i can rollback my entire system should anything go wrong.

it would be similar to what fedora is attempting:

https://fedoraproject.org/wiki/Features … kWithBtrfs

with this ability, one could simply append their kernel boot line with "rootflags=subvol=rollback" or similar, and they could boot into their / BEFORE the upgrade was performed.  with this, anyone could safely update their system with very little risk; the only exception being kernel updates because as of right now /boot MUST be on a separate, non-btrfs partition. however, packer could simply backup vmlinuz/kernel26.img in /boot and be on its merry way.

i will keep an eye on this and probably send a patch soon.

i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot".  i dont trust people from the aur, or myself.  i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar.  --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.

also a way to disable AUR or official repos completely would be good as others has semi-mentioned.  i like to keep the two separate

I will probably add an --auronly flag or something soon for that. Obviously if one wants to disable aur stuff for a particular action, they should just use pacman. If someone is running packer as root, and they have an opportunity to edit the PKGBUILD, what can you do really? Should I hold their hand? I guess I could prompt one more time in that case and ask them to please consider dropping to an unprivileged user or to give me a username to do so or something. I don't want to be too much of a hand holder is all. I agree with you though on the makepkg as root stuff, just realize that it is their system and such.

Last edited by bruenig (2010-01-08 05:53:10)

Offline

#50 2010-01-08 05:59:26

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: packer - New bash aur+pacman wrapper (replaces poorly written yaourt)

extofme wrote:

i havent ran this yet but after looking at the source i see a few things, but the one that jumped at me is the "makepkg --asroot".  i dont trust people from the aur, or myself.  i would recommend testing the working directory uid/gid, and trying to drop privileges, or asking the user to give a username to drop to or something similar.  --asroot could be bad bad... while developing PKGBUILDs for my own internal use i nearly botched a system or two with it.

also a way to disable AUR or official repos completely would be good as others has semi-mentioned.  i like to keep the two separate

Sorry, I really don't want to promote a "competitor" in this thread, but I have to mention that Bauerbill does exactly that. If it detects that it's running as root, it will change to a specified user before building packages (see the BuildAs/--build-as option... it prompts for a user if that isn't set though). It also only enables AUR support when the "--aur" option is passed. Building repo packages from source can be enabled separately.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

Board footer

Powered by FluxBB