cower and burp: fast and simple AUR {up,down}loading

falconindy · 2010-12-31 13:50:44

Right. It is related. It was an intentional change, but it appears that slow connections to the AUR are suffering from it. You can either keep both the timeout and the nosignal line as you have them, or remove them both. Either way is safe. I've also committed a change last night to -git that provides a --timeout flag. Setting this to 0 disables timeouts.

I'll be backporting the NOSIGNAL patch to 2.9.9, but nothing else is broken here, persay.

falconindy · 2011-03-16 20:09:50

3.0.0 released in celebration of pacman 3.5's release! Lots of bug fixes, and a few new features snuck in since the last tag...

Bugfixes
* Fixed an occasional crash when using SSL connections during large updates
* Fixed an occasional crash when specifying multiple updates
* Fixed regex handling to avoid over filtering results
* Added locale recognition for proper printing of utf-8 character

Features
* --format lets you custom format output from -m, -s, and -i (very similar to expac)
* --ignorerepo allows skipping a binary repo from being searched for packages during updates
* added --nossl option
* added a config file for a few persistent per user settings (see /usr/share/cower/config for details)

Some people may also notice a small speed improvement. On a technical level, I used to allocate a curl handle per job, and ignored the idea of recycling them. That behavior has been changed and now each thread maintains its own curl handle which is just cleared between jobs. This results in fewer DNS requests and snappier responses on large updates and -ii queries.

Also note that this is somewhat of a hybrid build. It will build against pacman 3.4.3 as well as 3.5. Please make sure to recompile if you upgrade to testing after building cower!

cower 3.0.0
Usage: cower <operations> [options] target...

 Operations:
  -d, --download          download target(s) -- pass twice to download AUR dependencies
  -i, --info              show info for target(s) -- pass twice for more detail
  -m, --msearch           show packages maintained by target(s)
  -s, --search            search for target(s)
  -u, --update            check for updates against AUR -- can be combined with the -d flag

 General options:
  -f, --force             overwrite existing files when downloading
  -h, --help              display this help and exit
      --ignore <pkg>      ignore a package upgrade (can be used more than once)
      --ignorerepo <repo> ignore a binary repo (can be used more than once)
      --nossl             do not use https connections
  -t, --target <dir>      specify an alternate download directory
      --threads <num>     limit number of threads created
      --timeout <num>     specify connection timeout in seconds

 Output options:
  -c, --color[=WHEN]      use colored output. WHEN is `never', `always', or `auto'
      --debug             show debug output
      --format <string>   print package output according to format string
      --listdelim <delim> change list format delimeter
  -q, --quiet             output less
  -v, --verbose           output more

JokerBoy · 2011-03-16 20:19:17

http://pastie.org/1679919

Last edited by JokerBoy (2011-03-16 20:31:00)

falconindy · 2011-03-16 20:25:41

Nope. just requires quotes aroound $PMCHECK in make.

JokerBoy · 2011-03-16 20:30:12

falconindy wrote:

quotes aroound $PMCHECK

what the.. i tried this & boom: the same error. i deleted the dir with cower PKGBUILD and redownloaded it again with your edit. voila, now it works. again, what the..

Last edited by JokerBoy (2011-03-16 20:30:43)

skottish · 2011-03-22 01:47:27

falconindy,

This is great. I'd been using Xyne's tools for quite a while. After he moved on, I needed something to check on package upgrades in AUR and to pull AUR tarballs down. I'm very impressed with the speed and clean UI. Very, very nice.

Thank you.

JokerBoy · 2011-03-23 11:38:37

seems to be something wrong with the verbose output.

cower --color=auto --target /home/bogdan/pkgbuilds --timeout 30 -vu

1.

:: :: Checking cower for updates...
Checking cairo-ubuntu for updates...
:: Checking dropbox for updates...
:: Checking dropbox-servicemenu for updates...
:: Checking flashplugin-prerelease for updates...

2.

:: :: :: Checking dropbox for updates...
:: Checking dropbox-servicemenu for updates...
Checking cairo-ubuntu for updates...
Checking cower for updates...
:: Checking flashplugin-prerelease for updates...

falconindy · 2011-03-23 12:24:50

Yes, that's been around since I rewrote cower with threads. I know of a few possible solutions...

edit: fixed

Last edited by falconindy (2011-03-23 13:01:41)

dolby · 2011-03-24 02:33:44

Since cower moved to community and then back at the AUR, the link of the stable version on the first post of this thread is wrong. It should be http://aur.archlinux.org/packages.php?ID=44921

falconindy · 2011-03-24 02:39:07

dolby wrote:

Since cower moved to community and then back at the AUR, the link of the stable version on the first post of this thread is wrong. It should be http://aur.archlinux.org/packages.php?ID=44921

Good point! Thanks, updated. Also, with pacman 3.5.1 moving to core, I've tagged cower 3.0.1 which contains a few minor bug fixes.

Last edited by falconindy (2011-03-24 02:39:50)

trapanator · 2011-03-24 06:56:52

I cannot compile cower. Here is the error:

cc -c -march=x86-64 -mtune=generic -O2 -pipe --std=c99 -g -pedantic -Wall -Wextra -Werror -DCOWER_VERSION=\"3.0.1\" cower.c
cc1: warnings being treated as errors
cower.c: In function ‘alpm_provides_pkg’:
cower.c:430:5: error: implicit declaration of function ‘alpm_find_satisfier’
cower.c: In function ‘resolve_dependencies’:
cower.c:1567:28: error: initialization makes pointer from integer without a cast
make: *** [cower.o] Error 1

falconindy · 2011-03-24 10:00:08

trapanator wrote:

I cannot compile cower. Here is the error:

cc -c -march=x86-64 -mtune=generic -O2 -pipe --std=c99 -g -pedantic -Wall -Wextra -Werror -DCOWER_VERSION=\"3.0.1\" cower.c
cc1: warnings being treated as errors
cower.c: In function ‘alpm_provides_pkg’:
cower.c:430:5: error: implicit declaration of function ‘alpm_find_satisfier’
cower.c: In function ‘resolve_dependencies’:
cower.c:1567:28: error: initialization makes pointer from integer without a cast
make: *** [cower.o] Error 1

Please update pacman.

JokerBoy · 2011-03-24 10:09:12

falconindy wrote:

Yes, that's been around since I rewrote cower with threads. I know of a few possible solutions...
edit: fixed

thanks

some notification for local packages newer than those in aur would be great also.

┌─[bogdan@pc][~]
└─╼ cower -vu pacman-color
:: Checking pacman-color for updates...
┌─[bogdan@pc][~]
└─╼ cower -ii pacman-color
Repository     : aur
Name           : pacman-color [installed]
Version        : 3.4.3-1
URL            : http://www.archlinux.org/pacman/
AUR Page       : https://aur.archlinux.org/packages.php?ID=11827
Depends On     : pacman=${pkgver}
Makedepends    : patch
Category       : system
License        : GPL
Votes          : 1600
Out of Date    : Yes
Description    : Command-line frontend for libalpm aka pacman with color patch

┌─[bogdan@pc][~]
└─╼ pacman -Q pacman-color
pacman-color 3.5.1-1

falconindy · 2011-03-24 10:13:33

JokerBoy wrote:

falconindy wrote:

Yes, that's been around since I rewrote cower with threads. I know of a few possible solutions...
edit: fixed

thanks

some notification for local packages newer than those in aur would be great also.

┌─[bogdan@pc][~]
└─╼ cower -vu pacman-color
:: Checking pacman-color for updates...
┌─[bogdan@pc][~]
└─╼ cower -ii pacman-color
Repository     : aur
Name           : pacman-color [installed]
Version        : 3.4.3-1
URL            : http://www.archlinux.org/pacman/
AUR Page       : https://aur.archlinux.org/packages.php?ID=11827
Depends On     : pacman=${pkgver}
Makedepends    : patch
Category       : system
License        : GPL
Votes          : 1600
Out of Date    : Yes
Description    : Command-line frontend for libalpm aka pacman with color patch

┌─[bogdan@pc][~]
└─╼ pacman -Q pacman-color
pacman-color 3.5.1-1

This is going to be chaotic with VCS apps, which are 99% of the time newer than what's in the AUR.

Last edited by falconindy (2011-03-24 10:21:43)

ngoonee · 2011-03-25 03:06:42

Just testing this out, why are the packages/package directories untarred with 777 permissions? Seems a very KISS solution, hoping to replace bauerbill

falconindy · 2011-03-25 03:16:34

Wait, all packages are untarred as 777? I know there's some oddly bundled packages on the AUR that were uploaded with 777 permissions, but this shouldn't be all of them. Admittedly, I'm not setting a umask anywhere in the download thread, but that shouldn't be an issue given that I'm just telling libarchive to unpack the taurballs with permissions preserved.

Last edited by falconindy (2011-03-25 03:18:56)

ngoonee · 2011-03-25 03:55:16

falconindy wrote:

Wait, all packages are untarred as 777? I know there's some oddly bundled packages on the AUR that were uploaded with 777 permissions, but this shouldn't be all of them. Admittedly, I'm not setting a umask anywhere in the download thread, but that shouldn't be an issue given that I'm just telling libarchive to unpack the taurballs with permissions preserved.

Ah, this only happens with unetbootin (which was the first package I tested with at that time, have now tested with all my packages and it seems only that one is affected). Probably just the uploader not using makepkg --source?

falconindy · 2011-03-25 03:59:24

Yeah, pretty much.

ngoonee · 2011-03-25 04:27:05

Thanks falconindy, will continue using this on my testing machine for a while. 'targetdir' is the main feature I was looking for .

mhertz · 2011-03-25 08:31:55

@falconindy

sorry for "stupid" question, but could you kindly tell me why you prefer to not let the app install the packages, instead of just downloading the pkgbuilds?

I believe it's for if wanting to modify the pkgbuild maybe, but then again, the pkgbuild could just be showed for editing optionally before the install?

I'm not trying to dismis your project at all, but i'm just really interested in why you and your users feel that this is the way to go for e.g. updating aur packages?

Honestly, then i'm asking because I would love to use a no BS kiss compiled(C) app for doing aur upgrades, and so cower intrueges me tremendously, but i'm just simply to dumb it seems to really understand it's designing philosophy, or however it's formulated in english...

Thanks in advance and I hope i'm not polutting your thread to much with this one stupid question

falconindy · 2011-03-25 10:13:32

I think its a fair question. Call it division of church and state, and following the unix philosophy. To expand on that...

- There's already a fantastic tool to build packages -- its called makepkg. Nothing is going to replace this, and any "full-featured" AUR helper that automates the process just shells out to makepkg and reads its return code to determine success or failure.
- Building a package from untrusted sources weighs in with a fair bit of responsibility on the user's part. cower is strictly a read only app aside from downloading, and I think this emphasis is important. Other apps prompt you to read/change the PKGBUILD, but I suspect a lot of people don't realize the importance of this step and breeze by it.
- I maintain a sizeable directory with all the foreign packages I build and install, with very few of them being identical to the PKGBUILDs in the AUR. While I try and submit back any useful changes to the maintainer, they don't always accept them, and they're not always appropriate. If I already have these scripts collected in one place, all I need is a tool to diff new PKGBUILDs against the old one (and I have such a tool), and then merge in appropriate changes and rebuild. Not something you can cleanly do with tools such as clyde or packer.

To that end, it's possible that somewhere off in the future I may build out some additional functionality in cower to allow for a wrapper written in some other language than C to tie together cower with makepkg. I make no promises that this tool will be created by my hands.

edit: since I received an email inquiring about this magical aur diff tool, I'll mention it here as well. It's not magical at all: https://github.com/falconindy/bin-scrip … er/aurdiff

Last edited by falconindy (2011-03-25 12:05:11)

mhertz · 2011-03-25 12:21:23

Ahh, I now see the intended use-case sceneario and your design goals much clearer now, and I thank you for the nice detailed explenation!

Much appreciated and thanks again!

CU, Martin.

ngoonee · 2011-03-25 20:59:59

falconindy wrote:

- I maintain a sizeable directory with all the foreign packages I build and install, with very few of them being identical to the PKGBUILDs in the AUR. While I try and submit back any useful changes to the maintainer, they don't always accept them, and they're not always appropriate. If I already have these scripts collected in one place, all I need is a tool to diff new PKGBUILDs against the old one (and I have such a tool), and then merge in appropriate changes and rebuild. Not something you can cleanly do with tools such as clyde or packer.

bauerbill has (used to have) a patch option, where patches to the packages would be stored in another abs-tree-like structure, and you could also set it to auto-patch on update. I used that heavily at one point, but ended up figuring that maintaining my own patched versions in the AUR made more sense.

litemotiv · 2011-03-25 22:34:35

Obama was wrong, cower is cool.

jOaNbE · 2011-03-29 08:40:49

Thank you very much for your efforts with this solid tool. It does exactly what I want (no need for colors though) with a very small system footprint. I breezed through the source code and I must say I envy your skills - that's some real clean code.

Arch Linux

#51 2010-12-31 13:50:44

Re: cower and burp: fast and simple AUR {up,down}loading

#52 2011-03-16 20:09:50

Re: cower and burp: fast and simple AUR {up,down}loading

#53 2011-03-16 20:19:17

Re: cower and burp: fast and simple AUR {up,down}loading

#54 2011-03-16 20:25:41

Re: cower and burp: fast and simple AUR {up,down}loading

#55 2011-03-16 20:30:12

Re: cower and burp: fast and simple AUR {up,down}loading

#56 2011-03-22 01:47:27

Re: cower and burp: fast and simple AUR {up,down}loading

#57 2011-03-23 11:38:37

Re: cower and burp: fast and simple AUR {up,down}loading

#58 2011-03-23 12:24:50

Re: cower and burp: fast and simple AUR {up,down}loading

#59 2011-03-24 02:33:44

Re: cower and burp: fast and simple AUR {up,down}loading

#60 2011-03-24 02:39:07

Re: cower and burp: fast and simple AUR {up,down}loading

#61 2011-03-24 06:56:52

Re: cower and burp: fast and simple AUR {up,down}loading

#62 2011-03-24 10:00:08

Re: cower and burp: fast and simple AUR {up,down}loading

#63 2011-03-24 10:09:12

Re: cower and burp: fast and simple AUR {up,down}loading

#64 2011-03-24 10:13:33

Re: cower and burp: fast and simple AUR {up,down}loading

#65 2011-03-25 03:06:42

Re: cower and burp: fast and simple AUR {up,down}loading

#66 2011-03-25 03:16:34

Re: cower and burp: fast and simple AUR {up,down}loading

#67 2011-03-25 03:55:16

Re: cower and burp: fast and simple AUR {up,down}loading

#68 2011-03-25 03:59:24

Re: cower and burp: fast and simple AUR {up,down}loading

#69 2011-03-25 04:27:05

Re: cower and burp: fast and simple AUR {up,down}loading

#70 2011-03-25 08:31:55

Re: cower and burp: fast and simple AUR {up,down}loading

#71 2011-03-25 10:13:32

Re: cower and burp: fast and simple AUR {up,down}loading

#72 2011-03-25 12:21:23

Re: cower and burp: fast and simple AUR {up,down}loading

#73 2011-03-25 20:59:59

Re: cower and burp: fast and simple AUR {up,down}loading

#74 2011-03-25 22:34:35

Re: cower and burp: fast and simple AUR {up,down}loading

#75 2011-03-29 08:40:49

Re: cower and burp: fast and simple AUR {up,down}loading

Board footer