cower and burp: fast and simple AUR {up,down}loading

lolilolicon · 2011-04-09 06:03:31

cower has evolved quite much since last time I checked. Good job falconindy.

Some features I'd like to see:

Option to sort --search and --msearch results by number of votes (descendent).
Add an indicator to signal "Out of Date" in search results, when color's off. Maybe just a simple * or [x] at the end.
For --msearch option, print only "Out of Date" packages, making life easier for AUR package maintainers.

These are what I consider useful options (which I implemented in dzlad back when learning a bit of Ruby).

Thanks for your work.

Mad Fish · 2011-04-09 13:58:21

Bug report: cower doesn't respect http_proxy environment variable.

falconindy · 2011-04-09 14:49:40

Mad Fish wrote:

Bug report: cower doesn't respect http_proxy environment variable.

Closed: not a bug.

connections by default use https, so you need to https_proxy or all_proxy. Alternatively, use --nossl or set NoSSL in the config.

falconindy · 2011-04-09 14:54:33

lolilolicon wrote:

cower has evolved quite much since last time I checked. Good job falconindy.
Some features I'd like to see:
Option to sort --search and --msearch results by number of votes (descendent).
Add an indicator to signal "Out of Date" in search results, when color's off. Maybe just a simple * or [x] at the end.
For --msearch option, print only "Out of Date" packages, making life easier for AUR package maintainers.
These are what I consider useful options (which I implemented in dzlad back when learning a bit of Ruby).
Thanks for your work.

Noted. The second I'm definitely interested in fixing. The first and the third can already be done with the --format flag and some other unix tools, so I'm not convinced I'll be adding those.

lolilolicon · 2011-04-09 15:42:46

falconindy wrote:

...can already be done with the --format flag and some other unix tools...

Yes, --format is indeed a good idea! Thanks for the tip.
Before you officially implement any of my requests, I can just use this hack:

search_sort_by_votes()
{ 
    cower -s "${1}" --format='aur/%n %v (%o)%t%%%d\n' | sort -nr -k2 -t'(' | sed 's/no%/\n    /;s/yes%/ [x]\n    /'
}
list_flagged_pkgs() 
{ 
    cower -m "${1}" --format='%n %t\n' | grep --color=auto ' yes$' | cut -d\  -f1
}

Looking forward to seeing request 2 implemented in the next release

falconindy · 2011-04-09 21:30:21

lolilolicon wrote:

Looking forward to seeing request 2 implemented in the next release

Done. I'd like to tag a new release soon anyways, since -git currently has proper and'ed searching. Maybe by the end of the weekend, depending on how work goes tomorrow.

Last edited by falconindy (2011-04-09 21:30:41)

falconindy · 2011-04-10 21:55:26

It's tagtacular sunday. Hrmm.. that would have been a lot more awesome on Tuesday. Anyways, new versions of burp and cower! Mostly bug fixes and some maintenance items...

Highlights

cower 3.0.2:

Searching is now and'ed, similar to how pacman works. Results will match all provided search terms.
A small tag <!> will be displayed when operating with color to denote an out of date package (thanks lolilolicon).
Some minor bug fixes and tweaks along the way as a result of the endless stream of questionable Bash that I unearth in the AUR.

burp 1.6.3

Nothing to see here. Just a lot of refactoring of old, bad code. Fixed a couple of bugs and a crash, too.

Enjoy! As always, bug reports can be posted on Github, or you might catch me on IRC in #archlinux.

skottish · 2011-04-15 01:16:51

I apologize if this has been answered here, but long threads are a pain if search terms are too general.

bauerbill had the --aur-only switch was very useful in cases like cdrtools where cdrkit has cdrtools as a provides line. The options listed in the man page for cower don't seem to cover this in any convenient way. Either I'm missing something or this is a feature request.

Last edited by skottish (2011-04-15 01:17:20)

falconindy · 2011-04-15 01:23:57

Whoa... that's fairly inconsistant on my part. 'cower -i cdrtools' will indeed show cdrtools in the AUR, but -d fails to grab it. I should be doing a direct query on the binary repos, not using a resolver.

In the meantime, you can get around this behavior by using 'cower -d cdrtools --ignorerepo extra'

skottish · 2011-04-15 01:30:58

falconindy wrote:

Whoa... that's fairly inconsistant on my part. 'cower -i cdrtools' will indeed show cdrtools in the AUR, but -d fails to grab it. I should be doing a direct query on the binary repos, not using a resolver.
In the meantime, you can get around this behavior by using 'cower -d cdrtools --ignorerepo extra'

Thanks for the quick reply.

I saw that option and it's not a big deal to use because I know where things are coming from, but it would be cool to not have to think about it. cdrtools is one of the few AUR packages that I use that I haven't rewritten for my needs, so this is an edge case where I discover these things.

falconindy · 2011-04-15 02:12:29

I realize now why cower behaves as it does for downloads, and it's a bit of a tricky situation. Instead, I think I'm going to change --ignorerepo to make the argument optional. If it's not specified, then all binary repos are ignored. This is only true of the command line option, and the config file option 'IgnoreRepo' remains unchanged. The command line opt will, of course, prevail over the config file.

Give me some time to test this to make sure I'm not breaking anything else in the process and I'll push it out.

edit: pushed

Last edited by falconindy (2011-04-15 12:39:32)

skottish · 2011-04-20 02:20:19

The curl upgrade in testing has seemed to caused some issues. I don't know enough about either to suggest where the problem lies:

~ > cower -u
error: curl: Server returned nothing (no headers, no data)

The error message is repeated.

Last edited by skottish (2011-04-20 02:23:31)

falconindy · 2011-04-20 02:26:31

hrmmm, cannot reproduce? That's really weird -- everything is kosher here. The only release day bug in curl was curl-config not reporting version correctly, but that definitely doesn't affect us. There was no soname bump either. Can you post an strace?

ngoonee · 2011-04-20 02:34:13

Works fine here as well, perhaps connectivity issue?

skottish · 2011-04-20 03:07:49

Another local phantom... I'll do some research and report my findings.

skottish · 2011-04-20 03:27:31

falconindy wrote:

hrmmm, cannot reproduce? That's really weird -- everything is kosher here. The only release day bug in curl was curl-config not reporting version correctly, but that definitely doesn't affect us. There was no soname bump either. Can you post an strace?

Whoops, of course:

http://pastebin.com/AtpkcBVM

falconindy · 2011-04-20 03:32:55

skottish wrote:

falconindy wrote:
hrmmm, cannot reproduce? That's really weird -- everything is kosher here. The only release day bug in curl was curl-config not reporting version correctly, but that definitely doesn't affect us. There was no soname bump either. Can you post an strace?
Whoops, of course:
http://pastebin.com/AtpkcBVM

Boring, since I didn't ask for strace -f (follow threads). Not sure it's going to yield anything anyways, but thanks for humoring me. If it happens again, keep something else nearby to see if maybe it's the AUR being silly, e.g.

aget () { 
    for pkg in "$@"; do
        if curl -s "https://aur.archlinux.org/packages/$pkg/$pkg.tar.gz" | tar xz 2> /dev/null; then
            echo ":: downloaded $pkg"
        else
            echo ":: $pkg not found"
        fi
    done
}

skottish · 2011-04-20 04:13:05

http instead of https, no?

falconindy · 2011-04-20 10:01:23

Well, adjust to your liking. If you're using --nossl with cower, then change it to http.

falconindy · 2011-04-20 21:30:41

@skottish: are you behind a proxy? Found this on the curl ML just now

https://github.com/bagder/curl/commit/c … b1358a5988

and reported on our end: https://bugs.archlinux.org/task/23863. This is definitely the cause -- I fired up squid and can recreate this. Patched curl works as expected.

Last edited by falconindy (2011-04-20 23:51:49)

skottish · 2011-04-21 02:02:35

falconindy wrote:

@skottish: are you behind a proxy? Found this on the curl ML just now
https://github.com/bagder/curl/commit/c … b1358a5988
and reported on our end: https://bugs.archlinux.org/task/23863. This is definitely the cause -- I fired up squid and can recreate this. Patched curl works as expected.

Beautiful. If I were to find out that yet another program that I use was doomed to brokenness by the world not cooperating with proxies (privoxy here), I would be forced barf violently. You stopped my impending violent barfing. I owe you a debt of gratitude my friend.

Thanks.

rekado · 2011-04-23 02:08:29

Just chiming in to say that I really enjoy using cower. I often find myself editing packages from the AUR, so cower with its clear restriction to downloading from the AUR is the best tool for the job. I never liked how pacman wrappers blurred the lines between the AUR and the official repos (as you can easily fall into the trap of dismissing warnings).
Thanks for sharing!

falconindy · 2011-04-26 15:07:28

FYI, cower 3.0.3 is in the AUR to build against yajl 2.0.0. However, there's an ugly bug in yajl that pops up when the json parser receives no input. In cower, this means when curl hits a timeout and has no response JSON. Not everyone will hit this, but I suggest lowering your thread usage (below 10) until this gets fixed upstream.

I've a bug report in upstream.

update: Bug is fixed in yajl 2.0.0-2, and I've pushed a small adjustment for cower-git as well.

Last edited by falconindy (2011-04-26 16:12:06)

anonymous_user · 2011-07-30 14:01:58

When I use cower to check AUR packages for updates, I get a lot of "Couldn't resolve host name" and "Timeout was reached" messages. After troubleshooting a bit, the cause seems to be the (high?) number of threads used by cower. I tweaked the cower config to have "MaxThreads = 1" (2 works too) and the errors went away.

I suppose this is why having a user config exists, however I have to ask if the default value works for everyone else (thus the problem is at my end) or if anyone else experienced this.

falconindy · 2011-07-30 14:25:46

I've heard this before, but not frequently enough for me to be inclined to change it. This isn't something that you encounter often; it should, realistically, be only when checking updates. cower is smart enough to use less than --threads if it doesn't need to.

Arch Linux

#76 2011-04-09 06:03:31

Re: cower and burp: fast and simple AUR {up,down}loading

#77 2011-04-09 13:58:21

Re: cower and burp: fast and simple AUR {up,down}loading

#78 2011-04-09 14:49:40

Re: cower and burp: fast and simple AUR {up,down}loading

#79 2011-04-09 14:54:33

Re: cower and burp: fast and simple AUR {up,down}loading

#80 2011-04-09 15:42:46

Re: cower and burp: fast and simple AUR {up,down}loading

#81 2011-04-09 21:30:21

Re: cower and burp: fast and simple AUR {up,down}loading

#82 2011-04-10 21:55:26

Re: cower and burp: fast and simple AUR {up,down}loading

#83 2011-04-15 01:16:51

Re: cower and burp: fast and simple AUR {up,down}loading

#84 2011-04-15 01:23:57

Re: cower and burp: fast and simple AUR {up,down}loading

#85 2011-04-15 01:30:58

Re: cower and burp: fast and simple AUR {up,down}loading

#86 2011-04-15 02:12:29

Re: cower and burp: fast and simple AUR {up,down}loading

#87 2011-04-20 02:20:19

Re: cower and burp: fast and simple AUR {up,down}loading

#88 2011-04-20 02:26:31

Re: cower and burp: fast and simple AUR {up,down}loading

#89 2011-04-20 02:34:13

Re: cower and burp: fast and simple AUR {up,down}loading

#90 2011-04-20 03:07:49

Re: cower and burp: fast and simple AUR {up,down}loading

#91 2011-04-20 03:27:31

Re: cower and burp: fast and simple AUR {up,down}loading

#92 2011-04-20 03:32:55

Re: cower and burp: fast and simple AUR {up,down}loading

#93 2011-04-20 04:13:05

Re: cower and burp: fast and simple AUR {up,down}loading

#94 2011-04-20 10:01:23

Re: cower and burp: fast and simple AUR {up,down}loading

#95 2011-04-20 21:30:41

Re: cower and burp: fast and simple AUR {up,down}loading

#96 2011-04-21 02:02:35

Re: cower and burp: fast and simple AUR {up,down}loading

#97 2011-04-23 02:08:29

Re: cower and burp: fast and simple AUR {up,down}loading

#98 2011-04-26 15:07:28

Re: cower and burp: fast and simple AUR {up,down}loading

#99 2011-07-30 14:01:58

Re: cower and burp: fast and simple AUR {up,down}loading

#100 2011-07-30 14:25:46

Re: cower and burp: fast and simple AUR {up,down}loading

Board footer