Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Barghest · 2010-08-17 17:15:13

It seems to be fixed...thx

Primoz · 2010-08-18 21:23:49

I was sure I had a similar problem before, but I haven't found anything like that in this or any other thread.
When I try to build the newest kernel26-ck it unpacks it nicely and starts to patch it, but then I get this error:

    Aborting...
ERROR: makepkg exited with an error (512)
WARNING: expected package does not exist: /tmp/bauerbill/build/aur/kernel26-ck/kernel26-ck-2.6.35.2-1-x86_64.pkg.tar.xz
--> scanning /tmp/bauerbill/build/aur/kernel26-ck for matching packages...
ERROR: no package found: /tmp/bauerbill/build/aur/kernel26-ck/kernel26-ck-2.6.35.2-1-x86_64.pkg.tar.xz
:: Abort? [Y/n] y

I asked on AUR page for kernel26-ck and they siad it has to be Bauerbill related. But I have no idea what it could be. It worked when the last update of kernel26-ck was made.
I don't see what the difference is. But for some reason it doesn't find the package to build...

Any ideas what coud be happening?

Xyne · 2010-08-18 21:53:00

Primoz wrote:

    Aborting...
ERROR: makepkg exited with an error (512)

The error is occurring in makepkg. Look for the cause further up in the output of makepkg and try running makepkg in the build directory to see if that works.

I haven't tested it here but I doubt that it's a bauerbill error if makepkg is aborting.

skottish · 2010-08-19 02:17:58

Xyne wrote:

I see an easy way to avoid the error by grepping for defined values on that line, but I want to understand why it's uninitialized in the first place.
I've added some checks further up in the code. Update perl-xyne-arch from my site and let me know if it resolves the error. If not, do you get the same error when running stand-alone reflector? What's the full output, including the mirrorlist itself?

Sorry for the slow response. This pesky life thing keeps getting in the way of fun stuff...

I'm no longer seeing the issue. Thanks Xyne.

k3ttc4r · 2010-08-19 18:18:06

Bauerbill is great. But I recently discovered that it uses 100% of my CPU (a 2GHz Duo) when doing a -Qu. I am suspecting that it might even be the cause of some apparently random system freezes I have been experiencing. During these, my computer becomes completely unresponsive for a couple of minutes, killing several programs, until it calmes down after a while.

I had a -SyQu printing the output to a text file in my hourly cron. I have removed it again, and I'm waiting to see if the freezes stop now. While such a freeze doesn't happen every hour, I cannot think of anything else that could be causing this, especially after I checked what a -Qu did to my CPU...

Xyne · 2010-08-20 02:22:15

@k3ttc4r
It's an issue with Perl threading. Each Perl thread copies everything in memory, which is why it uses a ridiculous amount of memory and CPU, depending on how many threads it needs.

I'm (slowly) working on an alternative approach, but for now I've made some changes to parallelization in perl-xyne-common and perl-xyne-arch. I've uploaded them to the repo on my site and will push them to [community] after some more testing.

Quick changelog:
* added "--threads <n>" option: <n> is an integer where 0 (and 1) disables threading, >=2 limits threads, <0 uses aggressive threading (applies to powerpill, bauerbill and whatever else subclasses powerpill)
* replaced threading with forking where return values can be ignored
* restructured code to make better use of worker threads

Test it with different values of <n> and let me know if it helps. On my system "bauerbill --threads 0 --aur -Qu" uses much less memory and nearly no cpu, but it takes noticeably longer. With 54 foreign packages, I'm getting the best speed around 10 threads, which I've set as the default for now.

When I update perl-xyne-*, I'll update the powerpill and bauerbill front-end packages to include new man pages and configuration files, but you can already add "Threads = <n>" to the existing configuration files.

Last edited by Xyne (2010-08-20 02:23:35)

dapolinario · 2010-08-20 03:20:57

"--rebase" is not working.

Xyne · 2010-08-20 06:03:22

Please update and try again.

k3ttc4r · 2010-08-20 06:39:34

Thank you Xyne.

I've just tried using --threads, and I noticed that if I use a value other than 10, it doesn't hog my CPU anymore, but that's probably because I am also not getting any upgradable AUR packages anymore, either, only those from the repos.

ngoonee · 2010-08-20 09:19:52

Just a note that adding user-agent=wget made a sourceforge download fail (package texlipse). Previously been using it with no issues. Not a big deal, just thought some may like to know.

Xyne · 2010-08-20 11:02:16

k3ttc4r wrote:

Thank you Xyne.
I've just tried using --threads, and I noticed that if I use a value other than 10, it doesn't hog my CPU anymore, but that's probably because I am also not getting any upgradable AUR packages anymore, either, only those from the repos.

If you mean that you all of your AUR packages are up-to-date, then that doesn't matter. It still needs to query the server to check if an update is available, so the lower CPU usage is definitely due to the code restructuring.

ngoonee wrote:

Just a note that adding user-agent=wget made a sourceforge download fail (package texlipse). Previously been using it with no issues. Not a big deal, just thought some may like to know.

Huh? Do you mean the aria2c user-agent? If so then it's either an Aria2 issue or a SourceForge issue.

ngoonee · 2010-08-20 12:05:41

Xyne wrote:

ngoonee wrote:
Just a note that adding user-agent=wget made a sourceforge download fail (package texlipse). Previously been using it with no issues. Not a big deal, just thought some may like to know.
Huh? Do you mean the aria2c user-agent? If so then it's either an Aria2 issue or a SourceForge issue.

Precisely, am only mentioning it here because about 5-10 pages back there was discussion on using it for bauerbill.

Xyne · 2010-08-20 12:18:48

ngoonee wrote:

Xyne wrote:
Huh? Do you mean the aria2c user-agent? If so then it's either an Aria2 issue or a SourceForge issue.
Precisely, am only mentioning it here because about 5-10 pages back there was discussion on using it for bauerbill.

Ah, ok, thanks.

Most posts in this thread come with expectations that I do something, so I wasn't sure that it was purely informational.

ngoonee · 2010-08-20 14:37:02

Xyne wrote:

ngoonee wrote:
Xyne wrote:
Huh? Do you mean the aria2c user-agent? If so then it's either an Aria2 issue or a SourceForge issue.
Precisely, am only mentioning it here because about 5-10 pages back there was discussion on using it for bauerbill.
Ah, ok, thanks.
Most posts in this thread come with expectations that I do something, so I wasn't sure that it was purely informational.

Ah yes, that seems obvious now that you mention it sorry for the confusion.

And I've said it more than once before, but bauerbill is awesome! I'm almost forgetting how to touch-type 'pacman'

k3ttc4r · 2010-08-22 04:42:39

Xyne wrote:

k3ttc4r wrote:
Thank you Xyne.
I've just tried using --threads, and I noticed that if I use a value other than 10, it doesn't hog my CPU anymore, but that's probably because I am also not getting any upgradable AUR packages anymore, either, only those from the repos.
If you mean that you all of your AUR packages are up-to-date, then that doesn't matter. It still needs to query the server to check if an update is available, so the lower CPU usage is definitely due to the code restructuring.

No. I did have upgradable packages, but they didn't show up on a "-Qu --threads=5" (or any other number other than 10). But they did show up when I used 10, though.

Anyways, I just upgraded your perl packages, and it works like a charm now, thanks

tri1976 · 2010-08-28 00:35:52

bauerbill doesn't seem to read the config file $XDG_CONFIG_HOME/powerpill/bauerbill.conf. I have editor path specified here as "Editor = /usr/bin/nano" but bauerbill always error "no valid editor specified"

Xyne · 2010-08-28 01:10:53

tri1976 wrote:

bauerbill doesn't seem to read the config file $XDG_CONFIG_HOME/powerpill/bauerbill.conf. I have editor path specified here as "Editor = /usr/bin/nano" but bauerbill always error "no valid editor specified"

This came up on the AUR page:

You're probably running it as root so it's checking root's $XDG_CONFIG_HOME/powerpill/bauerbill.conf and $HOME/.config/powerpill/bauerbill.conf. Here's a script that you can use to see which user it detects and which paths it checks:
http://aur.pastebin.com/WcJXsnuG
Either move the configuration file to where it should be or use the "--config" option to specify the file that you want to use.

tri1976 · 2010-08-28 01:23:00

"--config" did the trick. Thank you.

pho · 2010-09-06 11:10:59

rubyforge and sourceforge downloads fail for me now with bauerbill / aria2
Is there a way to temporary disable the aria2 and use wget instead?
Cause the timeout just didn't work for the downloads and it keeps at 0kb/s and download nothing but just hangs there

Xyne · 2010-09-06 11:39:14

@pho
Restructuring the code to support other download managers would be a pain and I don't have the time right now. Maybe you could use the "--save-metalink" option with a script to extract the URLs and use wget (or whatever you like) to finish the download.

You could also try setting a minimum download speed for aria2 to see if that helps. *edit: It may also be that SourceForge and RubyForge reject segmented downloads... try disabling them via the aria2c arguments to see if that works*

If not, double-check that it's correctly parsing the download link from the PKGBUILD (inspect the metalink using "--save-metalink" or "--get-metalink"). If it isn't, edit the PKGBUILD and simplify it. If the link is correct, contact the aria2 developer and maybe even someone at those sites to see if they discriminate against certain user agents.

pho · 2010-09-07 14:55:26

ok filed a bug to aria2:
https://sourceforge.net/tracker/?func=d … tid=813673

jdarnold · 2010-09-17 22:04:55

I had this problem before and it got fixed, but it seems to have gotten broken. I have the following for an editor:

Editor = /usr/bin/emacsclient --socket-name /tmp/emacs1000/server

in /etc/bauerbill.conf. It used to work (I need the socket because it runs as root and my emacs runs as my normal user), but now I get this error:

sh: /usr/bin/emacsclient --socket-name /tmp/emacs1000/server: No such file or directory

so whatever is spawning the editor is looking for the full editor line as the command and isn't finding it (obviously). So perhaps it changed from "shell(editor-lien)" to "exec(editor-line)". Not sure of the workaround.

Xyne · 2010-09-17 23:54:43

@jdarnold
It should be fixed now, but the source of the problem has been there for a long time ($editor = &which($editor)).

cap_sensitive · 2010-09-18 03:06:21

Hi Xyne:

Could you set an example about using "--autopatch" function? I'm rather confused about "PatchDir hierarchy".
Should I just place $pkgname.patch in PatchDir, or it needs certain "directory structure"?

thanks.

Last edited by cap_sensitive (2010-09-18 03:58:24)

ngoonee · 2010-09-18 05:59:02

Directory structure should match your directory structure in in BuildDir. Meaning there'd be a folder called extra with subdirectories with package names. Put the patches there.

So patching jpilot would be extra/jpilot/PKGBUILD.patch

AUR is just the folder named aur.

Arch Linux

#776 2010-08-17 17:15:13

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#777 2010-08-18 21:23:49

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#778 2010-08-18 21:53:00

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#779 2010-08-19 02:17:58

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#780 2010-08-19 18:18:06

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#781 2010-08-20 02:22:15

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#782 2010-08-20 03:20:57

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#783 2010-08-20 06:03:22

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#784 2010-08-20 06:39:34

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#785 2010-08-20 09:19:52

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#786 2010-08-20 11:02:16

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#787 2010-08-20 12:05:41

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#788 2010-08-20 12:18:48

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#789 2010-08-20 14:37:02

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#790 2010-08-22 04:42:39

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#791 2010-08-28 00:35:52

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#792 2010-08-28 01:10:53

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#793 2010-08-28 01:23:00

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#794 2010-09-06 11:10:59

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#795 2010-09-06 11:39:14

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#796 2010-09-07 14:55:26

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#797 2010-09-17 22:04:55

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#798 2010-09-17 23:54:43

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#799 2010-09-18 03:06:21

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

#800 2010-09-18 05:59:02

Re: Bauerbill: Powerpill + ABS, AUR, CPAN and Hackage support + more

Board footer