Powerpill - A wrapper and download accelerator for Pacman

Xyne · 2008-12-05 18:38:02

Afaik, the site has never actually been down. Some DNS servers just seem to "forget" about it which leads to transient resolver errors. I've added a direct link to my sig and to the site. When accessed through that link, the site will convert all links to direct ones which should solve the problem.

I've never done site mirroring before so I'm not sure how to show you how. Would you just need a list of resources to mirror? I could create a page to generate such a list.

Zariel · 2008-12-05 20:49:18

Could you not host it on github or so?

Xyne · 2008-12-05 22:09:35

I suppose that I could, but it just doesn't seem necessary. The only issue is the DNS one at present and that should fix itself. My ISP's DNS no longer has any trouble with the site (I couldn't access it at all before, then only sometimes) and for those that do the direct link should be enough for now.

I like the way that I have it set up at the moment because I'm able to automate most of it. If MindTooth does end up mirroring the site, then there should be no problem at all.

Btw, the powerpill PKGBUILD contains a line to handle DNS errors.

doorknob60 · 2008-12-09 04:20:24

Nice, it works well. My only complaints is it asks you stuff more than regular Pacman. Pacman asks you to confirm once, showing the packages and the size. I like that. I use the --noconfirm option in Powerpill because otherwise it asks me like 3 times which is a little annoying. Also, I set it to use pacman-color in the conf file, but it still seems to be using pacman. Any ideas on that one?

Xyne · 2008-12-09 04:55:44

Use "--nomessages" instead so that you don't suppress pacman messages.

There is a reason that powerpill has more dialogues than regular pacman. Powerpill needs to know which packages it's supposed to download so it emulates pacman's pkg selection dialogue. It then shows you what will be downloaded and asks for confirmation before continuing (perhaps that could be removed or maybe I'll add some more settings to the config file). Once the packages have been downloaded, it invokes pacman for installation operations. You could add "--noconfirm" to the end of the path to pacman in the powerpill config file, but that will suppress important pacman dialogues (which is also why I recommend using "--nomessages" instead... perhaps i should change that option to "--silent-powerpill").

Someone on yaourt's AUR page stated that they were using powerpill inside yaourt and around pacman-color, so it should work. Double-check that you haven't typo'd the setting and that the config file is in the right place (either ~/.config/powerpill/powerpill.conf or in whatever dir you specify with the --conf-dir option).

MindTooth · 2008-12-15 07:35:40

I have added the alias to my .bashrc file, but still Yaourt used the old pacman.
What can cause this?

Birger

joephantom · 2008-12-15 16:24:58

MindTooth wrote:

I have added the alias to my .bashrc file, but still Yaourt used the old pacman.
What can cause this?
Birger

++.

Xiong Chiamiov · 2008-12-16 01:07:07

Xyne wrote:

Someone on yaourt's AUR page stated that they were using powerpill inside yaourt and around pacman-color, so it should work.

That was me. Of note is that I've aliased pacman to pacman-color as well, although that shouldn't make a difference, as the powerpill configuration file uses a full path.

MindTooth wrote:

I have added the alias to my .bashrc file, but still Yaourt used the old pacman.
What can cause this?

Yaourt uses its own configuration file (at /etc/yaourtrc), which is where you specify the pacman binary to use. You'll have to change PacmanBin to /usr/bin/powerpill if you want to use powerpill instead of pacman or pacman-color.

The annoying thing about this is that it seems that you *must* specify the full path. This means that you cannot use flags automatically (such as the --silent-aria flag I requested), because yaourt complains about extra arguments in its config file.

As I was typing this, I began to think that a simple bash script could be used as the pacman binary, in which it simply calls powerpill and adds the appropriate options. A simple test shows that command-line options need to be passed in via a for loop or somesuch, and as I'm not very familiar with bash, I can't do that off of the top of my head. If no one beats me to it, I may hack something up (whether in bash, python, or the language I'm working on learning, perl) tonight in a few hours.

EDIT:
Alrighty, it was simpler than I thought. I created /usr/bin/powerpill-launcher, gave it executable permisions (chmod +x) and changed the pacman binary in yaourtrc to point to it, and it works just fine, as far as I can see.

#!/usr/bin/env bash

powerpill --silent-aria $@

I'm not sure if there's a "better" way, but as long as it works, I don't care too much.

Of course, you don't have to use --silent-aria; you can add or subtract whatever arguments you like. $@ is a bash expansion for all of the command-line arguments, so that whatever options get passed to the script get passed on to powerpill (and thus, pacman).

Last edited by Xiong Chiamiov (2008-12-16 03:55:09)

Xyne · 2008-12-16 05:10:16

Maybe yaourt changed something. I've added an option to powerpill.conf to deal with this. You can get more info on the powerpill info page but basically, do this:

Update powerpill then generate a new configuration file (it will load settings from the current one).

powerpill --gen-conf

Edit powerpill.conf and change the additional powerpill arguments line to:

ADDITIONAL_POWERPILL_ARGUMENTS = '--nomessages'

Edit the yaourtrc file to specify the full path to powerpill instead of pacman.
Remove the previous pacman/powerpill alias from your bashrc and clear the alias with

unalias pacman

Let me know if this solves the problem.

Last edited by Xyne (2008-12-16 05:23:08)

Xiong Chiamiov · 2008-12-16 06:19:29

Powerpill doesn't seem to be reading in the additional arguments specified in the config file, as far as I can tell. Running 'sudo powerpill --nomessages -S inkscape' just freezes (or rather, I'm guessing that it's waiting for input, but isn't writing a prompt, meaning I should be using the flag that says yes to everything), but having --nomessages in the config file and running 'sudo powerpill -S inkscape' doesn't exihibit that behavior. Changing the config option to --conf-dir still ends up with powerpill using my default config file, as well.

BTW, I noticed this when I was updated powerpill:

Do you want to update these packages [ w X [Y/n]

deltaecho · 2008-12-16 06:31:33

deltaecho wrote:

How difficult would it be to incorporate a feature that would begin installing packages as soon as all their dependencies are downloaded, while others are still downloading? If you/we could find a good algorithm for this, it would speed up the process even more - WHOOT!

Xyne wrote:

That's an interesting idea. The problem that I see with it is that powerpill lets pacman handle the installation of packages. When pacman is invoked, it creates a lock file to prevent multiple instances from manipulating the database at the same time (such as during installation). The only way around that that I can think of would be to incorporate some installation functionality into powerpill directly. Although it shouldn't be too difficult, I'm reluctant to do that because it opens the doors for bugs corrupting the local database and installed packages (probably wouldn't happen, but the possibility is there, plus I think others will be less inclined to use powerpill in that case).
Actually, it would probably be possible to install packages through a thread with pacman and wait for the thread to exit before launching another one to avoid simultaneous database access conflicts, but that made me realize the greatest limitation to doing this: the packages are not downloaded in any particular order in the metalink, so some dependencies may be the last packages to be downloaded, which nullifies this approach.
My last thought on this for now is that package installation doesn't take that long (for me at least) so I'm not sure that the complexity/benefit ratio would be worth it. I'd be interested in others' thoughts about this.
Last edited by Xyne (2008-10-18 16:38:06)

Brainstorming: As far as metalinking goes, you could parse the packages to download and separate them into groups depending on which depend / don't depend on each other, then create multiple metalinks with those and download them simultaneously. Then, as each metalink finishes downloading, begin installing those packages.

Edit: Maybe a metalink of metalinks?

Last edited by deltaecho (2008-12-16 06:32:46)

Xyne · 2008-12-16 06:32:36

Do you want to update these packages [ w X [Y/n]

I think that's a pacman error (or whatever you're using to wrap pacman or to wrap powerpill). The powerpill source code doesn't even contain the world "update".
If there's an error in your wrapper, maybe that's causing the problem.

Are you sure you're using the latest version btw and that you've regenerated your config file? It specifically says that it ignores certain options in the additional arguments string, including "--conf-dir". Remove them and try it again (I just realized that those arguments are not being filtered out so that's probably what's causing the problem in pacman).

MindTooth · 2008-12-16 07:15:37

Maybe a dum question, can I remove this output:

==> /home/mt/.config/powerpill/powerpill.conf loaded
==> Exiting powerpill and passing control to pacman...
PACMAN: Starting...

I added --nomessage to the powerpill.conf file.

Thank you for guiding the yaourt fix

Birger

Xyne · 2008-12-16 08:02:50

@Xiong Chiamiov (et al.)

Ok, I've fixed the argument parsing (take a look at the control structure at the top of the argument parsing function... fun fun fun).

@deltaecho
Interesting but I don't think it would be practical. If you download all the sub-metalinks simultaneously, it's exactly the same thing as using the main one. The order or metalinks in a metalink of metalinks would be just as random as the current order of files (metalink metalink metametametamememememmmmm....). If you download them sequentially, you end up taking longer and not making the most of bandwidth. I understand that the idea is to trick aria into preferentially downloading the deps but I think that as soon as you have multiple instances running, it doesn't matter in what order they were started.

I'll probably end up converting powerpill to python to take advantage of pyalpm[m] at some point. When that happens, I think I'll be able to install packages as soon as they're downloaded so that should speed it up too. Hmmm, one problem there though is that simultaneous downloading and installation will require a second shell or suppression of download status messages to enable installation dialogues etc.

Xyne · 2008-12-16 08:04:11

MindTooth wrote:

Maybe a dum question, can I remove this output:
==> /home/mt/.config/powerpill/powerpill.conf loaded
==> Exiting powerpill and passing control to pacman...
PACMAN: Starting...
I added --nomessage to the powerpill.conf file.
Thank you for guiding the yaourt fix
Birger

Hehe, I just noticed that myself while I was reworking the argument and configuration parsing. It's already been fixed.

MindTooth · 2008-12-16 08:18:03

Still shows the "==> /home/mt/.config/powerpill/powerpill.conf loaded" line. Is the on be default?

Birger

Xyne · 2008-12-16 08:45:36

MindTooth wrote:

Still shows the "==> /home/mt/.config/powerpill/powerpill.conf loaded" line. Is the on be default?
Birger

Messages are on by default and that one was displayed as the file was loaded but before it was parsed for the "--nomessages" argument. It's fixed now.

MindTooth · 2008-12-16 10:48:41

Thank you.

Birger

Xiong Chiamiov · 2008-12-16 19:07:46

Xyne wrote:

@Xiong Chiamiov (et al.)
Ok, I've fixed the argument parsing (take a look at the control structure at the top of the argument parsing function... fun fun fun).

Works like a charm. Thanks.

oddsocks · 2008-12-17 02:36:47

Sorry but I'm still adapting to Arch linux. I edited my /etc/yaourtrc file like this:

# a
# a
# ~/yaourtrc - Configuration for yaourt
#
# See yaourt(8) for more information
#
#
#
# AlwaysUpgradeDevel no
# AlwaysUpgradeAur no
# AlwaysForce no
# AurVoteSupport yes
# AutoSaveBackupFile no
# ColorMod Normal
# ColorMod LightBackGround
# ColorMod NoColor
# ColorMod TextOnly
# DontNeedToPressEnter yes
# EditPkgbuild yes
# ExportToLocalRepository /where/you/want
# ForceEnglish no
# LastCommentsNumber 5
# LastCommentsOrder asc
# NoConfirm no
PacmanBin /usr/bin/powerpill
# PkgbuildEditor gvim
# SearchInAurUnsupported yes
# ShowAurComment yes
# TmpDirectory /where/you/want
# UpdateTerminalTitle yes
#
# Define here your prefered Sourceforge mirror: (none surfnet ufpr heanet easynews umn switch belnet kent mesh optusnet jaist puzzle superb-east nchc superb-west)
# SourceforgeMirror heanet
#

but yaourt doesn't use powerpill. Where have I gone wrong?

Xyne · 2008-12-17 03:20:02

I don't really know because I still haven't installed yaourt.

If you've changed the config to point to a different binary but it still runs pacman, I suspect that the configuration file is being ignored. Is there any chance that you have a yaourtrc file in your home directory that's taking precedence? I don't even know if it creates them, but check if you have ~/.yaourtrc or ~/.config/yaourt/yaourtrc or something similar.

Does yaourt respect other settings in that file?

Hmmm, sorry I can't be of more help. Hopefully someone else will reply quickly.

oddsocks · 2008-12-17 05:27:44

Thanks for the quick reply Xyne. Still looking through and trying to fix the problem, but I think I may stop using yaourt until I figure this out. The fast downloads of powerpill is just too good to give up! Thank for making this by the way

MindTooth · 2008-12-17 09:50:30

Strange, after I started my machine today. Yaourt doesn't seem to use yaourt anymore. Just a note. Will debug this myself.

Birger

Xyne · 2008-12-17 11:26:42

MindTooth wrote:

Yaourt doesn't seem to use yaourt anymore.

I think I know what you meant, but for a second there my brain twitched.

Xiong Chiamiov · 2008-12-17 23:28:44

oddsocks wrote:

Sorry but I'm still adapting to Arch linux. I edited my /etc/yaourtrc file like this:
# a
# a
# ~/yaourtrc - Configuration for yaourt
#
# See yaourt(8) for more information
#
#
#
# AlwaysUpgradeDevel no
# AlwaysUpgradeAur no
# AlwaysForce no
# AurVoteSupport yes
# AutoSaveBackupFile no
# ColorMod Normal
# ColorMod LightBackGround
# ColorMod NoColor
# ColorMod TextOnly
# DontNeedToPressEnter yes
# EditPkgbuild yes
# ExportToLocalRepository /where/you/want
# ForceEnglish no
# LastCommentsNumber 5
# LastCommentsOrder asc
# NoConfirm no
PacmanBin /usr/bin/powerpill
# PkgbuildEditor gvim
# SearchInAurUnsupported yes
# ShowAurComment yes
# TmpDirectory /where/you/want
# UpdateTerminalTitle yes
#
# Define here your prefered Sourceforge mirror: (none surfnet ufpr heanet easynews umn switch belnet kent mesh optusnet jaist puzzle superb-east nchc superb-west)
# SourceforgeMirror heanet
#
but yaourt doesn't use powerpill. Where have I gone wrong?

That is the correct edit, so that means that one of several things has gone wrong:
a) Yaourt is using a different configuration file.
b) You made changes to the file, but they are not saved (you didn't edit it as root).
c) Yaourt is using powerpill, but you think it isn't.
Please don't take any of these as an insult on your intelligence; I'm just listing the possibilities.

A can be tested by changing something else (say, NoConfirm). B is simple - just do a cat or less on the file and see what it says. For C, the easiest way I can think of it is to change the PacmanBin to point to /usr/bin/pacman-color (assuming you've installed pacman-color), then see if you get colored output in pacman while using yaourt. If not, then it's still using /usr/bin/pacman.

Arch Linux

#51 2008-12-05 18:38:02

Re: Powerpill - A wrapper and download accelerator for Pacman

#52 2008-12-05 20:49:18

Re: Powerpill - A wrapper and download accelerator for Pacman

#53 2008-12-05 22:09:35

Re: Powerpill - A wrapper and download accelerator for Pacman

#54 2008-12-09 04:20:24

Re: Powerpill - A wrapper and download accelerator for Pacman

#55 2008-12-09 04:55:44

Re: Powerpill - A wrapper and download accelerator for Pacman

#56 2008-12-15 07:35:40

Re: Powerpill - A wrapper and download accelerator for Pacman

#57 2008-12-15 16:24:58

Re: Powerpill - A wrapper and download accelerator for Pacman

#58 2008-12-16 01:07:07

Re: Powerpill - A wrapper and download accelerator for Pacman

#59 2008-12-16 05:10:16

Re: Powerpill - A wrapper and download accelerator for Pacman

#60 2008-12-16 06:19:29

Re: Powerpill - A wrapper and download accelerator for Pacman

#61 2008-12-16 06:31:33

Re: Powerpill - A wrapper and download accelerator for Pacman

#62 2008-12-16 06:32:36

Re: Powerpill - A wrapper and download accelerator for Pacman

#63 2008-12-16 07:15:37

Re: Powerpill - A wrapper and download accelerator for Pacman

#64 2008-12-16 08:02:50

Re: Powerpill - A wrapper and download accelerator for Pacman

#65 2008-12-16 08:04:11

Re: Powerpill - A wrapper and download accelerator for Pacman

#66 2008-12-16 08:18:03

Re: Powerpill - A wrapper and download accelerator for Pacman

#67 2008-12-16 08:45:36

Re: Powerpill - A wrapper and download accelerator for Pacman

#68 2008-12-16 10:48:41

Re: Powerpill - A wrapper and download accelerator for Pacman

#69 2008-12-16 19:07:46

Re: Powerpill - A wrapper and download accelerator for Pacman

#70 2008-12-17 02:36:47

Re: Powerpill - A wrapper and download accelerator for Pacman

#71 2008-12-17 03:20:02

Re: Powerpill - A wrapper and download accelerator for Pacman

#72 2008-12-17 05:27:44

Re: Powerpill - A wrapper and download accelerator for Pacman

#73 2008-12-17 09:50:30

Re: Powerpill - A wrapper and download accelerator for Pacman

#74 2008-12-17 11:26:42

Re: Powerpill - A wrapper and download accelerator for Pacman

#75 2008-12-17 23:28:44

Re: Powerpill - A wrapper and download accelerator for Pacman

Board footer