Powerpill - A wrapper and download accelerator for Pacman

MetalheadGautham · 2009-06-08 23:47:15

I upgraded to the latest ppill from your website.

Error:

bash-3.2$ sudo ./powerpill -Syu
Can't locate Xyne/Arch/Powerpill.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.10.0 /usr/share/perl5/site_perl/5.10.0 /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl /usr/lib/perl5/current /usr/lib/perl5/site_perl/current .) at ./powerpill line 25.
BEGIN failed--compilation aborted at ./powerpill line 25.
bash-3.2$

Xyne · 2009-06-09 01:13:28

@MetalheadGautham
What's the output of the following command?:

pacman -Ql perl-xyne-arch

nonickknown · 2009-06-11 14:42:22

hi xyne
is it possible to get a real progressbar for powerpill?
like the pacman one...

[#############################------------------] 70%

Xyne · 2009-06-11 17:18:23

I might add a progress bar but I would want to keep some of the other information such as download rate and total size, which makes formatting the progress bar line unnecessarily complicated (it requires tput, fixed width fields which ETA is not, etc). The same thing goes for adding an extra option just to select a progress bar and ultimately any such change would reduce the information currently presented (the percentage is already displayed... having a bar of hashes is just "eye candy").

I'll keep it in mind though and implement it if I come up with a smooth way to do it.

Last edited by Xyne (2009-06-11 17:18:42)

MetalheadGautham · 2009-06-12 23:02:38

Xyne wrote:

@MetalheadGautham
What's the output of the following command?:
pacman -Ql perl-xyne-arch

error: package "perl-xyne-arch" not found

BTW I don't use the package. I just took the perl script out and stuffed it into my custom scripts folder.

Xyne · 2009-06-13 11:40:53

You need perl-xyne-arch, or at least a few of the Perl modules in it along with some in perl-xyne-common. Powerpill used to be a single monolithic messy script but I've broken it up into various modules that can be reused in different applications. If you look at the Powerpill script itself, it's only a very thin frontend to those modules now.

I recommend using the package as I often update perl-xyne-arch which contains the actual Powerpill code. If you wish to avoid the package, you will need the following either in the same directory as the script or in Perl's inclusion path:

Xyne/Arch/Pacman.pm
Xyne/Arch/PacmanConf.pm
Xyne/Arch/Powerpill.pm
Xyne/Arch/PowerpillConf.pm
Xyne/Arch/Reflector.pm
Xyne/Common/Functions.pm
Xyne/Common/Log.pm
Xyne/Common/Metalink.pm
Xyne/Common/Shell.pm

You will also need LWP::Simple which is contained in perl-libwww. You could try adding LWP/Simple.pm to Perl's inclusion path, but I believe that it's just a frontend to another module contained in the package.

As you can see, it would be much simpler to simply install the packages. The easiest way is by using the repo on my site.

Also note that the total installed size of perl-xyne-arch, perl-xyne-common and powerpill is only 200 K.

gee · 2009-06-14 21:33:59

Hi,

I have not been on Arch / using powerpill for a while, but the latest version won't work if I do not have sudo installed (even if I'm running it as root...):

root@gee: powerpill -v -Syu
Error: this operation requires root privileges
Either log in as root and rerun the application
or install and configure "sudo".
root@gee: pacman -S sudo
...
(1/1) installing sudo                               [###################################################################] 100%
root@gee: powerpill -v -Syu
:: Synchronizing package databases...
 testing is up to date
 core is up to date
 extra is up to date
 community is up to date
 archlinuxfr is up to date
Root      : /
...

I have everything up to date...

Thanks

Xyne · 2009-06-14 23:35:56

@gee
That's weird. What is the output of the following command when you are logged in as root?

perl -e 'print $ENV{"USER"}'

For now I've changed the code to issue a warning and then try anyway so it should work as long as the current user has the right privileges. Let me know if the issue persists with the latest version (perl-xyne-arch >= 0.64). I would still like to know why it fails to detect the user as root.

Last edited by Xyne (2009-06-14 23:36:35)

gee · 2009-06-15 06:33:12

good guess, it is my normal user.

Any reason for that?

the latest version of perl-xyne-arch only prints the warning which is better.
Before today I've never installed that per lib, aren't those files included with powerpill or reflector?

Thanks

Last edited by gee (2009-06-15 06:40:46)

Xyne · 2009-06-15 14:41:21

I don't know. I've tested it here by logging in as root with "su" in urxvt and it correctly detects the user as root. Check the output of "echo $USER" to see if the problem is with Perl or with the actual environment variable. Either way I have no idea what would cause the discrepancy.

Powerpill and reflector used to be monolithic but I've split them and my other Perl applications into modules to make it easier to maintain them, reuse code, etc. The most practical way to package them that I've found is to lump the Arch-related modules together in one package and the non-Arch-related modules in another. Powerpill, reflector, etc are very thin frontends to those modules and only contain a few basic files.

gee · 2009-06-15 16:06:29

So basically if I log as my normal user, then switch to root I get my normal user in those commands, but if I directly log in as root I get root returned...

Allamgir · 2009-06-15 20:56:47

When using powerpill I've noticed it makes the terminal window really messy. Is there a way you could constrain the downloading process to a single line that changes as progress is made?

Also, the progress line does not fit into 80 characters. By removing the "--" part of the "-->" arrows and the space between the arrow and the word "progress", it will fit in 80 characters.

for example, change:

--> progress: 5 of 8 files, 24.73 MiB of 191.01 MiB (12%), 0 B/s, ETA: N/A

to

>progress: 5 of 8 files, 24.73 MiB of 191.01 MiB (12%), 0 B/s, ETA: N/A

Xyne · 2009-06-15 23:54:59

@Allamgir
If you're getting spammed with aria2c's output, try passing the "--aria2-silent" option or add "Aria2Silent" to the options section of /etc/powerpill.conf.

Powerpill's progress is printed to a single line using "\r". I've shortened the progress line so that it should hopefully remain within 80 characters (no guarantees though as the number of packages and the ETA can stretch it out) in case that was the cause of the messy console.

Let me know if this fixes the issue for you.

MetalheadGautham · 2009-06-20 04:08:44

@Xyne: Hmm... Then I guess I will be using the package. BTW, you made a nice move by splitting the script. The original powerpill was one of the hardest code pieces to read

eNTi · 2009-07-06 15:22:39

hi. i've just installed powerpill from aur and i found, that the pre-installed config file says:

# maximum total download speed (0 = disabled)
# Use this to throttle the download.
# The speed is given as for lowest-speed-limit above.
max-download-limit=

the man file says:

       --max-download-limit=SPEED
           Set max download speed per each download in bytes/sec.  0 means unrestricted. You can append K or M(1K = 1024, 1M = 1024K). 
           To limit the overall download speed, use --max-overall-download-limit option. Default: 0

you're missing the "overall".

edit: also it seems aria has quite some problems with limiting the download speed for lots of small files, because it starts out with a high bandwith and only slowly reaches the "--max-overall-download-limit", which in turn results in bad lag, if someone else plays an online game. somehow this isn't satifactory...

Last edited by eNTi (2009-07-06 15:32:09)

Xyne · 2009-07-07 06:40:03

Thanks eNTi. I have no idea how that happened because I haven't changed the default powerpill.conf since I first wrote it. It has contained "max-overall-download-limit=0' the entire time and I don't see any way that could have changed during a source upload (and yet it did). I've double-checked both the binary package and the AUR package. It should be fixed now.

The issue with aria2's respecting the max overall download limit is beyond my control. The only suggestion that I have is to limit your overall number of connections and to limit the speed of each connection. You can use all of aria2's options in the powerpill configuration file and you can specify a configuration on the command line, so it would be possible to have a "powerpill.limited" config that you could use when your mates are gaming.

9nqksfhn · 2009-08-03 09:27:44

What does it do? Does it download from multiple mirrors in parralel?

Xyne · 2009-08-03 09:47:54

9nqksfhn wrote:

What does it do? Does it download from multiple mirrors in parralel?

It downloads from multiple mirrors in parallel and it uses segmented downloads to split large files. It can also retrieve and filter mirrors from GerBra's mirrorcheck page and add them to the pool at runtime.

Take a look at the comments in /etc/powerpill.conf to get an idea of what you can do.

*makes a mental note to add more info to the info page... there used to be more before*

9nqksfhn · 2009-08-03 09:55:57

Cool, but since your a Trusted User why isn't it in the community repo?

Xyne · 2009-08-03 10:17:34

9nqksfhn wrote:

Cool, but since your a Trusted User why isn't it in the community repo?

I'm waiting for support for the "any" architecture, which they're working on right now. Without that, I would have to do duplicate each package, hack PKGBUILDs, upload the same files twice for each update, etc.

Xyne · 2009-08-09 14:48:01

I've just moved powerpill into [community] along with perl-xyne-common and perl-xyne-arch. As long as I didn't do something wrong, it should start showing up on the repos as the resync.

Last edited by Xyne (2009-08-09 14:59:32)

zatricky · 2009-08-30 13:37:22

I just started using powerpill and I'm loving how fast it goes.

I was receiving this error due to permissions:

[ brendan@watricky : 15:20:01 : ~ ]
:) powerpill -Su
==> ERROR no writable cache directory specified

I "fixed" it with sudo chmod 777 /var/cache/pacman/pkg/.

If powerpill isn't running the download as root then I'd much rather that powerpill saved active downloads somewhere within ~ and then sudo mv finished downloads to /var/cache/pacman/pkg/ afterwards.

I think its a bad idea to have /var/cache/pacman/pkg world-writable. Strictly speaking, right now, powerpill doesn't work "out-the-box".

Is there a supported workaround for this or is it already planned for a future release?

Xyne · 2009-09-01 00:37:02

You can do one of the following:
a) run powerpill with sudo
b) run powerpill as root
c) use the "--cachedir" flag to specify a different directory

It's meant to be a pacman wrapper so I will not change the default download directory for packages. You can't download packages to that directory without root privileges using pacman either, so I don't understand why you would claim that it doesn't work "out-the-box".

I agree that making /var/cache/pacman/pkg world-writable is a bad idea but there is absolutely no reason to do that.

zatricky · 2009-09-01 03:59:48

The reason I say it doesn't work "out-the-box" is powerpill has sudo support built-in. It takes into account the fact that my account is unprivileged and that it will ask me for my sudo pass whenever it is appropriate.

This seems an appropriate place but here its not following through with sudo support. Call me a nitpicking permissions Nazi but I don't think "sudo powerpill" is the right way here at all.

Can I assume if I use the "--cachedir" parameter in future that the default /var/cache/pacman/pkg/ will never be touched ever again? If that is the case, its pretty much the same as chown'ing the /var/cache/pacman/pkg/ directory to my user.

Xyne · 2009-09-01 04:32:00

Powerpill, like makepkg, only uses sudo for pacman operations which require root privileges. This makes it possible to invoke powerpill itself without root privileges for various operations (e.g. downloading to alternative directories, metalink generation) while still making it possible to run pacman operations (e.g. database synchronization, package installation).

That is different from having powerpill check if it should have been run with sudo itself. In that case it would basically be saying to the user "here, let me type sudo for you... what's your password?". If your issue is essentially that you don't want to have to type an extra 5 characters or create an alias then I'm not really sure what to tell you. Why is it a better solution to chmod the pkg cache than to create an alias for powerpill (e.g. "alias pp='sudo powerpill'")? Why do you feel more comfortable having applications decide for you when they should be run with elevated privileges?

I'm not saying that I absolutely won't change this behavior, but it would take convincing from others. At the moment I consider this a non-issue and it feels like a request for hand-holding.

Arch Linux

#251 2009-06-08 23:47:15

Re: Powerpill - A wrapper and download accelerator for Pacman

#252 2009-06-09 01:13:28

Re: Powerpill - A wrapper and download accelerator for Pacman

#253 2009-06-11 14:42:22

Re: Powerpill - A wrapper and download accelerator for Pacman

#254 2009-06-11 17:18:23

Re: Powerpill - A wrapper and download accelerator for Pacman

#255 2009-06-12 23:02:38

Re: Powerpill - A wrapper and download accelerator for Pacman

#256 2009-06-13 11:40:53

Re: Powerpill - A wrapper and download accelerator for Pacman

#257 2009-06-14 21:33:59

Re: Powerpill - A wrapper and download accelerator for Pacman

#258 2009-06-14 23:35:56

Re: Powerpill - A wrapper and download accelerator for Pacman

#259 2009-06-15 06:33:12

Re: Powerpill - A wrapper and download accelerator for Pacman

#260 2009-06-15 14:41:21

Re: Powerpill - A wrapper and download accelerator for Pacman

#261 2009-06-15 16:06:29

Re: Powerpill - A wrapper and download accelerator for Pacman

#262 2009-06-15 20:56:47

Re: Powerpill - A wrapper and download accelerator for Pacman

#263 2009-06-15 23:54:59

Re: Powerpill - A wrapper and download accelerator for Pacman

#264 2009-06-20 04:08:44

Re: Powerpill - A wrapper and download accelerator for Pacman

#265 2009-07-06 15:22:39

Re: Powerpill - A wrapper and download accelerator for Pacman

#266 2009-07-07 06:40:03

Re: Powerpill - A wrapper and download accelerator for Pacman

#267 2009-08-03 09:27:44

Re: Powerpill - A wrapper and download accelerator for Pacman

#268 2009-08-03 09:47:54

Re: Powerpill - A wrapper and download accelerator for Pacman

#269 2009-08-03 09:55:57

Re: Powerpill - A wrapper and download accelerator for Pacman

#270 2009-08-03 10:17:34

Re: Powerpill - A wrapper and download accelerator for Pacman

#271 2009-08-09 14:48:01

Re: Powerpill - A wrapper and download accelerator for Pacman

#272 2009-08-30 13:37:22

Re: Powerpill - A wrapper and download accelerator for Pacman

#273 2009-09-01 00:37:02

Re: Powerpill - A wrapper and download accelerator for Pacman

#274 2009-09-01 03:59:48

Re: Powerpill - A wrapper and download accelerator for Pacman

#275 2009-09-01 04:32:00

Re: Powerpill - A wrapper and download accelerator for Pacman

Board footer