powerpill reborn: pacman wrapper for faster downloads

dapolinario · 2016-04-05 00:11:16

Broken again:

Packages: pm2ml-2016.4.4.3-1 python3-xcgf-2016.4.5-1 python3-xcpf-2016.4.5-1

$ sudo powerpill -Syu

04/04 21:09:40 [NOTICE] GID#d4a31538b54c7606 - Download já foi completado: /var/lib/pacman/sync/xyne-x86_64.db

04/04 21:09:40 [NOTICE] GID#3ca9e012fcb36154 - Download já foi completado: /var/lib/pacman/sync/xyne-x86_64.db.sig

04/04 21:09:40 [NOTICE] Download Completo: /var/lib/pacman/sync/xyne-x86_64.db

04/04 21:09:40 [NOTICE] Download Completo: /var/lib/pacman/sync/xyne-x86_64.db.sig

04/04 21:09:40 [NOTICE] GID#329ec9fb1b1372a9 - Download já foi completado: /var/lib/pacman/sync/extra.db

04/04 21:09:40 [NOTICE] Download Completo: /var/lib/pacman/sync/extra.db

04/04 21:09:41 [NOTICE] GID#685a6d2b2297412d - Download já foi completado: /var/lib/pacman/sync/core.db
[DL:0B][#51d759 0B/0B][#685a6d 0B/0B]                                          
04/04 21:09:42 [NOTICE] Download Completo: /var/lib/pacman/sync/core.db

04/04 21:09:42 [NOTICE] GID#51d759e2359809dc - Download já foi completado: /var/lib/pacman/sync/community.db

04/04 21:09:42 [NOTICE] Download Completo: /var/lib/pacman/sync/community.db

Resultados do Download:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
d4a315|OK  |       0B/s|/var/lib/pacman/sync/xyne-x86_64.db
3ca9e0|OK  |       0B/s|/var/lib/pacman/sync/xyne-x86_64.db.sig
329ec9|OK  |       0B/s|/var/lib/pacman/sync/extra.db
685a6d|OK  |       0B/s|/var/lib/pacman/sync/core.db
51d759|OK  |       0B/s|/var/lib/pacman/sync/community.db

Legenda de estados:
(OK):download completo.
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 975, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 966, in run_main
    return main(args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 956, in main
    powerpill.download_packages()
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 798, in download_packages
    self.download(pm2ml_args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 582, in download
    self.pm2ml.resolve_targets_from_arguments(pm2ml_pargs)
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 592, in resolve_targets_from_arguments
    only_needed=pargs.needed,
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 521, in resolve_targets
    a, b, orphans = self.determine_upgradable(downgrades=(upgrade > 1))
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 291, in determine_upgradable
    if self.check_aur:
AttributeError: 'Pm2ml' object has no attribute 'check_aur'

Xyne · 2016-04-05 00:22:52

Fixed that one too now. Thanks for the reports.

dapolinario · 2016-04-05 00:27:14

One more time (package pm2ml-2016.4.5-1):

$ sudo powerpill -Syu

... (suppressed by me)

Resultados do Download:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
ae6563|OK  |       0B/s|/var/lib/pacman/sync/xyne-x86_64.db
d957b8|OK  |       0B/s|/var/lib/pacman/sync/xyne-x86_64.db.sig
bc1b68|OK  |       0B/s|/var/lib/pacman/sync/community.db
cdd489|OK  |       0B/s|/var/lib/pacman/sync/core.db
a068d5|OK  |       0B/s|/var/lib/pacman/sync/extra.db

Legenda de estados:
(OK):download completo.
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 975, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 966, in run_main
    return main(args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 956, in main
    powerpill.download_packages()
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 798, in download_packages
    self.download(pm2ml_args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 582, in download
    self.pm2ml.resolve_targets_from_arguments(pm2ml_pargs)
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 594, in resolve_targets_from_arguments
    only_needed=pargs.needed,
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 525, in resolve_targets
    aur_pkgs |= b.ignore(ignored, ignoredgroups)
AttributeError: 'NoneType' object has no attribute 'ignore'

Xyne · 2016-04-05 00:55:47

I'm sorry. It's been a long, bad day and I was being lazy. I took the time to test it more carefully. Please let me know if it works with pm2ml 2016.4.5.2.

dapolinario · 2016-04-05 00:58:08

For me is perfect now! Thanks!

Xyne · 2016-04-05 01:33:22

dapolinario wrote:

For me is perfect now! Thanks!

*phew*

Thanks again for the testing and feedback!

lundibundi · 2016-04-20 19:50:22

Broken with the most recent update:
pm2ml 2016.4.6-1
powerpill 2016.4.5.1-1
python3-xcgf 2016.4.5-1
python3-xcpf 2016.4.5.2-1

sudo powerpill -Syu
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 937, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 928, in run_main
    return main(args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 894, in main
    pacman_conf = get_pacman_conf(pargs, powerpill_conf)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 371, in get_pacman_conf
    pacman_conf = pm2ml.PacmanConfig(pargs['pacman_config'])
AttributeError: module 'pm2ml' has no attribute 'PacmanConfig'

Xyne · 2016-04-20 21:07:04

lundibundi wrote:

powerpill 2016.4.5.1-1

  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 371, in get_pacman_conf
    pacman_conf = pm2ml.PacmanConfig(pargs['pacman_config'])

Line 371 of 2016.4.5.1-1 is

class ArgumentError(PowerpillError):

Your tracelog does not match the current version. Are you sure that you have updated all local packages?

lundibundi · 2016-04-20 21:19:37

Strangely enough, i have reinstalled all pack (powerpill, pyalpm, pm2ml, python3-xcgf, python3-xcpf) and it works. Sorry for bother.
Edit: i guess i've got an idea. I thinks aura as deps of pp updated pm2ml before it, but after that I've got an old pp that's incompatible with new pm2ml and aura tried to use it to download new pp which obviously resulted in error.

Last edited by lundibundi (2016-04-20 21:48:12)

Alad · 2016-04-20 22:08:51

Edit: i guess i've got an idea. I thinks aura as deps of pp updated pm2ml before it, but after that I've got an old pp that's incompatible with new pm2ml and aura tried to use it to download new pp which obviously resulted in error.

aura isn't very good at resolving dependencies - you shouldn't rely on it for anything more than trivial tasks. See the comparison table on the wiki.

Last edited by Alad (2016-04-20 22:09:52)

Xyne · 2016-04-20 22:11:38

<shameless self promotion> Why not use bauerbill instead? </shameless self promotion>

lundibundi · 2016-04-20 22:42:12

it's just that aura worked just fine up to now=).
Promotion worked gonna try it, but is there a way for smooth transition from aura to bauerbill? (it's kind of bothersome to reinstall all packages from aur)

Xyne · 2016-04-20 22:44:04

lundibundi wrote:

is there a way for smooth transition from aura to bauerbill? (it's kind of bothersome to reinstall all packages from aur)

Enable my repo and then run "pacman -Syu bauerbill"

dapolinario · 2016-05-18 14:42:52

New error with parameter --needed

Packages: powerpill 2016.4.5.1-1, pm2ml 2016.4.6-1, python3-xcgf 2016.4.5-1, python3-xcpf 2016.4.5.2-1

sudo powerpill -Syu --needed
Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 989, in <module>
    sys.exit(run_main())
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 974, in run_main
    return main(args)
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 954, in main
    powerpill.refresh_databases()
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 782, in refresh_databases
    self.download(pm2ml_args, dbs=True, force=(self.pargs['refresh'] > 1))
  File "/usr/lib/python3.5/site-packages/Powerpill.py", line 582, in download
    self.pm2ml.resolve_targets_from_arguments(pm2ml_pargs)
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 595, in resolve_targets_from_arguments
    only_needed=pargs.needed,
  File "/usr/lib/python3.5/site-packages/pm2ml.py", line 569, in resolve_targets
    for p in handle.get_localdb().pkgcache:
NameError: name 'handle' is not defined

Xyne · 2016-05-24 20:57:27

That should be fixed now. Sorry for the delay.

dapolinario · 2016-05-24 22:03:54

Thanks!

Xyne · 2016-07-08 00:11:14

Pseudo-Poll
Currently the powerpill and bauerbill configuration files are written in JSON. Before I release the new backend, I could change the configuration file format to YAML. What do you prefer, and why?

I'm used to JSON but YAML seems more versatile as a superset of JSON with comments, etc. The flatter structure is likely easier to read and write too, but maybe more prone to errors. For powerpill/bauerbill, the JSON files are not particularly complicated though, so this isn't really an issue (imo).

If you want to see what a configuration file might look like in YAML, you can download my json2yaml script and run e.g.

json2yaml /etc/powerpill/powerpill.json -

Alad · 2016-07-09 06:29:37

I'm not familiar with YAML but it looks to be easier to be parse in a shell with standard tools (awk, grep, e.a) than JSON. Then again I'd simply look at the format which is easiest/simplest to implement for bauerbill.

Xyne · 2016-07-09 14:03:36

Both are just as easy to implement. The standard library provides a module for JSON and there is a readily available non-standard module for YAML (python-yaml, which wraps the libyaml C library recommended by the YAML specification authors). The internal representation after parsing is the same. The only consequence for users is the additional dependency on python-yaml, which is 213 KiB once installed.

I'm leaning towards YAML.

Xyne · 2016-07-10 20:41:03

The new backend is available for testing here. It is not entirely backwards-compatible so please give it a try now and get ready to migrate your configuration files and scripts. Extension developers are most welcome

txtsd · 2016-11-07 05:16:38

powerpill doesn't seem compatible with libressl (2.5.0)

Is this something that can be easily fixed?

EDIT: Just needed to recompile curl.

Last edited by txtsd (2016-11-07 07:28:40)

x33a · 2016-12-22 14:46:05

Hey Xyne, I have recently started using powerpill and I couldn't find any option to show download sizes of individual packages before downloading. It seems that this option used to exist previously. Also, on a related note, is there an option to confirm before powerpill starts downloading the packages?

PS: It is super awesome and downloads are happening in the blink of an eye

Xyne · 2016-12-23 14:11:02

I've made a note to consider adding both features. In the next few days I hope to officially release the pacboy backend that's been sitting around in a test archive for ages. Once that goes live I'll be actively making changes to the code again.

dapolinario · 2017-01-06 12:11:56

Please, rebuilt powerpill against the new version of python3 in ArchLinux repositories.

x33a · 2017-01-07 13:29:19

@dapolinario, you are responsible for rebuilding AUR packages yourself.

Arch Linux

#226 2016-04-05 00:11:16

Re: powerpill reborn: pacman wrapper for faster downloads

#227 2016-04-05 00:22:52

Re: powerpill reborn: pacman wrapper for faster downloads

#228 2016-04-05 00:27:14

Re: powerpill reborn: pacman wrapper for faster downloads

#229 2016-04-05 00:55:47

Re: powerpill reborn: pacman wrapper for faster downloads

#230 2016-04-05 00:58:08

Re: powerpill reborn: pacman wrapper for faster downloads

#231 2016-04-05 01:33:22

Re: powerpill reborn: pacman wrapper for faster downloads

#232 2016-04-20 19:50:22

Re: powerpill reborn: pacman wrapper for faster downloads

#233 2016-04-20 21:07:04

Re: powerpill reborn: pacman wrapper for faster downloads

#234 2016-04-20 21:19:37

Re: powerpill reborn: pacman wrapper for faster downloads

#235 2016-04-20 22:08:51

Re: powerpill reborn: pacman wrapper for faster downloads

#236 2016-04-20 22:11:38

Re: powerpill reborn: pacman wrapper for faster downloads

#237 2016-04-20 22:42:12

Re: powerpill reborn: pacman wrapper for faster downloads

#238 2016-04-20 22:44:04

Re: powerpill reborn: pacman wrapper for faster downloads

#239 2016-05-18 14:42:52

Re: powerpill reborn: pacman wrapper for faster downloads

#240 2016-05-24 20:57:27

Re: powerpill reborn: pacman wrapper for faster downloads

#241 2016-05-24 22:03:54

Re: powerpill reborn: pacman wrapper for faster downloads

#242 2016-07-08 00:11:14

Re: powerpill reborn: pacman wrapper for faster downloads

#243 2016-07-09 06:29:37

Re: powerpill reborn: pacman wrapper for faster downloads

#244 2016-07-09 14:03:36

Re: powerpill reborn: pacman wrapper for faster downloads

#245 2016-07-10 20:41:03

Re: powerpill reborn: pacman wrapper for faster downloads

#246 2016-11-07 05:16:38

Re: powerpill reborn: pacman wrapper for faster downloads

#247 2016-12-22 14:46:05

Re: powerpill reborn: pacman wrapper for faster downloads

#248 2016-12-23 14:11:02

Re: powerpill reborn: pacman wrapper for faster downloads

#249 2017-01-06 12:11:56

Re: powerpill reborn: pacman wrapper for faster downloads

#250 2017-01-07 13:29:19

Re: powerpill reborn: pacman wrapper for faster downloads

Board footer