You are not logged in.

#276 2018-04-30 12:24:39

cradlemann
Member
Registered: 2017-07-14
Posts: 14

Re: powerpill reborn: pacman wrapper for faster downloads

Xyne wrote:
cradlemann wrote:

During my searches I found couple of bugs, using flake8 tool and want to fix them.

Which bugs in which files? All I see are cosmetic errors.

return platform.machine() - missing import platform in Powerpill.py file

Offline

#277 2018-06-01 07:43:36

cradlemann
Member
Registered: 2017-07-14
Posts: 14

Re: powerpill reborn: pacman wrapper for faster downloads

Hi Xyne,
Is it possible to ask confirmation before download everything in powerpill and bauerbill? And add to bauerbill posibility to edit PKGBUILD files? Used to do it very often, when I used yaourt.

Offline

#278 2018-06-01 19:50:39

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

@cradlemann
I've added it to the todo list.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#279 2018-10-20 01:53:44

raggerv8
Member
From: Sweden
Registered: 2010-07-01
Posts: 110
Website

Re: powerpill reborn: pacman wrapper for faster downloads

error: xyne-x86_64: missing required signature
error: database 'xyne-x86_64' is not valid (invalid or corrupted database (PGP signature))

Error on your end?

Edit: seems to be fixed

Last edited by raggerv8 (2018-10-21 00:12:06)


Brottweiler@#archlinux

Offline

#280 2018-11-02 14:10:25

x4fyr
Member
Registered: 2014-10-12
Posts: 5

Re: powerpill reborn: pacman wrapper for faster downloads

There is a problem, when there are many packages to download via rsync, as rsync has hard coded arg limit of 1000. Maybe this should be considered in powerpill, so that so many packages are not downloaded with a single command.

Offline

#281 2018-11-04 02:21:10

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

@x4fyr
Are you hitting a 1000 package limit on a regular upgrade? I'm just curious because you're the first person to report the issue.

I've added the dealing with the limit  to my todo list and will try to get to it soonish.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#282 2018-11-04 14:36:16

x4fyr
Member
Registered: 2014-10-12
Posts: 5

Re: powerpill reborn: pacman wrapper for faster downloads

It was a regular upgrade on a system I haven't used some months. I dont think too many people use powerpill+rsync and don't upgrade their system with many packages that infrequent. Imho it is a very special case, so no hurry needed.
I circumvented it, by first upgrading just core and then the rest.

The limit in rsync is actually not 1000 files but 1000 arguments. So it will be a bit less, as the options will count into the limit, too.

Offline

#283 2018-11-04 16:03:44

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

Rsync commands are now split and run in parallel to handle the limit.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#284 2018-12-20 22:42:04

Ropid
Member
Registered: 2015-03-09
Posts: 1,069

Re: powerpill reborn: pacman wrapper for faster downloads

I just noticed that when I try to install "nvidia-utils", powerpill tries to download a different package named "nvidia-340xx-utils" that has that nvidia-utils name in its "provides=..." PKGBUILD setting. Here's the output when I stop it with Ctrl-C:

$ sudo powerpill -S nvidia-utils
[#9dff34 2.0MiB/22MiB(9%) CN:4 DL:692KiB ETA:30s]^C                                                                                                                                                                                          
Download Results:
gid   |stat|avg speed  |path/URI
======+====+===========+=======================================================
9dff34|INPR|   535KiB/s|/var/cache/pacman/pkg/nvidia-340xx-utils-340.107-3-x86_64.pkg.tar.xz

Status Legend:
(INPR):download in-progress.

aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.

The powerpill version:

$ pacman -Qi powerpill
Name            : powerpill
Version         : 2018.11-1
...

Last edited by Ropid (2018-12-20 22:45:41)

Offline

#285 2018-12-24 01:48:54

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

Fixed in python3-xcpf-2018.12-1. Thanks for reporting the error.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#286 2019-01-10 08:05:21

zerophase
Member
Registered: 2015-09-03
Posts: 228

Re: powerpill reborn: pacman wrapper for faster downloads

@cradlemann
This won't help you edit packages manually, but I believe it is possible to set up a script bauerbill uses to modify specific packages. There's a section in the json for per package scripts, I believe.

@xyne
With the pacman update for delta3 should we still be using rsync, or both rsync and delta3 for the best download times?

Offline

#287 2019-01-10 13:52:39

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: powerpill reborn: pacman wrapper for faster downloads

There was no pacman update for xdelta3. xdelta3 has *always* been an optional dependency of the pacman binary, *if* you have a repository which provides delta packages, which the official ones do not.

Recently the pacman PKGBUILD was updated to reflect this longstanding fact, but that is merely advertisement, not implementation of a feature. wink


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

#288 2019-01-10 19:42:11

zerophase
Member
Registered: 2015-09-03
Posts: 228

Re: powerpill reborn: pacman wrapper for faster downloads

@eschwartz

Ah, alright. xdelta3 was just, recently, split off from pacman to a separate package, or updated to be listed as an optional dependency?  Yeah, there's just that one guy advertising on the forums about having an xdelta3 server setup.

@xyne
Is my hooks array setup correctly? I have the script in /etc/bauerbill/hooks/linux-ck/prebuild.

  "hooks" : {
    "directory" : ["/etc/bauerbill/hooks"],
    "arguments" : {
      "default": ["{PackageBase}", "{LastModified}"]
    },
    "commands" : {
      "prebuild" :  {
	"linux-ck" : ["linux-ck_modify.sh"],
        "default" : []
      }
    }
  }

Last edited by zerophase (2019-01-10 19:42:35)

Offline

#289 2019-01-18 19:33:55

raggerv8
Member
From: Sweden
Registered: 2010-07-01
Posts: 110
Website

Re: powerpill reborn: pacman wrapper for faster downloads

I started getting this today...

Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.7/site-packages/Powerpill.py", line 43, in <module>
    import pm2ml
  File "/usr/lib/python3.7/site-packages/pm2ml.py", line 38, in <module>
    import XCPF.PacmanConfig
  File "/usr/lib/python3.7/site-packages/XCPF/PacmanConfig.py", line 28, in <module>
    from pycman.config import pacman_conf_enumerator, _logmask, cb_log, LIST_OPTIONS, BOOLEAN_OPTIONS
ImportError: cannot import name 'pacman_conf_enumerator' from 'pycman.config' (/usr/lib/python3.7/site-packages/pycman/config.py)

I noticed another user had the same issue and they wrote about it on the AUR page, and you replied.

Xyne @ AUR wrote:

Can you compare the installed file at /usr/lib/python3.7/site-packages/Powerpill.py to Powerpill.py in the source archive?

My /usr/lib/python3.7/site-packages/Powerpill.py and the source archive Powerpill.py are identical.

Last edited by raggerv8 (2019-01-18 19:34:52)


Brottweiler@#archlinux

Offline

#290 2019-01-19 02:49:07

crazymanjinn
Member
Registered: 2019-01-19
Posts: 1

Re: powerpill reborn: pacman wrapper for faster downloads

I'm seeing the stack trace same thing as raggerv8 above.

It appears to be due to an update in pyalpm package. If I downgrade to version 0.8.4-2, then powerpill works without a problem. However, upgrading to 0.8.5-1 or newer, causes this to break with that same import error.

edit: looking into this more, it looks like pyalpm completely changed the format of pycman/config.py, just looking at relevant part of the diff to the import error in the stack trace:

diff --git a/pycman/config.py b/pycman/config.py
index f62a946..84f520e 100644
--- a/pycman/config.py
+++ b/pycman/config.py
@@ -63,65 +76,105 @@ BOOLEAN_OPTIONS = (
    'ShowSize',
    'UseDelta',
    'TotalDownload',
-   'CheckSpace'
+   'CheckSpace',
+   'VerbosePkgLists',
+   'ILoveCandy',
+   'Color',
+   'DisableDownloadTimeout',
 )
 
-def pacman_conf_enumerator(path):
-   filestack = []
-   current_section = None
-   filestack.append(open(path))
+class PacmanConfEnumeratorSession():
+
+   def __init__(self, path):
+       self.path = path

I guess python3-xcpf needs to be updated to handle all the changes in pyalpm?

Last edited by crazymanjinn (2019-01-19 03:03:50)

Offline

#291 2019-01-19 03:04:57

zerophase
Member
Registered: 2015-09-03
Posts: 228

Re: powerpill reborn: pacman wrapper for faster downloads

It's from all of the free floating config functions being placed in a class in pyalpm, PacmanConfig.

Offline

#292 2019-01-22 20:43:16

barraponto
Member
From: Campinas, Brazil
Registered: 2009-10-15
Posts: 29
Website

Re: powerpill reborn: pacman wrapper for faster downloads

Is there a repository I can make pull request to?

Offline

#293 2019-06-09 14:58:40

txtsd
Member
Registered: 2014-10-02
Posts: 96
Website

Re: powerpill reborn: pacman wrapper for faster downloads

Does not work with pacman-git.

λ powerpill -Syu
Traceback (most recent call last):
  File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.7/site-packages/Powerpill.py", line 41, in <module>
    import pyalpm
ImportError: /usr/lib/python3.7/site-packages/pyalpm.cpython-37m-x86_64-linux-gnu.so: undefined symbol: alpm_option_set_deltaratio

[CPU] AMD Ryzen 5 2400G
[iGPU] AMD RX Vega 11
[Kernel] linux-zen
[sway] [zsh] Arch user since [2014-09-01 02:09]

Offline

#294 2019-06-15 03:09:01

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

Thanks for the heads-up, but the error is in pyalpm. Please test pyalpm-git. If the error has not been fixed there, please report it to the pyalpm devs.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#295 2019-10-23 02:22:39

SolarAquarion
Member
Registered: 2014-05-18
Posts: 92

Re: powerpill reborn: pacman wrapper for faster downloads

/usr/share/makepkg/buildenv/buildflags.sh: line 28: build_options: readonly variable
/usr/share/makepkg/buildenv/compiler.sh: line 30: build_options: readonly variable
/usr/share/makepkg/buildenv/makeflags.sh: line 28: build_options: readonly variable

That's what i get when i do build.sh

Offline

#296 2019-10-25 05:25:45

Xyne
Administrator/PM
Registered: 2008-08-03
Posts: 6,963
Website

Re: powerpill reborn: pacman wrapper for faster downloads

@SolarAquarion
Thanks for notifying me of the errors, which were due to changes in makepkg in pacman 5.2.0. These have been fixed in bauerbill-2019.

For future bauerbill errors and feedback, please post in the Bauerbill thread.


My Arch Linux StuffForum EtiquetteCommunity Ethos - Arch is not for everyone

Offline

#297 2020-12-12 03:22:11

FirstAirBender
Member
Registered: 2019-09-13
Posts: 15

Re: powerpill reborn: pacman wrapper for faster downloads

Please use the full path to python when launching the application.

In the powerpill application, I see

python3 -m Powerpill "$@"

Today I got the error: /home/c/.pyenv/versions/3.9.1/bin/python3: No module named Powerpill
This because I manage some local python using pyenv, so I had to run

pyenv global system

for powerpill to work

I use yay as my AUR helper.

Last edited by FirstAirBender (2020-12-12 03:22:26)

Offline

#298 2020-12-12 10:21:56

V1del
Forum Moderator
Registered: 2012-10-16
Posts: 21,427

Re: powerpill reborn: pacman wrapper for faster downloads

It really isn't the job of the single application/utility to workaround your environment. Why are you running a system package within your local build/pyenv environment?

Offline

#299 2020-12-13 08:05:49

FirstAirBender
Member
Registered: 2019-09-13
Posts: 15

Re: powerpill reborn: pacman wrapper for faster downloads

V1del wrote:

It really isn't the job of the single application/utility to workaround your environment. Why are you running a system package within your local build/pyenv environment?

I wasn't trying to run it within the pyenv environment. I use pyenv for other python stuff. Yay always displays a warning when run with sudo, so I don't always use sudo to run it, which is why it picks up stuff from my user's environment.

Anyways after searching a bit, I created the following alias for running yay.

alias yay='env --ignore-environment sudo --login --user="$USER" yay '

Feel free to judge

Last edited by FirstAirBender (2020-12-13 08:11:00)

Offline

#300 2020-12-13 13:29:58

eschwartz
Fellow
Registered: 2014-08-08
Posts: 4,097

Re: powerpill reborn: pacman wrapper for faster downloads

Python software really should be getting launched via wrapper scripts looking like this:

#!/usr/bin/python3

import sys
from Powerpill import main

sys.exit(main())

There is zero reason to wrap everything in bash just to interpose another process -- unless wasting a few CPU cycles is a reason. Setuptools entry points handle this seamlessly and get installed during "python3 setup.py install", reducing the number of lines in a PKGBUILD.
Setuptools entry points also guarantee the #! shebang points to the absolute path of the python executable used to perform "python3 setup.py install", meaning no env lookup, meaning it is guaranteed to never run in a virtualenv, regardless of your interactive shell environment (unless it was installed from a virtualenv, in which case it will *always* run in that virtualenv).

Nevertheless, the bash entry point (and the one-file command line tool installed as a module to begin with) is a classic feature of Xyne software. wink

Anyway. Yeah. There's ways to fix this, and even ways to simplify packaging *and* be more correct about it which just fix this by accident.


Managing AUR repos The Right Way -- aurpublish (now a standalone tool)

Offline

Board footer

Powered by FluxBB