You are not logged in.
@oboedad55
I have updated Powerpill to ignore certain non-zero exit status values from Aria2.
If heftig does not supply signatures for his packages and databases then I do not understand why he recommends the SigLevel Optional.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
@oboedad55
I have updated Powerpill to ignore certain non-zero exit status values from Aria2.If heftig does not supply signatures for his packages and databases then I do not understand why he recommends the SigLevel Optional.
@Xyne: Thanks for the update. I changed heftig's repo SigLevel to Never and it seemed to get rid of the error.
Registered Linux user #436067
Offline
Hi xyne I'm getting an error downloading the community db - pacman downloads it no problem so I guess it's an aria2 issue?
[ERROR] CUID#10 - Download aborted. URI=ftp://mirror.us.leaseweb.net/archlinux/community/os/i686/community.db
Exception: [AbstractCommand.cc:355] errorCode=1 URI=ftp://mirror.us.leaseweb.net/archlinux/community/os/i686/community.db
-> [RequestGroup.cc:754] errorCode=1 Download aborted.
-> [DefaultBtProgressInfoFile.cc:278] errorCode=1 total length mismatch. expected: 1897972, actual: 1894679
It seems to occur whatever the server that's visited.
(I have set up some reflection option in the conf file to update the latest 50 and most recently updated 10 so it can change from one try to the next).
I have tried the reflector option of http and rsync (not sure that's relevant).
"...one cannot be angry when one looks at a penguin." - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle
Offline
@skanky
There is no way that Reflector would pull in an FTP mirror if the protocol option is set to "http" (and/or "rsync"). By default, powerpill only uses HTTP mirrors with aria2c. Rsync will not work at all with it and FTP often leads to errors when trying to emulate "HEAD" requests due to varying server configurations.
If you have an FTP mirror in /etc/pacman.d/mirrorlist then remove it, otherwise check the powerpill.json file again because you must have misconfigured it. If you need further help then post the file and I will take a look at it.
I am not entirely sure why this happens, but it's a problem between aria2c and the FTP server.
While considering this I realized that it would be useful to have an option to blacklist specific mirrors. I have therefore replaced the "--grep" option with "--include", and added an "--exclude" option to exclude mirrors by regex. I still strongly recommend using the protocol option to limit the output to HTTP. The new filters should be used to target specific mirrors.
edit: the updated package is only available in my repo. I will move it to [community] in a few days.
Last edited by Xyne (2013-01-04 02:22:05)
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Thanks for the response. I did have two ftp servers in mirrorlist, which was created using reflector (I had assumed that powerpill's reflector stuff overrode that). I have added the -p option to the call now and now only get http servers.
However I still get the error:
[ERROR] CUID#10 - Download aborted. URI=http://archlinux.mirrors.uk2.net/community/os/i686/community.db
Exception: [AbstractCommand.cc:355] errorCode=1 URI=http://archlinux.mirrors.uk2.net/community/os/i686/community.db
-> [RequestGroup.cc:754] errorCode=1 Download aborted.
-> [DefaultBtProgressInfoFile.cc:278] errorCode=1 total length mismatch. expected: 1899966, actual: 1894679
Here is my powerpill.json file as it currently is:
{
"aria2": {
"args": [
"--allow-overwrite=true",
"--always-resume=false",
"--auto-file-renaming=false",
"--check-integrity=true",
"--conditional-get=true",
"--continue=false",
"--file-allocation=none",
"--log-level=error",
"--max-concurrent-downloads=50",
"--max-connection-per-server=5",
"--min-split-size=5M",
"--show-console-readout=false",
"--split=10"
],
"path": "/usr/bin/aria2c"
},
"pacman": {
"config": "/etc/pacman.conf",
"path": "/usr/bin/pacman"
},
"pacserve": {
"server": null
},
"powerpill" : {
"ask" : true
},
"reflector": {
"args": [
"--protocol",
"http",
"--latest",
"50",
"--fastest",
"10"
]
},
"rsync": {
"args": [
"--no-motd",
"--progress"
],
"db only": true,
"path": "/usr/bin/rsync",
"servers": [
]
}
}
"...one cannot be angry when one looks at a penguin." - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle
Offline
Please post the output of
ls -l /var/lib/pacman/sync/
I suspect there may be a leftover community.db.aria2 file in there. That is the only thing that I can think of which would create any expectation of length. If that is the case, remove that file and community.db and run powerpill again.
I already have some commented code in powerpill to deal with leftover .aria2 files. I'll clean it up and add an option to purge them (or think of a better way to determine unrecoverable downloads).
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Yep that was it, and the fix fixed it.
Hopefully now that I've set reflector to always update mirrorlist with http, I won't hit that again.
Just one question though, how do the reflector settings work with powerpill, given that it still uses mirrorlist?
Again, many thanks for an excellent tool and your support.
"...one cannot be angry when one looks at a penguin." - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle
Offline
Reflector is used to augment the default mirror list instead of replacing it. Part of that is legacy from the original version of Powerpill. I considered it less safe to download databases from random (and thus potentially untrusted) mirrors so I restricted database downloads to the user's mirrorlist. Packages are checksummed against values in the database so pulling them from random servers seemed relatively safe.
With the advent of package signing I no longer restrict the list, although technically I probably should for unsigned database. I could easily recreate the old behavior now by changing just a few lines of code, but managing the distinction between signed and unsigned repo databases would probably require two separate database downloads. Given that only the official databases are affected by this, it's easier to just add an rsync server to the rsync section of the powerpill.json file. The official databases will then always be downloaded from a "trusted" mirror.
I have now added a new option to clean up leftover files: --powerpill-clean
If the length mismatch error occurs again, please let me know if that options solves it.
Thanks for the feedback.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
That all makes sense. Many thanks.
"...one cannot be angry when one looks at a penguin." - John Ruskin
"Life in general is a bit shit, and so too is the internet. And that's all there is." - scepticisle
Offline
Hey, xyne, will the bauerbill be back? Or it will now be the part of powerpill?
Offline
Hey, xyne, will the bauerbill be back? Or it will now be the part of powerpill?
I don't have any direct plans to resurrect bauerbill right now. I would like to, but there are some technical issues that need to be solved for "safely" automating AUR support and I just don't have the time to work on them. I'm hoping that we will eventually get some sort of package signing in the AUR that will eliminate some of the issues.
It wouldn't be that difficult to implement basic support with python3-aur, but I know if I start tinkering with that then I will keep tinkering and I just can't risk spending that time right now.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
I felt a need to tell you that the professional way you treat open source development is an inspiration, for me, at least.
Offline
I felt a need to tell you that the professional way you treat open source development is an inspiration, for me, at least.
Thanks! Although to be honest, it feels like I have a long way to go before my approach resembles anything even close to professional, but I'll gladly take the complement.
I think I'll add powerpill integration into Aura.
Let me know if you hit any snags with the integration. I haven't fully tested all options and I may have missed some. I have also implemented a custom group dialogue which may clash with some wrappers.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Finally got around to giving powerpill a go and I have no complaints. It is fast and it is an excellent application. Thank you.
Offline
Why every time I install a package, it downloads the package again?
Arch Linux: The power in your hands!
Offline
Why every time I install a package, it downloads the package again?
Maybe you are misinterpreting the output and only think it is downloading something twice.
Maybe you have misconfigured the "reflect databases" option.
Maybe you are just really unlucky and happen to upgrade just as the new packages are uploaded to the server.
Without more information, I cannot help you.
Useful information would be:
* the command output
* your powerpill.json file
* your pacman.conf file
Post them here in [code] tags or Pastebin them and post links.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Come on! First I just installed powerpill, did some configuration.
* the command output
danielr ~ $ sudo powerpill -S dvd+rw-tools
[sudo] password for danielr:
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
Resultados do Download:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
a5491c|OK | 188KiB/s|/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
09833b|OK | 18KiB/s|/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
Legenda de estados:
(OK):download completo.
resolvendo dependências...
procurando por conflitos inter-relacionados...
Alvos (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Tamanho Total Instalado: 2,81 MiB
Prosseguir com a instalação? [S/n] s
(2/2) verificando integridade do pacote [######################] 100%
(2/2) carregando arquivos do pacote [######################] 100%
(2/2) verificando conflitos de arquivo [######################] 100%
(2/2) verificando espaço em disco disponível [######################] 100%
(1/2) instalando cdrkit [######################] 100%
(2/2) instalando dvd+rw-tools
* powerpill.json
{
"aria2": {
"args": [
"--allow-overwrite=true",
"--always-resume=false",
"--auto-file-renaming=false",
"--check-integrity=true",
"--conditional-get=true",
"--continue=false",
"--file-allocation=none",
"--log-level=error",
"--max-concurrent-downloads=50",
"--max-connection-per-server=5",
"--min-split-size=5M",
"--show-console-readout=false",
"--split=10"
],
"path": "/usr/bin/aria2c"
},
"pacman": {
"config": "/etc/pacman.conf",
"path": "/usr/bin/pacman"
},
"pacserve": {
"server": null
},
"powerpill": {
"ask": true,
"reflect databases": false
},
"reflector": {
"args": [
"--protocol",
"http",
"--latest",
"50"
]
},
"rsync": {
"args": [
"--no-motd",
"--progress"
],
"db only": true,
"path": "/usr/bin/rsync",
"servers": []
}
}
* pacman.conf
#
# /etc/pacman.conf
#
# See the pacman.conf(5) manpage for option and repository directives
#
# GENERAL OPTIONS
#
[options]
# The following paths are commented out with their default values listed.
# If you wish to use different paths, uncomment and update the paths.
#RootDir = /
#DBPath = /var/lib/pacman/
#CacheDir = /var/cache/pacman/pkg/
#LogFile = /var/log/pacman.log
#GPGDir = /etc/pacman.d/gnupg/
HoldPkg = pacman glibc
# If upgrades are available for these packages they will be asked for first
SyncFirst = pacman
#XferCommand = /usr/bin/curl -C - -f %u > %o
#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u
#CleanMethod = KeepInstalled
Architecture = auto
# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
#IgnorePkg =
#IgnoreGroup =
#NoUpgrade =
#NoExtract =
# Misc options
#UseSyslog
#UseDelta
#TotalDownload
CheckSpace
#VerbosePkgLists
# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
#SigLevel = Optional TrustedOnly
# NOTE: You must run `pacman-key --init` before first using pacman; the local
# keyring can then be populated with the keys of all official Arch Linux
# packagers with `pacman-key --populate archlinux`.
#
# REPOSITORIES
# - can be defined here or included from another file
# - pacman will search repositories in the order defined here
# - local/custom mirrors can be added here or in separate files
# - repositories listed first will take precedence when packages
# have identical names, regardless of version number
# - URLs will have $repo replaced by the name of the current repo
# - URLs will have $arch replaced by the name of the architecture
#
# Repository entries are of the format:
# [repo-name]
# Server = ServerName
# Include = IncludePath
#
# The header [repo-name] is crucial - it must be present and
# uncommented to enable the repo.
#
# The testing repositories are disabled by default. To enable, uncomment the
# repo name header and Include lines. You can add preferred servers immediately
# after the header, and they will be used before the default mirrors.
#[testing]
#SigLevel = PackageRequired
#Include = /etc/pacman.d/mirrorlist
[core]
SigLevel = PackageRequired
Include = /etc/pacman.d/mirrorlist
[extra]
SigLevel = PackageRequired
Include = /etc/pacman.d/mirrorlist
#[community-testing]
#SigLevel = PackageRequired
#Include = /etc/pacman.d/mirrorlist
[community]
SigLevel = PackageRequired
Include = /etc/pacman.d/mirrorlist
[repo-ck]
SigLevel = PackageRequired
Server = http://repo-ck.com/$arch
[xyne-i686]
SigLevel = Required
Server = http://xyne.archlinux.ca/repos/xyne
# An example of a custom package repository. See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#SigLevel = Optional TrustAll
#Server = file:///home/custompkgs
Arch Linux: The power in your hands!
Offline
Come on! First I just installed powerpill, did some configuration.
* the command output
danielr ~ $ sudo powerpill -S dvd+rw-tools [sudo] password for danielr: [NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz [NOTICE] Download Completo: /var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz [NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz [NOTICE] Download Completo: /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz Resultados do Download: gid |stat|avg speed |path/URI ======+====+===========+======================================================= a5491c|OK | 188KiB/s|/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz 09833b|OK | 18KiB/s|/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz Legenda de estados: (OK):download completo. resolvendo dependências... procurando por conflitos inter-relacionados... Alvos (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4 Tamanho Total Instalado: 2,81 MiB Prosseguir com a instalação? [S/n] s (2/2) verificando integridade do pacote [######################] 100% (2/2) carregando arquivos do pacote [######################] 100% (2/2) verificando conflitos de arquivo [######################] 100% (2/2) verificando espaço em disco disponível [######################] 100% (1/2) instalando cdrkit [######################] 100% (2/2) instalando dvd+rw-tools
Ok, so then it was the first case:
Maybe you are misinterpreting the output and only think it is downloading something twice.
If it was downloading twice, you would see this:
resolving dependencies...
looking for inter-conflicts...
Targets (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Total Download Size: 0.66 MiB
Total Installed Size: 2.97 MiB
Net Upgrade Size: 0.00 MiB
Proceed with installation? [Y/n]
instead of
resolving dependencies...
looking for inter-conflicts...
Targets (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Total Installed Size: 2.97 MiB
Net Upgrade Size: 0.00 MiB
Proceed with installation? [Y/n]
what follows is the package verification and installation, not a download.
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Actually are downloading twice. Check the before and the after, the date changing.
danielr ~ $ ls -la /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
-rw-r--r-- 1 root root 77308 Fev 28 11:38 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
danielr ~ $ sudo powerpill -S dvd+rw-tools
[sudo] password for danielr:
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
Resultados do Download:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
8522a5|OK | 25KiB/s|/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
295376|OK | 91KiB/s|/var/cache/pacman/pkg/cdrkit-1.1.11-2-i686.pkg.tar.xz
Legenda de estados:
(OK):download completo.
resolvendo dependências...
procurando por conflitos inter-relacionados...
Alvos (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Tamanho Total Instalado: 2,81 MiB
Prosseguir com a instalação? [S/n] n
danielr ~ $ ls -la /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
-rw-r--r-- 1 root root 77308 Fev 28 14:10 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-i686.pkg.tar.xz
Arch Linux: The power in your hands!
Offline
Ok, that includes more information than what you had posted above.
I do not know what is going on. I suspect that there is something wrong with the existing file as both powerpill and aria2c should detect it and verify its integrity. Compare the checksums of the old and new files (md5sum, sha256sum, etc).
Last edited by Xyne (2013-03-01 08:26:37)
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Done! But the checks are identical.
daniel ~ $ sudo powerpill -S dvd+rw-tools
[ERROR] CUID#6 - Download aborted. URI=http://mirrors6.ustc.edu.cn/archlinux/extra/os/x86_64/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
Exception: [AbstractCommand.cc:304] errorCode=19 URI=http://mirrors6.ustc.edu.cn/archlinux/extra/os/x86_64/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
-> [AbstractCommand.cc:737] errorCode=19 CUID#6 - Name resolution for mirrors6.ustc.edu.cn failed:DNS server returned answer with no data
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
Resultados do Download:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
50b98f|OK | 66KiB/s|/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
0c21f2|OK | 259KiB/s|/var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
Legenda de estados:
(OK):download completo.
resolvendo dependências...
procurando por conflitos inter-relacionados...
Alvos (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Tamanho Total Instalado: 2,97 MiB
Prosseguir com a instalação? [S/n] n
daniel ~ $ md5sum /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
2207902cd6b351ddd8d678cfb7ad7b43 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
daniel ~ $ sha256sum /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
ba027e8ae95d4598fdc11778d99c2a0d5ccf0425f71073314fa7932897686176 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
daniel ~ $ sudo powerpill -S dvd+rw-tools
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
[NOTICE] Verificação concluida com sucesso. arquivo=/var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
[NOTICE] Download Completo: /var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
Resultados do Download:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
2dfc10|OK | 44KiB/s|/var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
d5d65d|OK | 160KiB/s|/var/cache/pacman/pkg/cdrkit-1.1.11-2-x86_64.pkg.tar.xz
Legenda de estados:
(OK):download completo.
resolvendo dependências...
procurando por conflitos inter-relacionados...
Alvos (2): cdrkit-1.1.11-2 dvd+rw-tools-7.1-4
Tamanho Total Instalado: 2,97 MiB
Prosseguir com a instalação? [S/n] n
daniel ~ $ md5sum /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz 2207902cd6b351ddd8d678cfb7ad7b43 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
daniel ~ $ sha256sum /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz ba027e8ae95d4598fdc11778d99c2a0d5ccf0425f71073314fa7932897686176 /var/cache/pacman/pkg/dvd+rw-tools-7.1-4-x86_64.pkg.tar.xz
Arch Linux: The power in your hands!
Offline
After spending some time on this, I have concluded that it's a bug in aria2c. It doesn't handle certain options correctly. I will contact the developer to get this fixed.
If you use the pacserve option you can avoid this issue. That is why I never noticed it. The rsync option should work as well for most packages.
edit: typo
Last edited by Xyne (2013-03-01 15:30:58)
My Arch Linux Stuff • Forum Etiquette • Community Ethos - Arch is not for everyone
Offline
Thanks!
Arch Linux: The power in your hands!
Offline
Using "makepkg -si" with powerpill (defined by $ PACMAN), when he installs the missing dependencies, they are not installed and it starts downloading the package target. Using without powerpill, everything happens as expected. I tested the package usb-creator.
Arch Linux: The power in your hands!
Offline