p2pacman - Bittorrent powered pacman wrapper

onny · 2013-05-15 02:52:06

Hey,
I read a lot about ideas regarding p2p support for pacman here in the forum, but I couldn't find any appropriate implementation yet. So I hacked together a quick POC (really bad code quality, a lot of hardcoded settings at this time) which might demonstrate the usefulness of such a feature:

1) p2pacman: Client program which synchronises the package db and performs a full system upgrade (equivalent to -Syu) while downloading every single package via torrent simultaneously. If there aren't enough peers or seeders the program fallbacks to normal direct download for the specific package. Note: For testing purpose I didn't included any webseeds yet, so a peer is required which already performed the same upgrade and therefore is seeding the same packages.

2) p2pacmand: Should be started via systemd, runs as a daemon and rescans the package cache for torrent files. All available packages will be seeded (local network and internet).

3) p2pacman-server: Creates a mirror for the ArchLinux repository and in addition to that generates info hashes for opentracker and torrent files.

If you want to test it, be sure you use a mirror which already offers torrent files in the format of: "package_xy.pkg.xz.torrent", e.g. add this to /etc/pacman.d/mirrorlist:

Server = http://mirror.project-insanity.org/$repo/os/$arch

AUR Link: https://aur.archlinux.org/packages/p2pacman-git/

Example usage:

[root@mail-pub ~]# p2pacman
:: Synchronizing package databases...
 testing is up to date
 core is up to date
 extra is up to date
 community-testing is up to date
 community is up to date
:: Starting full system upgrade...
downloading torrent metadata ...
Loading torrent file in thread
Loading torrent file in thread
starting torrents ...
downloading 2 package(s):
 - amule-10802-2-x86_64.pkg.tar.xz: 0.00% complete (down: 0.0 kb/s up: 0.1 kB/s peers: 0)
 - htop-1.0.2-2-x86_64.pkg.tar.xz: 100.00% complete (down: 4.9 kb/s up: 0.4 kB/s peers: 0)
downloading 1 package(s):
 - amule-10802-2-x86_64.pkg.tar.xz: 0.00% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0)
timeout reached, skipping torrent. idle time: 60
starting manual direct download of: amule-10802-2-x86_64.pkg.tar.xz
all downloads finished!
installing packages ...
loading packages...
resolving dependencies...
looking for inter-conflicts...

Packages (2): amule-10802-2  htop-1.0.2-2

Total Installed Size:   20.06 MiB
Net Upgrade Size:       0.31 MiB

:: Proceed with installation? [Y/n] Y
(2/2) checking keys in keyring                                               [###########################################] 100%
(2/2) checking package integrity                                             [###########################################] 100%
(2/2) loading package files                                                  [###########################################] 100%
(2/2) checking for file conflicts                                            [###########################################] 100%
(2/2) checking available disk space                                          [###########################################] 100%
(1/2) upgrading amule                                                        [###########################################] 100%
(2/2) upgrading htop                                                         [###########################################] 100%
New optional dependencies for htop
    lsof: show files opened by a process
    strace: attach to a running process

Disclaimer: Please note that at least the client requires root permissions for package operations and the code is definitely not safe, so this could be a security risk and harm your system. I'm not responsible for any damage, test it at your own risk.

Note: Unfortunately libtorrent bindings for python3 works, at least for me, only on x64 systems. Hope this bug will be soon resolved.

Feedback and contributions welcome

Last edited by onny (2013-05-15 02:53:11)

robotanarchy · 2013-05-30 19:23:40

This looks very interesting. We might try it in our local hacker space, where we have a rather bad internet connection, but 3 arch users that could exchange packages that way easily.

Is it possible to limit seeding to LAN?

EDIT: would these machines then discover each other automagically in the LAN or would we need some kind of tracker?

Last edited by robotanarchy (2013-05-30 19:26:14)

HalosGhost · 2013-05-30 19:28:14

Oooh, this is very interesting. P2P support for pacman would be very exciting. I look forward to seeing how this progresses.

All the best,

-HG

kdb424 · 2013-05-30 19:50:11

Looks interesting. Will test. I'm on an x86 chroot, so I'll give it a run for it's money.

hydrosIII · 2014-01-17 04:08:06

Hi, Im trying yo use p2pacman, however the current mirror

http://mirror.project-insanity.org/$repo/os/$arch

seems down, is there another mirror with torrent support???

good project!!

Arch Linux

#1 2013-05-15 02:52:06

p2pacman - Bittorrent powered pacman wrapper

#2 2013-05-30 19:23:40

Re: p2pacman - Bittorrent powered pacman wrapper

#3 2013-05-30 19:28:14

Re: p2pacman - Bittorrent powered pacman wrapper

#4 2013-05-30 19:50:11

Re: p2pacman - Bittorrent powered pacman wrapper

#5 2014-01-17 04:08:06

Re: p2pacman - Bittorrent powered pacman wrapper

Board footer