pacserve - easily share Pacman packages between computers

Xyne · 2013-04-14 04:32:13

@spychalski
Are you sure that you are using the latest version? The restart delay in the last release should solve that problem. If you have upgraded without re-installing then you need to run "systemctl --system daemon-reload".

@spacebison & nplatis
The environment file is indeed the issue. After some testing I moved the variables to the end of the file and it seems to work here. I didn't both digging through the systemd source code to check how it loads the file, but it clearly makes some unexpected assumptions about the file layout.

Try the latest version and let me know if it works.

nplatis · 2013-04-14 13:46:34

Tried with pacserve 2013.4.13.2-1.

status gives me

python2 /usr/bin/pacserve --pid-file --multicast

from which the actual pid file is missing! pacserve seems to start, but I don't think it starts correctly (it probably misuderstands --multicast for the pid-file).

If I edit /etc/pacserve/pacserve.service.conf and remove all comments, I get

python2 /usr/bin/pacserve --pid-file /run/pacserved.pid --multicast

and pacserve starts correctly.

EDIT: ... and after some discussion in the systemd-devel mailing list it was indeed a bug in their parser: https://bugs.freedesktop.org/show_bug.cgi?id=63477

Last edited by nplatis (2013-04-14 14:02:22)

Xyne · 2013-04-14 15:23:53

nplatis wrote:

EDIT: ... and after some discussion in the systemd-devel mailing list it was indeed a bug in their parser: https://bugs.freedesktop.org/show_bug.cgi?id=63477

I have removed all comments from that file for now and will restore them later when this bug is fixed upstream.

The last release was indeed creating a file named "--multicast". It should have been removed on exit but if it still remains in the system root directory, you can remove it manually if you are sure that no pacserve service is running.

spychalski · 2013-04-14 17:21:19

Sorry about the confusion, I fixed my issues by completely removing pacserve, rebooting and reinstalling. Everything is fine now!

spacebison · 2013-04-20 08:27:39

I noticed that there's a "RestartSet=15" option in the .service file. Shouldn't it be "RestartSec"?

Last edited by spacebison (2013-04-20 08:27:49)

Xyne · 2013-04-20 18:26:21

@spacebison
fixed, thanks

Xyne · 2013-05-10 21:00:31

update OP to reflect rewrite

onny · 2013-05-10 22:25:52

Wow, nice to see such a project emerging but I just wondering why we shouldn't use P2P technology like Bittorrent to spread package updates locally and on the internet?

karol · 2013-05-10 22:28:35

onny wrote:

Wow, nice to see such a project emerging

FYI, this isn't a new project, just a new thread https://bbs.archlinux.org/viewtopic.php?id=117094

onny wrote:

I just wondering why we shouldn't use P2P technology like Bittorrent to spread package updates locally and on the internet?

Please search before posting:
https://bbs.archlinux.org/viewtopic.php?id=9399
https://bbs.archlinux.org/viewtopic.php?id=68058
https://bbs.archlinux.org/viewtopic.php?id=90970
https://bbs.archlinux.org/viewtopic.php?id=91201
https://bbs.archlinux.org/viewtopic.php?id=115731
https://bbs.archlinux.org/viewtopic.php?id=125426

Last edited by karol (2013-05-10 22:30:15)

Xyne · 2013-05-10 22:55:17

karol wrote:

FYI, this isn't a new project, just a new thread https://bbs.archlinux.org/viewtopic.php?id=117094

Ah, there it is. I didn't have the forum ID in the project metadata, and I didn't find that thread when I searched, so I didn't think there was a thread.

merged

nomorewindows · 2013-05-13 15:28:54

I started having trouble with one of my pacserve clients. It locked up during pacman -Syu, had to Ctrl-Z to break out of it.
Journalctl -xn shows:

INFO: x.x.x.x: "GET /request/core/i686/pkg/systemd-204-1-i686.pkg.tar.xz HTTP/1.1" 303 -
INFO: x.x.x.x:  redirecting to pkg/systemd-204-1-i686.pkg.tar.xz

Then the next message has one more pkg added to the directory and keeps adding onto it and the messages keep repeating.
systemctl restart pacserve gives an error:
systemctl status pacserve

systemd[1]: pacserve.service: main process exited, code=exited, status=1/FAILURE
systemd[1]: pacserve.service entered a failed state.

Then python3 -mPacserve --multicast

/usr/bin/python3: No module named Pacserve

nplatis · 2013-05-13 16:33:45

Especially for the last error, see the discussion on aur: https://aur.archlinux.org/packages/pacserve/

nomorewindows · 2013-05-13 16:53:53

nplatis wrote:

Especially for the last error, see the discussion on aur: https://aur.archlinux.org/packages/pacserve/

Since I pull it from xyne's repos, I don't see the AUR messages. Made suggested change in /usr/bin/pacserve to python3 -mThreadedServers.Pacserve "$@" and had to stop remaining python3 process continuing to run with old arguments. Should be working now.

Xyne · 2013-05-13 18:46:51

@nomorewindows
The first error looks like you might be using an old /etc/pacman.d/pacserve file. The format has changed. You should have a pacserve.pacnew file. If you do then merge the changes, else post the file.

See my comment on the AUR page regarding the systemd service error and how to deal with it. I am working on a fix and will hopefully have it ready tonight.

The last error has also been fixed in the package.

Thanks for the feedback... it's only by breaking stuff that I find out who's using my packages.

nomorewindows · 2013-05-13 21:25:34

I don't have any /etc/pacman.d/pacserve.pacnew

It just reads:

Server = http://localhost:15678/pkg/?repo=$repo&arch&file=

Xyne · 2013-05-13 21:38:41

I've found the error and fixed it locally. I'll upload it as soon as I finish fixing something else.

Xyne · 2013-05-14 01:12:00

Done.

Xyne, on the AUR page, wrote:

All reported errors should be fixed now (redirection bugs, sync database corruption, systemd service errors).
To get everything working again, do the following (skip unneeded steps, obviously):
1) systemctl stop pacserve.service
2) systemctl status pacserve.service (note the PID)
3) ps -e python3 (check for matching PIDs )
4) kill -9 <pid> ...
5) remove pacserve from pacman.conf
6) pacman -Syyu
7) update pacserve and python3-threaded servers
8) systemd start pacserve.service
9) add pacserve back to pacman.conf
After that everything should work as before (or even better). Sorry for the inconvenience.

nomorewindows · 2013-05-14 20:21:39

Are you sure about pacserve from pacman.conf? Are you sure it isn't something in /etc/pacman.d/mirrorlist?
My clients are still getting 400/404 Error messages.

Last edited by nomorewindows (2013-05-14 20:26:11)

Xyne · 2013-05-14 20:32:12

nomorewindows wrote:

Are you sure about pacserve from pacman.conf? Are you sure it isn't something in /etc/pacman.d/mirrorlist?

For those who want to use Pacserve directly with Pacman, putting "Include = /etc/pacman.d/pacserve" in the mirrorlist will only enable Pacserve for the official servers, as those are the only ones that use the mirrorlist. To use it with all repos, it needs to be in each repo section, so those that do not "Include" the mirrorlist must include that line in pacman.conf.

Of course, you could just use the included "pacsrv" wrapper script if you have a local server running (which you should if you use Pacserve), which manages this transparently without editing pacman.conf.
Powerpill users do not need to deal with any of this as it detects local Pacserve servers automatically.

nomorewindows wrote:

My clients are still getting 400/404 Error messages.

Ah, I just was this now.

What version of pacserve and python3-threaded_servers do you have installed?
What is the output of "systemctl status pacserve.service"?
How have you enabled Pacserve?
Do you have other mirrors in the mirrorlist and in each repo section?

Last edited by Xyne (2013-05-14 20:35:39)

nomorewindows · 2013-05-15 14:10:10

Xyne wrote:

For those who want to use Pacserve directly with Pacman, putting "Include = /etc/pacman.d/pacserve" in the mirrorlist will only enable Pacserve for the official servers, as those are the only ones that use the mirrorlist. To use it with all repos, it needs to be in each repo section, so those that do not "Include" the mirrorlist must include that line in pacman.conf.
What version of pacserve and python3-threaded_servers do you have installed?
What is the output of "systemctl status pacserve.service"?
How have you enabled Pacserve?
Do you have other mirrors in the mirrorlist and in each repo section?

I had it in /etc/pacman.d/mirrorlist, but now that I've made this change it works again.
I've been using your repo, so it'd be up to date. Prior to this, the mirrorlist was working, but now switched over to including /etc/pacman.d/pacserve.

Myshkin · 2013-06-29 17:56:44

I'm probably missing something simple but it isn't working for me. On both machines I've done as per website:
systemctl enable pacserve-ports.service
systemctl start pacserve.service

and the website says it should "just work." Alas, it doesn't. The 2 computers don't seem to see each other. Journalctl on both computers shows only "announcing" messages and "redirecting" to external mirror messages. I don't see any "adding server" messages like I've seen in other posts on this thread.

I don't think it is a firewall problem because quickserve works fine. Am I correct that no additional configuration is needed if i use only pacsrv?

will.price94 · 2013-06-29 22:16:37

Have you tried adding 'Include = /etc/pacman.d/pacserve' to the top of /etc/pacman.d/mirrorlist ? I think I had problems until I did that. Please note that this will use pacserve repositories when using `pacman`, and not just `pacsrv`

Myshkin · 2013-06-30 04:04:13

Yes, I have tried that and still get "redirecting" to the first external mirror.
mirrorlist:

Server = http://192.168.1.102:15651

Include = /etc/pacman.d/pacserve

Server = http://ftp.snt.utwente.nl/pub/os/linux/archlinux/$repo/os/$arch
Server = http://mirror.nl.leaseweb.net/archlinux/$repo/os/$arch
Server = http://mirror.de.leaseweb.net/archlinux/$repo/os/$arch

The first line is for quickserve which does work. I stopped quickserve, rebooted both machines, repeated the ports init, restarted pacserve, and waited about 10 minutes on both machines for the next test.

journalctl:

Jun 28 17:52:25 knox systemd[1]: Started Pacserve.
Jun 28 17:52:28 knox pacserve[2360]: [2013-06-28 17:52:28 EAT] INFO: announcing presence via multicast (group: 224.3.45.67)
...
Jun 28 18:02:28 knox pacserve[2360]: [2013-06-28 18:02:28 EAT] INFO: announcing presence via multicast (group: 224.3.45.67)
Jun 28 18:02:46 knox pacserve[2360]: [2013-06-28 18:02:46 EAT] INFO: 127.0.0.1 redirecting to http://ftp.snt.utwente.nl/pub/os/linux/archlinux/extra/os/i686/xproto-7.0.24-1-any.pkg.tar.xz

Mirrorlist is the same on both computers look the same except for the quickserve entry.
Other computer (which does have xproto-7.0.24-1 in the cache) journalctl:

Jun 28 17:57:42 chanan002 pacserve[8465]: [2013-06-28 17:57:42 EAT] INFO: announcing presence via multicast (group: 224.3.45.67)
Jun 28 18:02:42 chanan002 pacserve[8465]: [2013-06-28 18:02:42 EAT] INFO: announcing presence via multicast (group: 224.3.45.67)
Jun 28 18:07:42 chanan002 pacserve[8465]: [2013-06-28 18:07:42 EAT] INFO: announcing presence via multicast (group: 224.3.45.67)

I've repeated this test both ways several times. Install first on knox but chanan still goes external for the same file. Install first on chanan, but knox still goes external. The computers are side by side, connected by synergy, and quickserve works fine. I'm stumped.

Xyne · 2013-06-30 15:42:05

"Include = /etc/pacman.d/pacserve" should be the first line in each repo section of pacman.conf if you are using pacserve directly with pacman, but that should not have any effect on the servers finding each other.

Check that the firewall ports are open:

$ sudo iptables -L
Chain INPUT (policy ...)
target     prot opt source               destination         
...
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:15678
ACCEPT     udp  --  anywhere             anywhere             PKTTYPE = multicast udp dpt:15679

If you use something else to manage your firewall then you may have to open the necessary ports yourself (15678 for TCP traffic, 15679 for UDP).

Restart each pacserve server (not the computer), one by one, and check if they detect each other in the output of "systemctl status pacserve.service". This is what you should see:

$ systemctl status pacserve.service
pacserve.service - Pacserve
   Loaded: loaded (/usr/lib/systemd/system/pacserve.service; enabled)
   Active: active (running) since Sun 2013-06-30 15:30:35 UTC; 2min 6s ago
 Main PID: 24290 (pacserve)
   CGroup: name=systemd:/system/pacserve.service
           ├─24290 /bin/bash /usr/bin/pacserve --multicast
           └─24291 python3 -mThreadedServers.Pacserve --multicast

Jun 30 15:30:35 Foo systemd[1]: Started Pacserve.
Jun 30 15:30:36 Foo pacserve[24290]: [2013-06-30 17:30:36 CEST] INFO: announcing presence via multicast (group: 224.3.45.67)
Jun 30 15:30:36 Foo pacserve[24290]: [2013-06-30 17:30:36 CEST] INFO: added http://192.168.0.57:15678/ (POST)
Jun 30 15:30:36 Foo pacserve[24290]: [2013-06-30 17:30:36 CEST] INFO: 192.168.0.57 "POST / HTTP/1.1" 200 -
...
Jun 30 15:32:39 Foo pacserve[24290]: [2013-06-30 17:32:39 CEST] INFO: POSTing to http://192.168.0.64:15678/
Jun 30 15:32:39 Foo pacserve[24290]: [2013-06-30 17:32:39 CEST] INFO: POSTing to http://192.168.0.64:15678/[unknown]
Jun 30 15:32:39 Foo pacserve[24290]: [2013-06-30 17:32:39 CEST] INFO: http://192.168.0.64:15678/ acknowledged POST

Also check if you can access each server from other computers directly in the browser via "http://...:15678/"

If that still doesn't work, try disabling your firewall if you are on a secure LAN just to see if it is a firewall setting.

Are the computers connected by cables or wifi? I've noticed that some wifi routers will not pass along UDP packets. You can try setting the "--peer ..." option in /etc/pacserve/pacserve.service.conf" to use other servers without multicast access.

Myshkin · 2013-06-30 18:13:13

Thanks for the replies.

My iptables look like yours.
After restart, still nothing but announcing.
Both machines can connect to each other using the browser.
Turned off all firewalls, no change.
Wired cables so N/A.
I know nothing about multicast, so i'll assume that's the problem and research it.

Arch Linux

#101 2013-04-14 04:32:13

Re: pacserve - easily share Pacman packages between computers

#102 2013-04-14 13:46:34

Re: pacserve - easily share Pacman packages between computers

#103 2013-04-14 15:23:53

Re: pacserve - easily share Pacman packages between computers

#104 2013-04-14 17:21:19

Re: pacserve - easily share Pacman packages between computers

#105 2013-04-20 08:27:39

Re: pacserve - easily share Pacman packages between computers

#106 2013-04-20 18:26:21

Re: pacserve - easily share Pacman packages between computers

#107 2013-05-10 21:00:31

Re: pacserve - easily share Pacman packages between computers

#108 2013-05-10 22:25:52

Re: pacserve - easily share Pacman packages between computers

#109 2013-05-10 22:28:35

Re: pacserve - easily share Pacman packages between computers

#110 2013-05-10 22:55:17

Re: pacserve - easily share Pacman packages between computers

#111 2013-05-13 15:28:54

Re: pacserve - easily share Pacman packages between computers

#112 2013-05-13 16:33:45

Re: pacserve - easily share Pacman packages between computers

#113 2013-05-13 16:53:53

Re: pacserve - easily share Pacman packages between computers

#114 2013-05-13 18:46:51

Re: pacserve - easily share Pacman packages between computers

#115 2013-05-13 21:25:34

Re: pacserve - easily share Pacman packages between computers

#116 2013-05-13 21:38:41

Re: pacserve - easily share Pacman packages between computers

#117 2013-05-14 01:12:00

Re: pacserve - easily share Pacman packages between computers

#118 2013-05-14 20:21:39

Re: pacserve - easily share Pacman packages between computers

#119 2013-05-14 20:32:12

Re: pacserve - easily share Pacman packages between computers

#120 2013-05-15 14:10:10

Re: pacserve - easily share Pacman packages between computers

#121 2013-06-29 17:56:44

Re: pacserve - easily share Pacman packages between computers

#122 2013-06-29 22:16:37

Re: pacserve - easily share Pacman packages between computers

#123 2013-06-30 04:04:13

Re: pacserve - easily share Pacman packages between computers

#124 2013-06-30 15:42:05

Re: pacserve - easily share Pacman packages between computers

#125 2013-06-30 18:13:13

Re: pacserve - easily share Pacman packages between computers

Board footer