Arch Linux

Shingoshi

Member

From: Salem, Oregon

Registered: 2007-02-24

Posts: 5

A non-obsoleting multiversioning package management filesystem.

I found this in another topic (http://bbs.archlinux.org/viewtopic.php?id=56108), and found it very interesting, since I have conspired to do the same thing myself. The question that has always kept my attention is, "how to eliminate packages from breaking others when being upgraded to versions other packages cannot use?

Much of this seems to be answered by the approach taken by GoboLinux, where each package essentially becomes a self-contained filesystem. Given such a system, it would also be possible to compile one master binary, and make deltas against it for all successive versions. Doing this would eliminate the extra overhead that such a system would otherwise impose, by having so many versions of the same package on the system at once. But then using the approach as stated by Allen below, multiple versions could and would coexist all in the same directory; /$pkgname. I think that if such an approach were to be fully implemented, even the PKGBUILDs would have to be versioned as PKGBUILD.$pkgver.

The next big question I have is how difficult would it be to have the binaries in packages patched live as they are running (or just before when called to run)? The idea is this. If we have a single master package upon which deltas are created for every successive version, it would be nice to patch the master package files as they are put into memory for execution.

http://bbs.archlinux.org/viewtopic.php? … 23#p427323

See section 6.3.2.3 on http://www.linuxfromscratch.org/lfs/vie … kgmgt.html

This is very easy to do with Arch given the PKGBUILDs are all made to be put in $pkgdir anyway...  In fact you could adjust makepkg to make pkgdir=/usr/pkg/$pkgname/$pkgver and not tar everything up.  Then you would just need something to put the symlinks into the / hierarchy - there are tools floating around to do this.  Note this cause big problems for people who have a separate /usr partition... but that makes little sense under this scheme

This was suggested as a method of dealing with the organization of PATHs in a system such as the one described here:

# make_doinst
# doinst.sh fragment:
if ! [ `grep $NAME` /etc/$distribution/program-paths ] ; then
echo "PATH=/programs-path:\$PATH" >> /etc/$distribution/program-paths
fi
# Then have your /etc/profile source the /etc/$distribution/program-paths file and then finally export the PATH:
. /etc/$distribution/program-paths
export PATH
make_doinst                                        # Step 14

This was meant to be used with another package building tool. But I think the principle applies to any the user chooses.

I would like to hear some feedback on this to see what others can or have come up with since the original discussion was posed here in: What you think about Gobolinux's FS layout? (http://bbs.archlinux.org/viewtopic.php?id=56108)

This is somewhat of a related topic (http://bbs.archlinux.org/viewtopic.php? … 36#p450436), at least for me. One of the things that I suggested to others would be the use of a community-wide distcc. The results of builds from the entire network of users would be distributed by deltas. The local ccache would be updated to the latest version of the master, providing new users the immediate benefit of ccache without having actually compiled any packages yet themselves. And using deltas would fundamentally eliminate the concern mentioned here:

So with every package we update (and note there are many per day) the entire repo gets rebuilt and I assume pushed for users to download in order to keep there system in sync with the Arch repos.  Think of the number of packages on your system.  That would require you to download (probably) several gigabytes of updates every time a single package was updated....

There would no longer be a need to download gigabytes to maintain each system in sync with the master. Of course this would also benefit from having the files of packages patched in memory as they were called. A live-patching system would really need to be created.

I currently operate a liquid-cooled quad-socket server which I built especially for this purpose. Having an automatic build system is something that I see as a necessity, not an option. So I have decided to build and operate my own, offering it to others as they choose to participate.

I guess there are other issues which would be detected and solved by such automatic build system. In the extreme case, after each new package committed to ABS, the whole set of packages should be recompiled to be sure that archlinux repositories are in a stable state. I know that this seems like a lot of work, but with many users contributing their spare CPU cycles (think: cloud computing, or Archlinux@Home) this would set archlinux to a higher level of quality.

Here's an image of the project (http://www.teamnexgen.org/forum/hardwar … -ever.html) at completion:  http://www.teamnexgen.org/forum/attachm … 3x1600.jpg

Xavian-Anderson Macpherson
Shingoshi

Last edited by Shingoshi (2009-02-01 20:32:52)