PKGBUILD template: svn versions without versioning data

Xyne · 2009-01-11 00:41:04

*EDIT*
The original post contained a bash function that could be dropped into a PKGBUILD. I've converted this function to a stand-alone perl script to facilitate maintainence (updating multiple PKGBUILDs across the AUR and elsewhere would quickly become a nightmare).

The script is named svn-export. The info page will provide more details, but basically by using svn-export and the PKGBUILD template below, you can keep the package sources up-to-date without needing to download all the versioning date. On some projects, especially older ones, the versioning data can reach several gigabytes, which makes for longer downloads and wasted space on the local machine because most users will not be committing changes and thus do not need the versioning data.

The PKGBUILD template can now be found on the svn-export info page.

Last edited by Xyne (2009-01-24 06:34:09)

EVRAMP · 2009-01-11 11:48:12

Hmmm.... nice script.

Xyne wrote:

(To the best of my search skills, there is no simple(r) way to do this. If you know of one, please let me know.)

yaourt -S package-svn
==> Building and installing package
==>  Yaourt has detected previous package-svn build. Do you want to use it (faster) ? [Y/n]
Y

And it downloads only changes in SVN/CVS/Git
All you need to do, is change TmpDirectory in /etc/yaourtrc (because /tmp is deleted after each reboot).

Last edited by EVRAMP (2009-01-11 11:51:46)

Xyne · 2009-01-11 12:55:02

Thanks EVRAMP, but I don't think you've understood exactly what my script does (my fault if I've been unclear).

Yaourt is almost certainly using a full svn checkout for the source, which means that inside every directory in the source directory, you'll find .svn directories which contain all of the svn versioning data. For most projects, that means that you have all the information about every single change made to the source since the project started, which can be larger than the actual current svn export. This can more than double the size of the local sources and initial download.

For most PKGBUILDs, this is completely unnecessary because most users will only be building the package and not committing changes to the svn tree.

My script doesn't do a checkout, it does an intelligent export to skip the svn data. After that it only gets the changes, as an svn update does, but again without the svn versioning data. After every run of makepkg, the local build directory will contain a fully up-to-date unversioned copy of the latest sources.

A comparison of a build directory using svn (via yaourt or anything else) and one using this will show a difference in size for any project beyond it's first revision.

Last edited by Xyne (2009-01-11 13:03:54)

shining · 2009-01-11 15:55:25

I found this question interesting, so I did some searching too. I found several people had the same need in the past, here are some relevant links :
http://svn.haxx.se/dev/archive-2004-11/0337.shtml
http://svn.haxx.se/users/archive-2004-12/0116.shtml
http://svn.haxx.se/users/archive-2005-06/0650.shtml
http://subversion.tigris.org/issues/show_bug.cgi?id=908

The only proposed workaround I found was using svn diff and patch to do an efficient update, but this apparently does not deal with directory changes. It looks like you figured out this issue yourself and at least tried to address it with a more advanced workaround.

A change that could apparently be accepted by subversion devs would be to either compress the text-base dir or even to make them optional, but this would probably require people to contribute patches and put some pressure in order to move ahead

It seems the best way would still be for every provider to also provide a rsync access or similar besides a svn access. Isn't it also in their interest to reduce the bandwidth load of initial checkouts by half? Also this problem probably applies to any version control system, not only svn.
I would think the number of users tracking latest version of oss without ever needing the versioned data is not negligible.

Xyne · 2009-01-11 17:02:51

I agree that most projects should provide rsync access (or rsync should incorporate some basic svn protocol). I also agree with the post in the link about adding support for "svn export -r X:Y". I don't understand why they haven't (my script could be modified to accept 2 revisions and export the changes).

Anyway, for the time being, this seems to work quite well, which is why I've shared it. I'd like to hear back from anyone who uses this to create an svn PKGBUILD.

Last edited by Xyne (2009-01-11 17:05:24)

Xyne · 2009-01-24 06:50:21

I've moved the bash function to a stand-alone perl script now for the sake of maintenance.

Arch Linux

#1 2009-01-11 00:41:04

PKGBUILD template: svn versions without versioning data

#2 2009-01-11 11:48:12

Re: PKGBUILD template: svn versions without versioning data

#3 2009-01-11 12:55:02

Re: PKGBUILD template: svn versions without versioning data

#4 2009-01-11 15:55:25

Re: PKGBUILD template: svn versions without versioning data

#5 2009-01-11 17:02:51

Re: PKGBUILD template: svn versions without versioning data

#6 2009-01-24 06:50:21

Re: PKGBUILD template: svn versions without versioning data

Board footer