Pacpatch

Fox · 2004-10-21 12:25:53

Hi @ all!
Maybe you read the threads "Securityfixes" and "pacpatch" at the mailinglist. I started developing a patching system for Arch Linux Packages

me @ the mailinglist wrote:

Client:
Pacpatch is patching system. It creates (if it's development is finished) patch between 2 Archlinux Packages by using xdelta (libxdelta or systemcall?). If pacman checks for updates, it checks if a package version is cached and if there is a cached version it executes to pacpatch, pacpatch checks for a patch, if there is parch it returns true, pacman applies the patch on the cached version and installs it. If there is no patch or the new package is smaller than the patch it pacman will download and install the package - not the patches. Pacpatch can use a chain of patches to patch from a older version than the version before the latest to the latest package.
Server:
The server automaticly generates patches when a new version of a package was uploaded. The clients can download this patches.
Some ideas are not perfect - so I'm happy about every help.

If you are interested in development please contact me.

http://sourceforge.net/projects/pacpatch

All ideas are welcome.

kth5 · 2004-10-21 12:34:05

it's not clear to me if you are trying to apply patches to the tar.gz files or rather replace specific files which were installed by some other package?

secondly, are you really thinking of automizing the patchcreation-mechanism?

Fox · 2004-10-21 12:56:35

kth5 wrote:

it's not clear to me if you are trying to apply patches to the tar.gz files or rather replace specific files which were installed by some other package?
secondly, are you really thinking of automizing the patchcreation-mechanism?

In the first developer release only patching of the compressed data is possible. But this will change in the next release.
I think it's to much work to generate the packages manualy.

Mr Green · 2004-10-21 13:26:06

I am sorry but what is the point?

I would like patches applied to packages (updates) to reduce download time & speed up updating of system....is this the idea....?

kth5 · 2004-10-21 13:34:11

Mr Green wrote:

I would like patches applied to packages (updates) to reduce download time & speed up updating of system....is this the idea....?

looks pretty much like it. though i think patching binaries is a rather odd thing to do.
overwriting some files installed by the to-be-updated package does make much more sense i think. only thing the server would have to do is to extract the updated package & the old one, then compare the files and repack newer ones into a proper patch. isn't that a lot easier and safer?

i still doubt that it will make any sense in the longrun though. the server's load will increase w/ every new patch applied to the initial release file. unless you release full packages so frequently that patches don't make any sense any more it will seriously haunt you whatever you do.

:edit:

btw, you do expect that the user has all the packages that actually can be patched in cache? im one of those guys who run a local mirror so i empty it everytime i install/upgrade something to save diskspace... i mean for what reason do you need cached versions of the software you already have installed unless you want to downgrade, which does not happen very often. :twisted:

Fox · 2004-10-21 14:09:26

Mr Green wrote:

I am sorry but what is the point?
I would like patches applied to packages (updates) to reduce download time & speed up updating of system....is this the idea....?

Yes. But I thought of security patches too, but this couldn't be done by the server automatically .

Fox · 2004-10-21 14:19:44

kth5 wrote:

Mr Green wrote:
I would like patches applied to packages (updates) to reduce download time & speed up updating of system....is this the idea....?
looks pretty much like it. though i think patching binaries is a rather odd thing to do.
overwriting some files installed by the to-be-updated package does make much more sense i think. only thing the server would have to do is to extract the updated package & the old one, then compare the files and repack newer ones into a proper patch. isn't that a lot easier and safer?
i still doubt that it will make any sense in the longrun though. the server's load will increase w/ every new patch applied to the initial release file. unless you release full packages so frequently that patches don't make any sense any more it will seriously haunt you whatever you do.
:edit:
btw, you do expect that the user has all the packages that actually can be patched in cache? im one of those guys who run a local mirror so i empty it everytime i install/upgrade something to save diskspace... i mean for what reason do you need cached versions of the software you already have installed unless you want to downgrade, which does not happen very often. :twisted:

1. It is not odd, because it uses xdelta.
2. Not all patches (I think 2 or 3 Version will be stored, but you can configure it) are stored on the server.
3. In the stable release it will extract the files from the package, generate patches of existing files and add/delte files which are new/ not contined anymore.
4. If you delete your cache it's your problem. For people like me which use testing packages a cache is very usefull.

Mr Green · 2004-10-21 14:32:30

I'm interested in the idea. ..I will keep an eye on it

i3839 · 2004-10-21 16:22:34

Do you have some stats for size reduction when using the patch compared to the real package? Something big like xorg or so will be nice.

Fox · 2004-10-21 16:24:29

i3839 wrote:

Do you have some stats for size reduction when using the patch compared to the real package? Something big like xorg or so will be nice.

Gentoo people have generated some stats for xdelta patches:
http://forums.gentoo.org/viewtopic.php?t=55210

i3839 · 2004-10-21 17:30:38

They're not mentioning xdelta there, only deltup. And they're using patches for source, not binaries. And I'm more interested in your stats anyway, because the last time I tried a binary diff on openssl it had almost no size difference after the compression.

Fox · 2004-10-22 14:23:45

i3839 wrote:

They're not mentioning xdelta there, only deltup. And they're using patches for source, not binaries. And I'm more interested in your stats anyway, because the last time I tried a binary diff on openssl it had almost no size difference after the compression.

I think before posting . Deltup (have a look at source) uses xdelta the same way as gpp (genpacpatch). AFAIK they use xdelta to patch the source packages not the source. You have to add the "-9" option to xdelta command. And I don't want to patch the compressed package. If I want to patch the files in the package and e.g. the glibc functions are included in binary you don't have to patch it, just the other the other changes have to be patched.

i3839 · 2004-10-22 19:33:10

I mean doing the diff before the compression gave significant size reduction, but after compressing that diff it didn't compress as well as the original package did, so almost all size reduction disappeared.

Doing a diff on the source (packed together or split up, doesn't matter) will always be way more succesfull than doing a diff on real binaries.

All I asked for is some size stats which you got after your diffing, that's all...

Fox · 2004-10-22 22:10:13

i3839 wrote:

I mean doing the diff before the compression gave significant size reduction, but after compressing that diff it didn't compress as well as the original package did, so almost all size reduction disappeared.
Doing a diff on the source (packed together or split up, doesn't matter) will always be way more succesfull than doing a diff on real binaries.
All I asked for is some size stats which you got after your diffing, that's all...

I have no stats but have a look:
http://deltup.sourceforge.net/
Some stats are listes there. I think they are realistic. But this depends on the changes if you for example diff a text and a binary file the patch will be bigger as the binary <-> binary patch. But if there no big changes the patch is very small in relation to the orginal package.

yak8998 · 2004-10-23 01:48:16

I'll be more than willing to help you test it, I'll be not too helpful at writing code, but I can probably do documentation and such for you np. Shoot me an email if you want, i think this is a good idea (at least from what I understand).

yak8998@ev1 net

Mr Green · 2004-10-23 08:33:13

A wiki guide might help us lesser mortals...

If you need any help please pm me

Fox · 2004-10-23 10:17:16

Ok!

Arch Linux

#1 2004-10-21 12:25:53

Pacpatch

#2 2004-10-21 12:34:05

Re: Pacpatch

#3 2004-10-21 12:56:35

Re: Pacpatch

#4 2004-10-21 13:26:06

Re: Pacpatch

#5 2004-10-21 13:34:11

Re: Pacpatch

#6 2004-10-21 14:09:26

Re: Pacpatch

#7 2004-10-21 14:19:44

Re: Pacpatch

#8 2004-10-21 14:32:30

Re: Pacpatch

#9 2004-10-21 16:22:34

Re: Pacpatch

#10 2004-10-21 16:24:29

Re: Pacpatch

#11 2004-10-21 17:30:38

Re: Pacpatch

#12 2004-10-22 14:23:45

Re: Pacpatch

#13 2004-10-22 19:33:10

Re: Pacpatch

#14 2004-10-22 22:10:13

Re: Pacpatch

#15 2004-10-23 01:48:16

Re: Pacpatch

#16 2004-10-23 08:33:13

Re: Pacpatch

#17 2004-10-23 10:17:16

Re: Pacpatch

Board footer