So do you only want deltas generated from the current version to one version back? or is N old versions to current version a better idea? or both?
I can add code for the first case, but I already have code for the second working.
I've lost track of what issue you're trying to solve at this point, so I'm not sure how to answer that. I would say, though, that I think repo-add's delta support should remain pretty basic; I don't want a ton of complex delta generation logic when it's easy enough for users to script that themselves with pkgdelta.
]]>This is entirely unintuitive, and furthermore I don't know when I would use it since we know at the time of generating it, how big it is relative to the package.
]]>Who created those? They are deltas going the wrong direction. I'd bet libalpm is just erroring on them
Why should it be erroring on "them"? The second delta is going in the right direction, the first delta was created with repo-add -d custom.db.tar.gz ${downgraded-package}.pkg.tar.xz because I wanted to add the old version and then add the new version to generate the delta going in the right direction, and I was too lazy to remove the -d flag.
At any rate, I removed the first delta.
$ bsdtar -tvf /var/lib/pacman/sync/custom.db | grep julia
drwxr-xr-x 0 eschwartz eschwartz 0 Dec 28 13:15 julia-docs-2:1.0.3-2/
-rw-r--r-- 0 eschwartz eschwartz 174 Dec 28 13:15 julia-docs-2:1.0.3-2/deltas
-rw-r--r-- 0 eschwartz eschwartz 502 Dec 27 18:43 julia-docs-2:1.0.3-2/desc
$ bsdtar -xOf /var/lib/pacman/sync/custom.db julia-docs-2:1.0.3-2/deltas
%DELTAS%
julia-docs-2:0.6.2-6_to_2:1.0.3-1-x86_64.delta 4c43982999af5fced09c3dcca384a1c5 654120 julia-docs-2:0.6.2-6-x86_64.pkg.tar.xz julia-docs-2:1.0.3-1-x86_64.pkg.tar.xz
$ cleanupdelta custom
$
This is definitely going in the right direction, and it's definitely operating on the copy of the db which is in pacman's syncdir, and cleanupdelta is definitely not listing this delta as unneeded due to offering a migration path from a previous pkgver to the current pkgver with pkgrel 1, whereas the repo only offers pkgrel 2 with no second delta to bridge pkgrel 1 and pkgrel 2.
]]>nkukard wrote:2. B is also removed from the mirrors when C comes along? how would we know that B existed? maybe I'm missing something?
Because we have deltas for B.
nkukard wrote:3. I cannot find anywhere in the pacman code that it uses deltas A->B when upgrading to from A->C, just A->C ... maybe I'm wrong and missed it?
I don't know where you see in alpm that it only uses a single delta. The delta operations use lists.
Ok, you're right, sorry ... I don't know how I missed that.
So do you only want deltas generated from the current version to one version back? or is N old versions to current version a better idea? or both?
I can add code for the first case, but I already have code for the second working.
]]>2. B is also removed from the mirrors when C comes along? how would we know that B existed? maybe I'm missing something?
Because we have deltas for B.
3. I cannot find anywhere in the pacman code that it uses deltas A->B when upgrading to from A->C, just A->C ... maybe I'm wrong and missed it?
I don't know where you see in alpm that it only uses a single delta. The delta operations use lists.
]]>Why does C make A -> B unusable? Use it to update A to B then update that to C.
1. If you have A -> B -> C , it is likely that the combined delta size of A->B and B->C that you would need to download would exceed the size of C. The delta of A->C would likely be smaller.
2. B is also removed from the mirrors when C comes along? how would we know that B existed? maybe I'm missing something?
3. I cannot find anywhere in the pacman code that it uses deltas A->B when upgrading to from A->C, just A->C ... maybe I'm wrong and missed it?
]]>Nonetheless, we cannot just keep N deltas around from A -> B, when C comes along the only delta that is really usabe is A -> C and B -> C. When D comes along A -> D, B -> D, C -> D.
I have POC set of changes working now with repo-add which keeps deltas around from the Nth version to the lastest and automatically removes the ones no longer applicable. I'm parsing the $pkgentry/deltas file and with a few lines of code its easy to determine what we need and what we don't.
Why does C make A -> B unusable? Use it to update A to B then update that to C.
]]>I have POC set of changes working now with repo-add which keeps deltas around from the Nth version to the lastest and automatically removes the ones no longer applicable. I'm parsing the $pkgentry/deltas file and with a few lines of code its easy to determine what we need and what we don't.
]]>$ bsdtar -tvf custom.db.tar.gz | grep julia
drwxr-xr-x 0 eschwartz eschwartz 0 Dec 27 21:34 julia-docs-2:1.0.3-2/
-rw-r--r-- 0 eschwartz eschwartz 174 Dec 27 21:34 julia-docs-2:1.0.3-2/deltas
-rw-r--r-- 0 eschwartz eschwartz 502 Dec 27 18:43 julia-docs-2:1.0.3-2/desc
$ bsdtar -xOf custom.db.tar.gz julia-docs-2:1.0.3-2/deltas
%DELTAS%
julia-docs-2:1.0.3-2_to_2:0.6.2-6-x86_64.delta 8ebc8a55fad0fd24f0e43fb2af4b097e 456156 julia-docs-2:1.0.3-2-x86_64.pkg.tar.xz julia-docs-2:0.6.2-6-x86_64.pkg.tar.xz
julia-docs-2:0.6.2-6_to_2:1.0.3-1-x86_64.delta 4c43982999af5fced09c3dcca384a1c5 654120 julia-docs-2:0.6.2-6-x86_64.pkg.tar.xz julia-docs-2:1.0.3-1-x86_64.pkg.tar.xz
$ cleanupdelta custom
$
The usage message is vastly confusing and attempts to figure out what -b does (it implies that to specify repositories to check, you must use "-b") have inevitably met with the help text... if it does expect the pacman.conf DBPath to initialize an alpm session then that makes sense, but I'm still left with "it prints nothing for me".
]]>Allan wrote:Such as the "cleanupdelta" script?
...
Mind explaining how to use it?
It basically calls alpm_pkg_unused_deltas for each package in a repository and prints the result.
Some example invocations:
cleanupdelta core extra
cleanupdelta extra
cleanupdelta -b /var/lib/pacman core
Such as the "cleanupdelta" script?
Which is not a script, it is a C executable with inscrutable help text that doesn't seem to do anything AFAICT. :)
Mind explaining how to use it?
]]>