Unable to do delta updates even after configuring as per wiki

sumitkbhardwaj · 2015-04-16 03:55:21

Hi All,

I have to live with bandwidth limited networks all the time in my country, hence I want to enable delta updates with pacman. I referred to the below wiki page :

https://wiki.archlinux.org/index.php/Deltup

So what I did is :

1. Added the URL http://delta.archlinux.fr/$repo/os/$arch at the top of my /etc/pacman.d/mirrorlist file.
2. Uncommented the "UseDelta = .7" line in /etc/pacman.conf
3. Installed xdelta3

Below are the first few lines of my mirrorlist file:

##
## Arch Linux repository mirrorlist
## Sorted by mirror score from mirror status page
## Generated on 2015-04-15
##
## Score: 0.3, Delta Archlinux.fr
Server = http://delta.archlinux.fr/$repo/os/$arch
## Score: 0.3, Denmark
Server = http://mirror.one.com/archlinux/$repo/os/$arch
## Score: 0.4, Germany
Server = http://archlinux.my-universe.com/$repo/os/$arch
## Score: 0.4, Netherlands
Server = http://ftp.nluug.nl/os/Linux/distr/archlinux/$repo/os/$arch
## Score: 0.4, France
Server = http://archlinux.polymorf.fr/$repo/os/$arch
.
.
.

But even after all this, very rarely I see pacman using deltas (by comparing the download size switching the UseDelta option on and off) and most of the time I get message like below:

 
[sumitb@sumitslaptop ~] $ sudo pacman -Syu
[sudo] password for sumitb: 
:: Synchronizing package databases...
 core                     125.8 KiB   200K/s 00:01 [######################] 100%
 extra                   1842.9 KiB   263K/s 00:07 [######################] 100%
 community                  2.6 MiB   184K/s 00:15 [######################] 100%
 multilib is up to date
 archlinuxfr is up to date
 infinality-bundle is up to date
:: Starting full system upgrade...
resolving dependencies...
looking for conflicting packages...

Package (1)          Old Version  New Version  Net Change  Download Size

community/filezilla  3.10.2-1     3.10.3-1       0.05 MiB       3.41 MiB

Total Download Size:    3.41 MiB
Total Installed Size:  14.49 MiB
Net Upgrade Size:       0.05 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages ...
error: failed retrieving file 'filezilla-3.10.3-1-x86_64.pkg.tar.xz' from delta.archlinux.fr : The requested URL returned error: 404
 filezilla-3.10.3-1-...     3.4 MiB   162K/s 00:22 [######################] 100%
(1/1) checking keys in keyring                     [######################] 100%
(1/1) checking package integrity                   [######################] 100%
(1/1) loading package files                        [######################] 100%
(1/1) checking for file conflicts                  [######################] 100%
(1/1) checking available disk space                [######################] 100%
(1/1) upgrading filezilla                          [######################] 100%

I manually checked the repo and the file is there allright, then why am I getting 404 error?? Can anybody please help me with setting up delta updates properly?

Thanks in advance!

loafer · 2015-04-16 05:15:18

The filezilla package in that repo is "filezilla-3.10.2-1-x86_64.pkg.tar.xz".

sumitkbhardwaj · 2015-04-16 05:22:44

@loafer : Oh, my bad! But in that case, is the delta library updated slower than other repos? Because if that is so, there won't be much benefit of having it, it will fail to provide delta packages most of the time. Am I right in saying that?

rezad · 2015-04-18 07:37:35

that repo doesn't have delta for all packages and even worse it doesn't support resume for download of packages so if with my shitty internet any timeout occurs it redownloads the whole packages.

sumitkbhardwaj · 2015-04-18 18:32:16

Hmm..By far, Fedora has the best implementation of delta updates. The presto system in fedora actually shows you how much you are going to save before doing every update, and Fedora project provides deltas for at least 90% of the packages from its official repos. In comparison to that Arch's delta system is quite new....and not that mature. Or may be its because due to the bleeding edge and rolling release nature of Arch...Anyway....

mojtabazn · 2015-04-18 18:57:37

Hi

change this

Uncommented the "UseDelta = .7"  line in /etc/pacman.conf

to just

UseDelta

sumitkbhardwaj · 2015-04-18 19:04:25

@mojtabazn OK, just done that. Let's see when the next updates arrive, right now I am up-to-date

karol · 2015-04-18 20:54:33

Most Archers seem to have decent internet connection so they don't need deltas.

WorMzy · 2015-04-18 21:01:58

Arch's delta system is anything but new, it's been around for a number of years now, it's just not widely implemented.

sumitkbhardwaj · 2015-04-19 04:14:38

@karol Well, then Arch doesn't have many users in developing countries. Anyway, at least in India, having delta updates is a big asset. Bandwidth is damn costly here and you have to pay higher for higher speeds, apart from the data cost itself.

@WorMzy Ya actually my bad choice of words, I exactly wanted to say what you said. But one thing I have noticed is that on the same system xdelta3 works slower than presto delta system while merging delta packages to create full packages.

Allan · 2015-04-19 07:02:11

sumitkbhardwaj wrote:

Hmm..By far, Fedora has the best implementation of delta updates. The presto system in fedora actually shows you how much you are going to save before doing every update, and Fedora project provides deltas for at least 90% of the packages from its official repos. In comparison to that Arch's delta system is quite new....and not that mature. Or may be its because due to the bleeding edge and rolling release nature of Arch...Anyway....

Deltas are relatively useless on rolling release distros. As noted, the one repo that does provide them notes that most package updates do not create deltas that are small enough to be of use.

On Fedora, there is no change in the toolchain, the libraries, etc between updates (only on upgrades between versions). So deltas are useful there.

Allan · 2015-04-19 07:10:43

sumitkbhardwaj wrote:

@karol Well, then Arch doesn't have many users in developing countries. Anyway, at least in India, having delta updates is a big asset. Bandwidth is damn costly here and you have to pay higher for higher speeds, apart from the data cost itself.

I'd wonder why they would choose a rolling release distro with its constant updates then...

@WorMzy Ya actually my bad choice of words, I exactly wanted to say what you said. But one thing I have noticed is that on the same system xdelta3 works slower than presto delta system while merging delta packages to create full packages.

I guess most of the time in our delta system is due to recompressing the package to check the signature of the recreated package file. We use xz compression which is slow, but makes your downloads smaller.

Presto does use bsdiff rather than xdelta3, but I am unable to find a good comparison of the two. I do know that bsdiff uses "insane" amounts of memory when creating diffs.

sumitkbhardwaj · 2015-04-19 12:07:01

@Allan The reason is to get latest updates quicker, and I don't mind paying extra for that but the thing is....if any amount of data (and by extension, my hard earned money) can be saved using methods like delta updates, I would like to go for it. And it is not that Fedora, or any other bleeding edge system like that dont update that much. When I was on Fedora, then also I use to get updates almost everyday. The only point I am trying to make is that delta updates make more sense to people like me with capped internet connections (both in terms of data download and speed) than those with unlimited/uncapped lines.

Hmm..never mind...I don't mind little bit or speed difference, its just something I noticed.

Allan · 2015-04-19 12:48:38

sumitkbhardwaj wrote:

And it is not that Fedora, or any other bleeding edge system like that dont update that much. When I was on Fedora, then also I use to get updates almost everyday.

Unless you were using Rawhide, those updates are bug fixes and not new versions (mostly - I know there are exceptions). Those sorts of updates work very well with deltas.

sumitkbhardwaj · 2015-04-19 18:54:05

Well, i don't think that's correct. Although its not the place to discuss things related to Fedora, but they do release new versions for many such "Exceptions". Only the packages that affect core user experience are not updated to Major versions. Minor version upgrades are still done in between releases.

Anyway, after removing the percentage parameter from UseDelta option in pacman.conf, now I am getting much more delta updates than before, so thanks everybody for all the help and insights. Will mark it as solved.

Arch Linux

#1 2015-04-16 03:55:21

Unable to do delta updates even after configuring as per wiki

#2 2015-04-16 05:15:18

Re: Unable to do delta updates even after configuring as per wiki

#3 2015-04-16 05:22:44

Re: Unable to do delta updates even after configuring as per wiki

#4 2015-04-18 07:37:35

Re: Unable to do delta updates even after configuring as per wiki

#5 2015-04-18 18:32:16

Re: Unable to do delta updates even after configuring as per wiki

#6 2015-04-18 18:57:37

Re: Unable to do delta updates even after configuring as per wiki

#7 2015-04-18 19:04:25

Re: Unable to do delta updates even after configuring as per wiki

#8 2015-04-18 20:54:33

Re: Unable to do delta updates even after configuring as per wiki

#9 2015-04-18 21:01:58

Re: Unable to do delta updates even after configuring as per wiki

#10 2015-04-19 04:14:38

Re: Unable to do delta updates even after configuring as per wiki

#11 2015-04-19 07:02:11

Re: Unable to do delta updates even after configuring as per wiki

#12 2015-04-19 07:10:43

Re: Unable to do delta updates even after configuring as per wiki

#13 2015-04-19 12:07:01

Re: Unable to do delta updates even after configuring as per wiki

#14 2015-04-19 12:48:38

Re: Unable to do delta updates even after configuring as per wiki

#15 2015-04-19 18:54:05

Re: Unable to do delta updates even after configuring as per wiki

Board footer