[SOLVED] Pacman -Sy (and -Syy and ...) very slow.

norswap · 2010-03-17 10:36:22

time sudo pacman -Syy gives me :

real 2m4.750s
user 0m0.780s
sys 0m4.510s

Precisions :
- I'm running this on a a 1.6Ghz Atom netbook. So it might just be that pacman is really crappy.
- Download speed is okay (6 megs connection - download speed ranging from 100 to 400 K/s in this particular case).
- Problem was present from day one.

The slowdown is mainly caused by community and extra repositories - after downloading database info there is a 1 minute freeze after those repositories.
I tried commenting them in pacman.conf which resulted in a 3 sec execution time. Since these repositories are way bigger than the others (in my case core and archlinuxfr), that comforts me in thinking that maybe pacman is crap. But I have to check anyway.

If it is indeed deemed "normal", isn't there any way to speed things up a little ? Some neat tweak would be nice. I am definitely willing to trade features for faster execution time.

Last edited by norswap (2010-03-21 19:00:46)

sand_man · 2010-03-17 10:39:18

Trying running pacman-optimize.
Also try clearing your cache if you don't want your old packages anymore 'pacman -Sc'

It also helps to have /var on a separate partition and speed depends on the filesystem used. But in general, any filesystem will do.

Last edited by sand_man (2010-03-17 10:40:17)

norswap · 2010-03-17 12:28:53

I did pacman-optimize already but redid it to be sure.
I also did the other command, but there was no speed increase at all.

I use XFS which is supposedly good at handling small files, so that shouldn't be a problem. I might give a run to the /var partition method however. In order to mount /var on a different partition, I just have to create the partition and then edit /etc/fstab right ?

matzo · 2010-03-17 13:32:16

I use XFS which is supposedly good at handling small files, so that shouldn't be a problem.

On the contrary, XFS is optimized for handling large files. The handling of small files is even one of the biggest weaknesses of this file system. At least with partitions formatted with the default settings. It is quite possible that here lies the cause of your problem.

Allan · 2010-03-17 13:32:32

What speed hard drive do you have? That seems really, really slow (at worst on my laptop takes 30sec and far, far less after the first run).

Note that pacman-3.4 (about a month or two away) will have some nice speed up in that area.

norswap · 2010-03-17 16:40:04

I might have been confused on the whole xfs thing But really, does it justify such a long time ? I wouldn't expect a difference of this magnitude on such a small (relatively speaking) set of files.

I did a bit of research, and filesystems comparisons I found said that while it is indeed best at handling for large files, it handles small files quite well (see http://www.debian-administration.org/articles/388). That being said, I don't know what credit to give those research, but they seem quite thorough.

The drive is 5400 rpm. The netbook is an Acer Aspire One A150X, full specs to be found here : http://en.wikipedia.org/wiki/Acer_Aspire_One It is noteworthy that the machine is generally speedy on other tasks.

And I'm looking forward to those speedups !

matzo · 2010-03-17 17:26:07

Hi,

I have just reconfigured 'pacman', so that the database resides on a partition that is formatted with XFS.
The execution of the command

time pacman -Syy

showed the following result:

:: Synchronizing package databases...
 core                      36.2K   19.6M/s 00:00:00 [#####################################################################################################################] 100%
 extra                    445.9K    2.2M/s 00:00:00 [#####################################################################################################################] 100%
 community                368.2K 1690.6K/s 00:00:00 [#####################################################################################################################] 100%

real    1m22.866s
user    0m0.100s
sys    0m0.670s

The harddrive makes 7200 revolutions per minute. My XFS-Partition is mounted with the parameters 'noatime' and 'logbufs=8'.

In contrast, the result on an EXT4 formated partition is as follows:

:: Synchronizing package databases...
 core                      36.2K  550.6K/s 00:00:00 [#####################################################################################################################] 100%
 extra                    445.9K 1794.2K/s 00:00:00 [#####################################################################################################################] 100%
 community                368.2K 1680.3K/s 00:00:00 [#####################################################################################################################] 100%

real    0m1.425s
user    0m0.083s
sys    0m0.403s

This drive is unfortunately a bit faster (10000 RPM), so that direct comparison can not be drawn. However, the time difference is enormous, so I think that for the most part, the file system is responsible.

broch · 2010-03-17 23:11:07

In addition, you can format xfs partition optimized for small files.

norswap · 2010-03-17 23:16:31

I fail ! In the process I totally overlooked saving the content of the old /var folder (working under the assumption it was all "temp" files) and now I can't get pacman to work (I get "error : database path is undefined"). Does that mean I have to reinstall arch or is there another path to redemption ?

Last edited by norswap (2010-03-17 23:17:25)

Daenyth · 2010-03-17 23:45:21

Get all the packages you installed from pacman.log and reinstall them using a live cd. Chroot in and start with pacman -Sf base, then work your way up. You should also back up /etc to save your configuration files.

norswap · 2010-03-18 00:05:22

I don't have the pacman.log anymore since it was in /var.

I just recreated manually /var/local/pacman, /var/log/pacman.log and /var/cache/pacman/pkg (and I also reinstalled pacman from source by sheer ignorance).
Now I just need to remember all packages I've installed (there's not too much of them). But it juste struck me now I also have to reinstall all the packages that I got from the install Well I'll do it, I don't have time to reinstall so I'll probably do it.

Btw, what are the package installed by default during the installation ? (I remember selecting base and base-devel groups - but are all packages in these groups installed by default ?- I think I didn't touch what was proposed).

Is there a reason why I should do this from a live cd and not from the system ?

On the bright side, pacman -Syy now only takes 11 seconds to complete

Arch Linux

#1 2010-03-17 10:36:22

[SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#2 2010-03-17 10:39:18

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#3 2010-03-17 12:28:53

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#4 2010-03-17 13:32:16

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#5 2010-03-17 13:32:32

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#6 2010-03-17 16:40:04

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#7 2010-03-17 17:26:07

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#8 2010-03-17 23:11:07

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#9 2010-03-17 23:16:31

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#10 2010-03-17 23:45:21

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

#11 2010-03-18 00:05:22

Re: [SOLVED] Pacman -Sy (and -Syy and ...) very slow.

Board footer