Pacman very slow with XFS, JFS

Gullible Jones · 2005-11-12 05:14:05

Pacman is bizarrely slow on both XFS and JFS, while just about everything else runs like greased lightning. This does not seem to be a fragmentation issue - it begins immediately after installation. The problem worsens after the installation of large packages, to the point that I must wait through several minutes of my hard drive cranking away before packages even start to download. Running pacman-optimize solves the problem, but only temporarily - installing more software with pacman will cause the sluggishness to return. On XFS volumes, xfs_fsr does not seem to help. This problem seems to me to be worse on JFS than on XFS (I have no idea why).

I've never see anything like this when using ext3 or ReiserFS. Anyone have an explanation?

T-Dawg · 2005-11-12 17:27:54

pacman -S reiserfs

iBertus · 2005-11-12 18:24:06

XFS and JFS are both server-centric filesystems and probably not optimized for lots of small files like ReiserFS. I don't really understand why ext3 is any faster, unless you're using non-default ext3 options.

Gullible Jones · 2005-11-12 19:17:34

Dude, I'm not talking about small slowdowns. When I do pacman -Sy, I have to wait about 2 minutes while my HD grinds away, *after* it's finished updating. And I mean grinds - my HDDs are quiet, and this is quite audible.

As I said before, everything else runs very, very fast - including ABS. But pacman causes the drives to do a *lot* of extra work. As I said before, pacman-optimize makes it a bit faster, but as soon as anything new is installed, or the pacman database is updated, the HDDs start grinding again. Again, xfs_fsr does *nothing*.

XFS may not be as fast as ReiserFS for large numbers of small files, but 2-3 minutes of extra grinding on an unfragmented drive goes well beyond the usual XFS issues. This stuff is not normal.

Gullible Jones · 2005-11-13 04:25:32

Okayy... Let me show you an example of this crap.

Searching for Kaffeine:

My computer wrote:

real 0m35.567s
user 0m0.596s
sys 0m0.856s

Yes, you are reading that right. It took 35 seconds.

Now a second time, immediately after:

[root@localhost proteus]# time pacman -Ss kaffeine
extra/kaffeine 0.7.1-3
Multimedia Player, based on Xine
real 0m1.297s
user 0m0.332s
sys 0m0.328s

And now a search for something completely different:

extra/totem-xine 1.2.0-2
GNOME2 integrated movie player based on Xine
real 0m0.780s
user 0m0.344s
sys 0m0.372s

And a search for something in /var/abs:

[root@localhost proteus]# time find /var/abs -name kaffeine
/var/abs/extra/multimedia/kaffeine
real 0m3.672s
user 0m0.020s
sys 0m0.132s

This problem seems to be a pacman-specific issue.

deficite · 2005-11-13 14:43:18

Those tests are pretty much moot, to tell you the truth. The point is that JFS/XFS are great for a couple huge files, but pacman deals with LOTS and LOTS of small files. Therefore you are on the wrong filesystem for what you are doing.

Have no fear! You can always just resize your partitions to give you space for a reiserfs, ext3, or reiser4 partition and mount it on /var. I would presonally redo the whole thing with ext3 or one of the reisers myself. XFS and JFS are good for what they do, like a SQL database or something, but I would never use it for a whole system IMHO.

Gullible Jones · 2005-11-13 19:40:41

Uh huh...

[root@localhost proteus]# time find /var/lib/pacman -name kaffeine-0.7.1-3
/var/lib/pacman/extra/kaffeine-0.7.1-3
real 0m0.090s
user 0m0.020s
sys 0m0.056s

If this is a shortcoming of XFS, please explain why find is so damned fast. Explain why ABS is so fast. Explain why pacman-optimize does *anything* to mitigate the problem.

Dusty · 2005-11-13 19:55:56

First explain to us why you insist on using a server filesystem on a workstation system and then complain that it isn't optimized for workstation activities.

Dusty

Gullible Jones · 2005-11-13 21:10:42

- It has an online defraggin utility.
- It was (and is) used on SGI's workstations as well as servers.
- It performs reasonably well on benchmarks such as these, generally better than ext3.

Moo-Crumpus · 2005-11-13 21:18:57

I use jfs, and it is fast. Maybe you should optimize your pacman files sometimes.

Moo-Crumpus · 2005-11-13 21:21:16

I use jfs, and pacman is fast. Maybe you should optimize your pacman files sometimes. Or your hd has some broken sectors, or is lazy.
Allthough not really a proof, here is the time for a searching:

time sudo pacman -Ss kaffeine
extra/kaffeine 0.7.1-3
    Multimedia Player, based on Xine

real    0m0.443s
user    0m0.212s
sys     0m0.200s

Even more senseless:

time sudo pacman -S kaffeine

Targets: kde-common-3.4.3-1 libsndfile-1.0.12-2
         jack-audio-connection-kit-0.100.0-3 arts-1.4.3-1 pcre-6.3-1
         openexr-1.2.2-4 glut-3.7-4 jasper-1.701.0-3 libidn-0.5.20-1
         kdelibs-3.4.3-1 libtheora-1.0alpha5-1 xine-lib-1.1.0-3 kaffeine-0.7.1-3

Total Package Size:   30.1 MB

Proceed with upgrade? [Y/n]

:: Retrieving packages from extra...
 kde-common-3.4.3-1       [################] 100%     115K   168.4K/s  00:00:00
 jack-audio-connection-ki [################] 100%     220K   276.4K/s  00:00:00
 arts-1.4.3-1             [################] 100%    1331K   559.9K/s  00:00:02
 openexr-1.2.2-4          [################] 100%     916K   510.0K/s  00:00:01
 jasper-1.701.0-3         [################] 100%     639K   473.3K/s  00:00:01
 libidn-0.5.20-1          [################] 100%     183K   243.4K/s  00:00:00
 kdelibs-3.4.3-1          [################] 100%   19586K   678.6K/s  00:00:28
 kaffeine-0.7.1-3         [################] 100%    2153K   605.8K/s  00:00:03

:: Retrieving packages from current...
 libsndfile-1.0.12-2      [################] 100%     332K   367.8K/s  00:00:00
 pcre-6.3-1               [################] 100%     313K   328.8K/s  00:00:00
 glut-3.7-4               [################] 100%     130K   213.7K/s  00:00:00
 libtheora-1.0alpha5-1    [################] 100%     196K   276.1K/s  00:00:00
 xine-lib-1.1.0-3         [################] 100%    4734K   648.6K/s  00:00:07

checking package integrity... done.
loading package data... done.
checking for file conflicts... done.
installing kde-common... done.
For more info on KDE please have a look at:
'http://wiki2.archlinux.org/index.php/KDE'
installing libsndfile... done.
installing jack-audio-connection-kit... done.

 --> The audio server comes with a daemon script that runs it as root.
 --> Configuration happens in /etc/conf.d/jack-audio-connection-kit and is
 --> documented there. If you like to run it as normal user you can
 --> download qjackctl and control the server with a gui.

 --> NOTE: Running jack as root to enable realtime capabilities isn't
 --> needed any longer. As of kernel-2.6.12 and pam > 0.80-2 you can
 --> achive the rights to run realtime on a per group basis, which is
 --> controlled by /etc/security/limits.conf

installing arts... done.
installing pcre... done.
installing openexr... done.
installing glut... done.
installing jasper... done.
installing libidn... done.
installing kdelibs... done.
installing libtheora... done.
installing xine-lib... done.
installing kaffeine... done.

real    1m5.597s
user    0m2.988s
sys     0m2.980s

Gullible Jones · 2005-11-13 21:36:15

I tried optimizing my pacman files. It worked, but for a damned short time.

But here's some interesting news: I just switched from the anticipatory IO scheduler to cfq, and now pacman is quite fast. :?

Gullible Jones · 2005-11-13 22:54:13

Hmmm, this is weird. After doing some stuff with pacman, the trouble is back, though not as bad as with the anticipatory scheduler.

Perhaps this is a fragmentation issue. It certainly does seems as though /var/lib/pacman is getting very fragmented very fast.

deficite · 2005-11-14 03:52:04

It's probably something to do with your disk cacheing. I was about to ask if you ran that find command after running the pacman -Ss, because the files would become cached and speed it up like crazy. That's why I said those tests were moot.

Once some good use of libidized pacman comes about, we may see some fast pacmanage. I do have to add that at current, pacman runs faster than apt-get did for me when I experimented with Ubuntu.

jerem · 2006-01-14 14:25:03

I find pacman to be VERY slow with ext3 too.

It takes 10 seconds for each operation, whereas with reiserfs all is done at lightning speed.

Haven't tried JFS or XFS yet.

Gullible Jones · 2006-01-14 18:22:13

Use tune2fs and e2fsck to make the filesystem directory-indexed, as explained here.

jerem · 2006-01-14 19:46:42

But I've used that option since I use ext3 !

Gullible Jones · 2006-01-14 21:18:18

Then try using pacman-optimize. That should solve it for a while.

jerem · 2006-01-16 21:57:41

It only optimizes for 2/3 pacman operations, then it slows down significantly again.

Is ext3 so much behind reiser in terms of performance ?

Gullible Jones · 2006-01-17 00:54:41

No, the problem seems to involve fragmentation of the pacman "database". For most things that sort of thing isn't a problem, but for pacman it is.

(Interestingly, I didn't get that problem on this install. It seems that rapid "fragmentation" occures in about half the ext3 installs you do, and does not affect the other half of installs using ext3. I have absolutely no idea why, and seriously wonder if I haven't gone a bit nuts.)

Moo-Crumpus · 2006-01-17 08:14:09

Funny, Gullible Jones. And yes, it is a hardly known feature of ext3: while formating a partition, it rolls the dice wether it should ever fragemtate a partition or not. If the result is a 1, it formats with /F what means fragmentate the partition in every case of data manipulations on it.
You can demand a new dice with the command "ext3-redice -f4", where -f forces ext3 to only fragmentate a partition if the result is larger then 4. -f can be a number between [2,4]. If the dice gives a 6, all data is jumbled.

Mwuhahaha...

You made my day.

To be honest, I had some sobering experiences with ext3, too. Sometimes it does well, sometimes it is a pain. Are you sure your hardware is ok?

Gullible Jones · 2006-01-17 11:49:24

Yep, hardware is fine. And you just made me fall off my chair.

dpc · 2006-01-22 01:02:32

Not very creative post ...
On the disk inside my laptop pacman is deadly slow. (ext3 with CFQ). Every package install, every single operation take ages. I get used to it, but I dream about real DB for pacman. It needs this.

T-Dawg · 2006-01-22 01:45:20

dpc wrote:

Not very creative post ...
On the disk inside my laptop pacman is deadly slow. (ext3 with CFQ). Every package install, every single operation take ages. I get used to it, but I dream about real DB for pacman. It needs this.

pacman-optimize should help with the slowness until we get a pacman with a quicker database system

Gullible Jones · 2006-01-22 04:35:00

Interesting, using CFQ makes Pacman a lot faster here.

Arch Linux

#1 2005-11-12 05:14:05

Pacman very slow with XFS, JFS

#2 2005-11-12 17:27:54

Re: Pacman very slow with XFS, JFS

#3 2005-11-12 18:24:06

Re: Pacman very slow with XFS, JFS

#4 2005-11-12 19:17:34

Re: Pacman very slow with XFS, JFS

#5 2005-11-13 04:25:32

Re: Pacman very slow with XFS, JFS

#6 2005-11-13 14:43:18

Re: Pacman very slow with XFS, JFS

#7 2005-11-13 19:40:41

Re: Pacman very slow with XFS, JFS

#8 2005-11-13 19:55:56

Re: Pacman very slow with XFS, JFS

#9 2005-11-13 21:10:42

Re: Pacman very slow with XFS, JFS

#10 2005-11-13 21:18:57

Re: Pacman very slow with XFS, JFS

#11 2005-11-13 21:21:16

Re: Pacman very slow with XFS, JFS

#12 2005-11-13 21:36:15

Re: Pacman very slow with XFS, JFS

#13 2005-11-13 22:54:13

Re: Pacman very slow with XFS, JFS

#14 2005-11-14 03:52:04

Re: Pacman very slow with XFS, JFS

#15 2006-01-14 14:25:03

Re: Pacman very slow with XFS, JFS

#16 2006-01-14 18:22:13

Re: Pacman very slow with XFS, JFS

#17 2006-01-14 19:46:42

Re: Pacman very slow with XFS, JFS

#18 2006-01-14 21:18:18

Re: Pacman very slow with XFS, JFS

#19 2006-01-16 21:57:41

Re: Pacman very slow with XFS, JFS

#20 2006-01-17 00:54:41

Re: Pacman very slow with XFS, JFS

#21 2006-01-17 08:14:09

Re: Pacman very slow with XFS, JFS

#22 2006-01-17 11:49:24

Re: Pacman very slow with XFS, JFS

#23 2006-01-22 01:02:32

Re: Pacman very slow with XFS, JFS

#24 2006-01-22 01:45:20

Re: Pacman very slow with XFS, JFS

#25 2006-01-22 04:35:00

Re: Pacman very slow with XFS, JFS

Board footer