pacman -Syu with 3.4 is very slow

Radioactiveman · 2010-07-01 15:46:26

Hi,
I've got a problem with Pacman 3.4.0-2, -Syu is very slow ...
:: Starting full system upgrade... takes ~ 60 seconds to complete.

sudo pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 there is nothing to do

If I executed -Syu onetime and do it again, it takes <5 seconds, after a reboot it's the same as before (60s).
I tried pacman-optimize and different mirrors, I also deleted the cache, but it's still slow. What could be the problem? Pacman 3.3 was so fast ...

skunktrader · 2010-07-01 15:56:12

I have noticed the same thing. If I run top at the same time as -Syu, there is no abnormal cpu load to account for the delay.

karol · 2010-07-01 16:36:28

I'm also using Pacman 3.4.0-2 and it's just as fast / slow as the 3.3.

Last edited by karol (2010-07-01 16:38:47)

Inxsible · 2010-07-01 17:17:00

no difference in speed at this end either

TigTex · 2010-07-01 17:40:09

for me pacman 3.4 is faster.

Try pacman-cage from AUR. It helps A LOT.

bananaoomarang · 2010-07-01 20:23:01

Try using Clyde. From my experience it's a whole lot faster.

Radioactiveman · 2010-07-01 21:58:04

I've tested pacman-cage, it's really fast but I asked what could be wrong with pacman and not how to avoid the problem.
Do the developers know the issue? I think it's not really reproduceable but skunktrader has the same regression of speed.

Allan · 2010-07-02 00:09:54

Where do you see the slow down? Extracting the downloaded repos is much, much, much faster in 3.4 with a -Sy.

Pierre · 2010-07-02 00:17:13

I have got the same problem. The dependency resolution takes a lot of io load and takes about minutes. (cpu load is zero during that...at first I even thought it was in an infinite loop)

Is there a way to get some output what pacman is actually doing here? (debug output does not show anything)

TigTex · 2010-07-02 01:48:23

Pacman is only updating your local repo (extracting the db). The activity is all in your HDD. If it's slow, maybe you should run pacman-optimize and do pacman -Syyu just to see if it speeds up a bit.If you have a slow hdd, try pacman-cage. Pacman is getting slow because "every day" there are more packages...

stqn · 2010-07-02 02:15:29

Pacman 3.3 took a lot of time after each repo package list download. Pacman 3.4 seems to have moved that processing time after ":: Starting full system upgrade..." is displayed. Takes a very long time here too. pacman-optimize doesn't help, or maybe only just right after it's done...

BTW it's strange because during this processing period, my cpu load is at 100%, but XFCE's cpufreq monitor tells me it is stuck at 600 MHZ, which is the lowest frequency of my processor... It should be going up if the load is more than 80%... (in conservative mode)

Edit: it doesn't take as much time if I have already run pacman -Syu since last boot, I think...

Edit2: this is on an ext3 partition, laptop HD about 30-60 MB/s, nothing special installed (no pacman-cage, clyde, whatever.)
Yesterday I did a pacman-optimize and a pacman -Syy. Today pacman -Syu "Starting full system upgrade" took 95 s to tell me there was nothing to upgrade. During that time, cpu load is at 100% (kinda, as explained above) and there is a lot of HD activity.

Last edited by stqn (2010-07-02 19:34:30)

TigTex · 2010-07-02 03:02:57

It takes less time the 2nd and consecutive times you run pacman -Syu because the local database is already cached. Pacman 3.4 is faster than 3.3 on my mediacenter pc, on my laptop i have pacman-cage and I didn't noticed any diference (it takes only 2 seconds reading the db).

The performance change was with pacman -Sy. Pacman -Su still has to read everything so it's still slow like older versions.

Diferent filesystems makes pacman faster/slower.

Last edited by TigTex (2010-07-02 03:03:44)

Radioactiveman · 2010-07-02 16:25:56

Allan wrote:

Where do you see the slow down? Extracting the downloaded repos is much, much, much faster in 3.4 with a -Sy.

As already said,':: Starting full system upgrade...' needs that long, not the extracting.

Pierre · 2010-07-02 16:33:35

I just checked. a cold -Syu takes 45s (no updates available). My ext4 on raid5 reads about 200MByte/s. On my old notebook with quite slow hd this takes forever.

maxi_jac · 2010-07-02 17:21:48

It also seems way longer when calculating the packages that needs to be updated with pacman 3.4 than it was with 3.3 for me.

lilsirecho · 2010-07-02 17:33:33

Just ran pacman -Syu and took 23 seconds to reply with no upgrade.

toofishes · 2010-07-02 22:19:35

maxi_jac wrote:

It also seems way longer when calculating the packages that needs to be updated with pacman 3.4 than it was with 3.3 for me.

The tradeoff with only doing partial extraction of the database is now the database gets fragmented across your filesystem a lot quicker than it did before. If you are a compulsive -Syu-er, you should 1) fix your habit 2) run pacman-optimize every 10 -Syu operations or so because the database is going to fragment.

Until we move to a single file solution, any time we have to wait on the disk we are going to have a slow grind.

Pierre · 2010-07-02 23:24:51

Running pacman-optimize does not improve anything for me.

olive · 2010-07-03 03:26:55

Radioactiveman wrote:

Hi,
I've got a problem with Pacman 3.4.0-2, -Syu is very slow ...
:: Starting full system upgrade... takes ~ 60 seconds to complete.
sudo pacman -Syu
:: Synchronizing package databases...
 core is up to date
 extra is up to date
 community is up to date
:: Starting full system upgrade...
 there is nothing to do
If I executed -Syu onetime and do it again, it takes <5 seconds, after a reboot it's the same as before (60s).
I tried pacman-optimize and different mirrors, I also deleted the cache, but it's still slow. What could be the problem? Pacman 3.3 was so fast ...

This is not new (at least for me) with pacman 3.4 but the explanation that pacman is slow on the first time is the following. Pacman has to read a lot of files (I have 9129 files in the sync database /var/lib/pacman sync). Accessing a lot of file take time. The second time you use pacman the kernel has cached the cont of the file in memory so that pacman does not have to read these on the disk, so that it is much faster. The same behaviour can be observed by a command like:

find / -name <some-file-name>

Slow at first, very fast the second time.

Allan · 2010-07-03 03:43:47

This would be the way to fix this issue:
http://wiki.archlinux.org/index.php/Pac … map#Tar-DB
Then the "files" would be all in one place and a lot quicker to read. Patches welcome and all...

Pierre · 2010-07-03 07:19:14

I think we are mixing things up here. At least in my case I know about the issue with all the small files pacman has to read. The point it that 3.4 seems to be a lot slower when calculating deps or searching for possible updates. So it either does quite a lot more chekcs which require more reads for example.

Allan · 2010-07-03 08:07:19

Just to clarify. There are no more checks done in pacman-3.4, just the area where all the small files are read has been effectively moved.

Now the database extraction only extracts the needed db files for new/updated packages onto your system. This speeds that up a lot and if you have no packages to update, an "-Syu" will be much faster.

A side effect of this is that now the db files for the packages that have not had an update are not in your systems cache and the dependency calculations need to read them in from scratch. This is the slow-down you have noticed

Pierre · 2010-07-03 12:25:09

I think I get it now. So previous versions extracted the whole db which also added those files into the cache? The "putting everything into a tar" approach might indeed help here.

stqn · 2010-07-03 15:31:11

Allan wrote:

Now the database extraction only extracts the needed db files for new/updated packages onto your system. This speeds that up a lot and if you have no packages to update, an "-Syu" will be much faster.

"pacman -Syu" is much slower than before, here. 95 seconds to tell me there is nothing to upgrade... (and that was the day after I did pacman -Syy and pacman-optimize.) (I've edited my post above to add this information.)

pacman -S non-existing-package also takes about the same time, which is more annoying.

lilsirecho · 2010-07-03 16:27:53

Allan:

Performed ..pacman-optimize yesterday...pacman -3.4.2-0

Today have this time report:

[root@n6re ~]# time pacman -Syu
:: Synchronizing package databases...
core is up to date
extra is up to date
community is up to date
:: Starting full system upgrade...
warning: vlc: local (1.1.0-1) is newer than extra (1.0.6-8)
there is nothing to do

real 0m2.010s
user 0m0.260s
sys 0m0.180s
[root@n6re ~]#
This indicates a problem I have been experiencing...no upgrades for two weeks!

Vlc was built from abs extra and installed via pacman -U under the new pacman package.

I am not sure what the statement "vlc package is newer than extra" means. Is it my listing of extra or the package in archlinux repo that is referenced.

Perhaps this conflict prevents any upgrade of other packages?

I read that pacman -U operates in a new way with the new pacman package. I note that it seems to download depends whether they already exist in file or not such as a local cachedir.

Is there a problem in the latest pacman related to the use of ..pacman -U with abs generated packages?

A previous post I made in this thread indicated 23 secs for ...pacman -Syu with same message resulting. Thus, pacman-optimize was effective in reducing the time. But I still cannot upgrade with Syu.

Last edited by lilsirecho (2010-07-03 16:28:22)

Arch Linux

#1 2010-07-01 15:46:26

pacman -Syu with 3.4 is very slow

#2 2010-07-01 15:56:12

Re: pacman -Syu with 3.4 is very slow

#3 2010-07-01 16:36:28

Re: pacman -Syu with 3.4 is very slow

#4 2010-07-01 17:17:00

Re: pacman -Syu with 3.4 is very slow

#5 2010-07-01 17:40:09

Re: pacman -Syu with 3.4 is very slow

#6 2010-07-01 20:23:01

Re: pacman -Syu with 3.4 is very slow

#7 2010-07-01 21:58:04

Re: pacman -Syu with 3.4 is very slow

#8 2010-07-02 00:09:54

Re: pacman -Syu with 3.4 is very slow

#9 2010-07-02 00:17:13

Re: pacman -Syu with 3.4 is very slow

#10 2010-07-02 01:48:23

Re: pacman -Syu with 3.4 is very slow

#11 2010-07-02 02:15:29

Re: pacman -Syu with 3.4 is very slow

#12 2010-07-02 03:02:57

Re: pacman -Syu with 3.4 is very slow

#13 2010-07-02 16:25:56

Re: pacman -Syu with 3.4 is very slow

#14 2010-07-02 16:33:35

Re: pacman -Syu with 3.4 is very slow

#15 2010-07-02 17:21:48

Re: pacman -Syu with 3.4 is very slow

#16 2010-07-02 17:33:33

Re: pacman -Syu with 3.4 is very slow

#17 2010-07-02 22:19:35

Re: pacman -Syu with 3.4 is very slow

#18 2010-07-02 23:24:51

Re: pacman -Syu with 3.4 is very slow

#19 2010-07-03 03:26:55

Re: pacman -Syu with 3.4 is very slow

#20 2010-07-03 03:43:47

Re: pacman -Syu with 3.4 is very slow

#21 2010-07-03 07:19:14

Re: pacman -Syu with 3.4 is very slow

#22 2010-07-03 08:07:19

Re: pacman -Syu with 3.4 is very slow

#23 2010-07-03 12:25:09

Re: pacman -Syu with 3.4 is very slow

#24 2010-07-03 15:31:11

Re: pacman -Syu with 3.4 is very slow

#25 2010-07-03 16:27:53

Re: pacman -Syu with 3.4 is very slow

Board footer