You are not logged in.

#1 2013-11-09 11:24:59

Box0
Member
From: Italy
Registered: 2011-09-21
Posts: 218

Pacman installation very slow for some packages

Lately pacman is acting a bit strange here. Some packages takes several minutes to install, even if they are small packages. An example of such packages is kdebase-runtime.

If a run a normal pacman -S kdebase runtime the installation process hangs for 2-3 minutes at this point:

(1/1) reinstalling kdebase-runtime                                                                     [############################################################] 100%

It happens every time I reinstall the package, and also when I upgrade it to a new version

If I run pacman -S --debug kdebase-runtime, I can see that it hangs at:

debug: executing "/usr/bin/bash" under chroot "/"

This happens when I run pacman via sudo, when I'm root and when I use a frontend like yaourt.

How can  I debug this issue?

Edit: here are some benchmarks I did with the time command before pacman (time pacman -S <package name>). Of course I'm doing this with already downloaded packages, so that the download time is not taken into account.

kdepimlibs (affected):

real    1m31.312s
user    0m1.110s
sys     0m0.433s

calibre (affected):

real    1m33.768s
user    0m3.457s
sys     0m0.593s

vlc (not affected):

real    0m4.129s
user    0m1.893s
sys     0m0.210s

Last edited by Box0 (2013-11-09 11:51:12)

Offline

#2 2013-11-09 12:23:37

apg
Developer
Registered: 2012-11-10
Posts: 123

Re: Pacman installation very slow for some packages

The problem isn't pacman, it's update-mime-database which is being called from the install scripts:
https://bugs.freedesktop.org/show_bug.cgi?id=70366

Offline

#3 2013-11-09 12:59:10

Box0
Member
From: Italy
Registered: 2011-09-21
Posts: 218

Re: Pacman installation very slow for some packages

Thanks for the link.

It seems shared-mime-info devs are suggesting that this needs to be fixed at the package manager level. Anyway, until they decide what's the best way to fix this, would it be safe to downgrade to version 1.1?

Because this bug can be very annoying if you have to update lots of pakages (say ~100).

Last edited by Box0 (2013-11-09 13:01:25)

Offline

#4 2013-11-17 02:35:06

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 5,662

Re: Pacman installation very slow for some packages

You might consider filing this against pacman. As a feature request. There are other reasons triggers would be useful. For example, right now the font cache is updated after each font package is installed, removed or upgraded. This can take a while if you install multiple font packages at once. The cache only really needs to be updated following the last transaction but I believe there is currently no way to implement that in Arch's packaging system. So the problem is a general one requiring a solution at the distro level even if this particular bug results in a configure option which could be used to avoid this particular delay.


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | grub2 | systemd | LVM2 on LUKS
Lenovo x121e | Intel(R) Core(TM) i3-2367M CPU @ 1.40GHz GenuineIntel | Intel Centrino Wireless-N 1000 | US keyboard with Euro | 320G 7200 RPM Seagate HDD

Offline

#5 2013-11-17 02:38:45

apg
Developer
Registered: 2012-11-10
Posts: 123

Re: Pacman installation very slow for some packages

There has been a feature request for hooks since 2005.

https://bugs.archlinux.org/task/2985

Offline

#6 2013-11-17 02:39:59

karol
Archivist
Registered: 2009-05-06
Posts: 25,428

Re: Pacman installation very slow for some packages

There's https://aur.archlinux.org/packages/pacmannrd/ , maybe you cna create a similar workaround.

Offline

#7 2013-11-17 03:07:11

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 5,662

Re: Pacman installation very slow for some packages

apg wrote:

There has been a feature request for hooks since 2005.

https://bugs.archlinux.org/task/2985

I 'm not sure that's the same feature. Doesn't that provide a way to run a given whatever whenever a particular package is upgraded or installed? Whereas this would provide a sort of triggering facility so that multiple packages could request a trigger but it would get triggered only once. Or have I misunderstood the hooks request?

EDIT: Is there any reason you couldn't theoretically have a package which installed no files but merely ran an install script? If so, is there a way to force a package to be installed last? Because if so, couldn't you just put the trigger action in a package to be installed/upgraded/reinstalled as the final step?

Last edited by cfr (2013-11-17 03:10:04)


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | grub2 | systemd | LVM2 on LUKS
Lenovo x121e | Intel(R) Core(TM) i3-2367M CPU @ 1.40GHz GenuineIntel | Intel Centrino Wireless-N 1000 | US keyboard with Euro | 320G 7200 RPM Seagate HDD

Offline

#8 2013-11-17 03:16:57

apg
Developer
Registered: 2012-11-10
Posts: 123

Re: Pacman installation very slow for some packages

cfr wrote:
apg wrote:

There has been a feature request for hooks since 2005.

https://bugs.archlinux.org/task/2985

I 'm not sure that's the same feature. Doesn't that provide a way to run a given whatever whenever a particular package is upgraded or installed? Whereas this would provide a sort of triggering facility so that multiple packages could request a trigger but it would get triggered only once. Or have I misunderstood the hooks request?

The hooks are not intended to be package specific if that's what you're getting at.  They would be triggered when certain packages/files were installed/upgraded/removed just like what you're talking about.  See https://wiki.archlinux.org/index.php/Us … cman_Hooks

cfr wrote:

EDIT: Is there any reason you couldn't theoretically have a package which installed no files but merely ran an install script? If so, is there a way to force a package to be installed last? Because if so, couldn't you just put the trigger action in a package to be installed/upgraded/reinstalled as the final step?

The only way to force order would be to have the trigger package depend on every single package that needs the trigger.

Offline

#9 2013-11-17 03:18:33

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 5,662

Re: Pacman installation very slow for some packages

But the idea of the triggers is that multiple packages would specify the same trigger yet pacman would only run the trigger once.

Yes, I thought order would be the problem.


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | grub2 | systemd | LVM2 on LUKS
Lenovo x121e | Intel(R) Core(TM) i3-2367M CPU @ 1.40GHz GenuineIntel | Intel Centrino Wireless-N 1000 | US keyboard with Euro | 320G 7200 RPM Seagate HDD

Offline

#10 2013-11-17 03:21:09

karol
Archivist
Registered: 2009-05-06
Posts: 25,428

Re: Pacman installation very slow for some packages

Instead of running it once per package, it could run once per 'pacman -S foo bar baz' command, but pacman doesn't support transactions.

Offline

#11 2013-11-17 03:23:47

apg
Developer
Registered: 2012-11-10
Posts: 123

Re: Pacman installation very slow for some packages

cfr wrote:

But the idea of the triggers is that multiple packages would specify the same trigger yet pacman would only run the trigger once.

If you look at the wiki page I linked, the current plan is for hooks to be able to specify whether they should be run per package or per transaction (as you would want for update-mime-database).  in fact, the ability to run something once for the entire transaction is the main appeal of hooks.

Offline

#12 2013-11-17 22:10:53

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 5,662

Re: Pacman installation very slow for some packages

I guess the problem was I had/have no idea what counts as a 'transaction' so that page really didn't tell me very much. Plus I read the bug as well which didn't make it sound like the same idea.

Anyway, it is all largely irrelevant since there's no sign anything like this will actually be implemented, is there?


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | grub2 | systemd | LVM2 on LUKS
Lenovo x121e | Intel(R) Core(TM) i3-2367M CPU @ 1.40GHz GenuineIntel | Intel Centrino Wireless-N 1000 | US keyboard with Euro | 320G 7200 RPM Seagate HDD

Offline

#13 2013-11-17 22:15:56

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman installation very slow for some packages

It is the same idea...    Also, the generic ini-style config file parser need for this has been committed to pacman-git.  So there is some progress.

Offline

#14 2013-11-17 23:46:05

cfr
Member
From: Cymru
Registered: 2011-11-27
Posts: 5,662

Re: Pacman installation very slow for some packages

Cool smile. Thank you for the update!


How To Ask Questions The Smart Way | Help Vampires

Arch Linux | x86_64 | GPT | EFI boot | grub2 | systemd | LVM2 on LUKS
Lenovo x121e | Intel(R) Core(TM) i3-2367M CPU @ 1.40GHz GenuineIntel | Intel Centrino Wireless-N 1000 | US keyboard with Euro | 320G 7200 RPM Seagate HDD

Offline

#15 2013-11-21 11:32:00

djomla
Member
Registered: 2013-05-27
Posts: 28

Re: Pacman installation very slow for some packages

I have problem 4 days in a row. I tried with mirror list change, last mirror list, tried also with various mirrors to put on top of list, from Germany, USA, Australia, France...none of them are working. When I try to do pacman -Syu , some of error that i got is:

error: failed retrieving file 'core.db' from mirror.de.leaseweb.net : Resolving timed out after 10519 milliseconds
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from mirror.de.leaseweb.net : Resolving timed out after 10519 milliseconds
error: failed to update extra (download library error)

This is my pacman.conf file:

#
# /etc/pacman.conf
#
# See the pacman.conf(5) manpage for option and repository directives

#
# GENERAL OPTIONS
#
[options]
# The following paths are commented out with their default values listed.
# If you wish to use different paths, uncomment and update the paths.
#RootDir     = /
#DBPath      = /var/lib/pacman/
#CacheDir    = /var/cache/pacman/pkg/
#LogFile     = /var/log/pacman.log
#GPGDir      = /etc/pacman.d/gnupg/
HoldPkg     = pacman glibc
#XferCommand = /usr/bin/curl -C - -f %u > %o
#XferCommand = /usr/bin/wget --passive-ftp -c -O %o %u
#CleanMethod = KeepInstalled
#UseDelta    = 0.7
Architecture = auto

# Pacman won't upgrade packages listed in IgnorePkg and members of IgnoreGroup
#IgnorePkg   = audacity
#IgnoreGroup =

#NoUpgrade   =
#NoExtract   =

# Misc options
#UseSyslog
Color
#TotalDownload
CheckSpace
#VerbosePkgLists

# By default, pacman accepts packages signed by keys that its local keyring
# trusts (see pacman-key and its man page), as well as unsigned packages.
SigLevel    = Required DatabaseOptional
LocalFileSigLevel = Optional
#RemoteFileSigLevel = Required

# NOTE: You must run `pacman-key --init` before first using pacman; the local
# keyring can then be populated with the keys of all official Arch Linux
# packagers with `pacman-key --populate archlinux`.

#
# REPOSITORIES
#   - can be defined here or included from another file
#   - pacman will search repositories in the order defined here
#   - local/custom mirrors can be added here or in separate files
#   - repositories listed first will take precedence when packages
#     have identical names, regardless of version number
#   - URLs will have $repo replaced by the name of the current repo
#   - URLs will have $arch replaced by the name of the architecture
#
# Repository entries are of the format:
#       [repo-name]
#       Server = ServerName
#       Include = IncludePath
#
# The header [repo-name] is crucial - it must be present and
# uncommented to enable the repo.
#

# The testing repositories are disabled by default. To enable, uncomment the
# repo name header and Include lines. You can add preferred servers immediately
# after the header, and they will be used before the default mirrors.

#[testing]
#Include = /etc/pacman.d/mirrorlist

[core]
Include = /etc/pacman.d/mirrorlist

[extra]
Include = /etc/pacman.d/mirrorlist

#[community-testing]
#Include = /etc/pacman.d/mirrorlist

[community]
Include = /etc/pacman.d/mirrorlist

# If you want to run 32 bit applications on your x86_64 system,
# enable the multilib repositories as required here.

#[multilib-testing]
#Include = /etc/pacman.d/mirrorlist

[multilib]
Include = /etc/pacman.d/mirrorlist

#[archlinuxfr]
#SigLevel = PackageOptional
#Server= http://repo.archlinux.fr/$arch

#[archlinuxcn]
#The Chinese Arch Linux communities packages.
#SigLevel = PackageOptional
#Server = http://repo.archlinuxcn.org/$arch

#[herecura-stable]
# additional apps not found in community
#SigLevel = PackageOptional
#Server = http://repo.herecura.be/herecura-stable/$arch

# An example of a custom package repository.  See the pacman manpage for
# tips on creating your own repositories.
#[custom]
#SigLevel = Optional TrustAll
#Server = file:///home/custompkgs

and last server that I`m using in mirror list file is Server = http://mirror.de.leaseweb.net/archlinux/$repo/os/$arch

I tried also with deleting all cache from pacman cache folder, with *.part file.

Last night was working fine, but this morning again, same issue.

Offline

#16 2013-11-21 11:37:55

arojas
Developer
From: Spain
Registered: 2011-10-09
Posts: 1,178

Re: Pacman installation very slow for some packages

djomla wrote:

I have problem 4 days in a row. I tried with mirror list change, last mirror list, tried also with various mirrors to put on top of list, from Germany, USA, Australia, France...none of them are working. When I try to do pacman -Syu , some of error that i got is:

Open a new thread, your post has nothing to do with the issue discussed here.

Offline

#17 2013-11-28 02:09:06

brando56894
Member
From: New Jersey, USA
Registered: 2008-08-03
Posts: 620

Re: Pacman installation very slow for some packages

I have noticed this also and was wondering if my SSD performance was degrading because it used to zip right along, now it slows to a crawl at some points.

Offline

#18 2013-12-04 22:00:26

Box0
Member
From: Italy
Registered: 2011-09-21
Posts: 218

Re: Pacman installation very slow for some packages

for the records, I just downgraded shared-mime-info to 1.1 and I plan to stick with it until a better solution shows up or something breaks in my box smile

Maybe it's because I do not update often (so when I do there are lots of packages to update) or maybe it's just me but I find this issue extremely annoying.

Offline

#19 2013-12-07 07:27:01

vvilp
Member
Registered: 2013-12-07
Posts: 1

Re: Pacman installation very slow for some packages

Glad to see this post
This bug makes me crazy

I found they have add  an environment variable to disable fdatasync() in shared-mime-info.

# time PKGSYSTEM_DISABLE_FSYNC=1 ./update-mime-database /usr/share/mime
Unknown media type in type 'all/all'
Unknown media type in type 'all/allfiles'
Unknown media type in type 'uri/mms'
Unknown media type in type 'uri/mmst'
Unknown media type in type 'uri/mmsu'
Unknown media type in type 'uri/pnm'
Unknown media type in type 'uri/rtspt'
Unknown media type in type 'uri/rtspu'

real    0m0.362s
user    0m0.294s
sys    0m0.068s

so we just need wait the new version of  shared-mime-info and add this VAR

BTW, when will this package upgrade .....?

Last edited by vvilp (2013-12-07 07:27:23)

Offline

#20 2013-12-07 10:11:01

Allan
Developer
From: Brisbane, AU
Registered: 2007-06-09
Posts: 10,409
Website

Re: Pacman installation very slow for some packages

Umm...  that has not made it to upstream git yet and has an objection to using environmental variables in the relevant bug report.   So don't expect an update yet.

Offline

#21 2013-12-20 17:01:45

artoo
Member
Registered: 2012-09-04
Posts: 175
Website

Re: Pacman installation very slow for some packages

The gentoo people have provided a patch.
On my gentoo-system, the annoyance is gone with patch & v1.2.

http://sources.gentoo.org/cgi-bin/viewv … iew=markup

I'll patch arch's shared-mime-info-1.2 and report the result, since I am unsure if the patch is related.

My harddisk is under heavy stress with 1.2 on arch, contrary to gentoo. The hdd heads go crazy for minutes.

Last edited by artoo (2013-12-20 17:05:25)

Offline

#22 2013-12-20 18:37:47

apg
Developer
Registered: 2012-11-10
Posts: 123

Re: Pacman installation very slow for some packages

artoo wrote:

The gentoo people have provided a patch.
On my gentoo-system, the annoyance is gone with patch & v1.2.

http://sources.gentoo.org/cgi-bin/viewv … iew=markup

I'll patch arch's shared-mime-info-1.2 and report the result, since I am unsure if the patch is related.

My harddisk is under heavy stress with 1.2 on arch, contrary to gentoo. The hdd heads go crazy for minutes.

That patch is not related to this issue.  Gentoo's is faster because they disable the fdatasync call at build time with

export ac_cv_func_fdatasync=no

as described in the bug report.

Offline

#23 2013-12-20 19:13:41

artoo
Member
Registered: 2012-09-04
Posts: 175
Website

Re: Pacman installation very slow for some packages

apg wrote:
artoo wrote:

The gentoo people have provided a patch.
On my gentoo-system, the annoyance is gone with patch & v1.2.

http://sources.gentoo.org/cgi-bin/viewv … iew=markup

I'll patch arch's shared-mime-info-1.2 and report the result, since I am unsure if the patch is related.

My harddisk is under heavy stress with 1.2 on arch, contrary to gentoo. The hdd heads go crazy for minutes.

That patch is not related to this issue.  Gentoo's is faster because they disable the fdatasync call at build time with

export ac_cv_func_fdatasync=no

as described in the bug report.


Yes, and me was thinking, it wouldn't build without glib patch.

Anyway, downgrade was the option of choice for the moment.

Offline

#24 2014-01-18 09:07:19

ssri
Member
Registered: 2010-02-16
Posts: 212

Re: Pacman installation very slow for some packages

Yeah, annoyingly this is a security feature rather than a bug despite mimeinfo.cache easily fixable.  Anyways, there's another option using libeatmydata

Rodrigo on AML tested it out:

Instead of a patch I've installed the libeatmydata package from the AUR and
added a file /usr/local/bin/update-mime-database that calls `exec eatmydata
update-mime-database "$@"`.

Before: 21.618 s.
After: 0.275 s.

A x100 improvement, that's something!

--
Rodrigo

Last edited by ssri (2014-01-18 09:11:59)

Offline

Board footer

Powered by FluxBB