[SOLVED] mandb rebuilds whole database everytime (again)

bernarcher · 2009-09-09 08:59:12

Since several days (after the latest man-db update 2009-09-05 ?) a mandb bug seems to show up again (see this thread).
Mandb rebuilds its whole database again in every run. See these timings of a run immediately done after a fresh db rebuild:

real    1m13.551s
user    0m23.655s
sys    0m55.560s

Does this happen to anybody else? If yes, is there a bug report already? (I can't search the bugs database currently.)

Last edited by bernarcher (2009-09-12 14:34:57)

shining · 2009-09-09 09:08:09

I would suggest contacting Colin about it : http://bugs.archlinux.org/user/3845
and maybe re-opening that bug after a second person has confirmed the problem : http://bugs.archlinux.org/task/14467

camphor · 2009-09-09 14:04:46

Having the same problem.

shining · 2009-09-09 14:06:20

request the reopening of the bug then.

ajonat · 2009-09-10 03:09:55

Same problem here, except I'm on an Athlon XP and mandb takes 5 minutes in which I can hardly do anything else. REALLY annoying.

man-db: 2.5.6-1

Last edited by ajonat (2009-09-10 03:19:12)

jdhore · 2009-09-10 04:54:26

I'm having the same problem. I'm on a decently fast, multi-core system, but rebuilding the db everynight at midnight is somewhat annoying considering it pegs one of my cores at about 50% actual CPU usage and 100% "I/O"

man-db 2.5.6-1

On ArchLinux i686

tomd123 · 2009-09-10 05:05:10

Heads up! If you run cron, then by default, there is a man-db script in /etc/cron.daily/man-db where it regenerates the man db.

shining · 2009-09-10 05:37:29

I guess we have enough confirmation now, no need for more. Did anyone contact Colin Watson ? And same here, no need for 10 person contacting him, only one, who can then update the others here.

bernarcher · 2009-09-10 06:37:50

According to the bugtracker http://bugs.archlinux.org/task/14467 a quest to re-open the bug has already been made on 2009-09-09.

jdhore · 2009-09-10 06:55:16

I took a look at the SVN for the man-db package. Besides the package version bump and the removal of 2 patches that appear to have been applied upstream, nothing in the packaging seems to have changed enough to cause this.

bernarcher · 2009-09-10 07:07:54

jdhore wrote:

I took a look at the SVN for the man-db package. Besides the package version bump and the removal of 2 patches that appear to have been applied upstream, nothing in the packaging seems to have changed enough to cause this.

I have no time to verify, but could you have a look at what those patches were? Could well be they had been not applied literally upstream.

jdhore · 2009-09-10 07:33:20

bernarcher wrote:

jdhore wrote:
I took a look at the SVN for the man-db package. Besides the package version bump and the removal of 2 patches that appear to have been applied upstream, nothing in the packaging seems to have changed enough to cause this.
I have no time to verify, but could you have a look at what those patches were? Could well be they had been not applied literally upstream.

It seems like one patch added support for man section 0, The other seemed to be for libdb/db_lookup.c, to make it slightly more sane, but it shouldn't have really affected functionality.

bernarcher · 2009-09-10 07:45:29

All right, if the bug would not get re-opened in time, I'd file a new bug this evening.

Allan · 2009-09-10 08:35:19

Bug report reopened.

bernarcher · 2009-09-11 07:37:27

The cause may well be not with man-db itself. At least downgrading to 2.5.5-2 (I should have done this before - mea culpa!) did not help.

Now that the bug report was reopend and information has been submitted we have to wait what Colin (hopefully soon) will find out.

AndyRTR · 2009-09-11 07:40:13

i wonder why you always poke our tracker and not the upstream developer who always helped very quickly. i'm not your postman to forward your bugs.

shining · 2009-09-11 07:55:02

AndyRTR wrote:

i wonder why you always poke our tracker and not the upstream developer who always helped very quickly. i'm not your postman to forward your bugs.

In this case, the upstream developer is registered in arch bugtracker, and was already following that bug before and resolved it.
The advantage I see is that everyone can follow what the status is, and to have one central place to inform all arch users.

In the common case where upstream developer is not registered to arch bug tracker, the same advantages still apply. But then of course, the users also need to report it upstream (ideally on upstream bugtracker, and then informing it on arch bugtracker).
Also it is never obvious whether an issue is purely upstream or not, sometimes it is only caused by interactions (sometimes complex) between all the different software arch provides.

You don't have to forward bugs. If users are not doing it themselves, it should not be too difficult to inform them you think the bug is upstream and that they should report there.

All the above is just my opinion. And while I spend a lot of time using looking and using bug trackers of different projects, I am not a package maintainer myself, so maybe I still lack experience and knowledge to know what is best

AndyRTR · 2009-09-11 08:26:01

Xavier, you are almost right. But too often users only report it to our tracker and leave all the rest up to the package maintainter. They could save us a lot of time and work preparing the fix/patch for us. When they don't help out they will have to live with a long delay if I have something else to do.

I'd prefer to look first in upstream trackers for known issues or ask upstream devs dierectly in #irc channels. Our tracker is primary for Arch related packaging bugs and only in a 2nd way to keep track of upstream issues.

That's just my opinion.

shining · 2009-09-11 09:11:41

AndyRTR wrote:

Xavier, you are almost right. But too often users only report it to our tracker and leave all the rest up to the package maintainter. They could save us a lot of time and work preparing the fix/patch for us. When they don't help out they will have to live with a long delay if I have something else to do.

That is very true, and is also explained there :
http://wiki.archlinux.org/index.php/Rep … or_Arch.3F

I'd prefer to look first in upstream trackers for known issues or ask upstream devs dierectly in #irc channels. Our tracker is primary for Arch related packaging bugs and only in a 2nd way to keep track of upstream issues.

The problems is that devs are pissed off both ways
downstream devs are pissed off by upstream issues and upstream devs are pissed off by downstream issues.
A very important aspect here is that there is one upstream for many downstream, so this seems to advice that, in doubt, look with downstream first before bothering upstream.

and see an example why this is sometimes not obvious : https://bugzilla.mozilla.org/show_bug.cgi?id=512940#c11

bernarcher · 2009-09-11 11:52:23

AndyRTR wrote:

i wonder why you always poke our tracker and not the upstream developer who always helped very quickly. i'm not your postman to forward your bugs.

So I shall feel guilty now?

Well, I surely read Upstream or Arch? in the Reporting Bug Guidelines. But it often requires more background as the normal user usually has. It is not always easy to decide where to report, especially if something looks like a regression from an already reported bug (actually, in this case, it appears to be something new).

I'd rather file a bug in the Arch bugtracker so it gets known after all. If I then was told to report upstream, I gladly will do. You need not be my postmaster. But in most cases someone more knowledgeable than me has to guide me.

Sorry for being incompetent at times. But I do not like flame wars whatsoever.

bernarcher · 2009-09-11 12:34:31

There appears to be a rather simple fix, as Colin wrote in the bugtracker:

Comment by Colin Watson (cjwatson) - Friday, 11 September 2009, 08:14 GMT
Thanks. This is a new cause of the same symptom, not as far as I can see a regression from the previous fix.
The proper fix in man-db is a little involved, and I'll have to think about it, but at least I have a test case now. In the meantime, your modprobe.d.5.gz is buggy anyway and ought to be fixed. I suggest that it be corrected as follows:
-.so modprobe.conf.5
+.so man5/modprobe.conf.5
(.so includes should always be relative to the *base* of the manual hierarchy, e.g. /usr/share/man in this case.) Fixing this will make this particular cause go away, although of course it's possible that there are other similar pages ...

Applying the change fixed the problem for me.
If this can be confirmed by somebody else I will mark this thread solved.

Last edited by bernarcher (2009-09-11 12:37:19)

jdhore · 2009-09-11 17:35:38

bernarcher wrote:

There appears to be a rather simple fix, as Colin wrote in the bugtracker:
Comment by Colin Watson (cjwatson) - Friday, 11 September 2009, 08:14 GMT
Thanks. This is a new cause of the same symptom, not as far as I can see a regression from the previous fix.
The proper fix in man-db is a little involved, and I'll have to think about it, but at least I have a test case now. In the meantime, your modprobe.d.5.gz is buggy anyway and ought to be fixed. I suggest that it be corrected as follows:
-.so modprobe.conf.5
+.so man5/modprobe.conf.5
(.so includes should always be relative to the *base* of the manual hierarchy, e.g. /usr/share/man in this case.) Fixing this will make this particular cause go away, although of course it's possible that there are other similar pages ...
Applying the change fixed the problem for me.
If this can be confirmed by somebody else I will mark this thread solved.

What exactly is supposed to be done there? Just move modprobe.conf.5 to man5/modprobe.conf.5 ? Edit a config file somewhere?

bernarcher · 2009-09-11 18:16:37

Uncompress /usr/share/man/man5/modprobe.d.5.gz, change the .so line (was the only one here) and recompress again. (Your editor may even do the uncompressing/compressing on its own.)

A prior file backup would be in order...

markg85 · 2009-09-13 13:16:27

Applying that fixed solved the issue for me as well. But i have another issue with mandb.. When it runs it says:

Purging old database entries in /usr/man...
Processing manual pages under /usr/man...
Purging old database entries in /usr/share/man...
Processing manual pages under /usr/share/man...
Purging old database entries in /usr/share/man/it.ISO8859-1...
Processing manual pages under /usr/share/man/it.ISO8859-1...
Purging old database entries in /usr/share/man/ru.KOI8-R...
Processing manual pages under /usr/share/man/ru.KOI8-R...
Purging old database entries in /usr/share/man/tr...
Processing manual pages under /usr/share/man/tr...
Purging old database entries in /usr/share/man/ja...
Processing manual pages under /usr/share/man/ja...
Purging old database entries in /usr/share/man/es...
Processing manual pages under /usr/share/man/es...
Purging old database entries in /usr/share/man/ru...
Processing manual pages under /usr/share/man/ru...
Purging old database entries in /usr/share/man/it...
Processing manual pages under /usr/share/man/it...
Purging old database entries in /usr/share/man/hu...
Processing manual pages under /usr/share/man/hu...
Purging old database entries in /usr/share/man/sv...
Processing manual pages under /usr/share/man/sv...
Purging old database entries in /usr/share/man/zh_CN...
Processing manual pages under /usr/share/man/zh_CN...
Purging old database entries in /usr/share/man/fr.ISO8859-1...
Processing manual pages under /usr/share/man/fr.ISO8859-1...
Purging old database entries in /usr/share/man/it.UTF-8...
Processing manual pages under /usr/share/man/it.UTF-8...
Purging old database entries in /usr/share/man/ru.UTF-8...
Processing manual pages under /usr/share/man/ru.UTF-8...
Purging old database entries in /usr/share/man/jp...
Processing manual pages under /usr/share/man/jp...
Purging old database entries in /usr/share/man/pl...
Processing manual pages under /usr/share/man/pl...
Purging old database entries in /usr/share/man/de.UTF-8...
Processing manual pages under /usr/share/man/de.UTF-8...
Purging old database entries in /usr/share/man/pt_BR...
Processing manual pages under /usr/share/man/pt_BR...
Purging old database entries in /usr/share/man/id...
Processing manual pages under /usr/share/man/id...
Purging old database entries in /usr/share/man/de...
Processing manual pages under /usr/share/man/de...
Purging old database entries in /usr/share/man/fr.UTF-8...
Processing manual pages under /usr/share/man/fr.UTF-8...
Purging old database entries in /usr/share/man/cs...
Processing manual pages under /usr/share/man/cs...
Purging old database entries in /usr/share/man/nl...
Processing manual pages under /usr/share/man/nl...
Purging old database entries in /usr/share/man/fr...
Processing manual pages under /usr/share/man/fr...
Purging old database entries in /usr/share/man/pl.ISO8859-2...
Processing manual pages under /usr/share/man/pl.ISO8859-2...
Purging old database entries in /usr/share/man/zh_TW...
Processing manual pages under /usr/share/man/zh_TW...
Purging old database entries in /usr/share/man/pl.UTF-8...
Processing manual pages under /usr/share/man/pl.UTF-8...
Purging old database entries in /usr/share/man/ko...
Processing manual pages under /usr/share/man/ko...
Purging old database entries in /usr/share/man/fi...
Processing manual pages under /usr/share/man/fi...
Processing manual pages under /usr/local/man...
0 man subdirectories contained newer manual pages.
0 manual pages were added.
0 stray cats were added.
0 old database entries were purged.

That's all just fine and works but how do i tell mandb to only update the english man pages and not all those other ones? i mean.. i can't read japanese, chinese, polish, german and all those other languages..

Arch Linux

#1 2009-09-09 08:59:12

[SOLVED] mandb rebuilds whole database everytime (again)

#2 2009-09-09 09:08:09

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#3 2009-09-09 14:04:46

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#4 2009-09-09 14:06:20

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#5 2009-09-10 03:09:55

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#6 2009-09-10 04:54:26

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#7 2009-09-10 05:05:10

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#8 2009-09-10 05:37:29

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#9 2009-09-10 06:37:50

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#10 2009-09-10 06:55:16

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#11 2009-09-10 07:07:54

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#12 2009-09-10 07:33:20

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#13 2009-09-10 07:45:29

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#14 2009-09-10 08:35:19

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#15 2009-09-11 07:37:27

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#16 2009-09-11 07:40:13

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#17 2009-09-11 07:55:02

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#18 2009-09-11 08:26:01

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#19 2009-09-11 09:11:41

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#20 2009-09-11 11:52:23

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#21 2009-09-11 12:34:31

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#22 2009-09-11 17:35:38

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#23 2009-09-11 18:16:37

Re: [SOLVED] mandb rebuilds whole database everytime (again)

#24 2009-09-13 13:16:27

Re: [SOLVED] mandb rebuilds whole database everytime (again)

Board footer