New: mlocate, faster drop-in for slocate

lydgate · 2006-02-14 17:46:02

mlocate is a magical tool that, rather than recreate the entire locate database every day, reuses your database. The result is that a first time updatedb with mlocate is slightly faster, and every subsequent one is much faster. To me there seems no reason not to use it, it's very reliable, and also I have a very loud harddrive and this makes the operation much quieter.

Read more here:
http://freshmeat.net/redir/mlocate/6237 … ge/mlocate

I just added a PKGBUILD for mlocate to the AUR, I would really appreciate some testing/attention from experienced users to tell me whether I did it correctly. I modelled it after the slocate PKGBUILD. Just search the AUR for mlocate and it's there. Vote if you use it! I think this should be an official part of the distro.

I will be replying to this posts with some tests in a few minutes.

EDIT: In AUR, also binary pkg here: (no longer have this webspace)

Last edited by lydgate (2007-04-19 09:15:24)

lydgate · 2006-02-14 18:02:49

(this is on a 60 G laptop HDD)

slocate run (every run will be this speed)
real 4m53.986s
user 0m3.854s
sys 0m20.781s

mlocate first run (while I was doing a few other things)
real: 5m13.455s
user: 0m1.551s
sys: 0m21.297s

mlocate second run
real: 0m39.835s
user: 0m0.583s
sys: 0m1.072s

mlocate third run
real: 0m1.211s
user: 0m0.556s
sys: 0m0.603s

Since I ran these tests consecutively, my filesystem is not changing much, but point is, every slocate updatedb takes as long as the first mlocate updatedb, with all the HD thrashing that entails.

As you can see, the first mlocate update is slower, but the later ones are much faster. Try this out and see if it stops your thrashing.

stonecrest · 2006-02-14 19:38:08

I'm trying this out right now (literally while I type this message) and it's very nice. Indeed it definitely is less cpu intensive, I was actually able to do other things while I ran updatedb. Normally my computer becomes very unresponsive when running it and I basically have to wait for it to finish.

I didn't do any speed comparisons though, maybe I will later tonight.

Nice find.

lucke · 2006-02-14 19:38:53

Yay. Always looked for something like this.

Thanks.

stonecrest · 2006-02-14 19:41:03

One word of advice.. maybe echo to the user that they have to put themselves in the mlocate group.

lydgate · 2006-02-14 20:06:15

stonecrest wrote:

One word of advice.. maybe echo to the user that they have to put themselves in the mlocate group.

I might do this in a few hours.

Actually I just run it from root, I just know that mlocate requires you to make a group. As a regular user can you use updatedb if you added yourself to the mlocate group? I'm not at my computer so I can't do this.

Cerebral · 2006-02-14 20:10:34

Just a thought, in your .install file, you should prefix the commands with /
eg, you have:

post_install() {
  sbin/ldconfig -r . &>/dev/null
  usr/sbin/groupadd -g 21 mlocate &>/dev/null
  echo "mlocate command is technically locate, but slocate is symlinked and still works."
}

and should probably have:

post_install() {
  /sbin/ldconfig -r . &>/dev/null
  /usr/sbin/groupadd -g 21 mlocate &>/dev/null
  echo "mlocate command is technically locate, but slocate is symlinked and still works."
}

jakob · 2006-02-14 20:13:15

wow, this is amazing...updating the db with 180 Gigs over 4 HDDs has never be fun but now... that what i always desired

Thanks a lot!

lydgate · 2006-02-14 20:19:10

Cerebral wrote:

Just a thought, in your .install file, you should prefix the commands with /
eg, you have:

post_install() {
  sbin/ldconfig -r . &>/dev/null
  usr/sbin/groupadd -g 21 mlocate &>/dev/null
  echo "mlocate command is technically locate, but slocate is symlinked and still works."
}

and should probably have:

post_install() {
  /sbin/ldconfig -r . &>/dev/null
  /usr/sbin/groupadd -g 21 mlocate &>/dev/null
  echo "mlocate command is technically locate, but slocate is symlinked and still works."
}

Hmm, i'm not sure. The slocate.install does it without the slashes which is why I thought it was right. Can you explain the difference?

Cerebral · 2006-02-14 20:28:25

Well, using it without the slashes assumes pacman cd's to / before executing the post_install() or whatever, whereas prefixing them with the / guarantees you run the one in /sbin/ldconfig.

At the moment, it probably doesn't make too big a difference, seeing how (I guess) pacman runs them from /... I'm just the kind of guy who prefers absolute pathnames.

lydgate · 2006-02-14 20:32:40

Cerebral wrote:

Well, using it without the slashes assumes pacman cd's to / before executing the post_install() or whatever, whereas prefixing them with the / guarantees you run the one in /sbin/ldconfig.
At the moment, it probably doesn't make too big a difference, seeing how (I guess) pacman runs them from /... I'm just the kind of guy who prefers absolute pathnames.

Gotcha, I will change it once I get back to my computer. You could tell Judd to do this in slocate as well

Cerebral · 2006-02-14 20:37:40

lydgate wrote:

Gotcha, I will change it once I get back to my computer. You could tell Judd to do this in slocate as well

Heh, like I said, if pacman cd's to / first, then all's good, and it's just my personal preference that absolute pathnames are better. Given that Judd probably knows the internals of pacman better than anyone, I'd say he'd be pretty knowledgeable on how to make .installs.
-=edit=- by the by, good find. I've got it installed on my sys now and it looks good!

lydgate · 2006-02-15 05:12:19

For the lazy and reckless, I've also put up a binary package in my webspace:

EDIT (no longer have this webspace)

Last edited by lydgate (2007-04-19 09:15:42)

patroclo7 · 2006-02-17 17:15:55

Cool, very cool. You have my vote.

shadowhand · 2006-02-17 17:30:23

Voted.

Romashka · 2006-02-17 18:10:39

COOL! A must have package!
Works really good. Voted.

stonecrest · 2006-02-17 22:44:15

lydgate wrote:

stonecrest wrote:
One word of advice.. maybe echo to the user that they have to put themselves in the mlocate group.
I might do this in a few hours.
Actually I just run it from root, I just know that mlocate requires you to make a group. As a regular user can you use updatedb if you added yourself to the mlocate group? I'm not at my computer so I can't do this.

Hmm, it appears it doesn't help. Although I haven't been able to figure out why. It spits out an error saying "locate: can not open `/usr/var/mlocate/mlocate.db': Permission denied" and the permissions of the file are:

-rw-r-----   1 root mlocate 2634364 2006-02-14 12:35 mlocate.db

Being in the mlocate group should be enough to read the file, no? Yet when I switch to slocate, file becomes owned by root:slocate, and I am able to use locate as a user.

Confused.

lydgate · 2006-02-17 22:49:08

stonecrest wrote:

Being in the mlocate group should be enough to read the file, no? Yet when I switch to slocate, file becomes owned by root:slocate, and I am able to use locate as a user.
Confused.

I'm sort of confused too. I get that message if I try to use locate when my user is not in the mlocate group. Once the user is in the mlocate group, (s)locate command works fine. The user should not be able to updatedb though, because this is something root should do--root can see all files.

So if you're trying to updatedb from regular user, it shouldn't let you. But it sounds like you are having problems locating from a regular user, and I don't know why. What do you get for ls -ld /usr/var/mlocate?

BTW, changing groups requires a logout. Type groups at a commandline to see what groups you're in. I'm guessing you've already done this but just in case, try it.

stonecrest · 2006-02-17 22:56:50

lydgate wrote:

I'm sort of confused too. I get that message if I try to use locate when my user is not in the mlocate group. Once the user is in the mlocate group, (s)locate command works fine. The user should not be able to updatedb though, because this is something root should do--root can see all files.

Yup. I'm not trying to updatedb, just use regular locate.

lydgate wrote:

So if you're trying to updatedb from regular user, it shouldn't let you. But it sounds like you are having problems locating from a regular user, and I don't know why. What do you get for ls -ld /usr/var/mlocate?

drwxr-x---  2 root root 4096 2006-02-14 12:33 /usr/var/mlocate/

lydgate wrote:

BTW, changing groups requires a logout. Type groups at a commandline to see what groups you're in. I'm guessing you've already done this but just in case, try it.

It doesn't require a logout if you use gpasswd -a, for example, as opposed to edited the /etc/group file. Or am I wrong? I haven't logged out but 'groups' does indeed show me in the mlocate group.

mlocate audio optical power users plugdev vmware

lydgate · 2006-02-17 22:59:35

Ok, I see the problem. As root, you should chown root:mlocate /usr/var/mlocate/

I don't know why the permissions on your folder are wrong; I tried removing the package and deleting the whole directory on my computer, and when I reinstalled it the permissions were right. If this seems like it is due to my .install script, let me know, but it works on my computer so . . .

Cerebral · 2006-02-17 23:00:13

change ownership of /usr/var/mlocate to root.mlocate - that's what it's set to on my system.
-edit- beat to the punch!

stonecrest · 2006-02-17 23:16:32

Hmm, I just removed the dir, reinstalled mlocate (v 0.12-1), and it recreated the dir with root:root permissions again. I don't know why it keeps doing that on my system..

But yes, changing the permissions of the dir fixed it.

lydgate · 2006-02-17 23:19:36

stonecrest wrote:

Hmm, I just removed the dir, reinstalled mlocate (v 0.12-1), and it recreated the dir with root:root permissions again. I don't know why it keeps doing that on my system..
But yes, changing the permissions of the dir fixed it.

If you wouldn't mind, try getting the PKGBUILD out of the AUR, 0.12-2, makepkg again (from regular user with fakeroot) and install again. Or were you using the binary package from my site?

stonecrest · 2006-02-17 23:36:29

lydgate wrote:

If you wouldn't mind, try getting the PKGBUILD out of the AUR, 0.12-2, makepkg again (from regular user with fakeroot) and install again. Or were you using the binary package from my site?

Tried the latest PKGBUILD and I got the same result: root.root permissions for the directory. And no, I wasn't using the binary package.

lydgate · 2006-02-17 23:40:23

stonecrest wrote:

lydgate wrote:
If you wouldn't mind, try getting the PKGBUILD out of the AUR, 0.12-2, makepkg again (from regular user with fakeroot) and install again. Or were you using the binary package from my site?
Tried the latest PKGBUILD and I got the same result: root.root permissions for the directory. And no, I wasn't using the binary package.

Thanks a lot, if any more experienced pkgbuilders could comment on whether this is due to my pkgbuild, i would be grateful.

Arch Linux

#1 2006-02-14 17:46:02

New: mlocate, faster drop-in for slocate

#2 2006-02-14 18:02:49

Re: New: mlocate, faster drop-in for slocate

#3 2006-02-14 19:38:08

Re: New: mlocate, faster drop-in for slocate

#4 2006-02-14 19:38:53

Re: New: mlocate, faster drop-in for slocate

#5 2006-02-14 19:41:03

Re: New: mlocate, faster drop-in for slocate

#6 2006-02-14 20:06:15

Re: New: mlocate, faster drop-in for slocate

#7 2006-02-14 20:10:34

Re: New: mlocate, faster drop-in for slocate

#8 2006-02-14 20:13:15

Re: New: mlocate, faster drop-in for slocate

#9 2006-02-14 20:19:10

Re: New: mlocate, faster drop-in for slocate

#10 2006-02-14 20:28:25

Re: New: mlocate, faster drop-in for slocate

#11 2006-02-14 20:32:40

Re: New: mlocate, faster drop-in for slocate

#12 2006-02-14 20:37:40

Re: New: mlocate, faster drop-in for slocate

#13 2006-02-15 05:12:19

Re: New: mlocate, faster drop-in for slocate

#14 2006-02-17 17:15:55

Re: New: mlocate, faster drop-in for slocate

#15 2006-02-17 17:30:23

Re: New: mlocate, faster drop-in for slocate

#16 2006-02-17 18:10:39

Re: New: mlocate, faster drop-in for slocate

#17 2006-02-17 22:44:15

Re: New: mlocate, faster drop-in for slocate

#18 2006-02-17 22:49:08

Re: New: mlocate, faster drop-in for slocate

#19 2006-02-17 22:56:50

Re: New: mlocate, faster drop-in for slocate

#20 2006-02-17 22:59:35

Re: New: mlocate, faster drop-in for slocate

#21 2006-02-17 23:00:13

Re: New: mlocate, faster drop-in for slocate

#22 2006-02-17 23:16:32

Re: New: mlocate, faster drop-in for slocate

#23 2006-02-17 23:19:36

Re: New: mlocate, faster drop-in for slocate

#24 2006-02-17 23:36:29

Re: New: mlocate, faster drop-in for slocate

#25 2006-02-17 23:40:23

Re: New: mlocate, faster drop-in for slocate

Board footer