KDE 4.13 Baloo sucking system resources.

Primoz · 2014-04-23 12:00:16

@V1del: Thanks. I guess I had to first set it to be mounted at log-in. The I logged out and back in again, thinking it will be automatically indexed, which it wasn't. Apparently it was automatically blacklisted, so I had to remove it from there.

Now baloo is indexing and yeah it uses up to 70% of CPU in short bursts, but it doesn't slow down or hog the resources to an extent, that I wouldn't be able to work normally.

Wilco · 2014-04-23 13:26:49

V1del wrote:

Yeah I find all the fuzz incredibly weird as well, seems like all the people complaining didn't use nepomuk anyway, and now have a huge problem with disabling the new engine for some, despite the myriad of possibilities mentioned pratically everywhere (except for a dedicated button), and decide the best course of action would be ranting all over the place instead of using one of the ways to disable it and go on with their lives, but whatever.
To enable indexing of external drives, you have to remove them from the blacklist in the Desktop Search KCM

It all depends on the amount of files in your home directory. At my work PC I have a lot of files and even on an SSD it keeps on using 100% of one CPU core for hours.

It makes no sense for KDE developers to enable this by default (considering the amount of bugs in the old Nepomuk) and to not include a enable/disable button

zosodk69 · 2014-04-23 13:30:15

V1del wrote:

...and decide the best course of action would be ranting all over the place instead of using one of the ways to disable it and go on with their lives, but whatever.

I think this is an unfair (and somewhat rude) characterization of a problem that you admit you've not experienced. I'm one of the people who's computer was rendered unusable after the 4.13 upgrade.

The CPU consumption wasn't necessarily the issue, rather the excessive thrashing of the hard drive. I initially let the software run for a day while using a separate machine, hoping the problem was simply an initial index that needed to be generated. After 24 hours I decided to exclude my home directory per the official instructions. This had no measurable effect as the baloo processes continued to monopolize my hard drive. Doing anything on the machine was utterly painful; saving a tiny text file in VIM took more then 15 seconds.

Ultimately the only relief for me was to use one of the back channel methods listed in this thread because there is no official way to disable this service, as was stated by one of the baloo authors:

There is no explicit “Enable/Disable” button any more. We would like to promote the use of searching and feel that Baloo should never get in the users way.

Slightly off topic, I'm terribly discouraged by the number of trolls in the Archlinux community as of late. I've used Arch exclusively since 2004. In the early days you'd never see toxic comments like "disable it and get on with your life" around here. I suppose as a community grows this sort of thing is to be expected though.

V1del · 2014-04-23 17:13:14

It may come off as rude, but I absolutely can't stand people that spread unconstructive FUD and talk shit about things people do hard work on, if you're not one of those people then I apologize. One may not use the advanced features of the semantic search, but trashing it as useless doesn't help anyone. Yeah I also find it weird that they didn't keep an enable/disable option, but there are literally 4 ways listed in this very thread on how to disable, there's no need for yet another "Hurrdurr this is a piece of crap, stupid KDE devs"-post, you've got your point across. There are obviously some cases which apparently need improvement maybe its also dependant on what kind of files you have around. As said I can't reproduce anything of the bad behaviour and I've let it index over 2TB of data.

zosodk69 wrote:

Ultimately the only relief for me was to use one of the back channel methods listed in this thread because there is no official way to disable this service, as was stated by one of the baloo authors:
There is no explicit “Enable/Disable” button any more. We would like to promote the use of searching and feel that Baloo should never get in the users way. However, we are smart about it and IF you add your HOME directory to the list of “excluded folders”, Baloo will switch itself off since it no longer has anything to index.

Uhm yeah... quoting things out of context... So there is this "official" way and they have noticed the backlash and are most likely gonna add a real disable option in the next release.

Last edited by V1del (2014-04-23 17:26:07)

zosodk69 · 2014-04-23 18:05:03

V1del wrote:

"Hurrdurr this is a piece of crap, stupid KDE devs"

Nothing in this thread even comes close to this. No one has made any personal attacks on any KDE devs.

V1del wrote:

Uhm yeah... quoting things out of context... So there is this "official" way and they have noticed the backlash and are most likely gonna add a real disable option in the next release.

Please read mine and others' posts acknowledging the official "solution" and its lack of effectiveness for us. Nothing was taken out of context. The complaint is that the service cannot be disabled (specifically, there's no official way to prevent baloo_* processes from spawning). I feel this is counter to the general spirit of KDE, which is generally criticized for being OVER configurable.

We know vocal community backlash is an effective way to incur change (google "Windows 8 start menu" or "GNOME 3 minimize button"). You yourself acknowledge that this backlash will likely be successful and that this community of outspoken individuals will eventually get there way.

I hope I'm not out of line by profiling you. I'm guessing (based on your use of words like "hurrdurr") that you're a collage student in your early 20s. Reflecting on that point in my life, I'm often embarrassed by my response to "social justice" issues. Lets reflect on your complaint: You agree that there should be an Enable/Disable option and also believe that the community backlash is the correcting factor, yet you are offended by the community backlash. Sir, I postulate that you are barking up the wrong tree and are dangerously close to looking like a fool.

mangus · 2014-04-23 18:49:25

V1del wrote:

Yeah I find all the fuzz incredibly weird as well, seems like most people complaining didn't use nepomuk anyway, and now have a huge problem with disabling the new engine for some reason, despite the myriad of possibilities mentioned practically everywhere (except for a dedicated button), and decide the best course of action would be ranting all over the place instead of using one of the ways to disable it and go on with their lives, but whatever.
To enable indexing of external drives, you have to remove them from the blacklist in the Desktop Search KCM

And tomorrow Linus Torvalds decides to remove all graphic drivers from the kernel because he thinks it's good and you are forced to buy an nvidia card.. (just saying..)
What kind of thought is that? Forcing things is always bad , especially in this opensource world that after 20 years I don't recognize anymore.
.I saw 'opensource linux enthusiast' android users sell wallpapers on playstore (!).. that's the sad state of things.
These are 'corporate style' ways of doing things are very far from the opensource spirit (Hey Gnome, I'm looking at you also..)

That said, I care for my SSD disks life and even if I can disable it , the first start will put uncalled pressure to my disks.
The developer has been bashed all the way on his blog anyway...

zosodk69 wrote:

Slightly off topic, I'm terribly discouraged by the number of trolls in the Archlinux community as of late. I've used Arch exclusively since 2004. In the early days you'd never see toxic comments like "disable it and get on with your life" around here. I suppose as a community grows this sort of thing is to be expected though.

@zosodk69 I think arch community is quite similar in terms of numbers , what happens is that time passes , societies change, internet and education changes.
Seriously, Are you that old?
like me?

Sorry for the OT. (I smell 'mangus, get a blog!')

Last edited by mangus (2014-04-23 19:26:03)

V1del · 2014-04-23 19:57:03

zosodk69 wrote:

V1del wrote:
"Hurrdurr this is a piece of crap, stupid KDE devs"
Nothing in this thread even comes close to this. No one has made any personal attacks on any KDE devs.

disable this ****

. Still, replacing the most hated KDE part, nepomuk, with another buggy replacement is uncalled for, nobody uses desktop indexing

Why KDE is forcing me using a useless, buggy, unpolished and unfinished file indexer,

(I'm aware that I may get accused of out-of-contextness for these but the meaning doesn't change)
To name a few plus most of the comments on the blog. It's the presentation that irks me, I have nothing against people having valid and personal reasons wanting to disable the semantic search. But then you can state so without bashing the project in the same turn, there are valid uses for this search (and all the IMO very useful semantic connections it brings) and most of these comments make me believe that they've never bothered with it anyway.

It also seems kinda nitpicky just to say its an inadequate resolution, just because there's a small baloo process remaining... they could've tacked that RAM into kdeinit4 or so and none would bat an eyelash.

I hope I'm not out of line by profiling you. I'm guessing (based on your use of words like "hurrdurr") that you're a collage student in your early 20s. Reflecting on that point in my life, I'm often embarrassed by my response to "social justice" issues. Lets reflect on your complaint: You agree that there should be an Enable/Disable option and also believe that the community backlash is the correcting factor, yet you are offended by the community backlash. Sir, I postulate that you are barking up the wrong tree and are dangerously close to looking like a fool.

Close, but I'm definitely too artistically inept to be a collage student

Again, I'm mostly annoyed at people unreasonably bashing on a project because it isn't for them. Maybe I should've clarified that a little better in my initial post. You don't seem to be one of those guys so rest assured I wasn't necessarily referring to you,

hamelg · 2014-04-23 20:11:20

Hello,

Here, the upgrade was very pleasant !
~/ was excluded automatically, baloo has taken my nepomuk setup and indexed my 2 subdirectories in my home directory :

~/DOC contains 2.8Gb of html and pdf files
~/WORK contains 500Mb of libreoffice files and others types of document

Baloo took 10 minutes to index these 2 folders.

kdepim is not installed on my system.

Last edited by hamelg (2014-04-23 20:13:27)

mangus · 2014-04-23 21:10:05

V1del wrote:

Again, I'm mostly annoyed at people unreasonably bashing on a project because it isn't for them. Maybe I should've clarified that a little better in my initial post. You don't seem to be one of those guys so rest assured I wasn't necessarily referring to you,

I'm glad it is for you. The only 'unreasonably' beheviour here is the developer's one, in my opinion.

Primoz · 2014-04-23 21:46:44

I don't know if the large amount of files is really to blame... I did a test and I have around 150.000 files on my home (most of them are in folders that aren't blacklisted) And I'm not sure if this includes the external disk with videos and music, which is another 64.000 files, which as I said before weren't indexed.
And the indexing of those 64.000 files took a mere minute (at most).

Again, I think the problem is, that I following some how-to guide or hack or something (can't remember) from KDE forums decided to allow Nepomuk both the memory and processor power for indexing and I did use it from almost the beginning. Yeah I did complain at first about how much it takes to index things etc. that's why I was recommended to increase the memory use for it, so the indexing would go faster. So my point being is: I guess it really boils down on whether you used file indexing before or not.

But option to allot more, or less memory to indexing should be reintroduced, because I think that helped me a lot.

Wilco · 2014-04-24 07:20:01

V1del wrote:

Again, I'm mostly annoyed at people unreasonably bashing on a project because it isn't for them. Maybe I should've clarified that a little better in my initial post.

That is a valid reason but one that doesn't apply here. It's not that baloo isn't for me, it's literally destroying my SSD and making my system crawl for hours every time I boot. Perfectly valid reasons for calling out on baloo. Especially for a piece of software that can't be disabled.

OT: again, I can confirm the hidden=true fix works just fine.

V1del · 2014-04-24 08:18:04

Your second response was a reasonable post that can help seeing were the problems are, your first was an unnecessary rant against both projects. If my post made you post a more valid, confirmable and thus fixable reason than "It sucks and it always has" then it has reached its goal. There's more to nepomuk/baloo than simply indexing your files for search, people that use those features find them really awesome, you don't, which is fine. Claiming the projects are useless and a waste of time for everyone is simply tactless.

That said I can confirm the disk wear to a certain degree, but that also stopped on my system after about an hour. But I do see why people with an SSD would want to disable it at this time.

mcloaked · 2014-04-24 09:16:57

People running a laptop on battery power might also want to maximise their battery use after they update, and not have it doing more than is necessary. Hence that may be another use case where it would be a valid reason not to have indexing taking up power, CPU and disk activity. The important point is that having the choice to enjoy a facility is great, but having the choice not to use a facility is just as important. Hence the optimal design is to allow users to make their own decisions about what is running on their devices. Mostly linux has in the past operated on a philosophy of giving people freedom, so let's hope that our freedoms are respected in the future.

V1del · 2014-04-24 09:51:30

The real taxing part of the indexing which indexes whole contents and semantic data about the files (and causes the disk wearing) is disabled while on battery

pogeymanz · 2014-04-24 13:49:19

V1del wrote:

The real taxing part of the indexing which indexes whole contents and semantic data about the files (and causes the disk wearing) is disabled while on battery

It didn't stop for me while on battery.

I think that the real performance suffering comes from people with a ton of files in their home directories. I have lots and lots of files.

ls /home/my_user -1UR | wc -l

Tells me that I have 327336 files in my home directory (including directories).

Indexing was murdering my laptop for DAYS. Even after the initial indexing, there were still baloo processes running often and making things very sluggish (I have an HDD). Then, when I disabled it by putting ~/ in the blacklist, the baloo_file_cleaner process ran for another few days. I finally just deleted .local/baloo/* and rebooted and everything is fine now.

EDIT: Included hidden files.
EDIT2: Un-included hidden files because they aren't indexed by baloo.

Last edited by pogeymanz (2014-04-24 15:12:35)

V1del · 2014-04-24 15:07:03

Then report a bug I suppose. Don't include hidden files, they aren't indexed. I also have a HDD didn't check how many files though, may do so later.

EDIT: 486823...

Last edited by V1del (2014-04-24 17:26:07)

lucke · 2014-04-24 18:20:12

A random observation: I updated an old laptop with a hdd from Kubuntu 12.04 to Kubuntu 14.04, they changed the default elevator to deadline (from cfq), the laptop was quite unusable responsiveness-wise until I changed the elevator to cfq, even though baloo_file wasn't reading much and other processes didn't want to do block io.

I didn't have problems with baloo on my Arch PCs with hdds (except for the noise when indexing for the first time). For me it is quite splendid.

Last edited by lucke (2014-04-24 18:21:51)

pogeymanz · 2014-04-24 18:23:29

V1del wrote:

Then report a bug I suppose. Don't include hidden files, they aren't indexed. I also have a HDD didn't check how many files though, may do so later.
EDIT: 486823...

So you have more files than me. And you say there wasn't much of an issue with baloo for you?

I wonder what the actual reason for some of us having such a bad experience is.

I have Akonadi disabled as well (and always have). I wonder if that was messing up baloo somehow?

bwmcn · 2014-04-26 17:30:35

I'll just put this here.

"
[13:14] <obsed> well baloo and others will get better in afew years after everyone has bug tested them, then they will be features users rave about
[13:14] <obsed> and if they let you disable them now then no one would bug test them enough
[13:14] <obsed> this is good for the progress of kde
[13:22] <obsed> mva: and baloo is better than the older nepomuk, its better at stopping you from disabling it
"

I hope this way sarcastic.

Last edited by bwmcn (2014-04-26 17:36:47)

xgdgsc · 2014-04-27 14:53:45

What I experienced is it takes more than 6G , freezing my 8G ram machine(heavily swapping) for a while the first time it indexes (and notify me the existence of it in this way). Then accumulates to about 2.6G before the process exit several times afterwards. Now it baloo_file_cleaner seem to has a small ram usage with constant 1%~2% CPU usage and a baloo_file_extractor process spawned frequently taking 1% CPU, which makes me a bit uncomfortable.

V1del · 2014-04-27 15:32:12

The file_extractor processes will stop once it has properly indexed all data, which may take a while initially

Crozepp · 2014-04-27 17:50:47

I recommend people to try out milou from AUR. You can put it on system tray and it's better than Krunner and more comfortable than dolphin/search.

EDIT: plasma-desktop crashes when I try to log in with milou widget enabled, so nevermind.

Last edited by Crozepp (2014-04-27 19:55:52)

palmaway · 2014-04-29 09:27:20

V1del wrote:

Yeah I find all the fuzz incredibly weird as well, seems like most people complaining didn't use nepomuk anyway, and now have a huge problem with disabling the new engine for some reason, despite the myriad of possibilities mentioned practically everywhere (except for a dedicated button), and decide the best course of action would be ranting all over the place instead of using one of the ways to disable it and go on with their lives, but whatever.

So you complain that users "rant all over the place" instead of disabling it and go on with their lives, and yet you decide that the best course of action would be to rant about that? How about going on with your life instead?

Last edited by palmaway (2014-04-29 09:28:07)

palmaway · 2014-04-29 09:36:22

Regarding baloo itself, the post by the developer linked above, and his replies in this bug report (which i recommend people register and comment on) show that the "official" position is to have distributions let their users "disable" baloo by letting them uninstall it. It appears that this is not at all possible in the current Archlinux, as kactivities (and therefore kdebase-runtime), kdegraphics-gwenview and kdepim-libkdepim depend on it. Any chance we'll see baloo unlinked from those packages in the future?

palmaway · 2014-04-30 17:24:21

Added bug asking to split the baloo package into two so that indexing functionality can be removed without breaking KDE (as suggested by the baloo dev):
https://bugs.archlinux.org/task/40161

Please vote!

Arch Linux

#26 2014-04-23 12:00:16

Re: KDE 4.13 Baloo sucking system resources.

#27 2014-04-23 13:26:49

Re: KDE 4.13 Baloo sucking system resources.

#28 2014-04-23 13:30:15

Re: KDE 4.13 Baloo sucking system resources.

#29 2014-04-23 17:13:14

Re: KDE 4.13 Baloo sucking system resources.

#30 2014-04-23 18:05:03

Re: KDE 4.13 Baloo sucking system resources.

#31 2014-04-23 18:49:25

Re: KDE 4.13 Baloo sucking system resources.

#32 2014-04-23 19:57:03

Re: KDE 4.13 Baloo sucking system resources.

#33 2014-04-23 20:11:20

Re: KDE 4.13 Baloo sucking system resources.

#34 2014-04-23 21:10:05

Re: KDE 4.13 Baloo sucking system resources.

#35 2014-04-23 21:46:44

Re: KDE 4.13 Baloo sucking system resources.

#36 2014-04-24 07:20:01

Re: KDE 4.13 Baloo sucking system resources.

#37 2014-04-24 08:18:04

Re: KDE 4.13 Baloo sucking system resources.

#38 2014-04-24 09:16:57

Re: KDE 4.13 Baloo sucking system resources.

#39 2014-04-24 09:51:30

Re: KDE 4.13 Baloo sucking system resources.

#40 2014-04-24 13:49:19

Re: KDE 4.13 Baloo sucking system resources.

#41 2014-04-24 15:07:03

Re: KDE 4.13 Baloo sucking system resources.

#42 2014-04-24 18:20:12

Re: KDE 4.13 Baloo sucking system resources.

#43 2014-04-24 18:23:29

Re: KDE 4.13 Baloo sucking system resources.

#44 2014-04-26 17:30:35

Re: KDE 4.13 Baloo sucking system resources.

#45 2014-04-27 14:53:45

Re: KDE 4.13 Baloo sucking system resources.

#46 2014-04-27 15:32:12

Re: KDE 4.13 Baloo sucking system resources.

#47 2014-04-27 17:50:47

Re: KDE 4.13 Baloo sucking system resources.

#48 2014-04-29 09:27:20

Re: KDE 4.13 Baloo sucking system resources.

#49 2014-04-29 09:36:22

Re: KDE 4.13 Baloo sucking system resources.

#50 2014-04-30 17:24:21

Re: KDE 4.13 Baloo sucking system resources.

Board footer