Politics and kernel schedulers

redden0t8 · 2011-04-08 14:01:30

I understand that there are a lot of politics involved, and that BFS will never be integrated into the mainline kernel, but this quote has me really curious:

http://ck.kolivas.org/patches/bfs/bfs-faq.txt wrote:

...The only way is to rewrite it to work that way, or to have more than one scheduler in the kernel. I don't want to do the former, and mainline doesn't want to do the latter...

Why is mainline opposed to having more than one scheduler? It seems very un-linuxy to force choices on people. Or is CK just exaggerating, and mainline might actually consider a second scheduler if one existed without the associated politics?

Gusar · 2011-04-08 14:29:19

Nope, he's not exaggerating, the main Linux people do not want more than one CPU scheduler in the kernel. Way back when I read about their reasons, but I don't recall anymore what they were. I think the gist was that they don't want specific purpose schedulers, their view is that there should be and can be one scheduler that fits all.
And as for why this one scheduler is CFS and not BFS... I wouldn't call it "politics", I'd call it pure and simple personality clashes.

ngoonee · 2011-04-08 14:49:16

Mainline has several reasons, some more reasonable than others. Based on his posts and general communication, I would think CK would be very difficult to work with, so that's a huge contributing factor. His patches are awesome though .

fsckd · 2011-04-08 19:07:11

ngoonee wrote:

Based on his posts and general communication, I would think CK would be very difficult to work with, so that's a huge contributing factor.

While maintaining -bfs in AUR, on several occasions I have been given assistance from con via irc. Others (graysky iirc) state he's easy to approach by email. I don't know why there is bad blood between him and the kernel devs but I would not blame his personality. He is by trade an anaesthetist which is a profession that requires excellent interpersonal skills.

redden0t8 · 2011-04-08 19:21:03

Gusar wrote:

Way back when I read about their reasons...

Do you by any chance remember where you read this? I've been looking around but can't find any technical reasoning.

eduardo.eae · 2011-04-08 20:23:36

I think it would be better to have several cpu schedulers in mainline. We already have several I/O schedulers. Having one scheduler for devices with 1 cpu at 500Mhz, and the same scheduler for devices with 16 cpus at 3Ghz (Just talking about home pcs) doesn't seem right to me.

jnguyen · 2011-04-08 20:25:51

fsckd wrote:

ngoonee wrote:
Based on his posts and general communication, I would think CK would be very difficult to work with, so that's a huge contributing factor.
While maintaining -bfs in AUR, on several occasions I have been given assistance from con via irc. Others (graysky iirc) state he's easy to approach by email. I don't know why there is bad blood between him and the kernel devs but I would not blame his personality. He is by trade an anaesthetist which is a profession that requires excellent interpersonal skills.

The importance of relationships is quite evident in the battle between LIO and SCST to become the mainline replacement for STGT (an SCSI target implementation). SCST had been pushing for mainline inclusion for 2-3 years and had the larger userbase among the two, but the LIO developers seemed to work better with the existing kernel community and in the end was chosen as the best candidate. This is what SCSI maintainer had to say:

James Bottomly wrote:

Look, let me try to make it simple: It's not about the community you bring to the table, it's about the community you have to join when you become part of the linux kernel. The interactions in the wider community are critical to the success of an open source project. You've had the opportunity to interact with a couple of them: sysfs we've covered elsewhere, but in the STGT case you basically said, here's our interface, use it. LIO actually asked what they wanted and constructed something to fit. Why are you amazed then when the STGT people seem to prefer LIO?

This is what LWN.net concluded in their article:

LWN.net wrote:

While the decision was contentious, it is yet another example of the difficulty of getting something merged without being able to cooperate with the kernel development community.

It would appear that technical merit alone is not the only criteria, and rightly so, as the community is just as important as the product.

Some might say CK's relationship with the kernel team (i.e. Ingo Molnar) is a little shaky (and I'm in no position to judge either side), but there's no doubt that his patches improve desktop responsiveness. In fact I've now put up kernel26-ck-ccs (with TOMOYO Linux) in AUR, as I haven't found anything better to stop firefox from freezing.

Last edited by jnguyen (2011-04-08 20:27:32)

sonay · 2011-04-08 21:16:14

Is BFS really better for desktops? If so, I am planning to use it on my netbook, would it worth the trouble?

Gusar · 2011-04-08 21:34:58

sonay wrote:

Is BFS really better for desktops? If so, I am planning to use it on my netbook, would it worth the trouble?

Especially on a netbook, YES! I haven't tried the recently released 0.376 yet, but reading CK's blog it's an improvement over the already very good 0.363 which I'm using now.

ngoonee · 2011-04-09 00:52:28

Gusar wrote:

sonay wrote:
Is BFS really better for desktops? If so, I am planning to use it on my netbook, would it worth the trouble?
Especially on a netbook, YES! I haven't tried the recently released 0.376 yet, but reading CK's blog it's an improvement over the already very good 0.363 which I'm using now.

Echo that, I haven't used anything else for a while.

Seems to me CK's a bit of a lone ranger, so perhaps its best for everyone that he's not directly involved in the kernel crew.

graysky · 2011-04-09 14:12:28

fsckd wrote:

While maintaining -bfs in AUR, on several occasions I have been given assistance from con via irc. Others (graysky iirc) state he's easy to approach by email. I don't know why there is bad blood between him and the kernel devs but I would not blame his personality. He is by trade an anaesthetist which is a profession that requires excellent interpersonal skills.

+1. CK is very friendly/helpful and responsive via email. I have engaged in several long conversation threads with him over the past months.

sonay wrote:

Is BFS really better for desktops? If so, I am planning to use it on my netbook, would it worth the trouble?

Another "yes" vote. There is no trouble using this kernel. You can compile it up from kernel26-ck in the AUR (see my sig) or use a pre-compiled binary package from the kernel26-ck repo (see wiki link in my sig).

BTW, bfs v0.376 is great. Con stated that he plans to finalize it as rebranded v0.4 over the weekend on his blog so stay tuned. FYI, you can get v0.376 in the kernel26-ck in the AUR via a flag in the PKGBUILD. Although bfs is designed to be about latency, not throughput, here is proof positive that, for x264 encoding anyway, bfs boasts statistically significant gains in throughput over mainline, and v0.376 is superior to v0.363:

CPU: Intel Xeon X3360 @ 8.5x400=3.40 GHz (4 cores/4 threads)
Linux version: Arch x86_64

On the other hand, when compiling the filezilla-0.3.4 package on the same quad machine, there is no difference between mainline (make -j5) and bfs376 (make -j4). If you run the mainline using the same make -j4 as you do with bfs, there IS a difference. Speaks to the inefficiencies of mainline @ high loads.

Last edited by graysky (2011-04-09 15:01:30)

redden0t8 · 2011-04-11 15:40:42

So I found the lkml thread with the original discussion. I'll link the most interesting messages:

Ingo Molnar's thoughts on early BFS, he doesn't really seem to get the point:
http://marc.info/?l=linux-kernel&m=125227082723350&w=2

Someone who actually gets the point replies:
http://marc.info/?l=linux-kernel&m=125229476821337&w=2

CK's rebuttal, he's obviously given up on ever having anything merged into the kernel again:
http://marc.info/?l=linux-kernel&m=125229579622556&w=2

The thread then goes on and on trying to come up with a benchmark. CK is long gone. It basically goes:

Someone comes up with a benchmark that shows BFS is "better"
Ingo modifies it so that it is no longer testing the right thing (ie latency on cheap hardware vs total throughput on $$ hardware)
Ingo's results show CFS is "better"
Rinse and repeat

I gave up reading after a while. I did some more lkml searches and the pattern seems to be pretty consistent. I think it can be summed up as follows:

CK has long since given up, so he's not "nice" to the kernel people anymore. At the same time, though, he has no problem admitting that BFS is not optimal for every workload.
IM is always superficially nice. He has no problem, though, exploiting the fact that interactivity (BFS's strong point) is notoriously hard to benchmark, and that total throughput (CFS's strong point) is easy to benchmark. He repeatedly takes advantage of this to "prove" CFS is better than BFS.
Annecdotal evidence from end users seems to ubiquitously show BFS provides a better experience. It would be really interesting to see a blind study on this.

Unless there is actually such a study, let's assume for a second that BFS does provide a better user experience through improved latency. That means that a very common usage case of desktop/laptop/mobile device is suffering. If there is a valid reason for the "one scheduler" thing, then the choice of CFS makes sense as it performs better in the interactivity case than BFS does in the N_CPUS >> 16 case. But at least from a naive perspective, it seems silly to think that the same scheduler would be optimal for both a single core ARM phone and a 4096 CPU supercomputer. It seems like the end user would be much better served by getting a choice in scheduler.

So unless someone can point me to a technical reason why there should only be one scheduler, I'm forced to conclude it is just politics (or "personality clashes", or whatever your choice of wording is ).

ngoonee · 2011-04-11 16:26:58

redden0t8 wrote:

So unless someone can point me to a technical reason why there should only be one scheduler, I'm forced to conclude it is just politics (or "personality clashes", or whatever your choice of wording is ).

Isn't that what almost everyone has been saying?

eldragon · 2011-04-11 19:22:53

redden0t8 wrote:

[*]Annecdotal evidence from end users seems to ubiquitously show BFS provides a better experience. It would be really interesting to see a blind study on this.[/*]

hmmm, placebo effect?

ive used bfs for a while, and when testing rc versions (where bfs is usually unavailable), i came to the conclusion that for every day use, there isnt even the slightest difference.
but that was my perception on a crappy core duo.

archman-cro · 2011-04-11 19:43:25

eldragon wrote:

ive used bfs for a while, and when testing rc versions (where bfs is usually unavailable), i came to the conclusion that for every day use, there isnt even the slightest difference.
but that was my perception on a crappy core duo.

Me too, I don't see an evident improvement on my Celeron M 1.7. I've been using bfs for quite some time, but switched to cfs, since I didn't want to bother patching the kernel with bfs, anymore, and guess what, I didn't experience any performance decrease.

pogeymanz · 2011-04-11 20:52:35

You really didn't notice any difference at all when you are compiling something with makepkg and still trying to get other work done?

The difference is highly noticeable here. It is beyond wishful thinking.

litemotiv · 2011-04-11 21:01:07

pogeymanz wrote:

You really didn't notice any difference at all when you are compiling something with makepkg and still trying to get other work done?
The difference is highly noticeable here. It is beyond wishful thinking.

Did you compare it with the new task-grouper in kernel .38?

brebs · 2011-04-11 21:30:10

Here's a similar thread.

I'm a little disappointed that no-one here has mentioned ionice and schedtool, for *proper* testing involving BFS. Examples:

alias mplayer="ionice -c2 nice -n -4 /usr/bin/mplayer"
alias make='ionice -c3 schedtool -D -e /usr/bin/make'

Army · 2011-04-11 21:32:08

Same here, I have a Pentium-M 1,7GHz Laptop with 512MB RAM and what I experienced with BFS is, that it performs even worse than CFS. I didn't measure any compilation durations etc., but when mpd begins to stutter when I compile something while I rip a movie with BFS, what doesn't happen with CFS, then I'm convinced that CFS is the better choice for me. Simple as that.
But since I'm in the minority here and there are many people who love BFS, it's good that it's there.

brebs · 2011-04-11 21:53:02

Army wrote:

mpd begins to stutter when I compile something while I rip a movie

Run mpd with niceness of e.g. -2, and the compiler and ripper with a niceness of e.g. 10. And don't forget ionice, as I demonstrated above.

Not sure why BFS would appear worse, but it could be just luck - try e.g.:

echo 6 > /proc/sys/kernel/rr_interval

The old default was 9, but ck's blog recently mentioned that the default will change to 6. I had previously found (several months ago) that 6 seemed a bit better.

To be able to run with e.g. niceness of -2 as a user, add to /etc/security/limits.d/idontcarewhatthefilenameis.conf the following:

yourusername           -   nice            -2

eduardo.eae · 2011-04-11 22:29:39

Setting nice also sets ionice. 'man ionice'.
So 'schedtool -D -n 5 makepkg' or whatever, can do all the work

Gusar · 2011-04-12 09:04:12

You shouldn't need to mess with nice and ionice. That's kinda the point of BFS.

brebs · 2011-04-12 09:51:29

Gusar wrote:

You shouldn't need to mess with nice and ionice. That's kinda the point of BFS.

Nonsense. See the guy himself.

Schedulers do not psychically know the relative importance of each process, in order to improve the user experience.

It would be nice if e.g. mplayer were automatically made more important, which is why I stated in the Gentoo thread that ulatencyd looks interesting.

For me, a bit of ionice/nice/schedtool in scripts is sufficient, with BFS, so I haven't been tempted enough to play with ulatencyd *yet*.

redden0t8 · 2011-04-12 12:46:21

ngoonee wrote:

redden0t8 wrote:
So unless someone can point me to a technical reason why there should only be one scheduler, I'm forced to conclude it is just politics (or "personality clashes", or whatever your choice of wording is ).
Isn't that what almost everyone has been saying?

Yeah, I guess I was just naively hoping that there was actually a good reason

ngoonee · 2011-04-13 01:23:59

redden0t8 wrote:

ngoonee wrote:
redden0t8 wrote:
So unless someone can point me to a technical reason why there should only be one scheduler, I'm forced to conclude it is just politics (or "personality clashes", or whatever your choice of wording is ).
Isn't that what almost everyone has been saying?
Yeah, I guess I was just naively hoping that there was actually a good reason

Politics IS a good reason. Inter-personal relationships are just as important as technical competency in almost every team effort, after all.

Arch Linux

#1 2011-04-08 14:01:30

Politics and kernel schedulers

#2 2011-04-08 14:29:19

Re: Politics and kernel schedulers

#3 2011-04-08 14:49:16

Re: Politics and kernel schedulers

#4 2011-04-08 19:07:11

Re: Politics and kernel schedulers

#5 2011-04-08 19:21:03

Re: Politics and kernel schedulers

#6 2011-04-08 20:23:36

Re: Politics and kernel schedulers

#7 2011-04-08 20:25:51

Re: Politics and kernel schedulers

#8 2011-04-08 21:16:14

Re: Politics and kernel schedulers

#9 2011-04-08 21:34:58

Re: Politics and kernel schedulers

#10 2011-04-09 00:52:28

Re: Politics and kernel schedulers

#11 2011-04-09 14:12:28

Re: Politics and kernel schedulers

#12 2011-04-11 15:40:42

Re: Politics and kernel schedulers

#13 2011-04-11 16:26:58

Re: Politics and kernel schedulers

#14 2011-04-11 19:22:53

Re: Politics and kernel schedulers

#15 2011-04-11 19:43:25

Re: Politics and kernel schedulers

#16 2011-04-11 20:52:35

Re: Politics and kernel schedulers

#17 2011-04-11 21:01:07

Re: Politics and kernel schedulers

#18 2011-04-11 21:30:10

Re: Politics and kernel schedulers

#19 2011-04-11 21:32:08

Re: Politics and kernel schedulers

#20 2011-04-11 21:53:02

Re: Politics and kernel schedulers

#21 2011-04-11 22:29:39

Re: Politics and kernel schedulers

#22 2011-04-12 09:04:12

Re: Politics and kernel schedulers

#23 2011-04-12 09:51:29

Re: Politics and kernel schedulers

#24 2011-04-12 12:46:21

Re: Politics and kernel schedulers

#25 2011-04-13 01:23:59

Re: Politics and kernel schedulers

Board footer