omg! Con Kolivas is back!

shining · 2009-09-12 18:02:11

cerbie wrote:

That's a fix to one small bug. They can want one scheduler to rule them all, but it keeps not working right. The solution, which could retire BFS, would be to allow selecting of scheduler as a config option, either fully hot, at boot-time, or by config, in the mainline kernel. I really don't get why there has constantly been such resistance to the idea.

iirc, one of the argument was that if there is more than one schedulers, then they would all get less tested. And they would require people to maintain each of them.
But I also think it would be cool to have like two schedulers available at config time, maybe one more server oriented and the second more desktop oriented.

Anyway it is easy to say that the mainline scheduler does not work right, but did you ever do anything about it ? I am sure no one here did.
Ingo explicitly said he also cares about interactivity, and would like to receive feedback from users who have problems.

But I have been curious since this thread started : which problems do people have with mainline scheduler that is addressed by bfs ?
And I don't want to hear things like "it feels faster/snappier/greater/whatever" because this is all just placebo.
Something more noticeable like : now I can compile stuff in the background and my audio does not skip anymore
(but I do this all the time using CFS without any problems)

cerbie · 2009-09-12 19:31:16

I'd even take one scheduler with some tuning to sacrifice throughput, if possible.

I went to realtime kernels (Ingo Molnar again). Sometimes buggy, sometimes OK (I think it has to do with the near-constant influx of patches). Not an ideal solution, as I don't need any fancy real-time stuff.

Problem: if a browser went to swap for anything (w/o running KDE, nothing else will use up all my RAM, and it occured with all browsers I tried), the display would hang for 10-300 seconds (I timed one to ~5m40s), and audio would skip when it all came back. No clue exactly what happened (I could only keep confirming what wasn't happening), except that the display freezes seemed to last until 10-20 seconds after the last HDD access, did not occur with rt kernels, and do not occur with .30 bfs (I haven't tried any .31, yet).

Problem: several specific GUI applications start(ed) slowly, such as xarchiver, thunar, and bashrun. No idea why it's not an I/O change that fixes it. Realtime kernels, going back kernel versions (I tried 2.6.26.? before rt), and now bfs fixed it. Magic to me.

Problem: Xarchiver would slow the PC down a great deal while extracting. I didn't think of it as a fixable problem (without replacing the 8-year-old 2.5" HDD, anyway) until my first rt kernel use made it all but vanish during normal use.

On my C2D desktop, though, no problems at all. I think it's a little snappier with various recommended desktop settings used (1000Hz, no dynticks, preemptible RCU, etc.), but it's little enough that I can say, "it's easier to make a custom kernel than it is to rule out placebo," and it is still vanilla (+nVidia).

Last edited by cerbie (2009-09-13 00:49:59)

nous · 2009-09-13 00:08:13

I'd been plagued for a year and a half, since I bought a Dell Vostro 1500 (Core 2 Duo / 2GHz, NVidia 8600M GT/256M, 4GB RAM), with poor performance. I mean, it's inexcusable for a system like this, operating on a hard disk that can deliver 58MB/s as indicated by hdparm, to skip audio in amarok and frames in mplayer whenever some disk-intensive task was performing or firefox was peaking at 99% CPU, not to talk about delays in xfce4 compositing or compiz even when idling.

Yesterday I patched a vanilla 2.6.31 tarball with bfs211 and rebooted. I fired up compiz and IMMEDIATELY saw the difference. Everything was fluid, fast and responsive like thought. My favourite test was alt/ctrl-tabbing in compiz (normal or ring switch), where it took sometimes half a second to respond. Not anymore! I began a -j2 compile of the kernel tree to keep both cores at 100% and a "find /", fired-up a HD movie in mplayer, amarok, firefox, kde4 hogs... and pressed alt-tab.

The windows must have switched before the key press! I then started wobblying windows around, rotating the cube, activated the "jackdaw" and "jess" visualizations of amarok resized to 1/4 of the screen each, and the computer, besides losing some fluidity in compiz operations due to heavy load on both the CPU and the GPU, remained responsive and obedient to every command and test I could devise. This is the important point: not a single frame from mplayer was lost, not a single skip from amarok was heard, every key press was immediately executed.

You might think I'm overly enthusiastic or biased, but no.

I just was sick and tired of having a delayed response desktop on a really fast machine.

Last edited by nous (2009-09-13 00:09:22)

Bogart · 2009-09-13 10:19:35

nous wrote:

I'd been plagued for a year and a half, since I bought a Dell Vostro 1500 (Core 2 Duo / 2GHz, NVidia 8600M GT/256M, 4GB RAM), with poor performance. I mean, it's inexcusable for a system like this, operating on a hard disk that can deliver 58MB/s as indicated by hdparm, to skip audio in amarok and frames in mplayer whenever some disk-intensive task was performing or firefox was peaking at 99% CPU, not to talk about delays in xfce4 compositing or compiz even when idling.
Yesterday I patched a vanilla 2.6.31 tarball with bfs211 and rebooted. I fired up compiz and IMMEDIATELY saw the difference. Everything was fluid, fast and responsive like thought. My favourite test was alt/ctrl-tabbing in compiz (normal or ring switch), where it took sometimes half a second to respond. Not anymore! I began a -j2 compile of the kernel tree to keep both cores at 100% and a "find /", fired-up a HD movie in mplayer, amarok, firefox, kde4 hogs... and pressed alt-tab.
The windows must have switched before the key press! I then started wobblying windows around, rotating the cube, activated the "jackdaw" and "jess" visualizations of amarok resized to 1/4 of the screen each, and the computer, besides losing some fluidity in compiz operations due to heavy load on both the CPU and the GPU, remained responsive and obedient to every command and test I could devise. This is the important point: not a single frame from mplayer was lost, not a single skip from amarok was heard, every key press was immediately executed.
You might think I'm overly enthusiastic or biased, but no.
I just was sick and tired of having a delayed response desktop on a really fast machine.

The thing is that it is people like you who can really help to fix those problems in the mainline scheduler. You have visible and reproducible problems with it. I'm sure that if you report them, the maintainers would be very glad to investigate which is the problem and get it fixed. But they do need your feedback. They can't solve problems magically.

Since you have an Intel CPU, have you tried the NO_NEW_FAIR_SLEEPERS thing that's mentioned in this thread? It seemed to solve these kind of problems for some people with Intel CPUs.

pawels64 · 2009-09-13 11:14:04

Bogart wrote:

nous wrote:
I'd been plagued for a year and a half, since I bought a Dell Vostro 1500 (Core 2 Duo / 2GHz, NVidia 8600M GT/256M, 4GB RAM), with poor performance. I mean, it's inexcusable for a system like this, operating on a hard disk that can deliver 58MB/s as indicated by hdparm, to skip audio in amarok and frames in mplayer whenever some disk-intensive task was performing or firefox was peaking at 99% CPU, not to talk about delays in xfce4 compositing or compiz even when idling.
Yesterday I patched a vanilla 2.6.31 tarball with bfs211 and rebooted. I fired up compiz and IMMEDIATELY saw the difference. Everything was fluid, fast and responsive like thought. My favourite test was alt/ctrl-tabbing in compiz (normal or ring switch), where it took sometimes half a second to respond. Not anymore! I began a -j2 compile of the kernel tree to keep both cores at 100% and a "find /", fired-up a HD movie in mplayer, amarok, firefox, kde4 hogs... and pressed alt-tab.
The windows must have switched before the key press! I then started wobblying windows around, rotating the cube, activated the "jackdaw" and "jess" visualizations of amarok resized to 1/4 of the screen each, and the computer, besides losing some fluidity in compiz operations due to heavy load on both the CPU and the GPU, remained responsive and obedient to every command and test I could devise. This is the important point: not a single frame from mplayer was lost, not a single skip from amarok was heard, every key press was immediately executed.
You might think I'm overly enthusiastic or biased, but no.
I just was sick and tired of having a delayed response desktop on a really fast machine.
The thing is that it is people like you who can really help to fix those problems in the mainline scheduler. You have visible and reproducible problems with it. I'm sure that if you report them, the maintainers would be very glad to investigate which is the problem and get it fixed. But they do need your feedback. They can't solve problems magically.
Since you have an Intel CPU, have you tried the NO_NEW_FAIR_SLEEPERS thing that's mentioned in this thread? It seemed to solve these kind of problems for some people with Intel CPUs.

Exactly. It seems CFS get some fine tuning and should be even better on desktops. It's not fair to compare BFS which isn't stable, is lack of many features and it's quite easy to some processes get caught in infinite loops, it's more probable it "introduces" some race conditions etc. However, it's good to have opportunity to test those both schedulers. Linux and end users benefit from this The best option is to have any problem resolved in the mainline and if someone suffers from some problems report this. Don't forget to test NO_NEW_FAIR_SLEEPERS before.

Last edited by pawels64 (2009-09-13 14:16:34)

attila · 2009-09-13 11:29:11

Bogart wrote:

The thing is that it is people like you who can really help to fix those problems in the mainline scheduler. You have visible and reproducible problems with it. I'm sure that if you report them, the maintainers would be very glad to investigate which is the problem and get it fixed. But they do need your feedback. They can't solve problems magically.

This sounds a little bit unfair. Con brings something on the route for people which use linux on the desktop but the same people should help the mainstream scheduler. In the best case we will get the choice to use what we want and both scheduler coexists in the kernel.:)

SpeedVin · 2009-09-13 18:46:38

Now I'm using kernel26-bfs and I see the diffrence on my C2Q (better performace).

pawels64 · 2009-09-13 18:50:39

SpeedVin wrote:

Now I'm using kernel26-bfs and I see the diffrence on my C2Q (better performace).

Have you tried NO_NEW_FAIR_SLEEPERS? Btw. try running some game like Urban Terror with Amarok or something else in the background. Twice less fps here with bfs and no fps drop with cfs.

Last edited by pawels64 (2009-09-13 18:51:25)

SpeedVin · 2009-09-13 19:27:17

pawels64 wrote:

SpeedVin wrote:
Now I'm using kernel26-bfs and I see the diffrence on my C2Q (better performace).
Have you tried NO_NEW_FAIR_SLEEPERS? Btw. try running some game like Urban Terror with Amarok or something else in the background. Twice less fps here with bfs and no fps drop with cfs.

No I don't enable this otpion.
I tested it like that (compilation/uzbl/mpd/urxvt) and all was quite faster.
I use 220 path

nous · 2009-09-13 19:44:11

Bogart wrote:

nous wrote:
I'd been plagued for a year and a half, since I bought a Dell Vostro 1500 (Core 2 Duo / 2GHz, NVidia 8600M GT/256M, 4GB RAM), with poor performance. I mean, it's inexcusable for a system like this, operating on a hard disk that can deliver 58MB/s as indicated by hdparm, to skip audio in amarok and frames in mplayer whenever some disk-intensive task was performing or firefox was peaking at 99% CPU, not to talk about delays in xfce4 compositing or compiz even when idling.
>snip<
You might think I'm overly enthusiastic or biased, but no.
I just was sick and tired of having a delayed response desktop on a really fast machine.
The thing is that it is people like you who can really help to fix those problems in the mainline scheduler. You have visible and reproducible problems with it. I'm sure that if you report them, the maintainers would be very glad to investigate which is the problem and get it fixed. But they do need your feedback. They can't solve problems magically.
Since you have an Intel CPU, have you tried the NO_NEW_FAIR_SLEEPERS thing that's mentioned in this thread? It seemed to solve these kind of problems for some people with Intel CPUs.

You're absolutely right. I haven't and I haven't.

I hereby solemnly apologize for my utter stupidity and my temerity to expect the obvious: decent performance on a fast machine, after 18 years of kernel development. I shall whip my ignorant butt with a silver-plated flagellum 'til the words sched-latency-ns, sched-min-granularity-ns, sched-wakeup-granularity-ns and NO_NEW_FAIR_SLEEPERS are deeply and indelibly incised on both cheeks, lest I ever forget them again and question the sheer perfection of a scheduler that can scale to bazillions of cpus but skips frames on a 2GHz dual core when I move a window or crond wakes up.

It's all my fault. I should've reported this to the maintainers who tested bfs on a dual quad-core hyperthreaded box and reached a unanimous verdict about its performance.

Have you ever googled through the lkml archives for "sound stutters" or "frame skips"? And I don't mean in the olden days of i486. What's more interesting is that suddenly the scheduler maintainers seem to have started paying attention to reports of scheduler misbehaviour, as if the bfs is going to steal customers from them. And, lo and behold, they propose solutions despite the fact that their benchmarks show a crushing superiority of cfs over bfs and even though they "can't reproduce the latency" on their fancy and shiny 128-core/64GB boxes.

Pardon me, but I was under the impression that the friggin' defaults of a OS component should favour the majority of its users who happen to own less than 2 cores and those laymen who might just know only how to boot into their desktops wanting to enjoy some multimedia, without echoing nanoseconds to /proc/sys/kernel/* first.

I especially liked the "people like you" salutation, it made me feel like I've purloined some helpless granny's purse.

Peace and Love and B'f'S.

pawels64 · 2009-09-13 20:44:23

@SpeedVin

Can you try running some 3D game with Amarok or something like this in the background? My fps are much lower with bfs when playing game and listening to the music same time.

@nous

Haha stop making yourself fool. Actually Ingo took care even if someone posted report without any numbers. Post a bug report, but before try NO_NEW_FAIR_SLEEPERS and stop saying bullshit. Such geeks make me laugh...

Last edited by pawels64 (2009-09-13 20:46:12)

nous · 2009-09-13 21:37:17

pawels64 wrote:

@nous
Haha stop making yourself fool. Actually Ingo took care even if someone posted report without any numbers. Post a bug report, but before try NO_NEW_FAIR_SLEEPERS and stop saying bullshit. Such geeks make me laugh...

Why should I or anyone else take any of these (including ourselves) seriously?

I actually did try NO_NEW_FAIR_SLEEPERS and the /proc/sys/kernel stuff which indeed improved responsiveness but not to the level of BFS. What I want to say is, we're missing the point here. It shouldn't take a rogue anaesthesiologist's patches to mobilize the system.

A desktop/multimedia-oriented scheduler should have been a kernel option years ago.

skottish · 2009-09-13 21:55:17

How close this thread is to closing. I can barely believe how some of the posts are getting to be so painfully, ridiculously useless.

pyther · 2009-09-13 22:10:15

Use what works for you.

I was successfully able to compile a bfs kernel and it seems that the system is a little bit more responsive under heavy loads. However in my normal activities I can't notice a difference.

sand_man · 2009-09-13 23:10:51

One thing I have noticed with bfs kernel is that when I try to compile a kernel, I get the "stuck" process problem when either the kernel is compiling or sometimes even when the kernel is extracting.
Now I would call that "heavy load" and it obviously can't handle it very well...

Army · 2009-09-14 08:27:11

I generally really like the idea behind bfs and I absolutely want to use it.

At 2min past midnight, cron does its every day task. Me watching a video in mplayer right at this moment shows a big difference between bfs and cfs. With bfs, mplayer hangs, shows about 1 frame per 3-4 seconds, with cfs everything is fine.

So I can't use bfs right now, but I will in the future, as soon as it's improved.

Oh and by the way, the problem with shell-fm's high cpu load is not because of bfs (I mentioned this earlier in this thread), it's the same with cfs. So it's only because of the higher frequency.

Last edited by Army (2009-09-14 08:28:44)

ugkbunb · 2009-09-14 13:52:43

pawels64 wrote:

@SpeedVin
Can you try running some 3D game with Amarok or something like this in the background? My fps are much lower with bfs when playing game and listening to the music same time.

Ive been playing HON with Amarok playing with no noticable slowdown... although I am not sure how to output FPS in HON... if you know how I will gladly run the test for you.

SpeedVin · 2009-09-14 17:07:05

Hello again .
I just noticed when I compiling some programs I got 100% use of CPU.
And here are my question if CPU will work radomly (once,twiece a week for some time like 5 or 7 min.) I am shortering life of my CPU?
But all thing compilates faster

Last edited by SpeedVin (2009-09-14 17:07:36)

agapito · 2009-09-14 21:35:11

bfs_scheduler_benchmarks

http://www.phoronix.com/scan.php?page=a … arks&num=1

mutlu_inek · 2009-09-14 21:48:06

agapito wrote:

bfs_scheduler_benchmarks
http://www.phoronix.com/scan.php?page=a … arks&num=1

My reply to the discussion about this benchmark, sorry for simply re-posting it here:

Many have hinted at the problem with this benchmark and with Ingo Molnar's treatment of Con's contribution: Con is not interested in pushing disk throughput or cpu performance. The point of BFS is to have a system that does not skip, lock or drop frames during high cpu load. Despite great advances in scheduling and despite faster and faster cpus, even multi-core systems, we still experience latency when several things are happening at the same time. These could be cron jobs like man-db or updatedb blocking the disk, compilations running in the background or many other uses of the cpu that should be rather fast, but _never_ block the user interface. So far optimizations have intended to make the system fast is absolute terms and to be fair in the sense that all processes get equal access to the cpu, but what we need is that certain processes need to take a (short) break if we do something with our mouse, the keyboard, or while watching videos and listening to music.
Benchmarks measuring performance are detrimental to Con's effort as they reinforce the way things have been done before: get the last bit out of the cpu, at the expense of the user experience. Being fast in absolute terms is making my experience of the system worse as I am annoyed by unresponsive UIs or dropped frames while I am unable to measure whether a certain process took a few more seconds to complete. I am really surprised that the BFS did so well in this test. Nonetheless, this does not speak for or against the BFS as its goal is completely unrelated.
[...]
I do not expect Phoronix or anyone else here to develop a new test just for BFS. I don't even know whether BFS is actually better than CFS. But if I was to benchmark BFS, I would put a huge disclaimer at the beginning and the end of the test in order to put in perspective that the benchmarks don't actually address the issue at all.
Con Kolivas is, of course, pointing out that we s/could use Interbench. He wrote the test himself (before creating BFS) and also uses it to test his work. The benchmark is available for most distributions, AFAIK.

Maybe we should have a go at interbench, it is in the AUR. I will see whether I will have some time at my hands later tonight.

sand_man · 2009-09-14 23:30:15

Oh great, another quality phoronix benchmark

How about some real world tests? Like doing 10 things at once? Not how long apache takes to compile...

Last edited by sand_man (2009-09-14 23:32:01)

flamelab · 2009-09-15 07:42:09

The whole thing about BFS is desktop responsiveness.

That cannot be measured. It something that you can only see with your eye, not with a sterile, meaningless benchmark.

SpeedVin · 2009-09-15 14:43:34

I know one thing from this test BFS aimed to be low latency and to take all of CPU (100%) that's why on heavy load thing/apps are faster and I like BFS too becuse it is more responsive on Multi-core proccesor

Last edited by SpeedVin (2009-09-19 12:46:25)

ngsupb · 2009-09-18 19:52:10

I got bfs kernel install. Any problem here.

Arch worked very fast for me after Ubuntu, dunno if something changed after I compiled kernel with bfs, I don't mind if it works better now, even it isn't noticeable .

Ah well, it is noticeable thought.
on heavy load I get the same fps on glxgears as without any load, it dropped significantly before under load.

cerbie · 2009-09-19 10:15:37

sand_man wrote:

Oh great, another quality phoronix benchmark
How about some real world tests? Like doing 10 things at once? Not how long apache takes to compile...

We need some *n*x test suite(s) akin to Sysmark, or SPEC*, that include batteries of real-world-like scripted stuff, and have some of the score involve wait time servicing UI requests initiated by the [virtual] user, while running many tasks.

Isolated benchmarks may be able to help, but they will show what you want them to, more often than not.

Last edited by cerbie (2009-09-19 10:20:13)

Arch Linux

#126 2009-09-12 18:02:11

Re: omg! Con Kolivas is back!

#127 2009-09-12 19:31:16

Re: omg! Con Kolivas is back!

#128 2009-09-13 00:08:13

Re: omg! Con Kolivas is back!

#129 2009-09-13 10:19:35

Re: omg! Con Kolivas is back!

#130 2009-09-13 11:14:04

Re: omg! Con Kolivas is back!

#131 2009-09-13 11:29:11

Re: omg! Con Kolivas is back!

#132 2009-09-13 18:46:38

Re: omg! Con Kolivas is back!

#133 2009-09-13 18:50:39

Re: omg! Con Kolivas is back!

#134 2009-09-13 19:27:17

Re: omg! Con Kolivas is back!

#135 2009-09-13 19:44:11

Re: omg! Con Kolivas is back!

#136 2009-09-13 20:44:23

Re: omg! Con Kolivas is back!

#137 2009-09-13 21:37:17

Re: omg! Con Kolivas is back!

#138 2009-09-13 21:55:17

Re: omg! Con Kolivas is back!

#139 2009-09-13 22:10:15

Re: omg! Con Kolivas is back!

#140 2009-09-13 23:10:51

Re: omg! Con Kolivas is back!

#141 2009-09-14 08:27:11

Re: omg! Con Kolivas is back!

#142 2009-09-14 13:52:43

Re: omg! Con Kolivas is back!

#143 2009-09-14 17:07:05

Re: omg! Con Kolivas is back!

#144 2009-09-14 21:35:11

Re: omg! Con Kolivas is back!

#145 2009-09-14 21:48:06

Re: omg! Con Kolivas is back!

#146 2009-09-14 23:30:15

Re: omg! Con Kolivas is back!

#147 2009-09-15 07:42:09

Re: omg! Con Kolivas is back!

#148 2009-09-15 14:43:34

Re: omg! Con Kolivas is back!

#149 2009-09-18 19:52:10

Re: omg! Con Kolivas is back!

#150 2009-09-19 10:15:37

Re: omg! Con Kolivas is back!

Board footer