The kernel patch that does wonders

Fruity · 2010-11-20 04:33:13

Thanks for the input karol, will bear in mind to always fully read man pages before I post questions on any topic in future.

Seriously, why post just to say 'go learn', Allan and bangkok both gave me useful answers...

Damn. Going to wonder off this thread now, eesh.

Allan · 2010-11-20 05:58:18

You were just lucky I was in a helpful mood... most of the time I would have pointed you to the man page too.

handy · 2010-11-20 10:22:32

Basn wrote:

handy wrote:
I did the following to run the this cgroup stuff, courtesy of Vivek Goyal's post here:
http://lkml.org/lkml/2010/11/16/413
It is working, but I haven't been using it long enough to really notice any difference at this stage...
Uhm, i think i got it working looking at that, but how does it work do you have to have it for each terminal etc? i dont really get it

Most of us Arch users who are (generally) not really masters of the Linux system (love it as we do) are finding this stuff a LOT tricky. I know that we are not alone in the distro world & that this problem is caused by a whole LOT of disinformation amongst the entire Linux kernel based distro community. Spin & disinformation in the world of the distros is rife at the mo'.

We must research our way out of this. Don't we?

My understanding is that basically, the bottom line is that people who do extremely intensive CPU work are going to be grateful for this patch. The rest of us won't even notice that it exists, be it in our kernel (as it will be) or in our ~/.bashrc & such...

Anyway, that's how I see it.

Zom · 2010-11-20 11:23:16

Most of us probably won't notice it, no. But if you run stuff like MPD in its own TTY, while compiling stuff in another TTY, your audio probably won't stutter. I suspect the same could be said if you run pulseaudio, a web browser, a music player and a movie player in different terminals, then you probably won't have flash or the movie player hogging the system, so you could still listen to music while watching a movie while playing farmville or something like that.

Linus also theorised about adding another flag to the .desktop files, something like "launch in individual cgroup", which would let you launch programs in a different group and thus not be able to hog the system. But as of now, most desktop systems (including the window managers) don't launch their programs in a separate tty or pts, so it won't matter AFAIK.

E: That said, I'm still glad to see this kind of development making it into the mainline kernel. It's nice to minimise the need for patches for desktop systems, IMO.

E2: VVVV

handy wrote:

@Zom: Just do the easy .bashrc thing & see what it does for you? After which you will be able to speak from experience.

I don't want to, and neither do I need to. I don't use CFS. This is just what I gathered from reading the whole mailing list yesterday. Everything, as usual, is just AFAIK of course.

Last edited by Zom (2010-11-20 13:15:06)

handy · 2010-11-20 12:51:11

@Zom: Just do the easy .bashrc thing & see what it does for you? After which you will be able to speak from experience.

ngoonee · 2010-11-20 13:41:24

Fruity wrote:

karol wrote:
I suggest you learn what patching means, what that command does and only then try to have some fun with the kernel or other apps.
Odd, I learnt something from Allans answer. Its what forums are for, no? Ask a question relevant to a thread, get a reply of use. Yes? I've always found elitist remarks about go learn elsewhere, kind of sad It's what forums are for, to spread knowledge and ask for answers on topics you dont know.
If I knew more, I wouldnt of asked. I'll strive for omniscience just for you karol.

Other forums may be for that. This forum is for that, but within boundaries. One of the boundaries is that you only ask a question after having done some homework. The homework involves google and man-page searching, at the least.

Ask an informed question and you'll find tons of quality answers. Ask a question which indicates you didn't bother to research and you'll more often than not get an answer like karol's. This is expected behaviour in this forum, Allan was just being nice.

Repeating information available elsewhere in a piece-wise manner is not spreading knowledge, it is simply increasing noise

litemotiv · 2010-11-20 15:17:04

ngoonee wrote:

Repeating information available elsewhere in a piece-wise manner is not spreading knowledge, it is simply increasing noise

Yea, you're now repeating this.

ngoonee · 2010-11-20 16:21:12

litemotiv wrote:

ngoonee wrote:
Repeating information available elsewhere in a piece-wise manner is not spreading knowledge, it is simply increasing noise
Yea, you're now repeating this. http://gathering.tweakers.net/global/sm … ocrite.gif

Oh. Yes, that's true. *hangs head in shame and goes off to bed*

litemotiv · 2010-11-20 17:03:38

ngoonee wrote:

litemotiv wrote:
Yea, you're now repeating this. http://gathering.tweakers.net/global/sm … ocrite.gif
Oh. Yes, that's true. *hangs head in shame and goes off to bed*

Aw..

thestinger · 2010-11-20 17:27:25

Zom wrote:

Most of us probably won't notice it, no. But if you run stuff like MPD in its own TTY, while compiling stuff in another TTY, your audio probably won't stutter. I suspect the same could be said if you run pulseaudio, a web browser, a music player and a movie player in different terminals, then you probably won't have flash or the movie player hogging the system, so you could still listen to music while watching a movie while playing farmville or something like that.
Linus also theorised about adding another flag to the .desktop files, something like "launch in individual cgroup", which would let you launch programs in a different group and thus not be able to hog the system. But as of now, most desktop systems (including the window managers) don't launch their programs in a separate tty or pts, so it won't matter AFAIK.
E: That said, I'm still glad to see this kind of development making it into the mainline kernel. It's nice to minimise the need for patches for desktop systems, IMO.
E2: VVVV
handy wrote:
@Zom: Just do the easy .bashrc thing & see what it does for you? After which you will be able to speak from experience.
I don't want to, and neither do I need to. I don't use CFS. This is just what I gathered from reading the whole mailing list yesterday. Everything, as usual, is just AFAIK of course.

You don't have to run the interactive apps in a terminal to benefit from the patch. Only the non-interactive, CPU hogging applications need to be in a tty. However, it will only help when things have a lot of threads/processes. Your use cases involve 1 process/thread hogging the CPU, and you'll actually slow down interactive GUI apps by running them in cgroups (they won't be able to fork a bunch of processes and hog the CPU though, but when does that happen?).

This doesn't make CFS any better at stopping a process from causing latency, that would require rewriting the scheduler (BFS), and CFS is already pretty damn good at handling one intensive process. cgroups just groups processes in one tty together, and stops that group from getting all the resources just because it has more jobs (which is what usually happens). This makes sense because a tty should only have 1 interactive program at a time.

Also, cgroups have never had anything to do with ttys, and something doesn't need to be in a tty to be put in a cgroup.

Last edited by thestinger (2010-11-20 17:40:40)

broch · 2010-11-20 17:44:52

Allan wrote:

Fruity wrote:
I'm also curious to know: Lets say 2.6.37 has this patch added, what would happen if I then tried to build a custom kernel with 2.6.37 and included this patch again in the build. Would it do nothing, cause harm, etc?
the patch command would likely fail, and makepkg would bail.

actually nothing should happen, edit patch line(s) in PKGBUILD to ignore installed patches when patch command is run (-N flag set to ignore patches that seem to be reversed or applied already). That is all.

Zom · 2010-11-20 18:39:41

thestinger wrote:

You don't have to run the interactive apps in a terminal to benefit from the patch. Only the non-interactive, CPU hogging applications need to be in a tty. However, it will only help when things have a lot of threads/processes. Your use cases involve 1 process/thread hogging the CPU, and you'll actually slow down interactive GUI apps by running them in cgroups (they won't be able to fork a bunch of processes and hog the CPU though, but when does that happen?).
This doesn't make CFS any better at stopping a process from causing latency, that would require rewriting the scheduler (BFS), and CFS is already pretty damn good at handling one intensive process. cgroups just groups processes in one tty together, and stops that group from getting all the resources just because it has more jobs (which is what usually happens). This makes sense because a tty should only have 1 interactive program at a time.

I figured, since you both want your application and your desktop to be responsive, you'd need to have each application in a separate cgroup. I mean, to both keep your music playing, your desktop to be responsive and to keep flash from hogging all the available CPU, as it tends to do.

(My netbook experiences this whenever I do something flash-related.)

I guess that'd be more aimed towards the scheduler itself than this particular patch though.

thestinger wrote:

Also, cgroups have never had anything to do with ttys, and something doesn't need to be in a tty to be put in a cgroup.

No, that's true, but this patch is aimed towards different TTY/PTS AFAIK. Cgroups are already in the kernel, and this feature is already there, just not activated in this way. Hence my smaller note about Linus wanting to include a new desktop specification for launching stuff in a different cgroup.

As another footnote, Lennart also mention that systemd pretty much does this already. It's not complete yet, and we'll see how well it works when it's getting nearer completition, though it's still interesting to note.

cgarcia · 2010-11-23 18:04:10

Zom wrote:

Most of us probably won't notice it, no. But if you run stuff like MPD in its own TTY, while compiling stuff in another TTY, your audio probably won't stutter. I suspect the same could be said if you run pulseaudio, a web browser, a music player and a movie player in different terminals, then you probably won't have flash or the movie player hogging the system, so you could still listen to music while watching a movie while playing farmville or something like that.
Linus also theorised about adding another flag to the .desktop files, something like "launch in individual cgroup", which would let you launch programs in a different group and thus not be able to hog the system. But as of now, most desktop systems (including the window managers) don't launch their programs in a separate tty or pts, so it won't matter AFAIK.
E: That said, I'm still glad to see this kind of development making it into the mainline kernel. It's nice to minimise the need for patches for desktop systems, IMO.
E2: VVVV
handy wrote:
@Zom: Just do the easy .bashrc thing & see what it does for you? After which you will be able to speak from experience.
I don't want to, and neither do I need to. I don't use CFS. This is just what I gathered from reading the whole mailing list yesterday. Everything, as usual, is just AFAIK of course.

It seems that this patch doesn't adds nothing useful that could not be already achievable by running CPU intensive task within a nice or schedtool -D wrapper. Is it really so difficult to type six more letters before a high-workload compilation?. In addition, these comands give one full control of the priority, instead of relying on automated hacks which may be clever, not so clever, or too much clever (beyond the necessary point).

One reason because I like Linux is just because I'm the computer owner, instead of being owned by the OS. The same reason because I switched from Ubuntu to Arch, from kde 3 to gnome (running away from kde 4) and finally from gnome to xfce. I hate thousands of automated and not wanted tasks and processes doing for me things I don't need, I don't known and I don't control.

By the way, Phoronix's "The ~200 Line Linux Kernel Patch That Does Wonders" tabloid makes me remember their old "The Huge Disaster Within The Linux 2.6.35 Kernel" news headline.

Last edited by cgarcia (2010-11-23 18:09:36)

thestinger · 2010-11-23 18:27:25

The newest version of the patch now uses "sessions" instead of ttys, whatever that means.

Zom · 2010-11-23 20:06:52

cgarcia wrote:

It seems that this patch doesn't adds nothing useful that could not be already achievable by running CPU intensive task within a nice or schedtool -D wrapper. Is it really so difficult to type six more letters before a high-workload compilation?. In addition, these comands give one full control of the priority, instead of relying on automated hacks which may be clever, not so clever, or too much clever (beyond the necessary point).
One reason because I like Linux is just because I'm the computer owner, instead of being owned by the OS. The same reason because I switched from Ubuntu to Arch, from kde 3 to gnome (running away from kde 4) and finally from gnome to xfce. I hate thousands of automated and not wanted tasks and processes doing for me things I don't need, I don't known and I don't control.
By the way, Phoronix's "The ~200 Line Linux Kernel Patch That Does Wonders" tabloid makes me remember their old "The Huge Disaster Within The Linux 2.6.35 Kernel" news headline.

No, of course not. This functionality is also already in the kernel.

However, adding the nice or schedtools way of doing it is left to the user, which Linus wants to avoid. Whenether it's "wrong" or "right", I agree with that decision. I see this as a good way, eventually, to make things snappier for the desktop user, and I don't think it has a negative impact on the system either way.

Of course, personally I'll keep using BFS, but I still support the decision.

eduardo.eae · 2010-11-23 22:39:38

BFS with schedtool is great.

I have all my high load apps aliased to schedtool -D -n 19 -e {app}

-D is :set idleprio
-n 19 :set niceness(the niceness also aplies the io scheduler `man ionice`)

I tested with 40x cpuburn in my dual core and it was responsive as if there's no load at all

Army · 2010-11-23 22:54:56

This patch does make a difference on my laptop, I realized it after the recent update of the kernel, which brought me back to a kernel without this patch. Since bfs doesn't do good jobs for me, I'll rebuild the newest kernel with it again.

ngoonee · 2010-11-24 03:34:57

I'm currently compiling a kernel with make -j2 on a dual core, and browsing forums/chatting/playing music isn't an issue. BFS-patch, I guess

Basically, I'm in no hurry to test this out, if it gets in to 2.6.37 or 2.6.38, great.

slytux · 2010-11-25 18:37:07

More headlines:

http://www.h-online.com/open/features/K … 41390.html

dyscoria · 2010-11-26 22:35:10

Octoploid wrote:

Here's what I use right now.
...

Thanks to Octoploid for the info...took me a while to find a solution that worked.

I changed it slightly to enable cgroups only when required:

/etc/rc.local

mount -t cgroup cgroup /cgroup -o cpu
chmod 0700 /cgroup
echo "/usr/local/sbin/rm_cgroup" > /cgroup/release_agent

~/.bashrc

createcgroup () {
    mkdir -m 0700 /cgroup/$$
    echo $$ > /cgroup/$$/tasks
    echo 1 > /cgroup/$$/notify_on_release
}

/sbin/rm_cgroup

#!/bin/sh
rmdir /cgroup/$1

Run as root:

# mkdir /cgroup
# chmod +x /usr/local/sbin/rm_cgroup
# reboot

Run "createcgroup" from any terminal/xterm when required.

Wanted an easy (admittedly lesser) alternative to BFS/CK, and this is about as easy as it gets!

Last edited by dyscoria (2010-12-05 17:22:39)

gbj13 · 2010-12-04 16:06:48

I want to thank Octoploid for the info he provided. I am running my setup the same as he does but I get this errror message when I restart the box.

cgclear failed with No such file or directory

So something must be wrong with my setup, but unfortunatly google/forums do not help. Could anyone give me any advice? It would be much appreciated, thank you.

My setup is exactly the same i.e.

.bashrc

if [ "$PS1" ] ; then                                                                                                                               
    mkdir -m 0700 /cgroup/$$
    echo $$ > /cgroup/$$/tasks
    echo 1 > /cgroup/$$/notify_on_release
fi

rc,local

mount -t cgroup cgroup /cgroup -o cpu
chmod 0777 /cgroup
echo "/sbin/rm_cgroup" > /cgroup/release_agent

/sbin/rm_cgroup

#!/bin/sh
rmdir /cgroup/$1

dyscoria · 2010-12-05 17:32:15

Oh oops, I put some incorrect chmod values in my post up there

They should be 0777 if you want normal users to be able to manage cgroups using my script.

Also, I'm not using libcg which is what I assume you are using judging by that error message. Octopoids post and my post have nothing to do with libcg.

Last edited by dyscoria (2010-12-05 17:33:34)

slytux · 2010-12-07 03:42:24

Mike Galbraith has made patches for Linux 2.6.36.1 and 2.6.37-rc4 available:

http://thread.gmane.org/gmane.linux.kernel/1070535

Cdh · 2010-12-07 16:10:46

thestinger wrote:

3804 tty1 00:00:00 startx
3825 tty1 00:00:00 openbox
it's because you use startx, so all your processes are running under that virtual terminal
I start X with xinit in inittab
x:5:once:/bin/su thestinger -l -c "/usr/bin/xinit ~/.config/xinitrc -- /usr/bin/X -nolisten tcp vt04 >/dev/null 2>&1"
2902 ? 00:00:53 awesome
Most people use display managers, and their desktop apps won't be under a tty afaik.

I tried that and then pm-suspend didn't work properly anymore:
Suspending via acpi event worked first, but when I issued pm-suspend the screen flickered and then it just hang. After killing pm-suspend then suspending via acpi event didn't work anymore either...
Now I reverted to startx and it works properly again.

Anyone experiencing something similar?

x:5:once:/bin/su chris -l -c "/bin/bash --login -c startx >/dev/null 2>&1"
#x:5:once:/bin/su chris -l -c "/usr/bin/xinit ~/.xinitrc -- /usr/bin/X -nolisten tcp vt04 >/dev/null 2>&1"

~/.xinitrc

exec dbus-launch ck-launch-session /usr/bin/startxfce4

I don't want to use a login manager.

thestinger · 2010-12-07 17:05:00

Cdh wrote:

thestinger wrote:
3804 tty1 00:00:00 startx
3825 tty1 00:00:00 openbox
it's because you use startx, so all your processes are running under that virtual terminal
I start X with xinit in inittab
x:5:once:/bin/su thestinger -l -c "/usr/bin/xinit ~/.config/xinitrc -- /usr/bin/X -nolisten tcp vt04 >/dev/null 2>&1"
2902 ? 00:00:53 awesome
Most people use display managers, and their desktop apps won't be under a tty afaik.
I tried that and then pm-suspend didn't work properly anymore:
Suspending via acpi event worked first, but when I issued pm-suspend the screen flickered and then it just hang. After killing pm-suspend then suspending via acpi event didn't work anymore either...
Now I reverted to startx and it works properly again.
Anyone experiencing something similar?
x:5:once:/bin/su chris -l -c "/bin/bash --login -c startx >/dev/null 2>&1"
#x:5:once:/bin/su chris -l -c "/usr/bin/xinit ~/.xinitrc -- /usr/bin/X -nolisten tcp vt04 >/dev/null 2>&1"
~/.xinitrc
exec dbus-launch ck-launch-session /usr/bin/startxfce4
I don't want to use a login manager.

I don't have the same problem, but starting X that way (not from a login manager or a VT) will stop consolekit from thinking it's a local session. You either have to use the startx method (which breaks the tty version of this patch) or a login manager.

Arch Linux

#101 2010-11-20 04:33:13

Re: The kernel patch that does wonders

#102 2010-11-20 05:58:18

Re: The kernel patch that does wonders

#103 2010-11-20 10:22:32

Re: The kernel patch that does wonders

#104 2010-11-20 11:23:16

Re: The kernel patch that does wonders

#105 2010-11-20 12:51:11

Re: The kernel patch that does wonders

#106 2010-11-20 13:41:24

Re: The kernel patch that does wonders

#107 2010-11-20 15:17:04

Re: The kernel patch that does wonders

#108 2010-11-20 16:21:12

Re: The kernel patch that does wonders

#109 2010-11-20 17:03:38

Re: The kernel patch that does wonders

#110 2010-11-20 17:27:25

Re: The kernel patch that does wonders

#111 2010-11-20 17:44:52

Re: The kernel patch that does wonders

#112 2010-11-20 18:39:41

Re: The kernel patch that does wonders

#113 2010-11-23 18:04:10

Re: The kernel patch that does wonders

#114 2010-11-23 18:27:25

Re: The kernel patch that does wonders

#115 2010-11-23 20:06:52

Re: The kernel patch that does wonders

#116 2010-11-23 22:39:38

Re: The kernel patch that does wonders

#117 2010-11-23 22:54:56

Re: The kernel patch that does wonders

#118 2010-11-24 03:34:57

Re: The kernel patch that does wonders

#119 2010-11-25 18:37:07

Re: The kernel patch that does wonders

#120 2010-11-26 22:35:10

Re: The kernel patch that does wonders

#121 2010-12-04 16:06:48

Re: The kernel patch that does wonders

#122 2010-12-05 17:32:15

Re: The kernel patch that does wonders

#123 2010-12-07 03:42:24

Re: The kernel patch that does wonders

#124 2010-12-07 16:10:46

Re: The kernel patch that does wonders

#125 2010-12-07 17:05:00

Re: The kernel patch that does wonders

Board footer