Arch Linux

Bralkein

Member

Registered: 2004-10-26

Posts: 354

Prioritising tasks based on their purpose

I was just thinking, it would be really cool if I could somehow group different tasks, based on what they do, and then set their priority for access to different resources by group.

For example, I am currently doing some experiments with digital video. Every time I encode something or whatever, I have to nice the process down to a low priority, otherwise my music will start jumping. Sure, I can just alias the commands I use so that they get niced to the priority I want, but then I have to remember to do that for every single tool I use, and if I use a new tool, I have to remember to do it for that before I use it. Also, I think the number of aliases I ended up with would quickly become less than manageable.

It would be really cool if there were some way I could have groups of tasks, like a group for encoders, a group for compilers, things that take lower priority to my desktop activities; and another group for media players, word processors etc., which I would want to be more responsive. It would be even cooler if I were able to assign different priorities for different resources, i.e. let one process have a low priority on the CPU, but a high priority on the HDD or whatever.

Is this at all possible, or is this just a fantasy? Even if the latter case is true, I still think it's an interesting idea. Thoughts, anyone?

augustob

Member

From: Florianópolis, Brazil

Registered: 2006-03-17

Posts: 135

Re: Prioritising tasks based on their purpose

I think that you may be cutting into the kernel's job here; scheduling is one of its top responsibilities, along with memory management.

If the scheduler is working with a simple algorithm, if your encoder has the same priority than your highly interactive tasks, they'd all get the same amount of processor time if they needed it, but they don't. The problem is you don't want equal processor time, you want small amounts processor time (refreshing screen, processing input) right NOW.

I think nicing the heavy stuff down to a lower priority level isn't enough, 'cause you'd be wasting processor time scheduling tasks that have nothing to do. The way to go would be making sure the interactive stuff gets scheduled more often, decreasing response time. In my opinion the problem stems from the fact that classic schedulers haven't been designed with desktop computers in mind; they wanna get the most out of the CPU.

The long term solution would be something along the lines of the event-driven embedded kernels that are popular today: everything runs in an interrupt, and the ever-so-common 'Task' abstraction isn't the central character anymore. The problems that arise from that are many, but at least your keypresses will be taken care of immediately

Even then you'd run into classic real-time scheduling problems, such as priority inversions (high priority tasks asks for mutex'ed resource a lower-priority task is holding). This is all very textbook stuff, check your local copy of The Tanenbaum for a more scientific discussion.

Oh God it appears that I've gone on a rant! :oops:

Bralkein

Member

Registered: 2004-10-26

Posts: 354

Re: Prioritising tasks based on their purpose

If the scheduler is working with a simple algorithm, if your encoder has the same priority than your highly interactive tasks, they'd all get the same amount of processor time if they needed it, but they don't. The problem is you don't want equal processor time, you want small amounts processor time (refreshing screen, processing input) right NOW.

I think nicing the heavy stuff down to a lower priority level isn't enough, 'cause you'd be wasting processor time scheduling tasks that have nothing to do. The way to go would be making sure the interactive stuff gets scheduled more often, decreasing response time.

Thanks for your reply!

I have compiled my kernel to use a low-latency desktop scheduling policy (I think), but this doesn't really seem to do the trick. The fact remains that if I nice the encoding processes to a lower priority, then my CPU usage remains at 100%, but whenever I perform some interactive task, it feels much more responsive than it would if I didn't nice the encoding process and just left it at normal priority.

What you're talking about seems to be the handling of differently-prioritised tasks, which is interesting, but what I am talking about is a way of having processes that you run automatically assigned a priority based on some kind of user-defined policy. This does not seem to be an entirely kernel-level problem.

Imagine you have two people using a computer simultaneously, and they are both given half of the computer's resources to use. Let's call them Jack and Jill. They are both doing some word-processing, and also listening to some mp3s. Jack is really obsessed with music, so he doesn't want it to skip at all. He sets his process priority policy up so that media player software is a very high-priority task, and word-processing is a normal-priority task. Jill, on the other hand, hates being at a computer. She likes listening to music, but she'd rather get her essay done ASAP, and it doesn't matter if the music skips. She sets the priority of office productivity applications to be high, because she wants any processor-intensive work her word processor does to be completed ASAP, never mind if the mp3 skips, to minimise the amount of time spent at the computer.

It's a bad example I guess, but do you see what I'm getting at?

sh__

Member

Registered: 2005-07-19

Posts: 272

Re: Prioritising tasks based on their purpose

I think you might be looking for something like schedtoold. It is a simple daemon that uses schedtool to adjust the scheduling policy of a process. It is in the early stage of development but looks quite interesting. It will probably work best with -ck patched kernels (like beyond).

augustob

Member

From: Florianópolis, Brazil

Registered: 2006-03-17

Posts: 135

Re: Prioritising tasks based on their purpose

The fact remains that if I nice the encoding processes to a lower priority, then my CPU usage remains at 100%, but whenever I perform some interactive task, it feels much more responsiveat?

That's because the interactive task goes from the processor to the head of the ready queue in the scheduler more often than the lower-priority, processor-intensive one. (Obs: If I recall correctly, Linux has a fairness algorithm that does some fiddling with this order, but I believe this is essentially correct).

In fact, it's scheduled so much that sometimes it gets the processor and yields it without doing anything at all, wasting precious CPU cicles!

The ideal scenario would be when the interactive tasks got the processor as soon as they needed it (Jill pressed a key on the keyboard), and not any sooner (her text editor was scheduled over the mp3 player unnecessarily, making the song skip).

Realize we're not disagreeing, just suggesting different things; you want a system to classify tasks by priority in Linux, and my perfect-world suggestion suggests two classes of tasks: interactive ones, that are purely interrupt driven, and processor-intensive ones, that get all the processor that's left.

Your suggestion can be implemented right now, in Linux, and probably will be the one we use until we die, but you are wasting a few cicles doing unnecessary context switching to tasks that don't want to be active. As time progresses this is less and less of an issue, because we'd rather buy more processing power than worry about these things.

My suggestion would involve some serious structural changes to any current operating system, and is far too academic (and admittedly crackpot) to get any serious attention.

If you have any serious scientific inclination, there are a few scheduler/memory management QoS papers that discuss a solution for this. But as always, you compromise overall performance to guarantee that the applications will be more responsive (or hog the CPU). This discussion is endless, albeit very interesting. Thanks for giving me something to blabber about in between meetings!

Bralkein

Member

Registered: 2004-10-26

Posts: 354

Re: Prioritising tasks based on their purpose

That schedtoold looks kind of like what I am talking about, but imagine if all binaries could be somehow categorised, and you could set priorities for the different categories so that you don't have to go over every binary in your system individually and set it up how you want. Maybe using nice levels is not that efficient, but as long as it made my desktop more pleasant to use, I wouldn't mind. My CPU is mostly idle anyway.

It would be cool to also have a profile that could be activated when the computer has gone for 5 minutes or so without user input, similar to how a screensaver works. This way, you could have your encoding, compiling, etc. niced to a low priority as you use the PC, then when you go to watch some TV or whatever, those processes will be niced back to normal priority.