kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

graysky · 2009-11-03 23:35:10

It's due to a tweaked scheduler in the 2.6.32 codebase. I measured a 53 % speed boost comparing 2.6.31-5 to 2.6.32-rc6 on my quad. See my post over at the handbrake forum here for details and see this page too.

The test encode was a 60 fps 720p (1280x720) encode using the "High Profile" on handbrake snapshot 2907 (which you can get in the AUR here):

Arch x86_64 (kernel 2.6.31.5): avg 15.99 fps
Arch x86_64 (kernel 2.6.32-rc6): avg 24.45 fps
Linux (kernel 2.6.31-bfs): avg 24.49 fps
Win XP 64-bit edition: avg 20.41 fps

Linux is 53 % faster w/ the new kernel and 20 % faster than Windows. Can't wait for 2.6.32 to go final!

Last edited by graysky (2009-12-06 10:57:58)

foutrelis · 2009-11-03 23:53:04

Nice cooperation between the x264 developer and the kernel hacker.

Ranguvar · 2009-11-04 00:23:26

Already read it on Dark Shikari's blog, yeah

whaevr · 2009-11-04 00:51:01

I'm excited for the next kernel release now!

graysky · 2009-11-04 00:55:14

...now if someone could fix that damn system slow-down while copying files on the x86_64 kernel.

Described in numerous threads:

http://bbs.archlinux.org/viewtopic.php?id=71131
http://bbs.archlinux.org/viewtopic.php?id=70525

Last edited by graysky (2009-11-04 00:56:06)

whaevr · 2009-11-04 00:57:31

Would this in turn also boost performance in something like folding@home also? For the smp clients anyway..

Last edited by whaevr (2009-11-04 00:57:50)

Ranguvar · 2009-11-04 01:34:50

Dunno. Test and see, if you dare
You could probably cherry-pick that commit and apply it as a patch to 2.6.31.5.

skottish · 2009-11-04 02:17:40

That's freakin' nuts!

tomd123 · 2009-11-04 03:31:41

too bad I don't encode anything is it just a gain in encoding?

Last edited by tomd123 (2009-11-04 03:35:06)

MP2E · 2009-11-04 06:23:52

Epic! I always encode things in x264(though through FFMPEG, though that shouldn't make a difference since it uses the x264 library, should it?) but I'm a user of BFS. Since you are referring to a scheduler change, will this affect me?

bangkok_manouel · 2009-11-04 07:44:17

MP2E wrote:

Epic! I always encode things in x264(though through FFMPEG, though that shouldn't make a difference since it uses the x264 library, should it?) but I'm a user of BFS. Since you are referring to a scheduler change, will this affect me?

you'll need to go back to the mainline CPU sched (CFS)

Ranguvar · 2009-11-04 07:47:26

bangkok_manouel wrote:

MP2E wrote:
Epic! I always encode things in x264(though through FFMPEG, though that shouldn't make a difference since it uses the x264 library, should it?) but I'm a user of BFS. Since you are referring to a scheduler change, will this affect me?
you'll need to go back to the mainline CPU sched (CFS)

But this bug was noticed in the first place because BFS kicked CFS's ass with x264, so sticking with BFS is fine

@tomd123: It's a general improvement in the scheduler, which should translate to other workloads as well. x264 got a huge boost, not everything will get such a huge increase in speed.

.:B:. · 2009-11-04 09:14:28

If you use BFS, I don't think it will affect you, if this is the CFS scheduler we're talking about (and I think we are).

I'm still a bit confused on kernel schedulers, you have I/O and CPU schedulers, and I don't think they're the same (CFS being a CPU scheduler and CFQ e.g. being an I/O one). If I get that right .

Themaister · 2009-11-04 13:10:05

That is really sweet I noticed good boosts even with a really simple threaded app I made myself It utilizes the CPU a lot better now. (Core i7 @ 3.33GHz)

Last edited by Themaister (2009-11-04 13:24:22)

iBertus · 2009-11-04 15:46:25

Ahhh.. drooling at the Core i7 @ 3.33 Ghz! The new i7's are way faster than my old harperstown Xeons and use less power! A person working with me just built a Core i7 workstation w/ 12GB of memory and a stack of solid-state drives for RAID0. I haven't gotten my hands on it yet but it'll probably scream!

Themaister · 2009-11-04 16:39:22

It screams for encoding and such, it's wonderful Obliterates my old Core2Duo xD

graysky · 2009-11-04 20:39:31

I tried using the same encode using speedvin's PKGBUILD for the 2.6.31-bfs and got a similar result (certainly within error) to the 2.6.32-rc6 kernel

Arch x86_64 (kernel 2.6.32-rc6): avg 24.45 fps
Arch x86_64 (kernel 2.6.31-304.5-bfs): avg 24.49 fps

Last edited by graysky (2009-11-04 20:39:47)

skottish · 2009-11-05 01:52:40

graysky wrote:

I tried using the same encode using speedvin's PKGBUILD for the 2.6.31-bfs and got a similar result (certainly within error) to the 2.6.32-rc6 kernel
Arch x86_64 (kernel 2.6.32-rc6): avg 24.45 fps
Arch x86_64 (kernel 2.6.31-304.5-bfs): avg 24.49 fps

That's more than within error, it's essentially identical since your system does other things.

Ranguvar · 2009-11-05 02:57:03

B wrote:

If you use BFS, I don't think it will affect you, if this is the CFS scheduler we're talking about (and I think we are).

Yes, it won't affect you. I was just saying there's no reason to switch from BFS to CFS for this if BFS works fine for you

graysky · 2009-11-05 08:52:48

skottish wrote:

That's more than within error, it's essentially identical since your system does other things.

Hmm... within error meaning it's identical within the limits of the test.

skottish · 2009-11-06 03:41:02

You know what's really funny about this? No matter how much of an outsider Con Kolivas is to the mainline kernel, he somehow frequently seems to get his ideas implemented.

ngoonee · 2009-11-06 05:56:56

skottish wrote:

You know what's really funny about this? No matter how much of an outsider Con Kolivas is to the mainline kernel, he somehow frequently seems to get his ideas implemented.

Not very surprising, to me. His ideas are good. His personality/attitude when dealing with the other devs (and their attitude as well I guess) isn't linked to his intelligence and innovation at all. Both are important on a team project, which is basically what the kernel is (on a big scale).

A pity, really. Then again, without his personality he may not be having his ideas....

Arch Linux

#1 2009-11-03 23:35:10

kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#2 2009-11-03 23:53:04

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#3 2009-11-04 00:23:26

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#4 2009-11-04 00:51:01

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#5 2009-11-04 00:55:14

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#6 2009-11-04 00:57:31

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#7 2009-11-04 01:34:50

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#8 2009-11-04 02:17:40

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#9 2009-11-04 03:31:41

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#10 2009-11-04 06:23:52

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#11 2009-11-04 07:44:17

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#12 2009-11-04 07:47:26

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#13 2009-11-04 09:14:28

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#14 2009-11-04 13:10:05

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#15 2009-11-04 15:46:25

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#16 2009-11-04 16:39:22

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#17 2009-11-04 20:39:31

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#18 2009-11-05 01:52:40

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#19 2009-11-05 02:57:03

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#20 2009-11-05 08:52:48

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#21 2009-11-06 03:41:02

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

#22 2009-11-06 05:56:56

Re: kernel 2.6.32 will boost x264 encode speed BIG TIME for multicore CPUs

Board footer