kernel26mm and the new extra/kernels

dp · 2004-08-03 22:37:53

hi all,

i made a pkg of the "Andrew Morton's -mm Kernel" for archlinux

the pkg is called kernel26mm and this is the kernel26 with the mm patches

more info about "-mm kernels" here:

http://kerneltrap.org/forum/linux/kernel/2.6/mm

the -mm kernel is not the official and standard kernel for arch - that's why it is not in "current" or "release" (you don't get support for it, but also not for the standard one, so it's more or less the same trouble you are in, but nontheless i wanted to mention it - any feedback is welcome and we can discuss in the forums about the mm kernels, anyways :-) )

some people say, that the mm is less stable, some say, it is more stable than the normal one - i'm one of the second ones and that's why i made the pkg

to install, -Sy it and then add simply an additional entry to your bootloader (by copying the one you already have for kernel26 by modifying the vmlinuz26 to vmlinuz26mm) - why additional? very easy: to make sure your kernel26 still boots, if the mm doesn't ;-)

have fun,

dp

jlvsimoes · 2004-08-03 23:13:51

wath about ck and wolk are they not good

dp · 2004-08-03 23:43:31

jlvsimoes wrote:

wath about ck and wolk are they not good

ck i dont know - only heard about from a RSS i dont remember - what is it about?

wolk/folk i never used or seen used, but heard the idea - the thing with wolk/folk is that the idea is somehow a little bit funny: my interpretation of the things i read aobut it: "put a lot of things in one kernel without a special target and hope it works"

please, feel free to tell me about them and we can have a look at them - i have a new firewire-box where i put my old laptop-hdd as external harddrive, so theoretically, if there are good reasons, i can add another kernel to extra/kernels

why mm? i was experimenting with string-compare alghorithms for a program i'm writing for myself for pattern-search in the genomes (biology) - the thing is, that such a genome is uncompressed about 100-1000MB big ASCII file and if i search for all against all sequences, it uses swap (i have 768mb ram and 2gb swap) - the trouble i had was that somehow if i use 2.6.normal, it becomes very instable and either crashes or becomes inaccessible for hours --- then i visited someone who was doing very similar things as me and she was using gentoo with the mm kernel without problems, so i decided to use a mm kernel too - compiled, tried, great, made pkg - now we are here :-)

i dont know a lot about the trees of the kernels that exist, except that they exist and the mm works fine for now --- searching the web now i found this:

http://members.optusnet.com.au/ckolivas/kernel/
wow, sounds good - anyone tried?

http://sourceforge.net/projects/wolk
http://wolk.sourceforge.net/
hmm, not much info about wolk :-(

i'm especially interested about how they compare to the normal and mm kernel

EDIT: i had a look at ck ... hey, it is a really small patch
http://ck.kolivas.org/patches/2.6/2.6.7/
or am i wrong?
(the mm patches are about 11MB compared to 130KB there is a huge difference)

EDIT2:
http://ck.kolivas.org/patches/2.6/2.6.8/2.6.8-rc2-mm2/
does that mean we can add ck to mm without messing around?

EDIT3: hard_swapping:
http://ck.kolivas.org/patches/2.6/2.6.8 … iness.diff

-    swap_tendency = mapped_ratio / 2 + distress + vm_swappiness;
+    swap_tendency = mapped_ratio + vm_swappiness;

why is ck ignoring distress?

EDIT4 (the last to this post, hopefully):
i dont see the point with ck - it's mainly a play with numbers and default values

wdemoss · 2004-08-04 00:37:03

my understanding of the ck patch set it that it is targeted at improving desktop gui responsivness. There are reports of it doing so, however, unless you use a heavier desktop (gnome or kde) there probably wont be that much difference in Arch.

-wd

DigitalSin · 2004-08-04 01:14:45

OWNS :pb

You can pick and choose from his split-out patchsets, and he provides optimizations for either server or desktop (you choose).

Bugfixes (and new features) round it out nicely, and with his affinity patchs you can use SCHED_ISO with schedtool. Other patches make a claim for similar functionality, but ck kernels are the only ones to ever work right (or well for that matter) in this respect, as normal user. Oh - and batch scheduling is sweet.

What I especially like about ck is the customizability he allows you - it's what it's all about...

You can push it to the edge and still be stable while bleeding hehe. Cannot say enough good things.

IceRAM · 2004-08-04 09:55:13

http://kerneltrap.org/node/view/875

Beyond that, desktop users will be excited to feel the Con Kolivas [interview] interactivity patches [forum] in action, and all of Con's patches are merged into Andrew's patchset. Or, perhaps you're interested in trying Jens Axboe's Complete Fair Queuing disk I/O scheduler [story] [story], also present. And real time OS fans will enjoy the included real time enhancements.

dp · 2004-08-04 13:17:00

http://lwn.net/Kernel/Patches

dp · 2004-08-04 13:27:36

DigitalSin wrote:

OWNS :pb
You can pick and choose from his split-out patchsets, and he provides optimizations for either server or desktop (you choose).
Bugfixes (and new features) round it out nicely, and with his affinity patchs you can use SCHED_ISO with schedtool. Other patches make a claim for similar functionality, but ck kernels are the only ones to ever work right (or well for that matter) in this respect, as normal user. Oh - and batch scheduling is sweet.
What I especially like about ck is the customizability he allows you - it's what it's all about...
You can push it to the edge and still be stable while bleeding hehe. Cannot say enough good things.

ok, what i found out about ck is, that a lot of things from ck goes to mm - the rest can be added to the mm kernel or the normal kernel easily

customizabiliity is ok, but having for every option an own pkg is not a very efficient way in my eyes (call me lazy) --- especially if you want to tweak the settings, you must apply the ck patches manually and also edit the files by hand, so it's not that good to have a general pkg (becauese, everybody will want to rebuild it him/herself) and sooner or later the things flow into the mm kernel

DigitalSin · 2004-08-04 17:36:56

dp wrote:

ok, what i found out about ck is, that a lot of things from ck goes to mm - the rest can be added to the mm kernel or the normal kernel easily

What I was getting at is there is far more in mm compared to ck, and from my experience with it that is how/why it is seems unstable for me (Andrew also puts more 'experimental' routines, or tries bleeding things a lot more). Don't misunderstand me though, because Morton does some really great things.
=8)

dp wrote:

customizabiliity is ok, but having for every option an own pkg is not a very efficient way in my eyes (call me lazy) --- especially if you want to tweak the settings, you must apply the ck patches manually and also edit the files by hand, so it's not that good to have a general pkg (becauese, everybody will want to rebuild it him/herself) and sooner or later the things flow into the mm kernel

Of course I agree it would be quite ludicrous to have 10's of kernels merely for convenience.

IMHO, If you wanted to you could use split-out patches and just comment-out particular liners in the PKGBUILD (so people could uncomment a line for each patch they wanted - or a group of lines for a particular purpose).

This would mean more work, and you're the one doing it, I don't tell people what to do - you do what you want. I'm only pointing out the option, because I think that if anything ArchLinux needs TWO sets:
1)UNI
2)SMP
Yes smp kernels will work on <some or most> configs, but it doesn't seem right to me [talking about v2.6 in CURRENT].

So far as split-out sets being more work - seems <to me> it would be easier to use them, so you could pick snipets from either <or any> branch to put in, while leaving-out anything not quite 'ready for primetime' <or anything else i.e. CAN>.

FYI I've had problems in the past with mm on boxes that have less or equal 256 MB ram.

dp · 2004-08-04 18:44:53

DigitalSin, thanx a lot for sharing your infos

about the smp/uni discussion, i dont know; personally i dont need smp, but it is also not hurting (or?) --- if we would use uni/smp, then imagine, we would have not 4 but 8 default kernels (scsi/ide, 24/26, smp/uni)

the ck is not one file and not that clean to include into a PKGBUILD, especially, because it varies and ck-patches go to mm or to vanilla, so every release, you need to check what patches to include

the mm i choosen, because it is a main tree of the kernel devs and it suits me (i have 768mb ram 2gb swap and need all of it for my calculations unfortunately and the vanilla crashes often being under heavy load for long - the mm works)

i had in mind to change the .config for the mm kernel, but decided to use the kernel26 config for the start --- main changes in the config, we can discuss, but they must be "good" for the mayority of us (e.g. i0m not going to disable xfs or reiser or other things like that) --- about smp/uni i dont knwo enough to decide: does a uni run ok on a hyperthreading cpu? what are the main differences and the main reason to use smp or uni?

i will go over the config for the 2.6.8-mm release and make some modifications (mainly enable more modules that are new in mm), but still being compatible with all archlinux customs

ideas always welcome

DigitalSin · 2004-08-04 19:36:54

Ahhh dp the ide/scsi blech.. I guess it is simpler to split the two out then to have the waste and lagtime in booting etc. of other distrobutions' way of handling scsi. Having forgotten about this, that's another bag all together, but there are other ways of doing it.

For HyperThreading to be used in a P4 on Linux, SMP must be enabled.

The reason I brought it up is that it is less optimised for UNI, since it contains a whole lot of CPU synchronisation code which is useless on UNI systems, and it also affects all code that is built on the box. Since all kernels are SMP on Arch, it's entirely possible any resulting code built could end up affected. I haven't had any real issues regarding this so far, but that doesn't mean they won't arise

I can hear the naysayers already, but what needs be realised is C/C++ progs/libs have specific opts etc that end up being enabled when the kernel + headers declare themselves to be SMP.

Guess Arch caters to the SMP crowd, but I can see the other side where suppose if you have SMP or use HT on a P4 you would want me to ST*U
hehe
Don't recall seeing any other distro's that do it this way, quite interesting..

dp · 2004-08-04 20:26:01

DigitalSin wrote:

For HyperThreading to be used in a P4 on Linux, SMP must be enabled.

this can be seen as the main reason why to keep smp enabled

thanx for making this clear

Xentac · 2004-08-04 22:22:31

DigitalSin wrote:

Guess Arch caters to the SMP crowd, but I can see the other side where suppose if you have SMP or use HT on a P4 you would want me to ST*U

More like Arch caters to the configure-your-own-kernel-we-just-offer-one-that-suites-the-most-general-case. Our kernel has as many things enabled as we can (and enabled as modules) so that we can cover the widest range of configurations. If you're one to complain about the stock kernel config, then you're one to compile your own kernel.

The only reason dp is distributing this one is because he uses it and figured that other people might want to use it too.

dp · 2004-08-05 00:33:02

Xentac wrote:

DigitalSin wrote:
Guess Arch caters to the SMP crowd, but I can see the other side where suppose if you have SMP or use HT on a P4 you would want me to ST*U
More like Arch caters to the configure-your-own-kernel-we-just-offer-one-that-suites-the-most-general-case. Our kernel has as many things enabled as we can (and enabled as modules) so that we can cover the widest range of configurations. If you're one to complain about the stock kernel config, then you're one to compile your own kernel.
The only reason dp is distributing this one is because he uses it and figured that other people might want to use it too.

well spoken :-)

jlvsimoes · 2004-08-05 19:45:01

i used wolk patchset on 2.4
now that i moved to 2.6 i use the ck patchset im happy with it be the way wolk kernel pkgbuild as been laong for a year on incoming now
and wolk actualy works

ravster · 2004-08-06 01:00:44

DigitalSin wrote:

I can hear the naysayers already, but what needs be realised is C/C++ progs/libs have specific opts etc that end up being enabled when the kernel + headers declare themselves to be SMP.

So, like, what happens if the kernel is manually configured? Do these options still get activated? And if they do, would they mess around with the comp? Basically what I'm asking is, are these options enabled at compile-time or run-time(cause if its run-time, then there shouldn't be a problem)?

Don't recall seeing any other distro's that do it this way, quite interesting..

Do you mean the SMP configuration?(just asking out of curiosity)

tranquility · 2004-08-06 20:13:13

Hi people, I pacman -S'ed kernel26mm and copied grub's menu.lst entry for the regular kernel, but I get a kernel panic when I try to boot the mm one. Seems like he can't find my root partition. I'm not sure why, but maybe it's because I have a SATA drive, dunno. Any how, here's my menu.lst:

# Config file for GRUB - The GNU GRand Unified Bootloader
# /boot/grub/menu.lst

# DEVICE NAME CONVERSIONS 
#
#  Old /dev    DevFS                    Grub
# -----------------------------------------------
#  /dev/fd0    /dev/floppy/0            (fd0)
#  /dev/hda    /dev/discs/disc0/disc    (hd0)
#  /dev/hdb2   /dev/discs/disc1/part2   (hd1,1)
#  /dev/hda3   /dev/discs/disc0/part3   (hd0,2)
#

#  FRAMEBUFFER RESOLUTION SETTINGS
#     +----------------------------------------+
#          | 640x480 800x600 1024x768 1280x1024
#      ----+-----------------------------------
#      256 |   0x301   0x303    0x305     0x307
#      32K |   0x310   0x313    0x316     0x319
#      64K |   0x311   0x314    0x317     0x31A
#      16M |   0x312   0x315    0x318     0x31B
#     +----------------------------------------+

# general configuration:
timeout   5
default   2
color light-blue/black light-cyan/blue

# boot sections follow
# each is implicitly numbered from 0 in the order of appearance below
#-*

# (0) Arch Linux
title  Arch Linux  [/boot/vmlinuz26]
root   (hd0,4)
kernel /boot/vmlinuz26 root=/dev/discs/disc0/part5 ro hdg=noprobe vga=773 devfs=nomount

# Arch Linux kernel mm
title  Arch Linux mm [/boot/vmlinuz26]
root   (hd0,4)
kernel /boot/vmlinuz26mm root=/dev/discs/disc0/part5 ro hdg=noprobe vga=773 devfs=nomount

title Windows XP
root (hd0,0)
makeactive
chainloader +1

Thanks in advance for your input. Cheers, Joao.

dp · 2004-08-06 20:52:50

i dont have sata, so i didnt know that ... the sata support for linux will change in the futur, and the mm kernel already has these new things inside

as some heavy changes were done from 2.6.7-mm4 to mm5 and then reversed in mm6 and then again enabled in mm7, i would suggest you to wait for the 2.6.8-mm1 to try it. i didn't change the config file for the mm kernel (just taken the kernel26 config), but now, especially for sata, i see that i need to change the configs for the mm tree (and will do this for the 2.6.8-mm1)

DigitalSin · 2004-08-06 21:33:25

Xentac wrote:

If you're one to complain about the stock kernel config, then you're one to compile your own kernel.

eheh Fair enough.

FYI anyone who updates to current and requires a 3rd party module that needs to be built from source will likely have to rebuild the kernel anyway.

Happened to me, as the kernel pkg in current is not rebuilt against new gcc and glibc as they become available.

Probably not a problem with %100 of all 3rd party modules, but there are also security flaws in vanilla 2.6.7 that forced me to rebuild anyhow.

DigitalSin · 2004-08-06 21:38:40

tranquility wrote:

Hi people, I pacman -S'ed kernel26mm and copied grub's menu.lst entry for the regular kernel, but I get a kernel panic when I try to boot the mm one. Seems like he can't find my root partition. I'm not sure why, but maybe it's because I have a SATA drive, dunno.

When I checked out 2.6.8-rc2-mm2, there were two SATA drivers. The old one is in the IDE config. The newest one is in the SCSI config.

Might be a new addition, but worth checking on (sorry, I haven't compiled 2.6.7-mm7).

DigitalSin · 2004-08-06 21:48:28

dp wrote:

(i have 768mb ram 2gb swap and need all of it for my calculations unfortunately and the vanilla crashes often being under heavy load for long - the mm works)

Have you considered using three swaps set at same prio, all on different drives? Example, you could use very small, very fast, scsi drives. In this way you could parallelize swap, just like a RAID 0 stripe.

Like:
/path/to/drive1partition# none swap sw,pri=3 0 0
/path/to/drive2partition# none swap sw,pri=3 0 0
/path/to/drive3partition# none swap sw,pri=3 0 0

Just an idea, if your swap is bottlenecking your sys...

I couldn't find the article from IBM, but a mirror on this is here:
http://www.fiveanddime.net/ss/swap.htm

dp · 2004-08-06 22:15:59

DigitalSin wrote:

dp wrote:
(i have 768mb ram 2gb swap and need all of it for my calculations unfortunately and the vanilla crashes often being under heavy load for long - the mm works)
Have you considered using three swaps set at same prio, all on different drives? Example, you could use very small, very fast, scsi drives. In this way you could parallelize swap, just like a RAID 0 stripe.
Like:
/path/to/drive1partition# none swap sw,pri=3 0 0
/path/to/drive2partition# none swap sw,pri=3 0 0
/path/to/drive3partition# none swap sw,pri=3 0 0
Just an idea, if your swap is bottlenecking your sys...
I couldn't find the article from IBM, but a mirror on this is here:
http://www.fiveanddime.net/ss/swap.htm

very cool idea!!!
unfortunately i have only one laptop (2ghz) with one internal hdd (ide, 5400rpm), one external hdd (connectd over firewire, old-laptop--hdd) and an mp3-player (256mb) as storage :-( swapping is only possible on the fw and the intern hdd and i use 2x1gb with higher priority for the external drive (because the internal is under heavy load while reading and writing the data computed)

my old tower-computer is a 266mhz p2 (a compaq persario 4890 - upgraded to the limit) that is a "way" to slow for my cpu-intensive comparings, but i plan to buy parts to build a new tower for at-home-power-computing when i have the money for something like this (unfortunately i cannot afford the cpu at the moment, and buying the rest without the cpu is stupid, because the prices go down regularly) --- and as my hobby/half-profession is photography, i want on this machine this display:
http://www-132.ibm.com/webapp/wcs/store … &storeId=1
(QUXGA-W - no need to resize the pics ever: they can be shown 1:1), so i need to work besides learning for the uni a lot :-( --- well, it can take some years, but i hope that the prices goes down in the same time :-) [the student-price for this monitor is only 60% of the price written on this url, but still a way tooo much]

Xentac · 2004-08-06 23:50:39

DigitalSin wrote:

FYI anyone who updates to current and requires a 3rd party module that needs to be built from source will likely have to rebuild the kernel anyway.
Happened to me, as the kernel pkg in current is not rebuilt against new gcc and glibc as they become available.

This part is not by design. We put in a lot of work making the kernel package build as many third party modules out of the box. Something like a mismatched gcc or glibc should be followed by a rebuild of the kernel package.

Submit a bug report for that one.

tranquility · 2004-08-07 09:48:35

Ok, thanks for your input dp. I'll be sure to try when 2.6.8-mm1 gets released. But it really is weird because the kernel config is the same.

Cheers, Joao.

EDIT: Well I was going through the /boot/kconfig26mm and found this:

# CONFIG_BLK_DEV_IDE_SATA is not set
[...]
# CONFIG_SCSI_SATA is not set

Isn't this a probable cause for me not being able to boot kernel26mm ? The stock arch 2.6.7 kernel has CONFIG_BLK_DEV_IDE_SATA set by the way.

tranquility · 2004-08-07 09:54:27

DigitalSin wrote:

tranquility wrote:
Hi people, I pacman -S'ed kernel26mm and copied grub's menu.lst entry for the regular kernel, but I get a kernel panic when I try to boot the mm one. Seems like he can't find my root partition. I'm not sure why, but maybe it's because I have a SATA drive, dunno.
When I checked out 2.6.8-rc2-mm2, there were two SATA drivers. The old one is in the IDE config. The newest one is in the SCSI config.
Might be a new addition, but worth checking on (sorry, I haven't compiled 2.6.7-mm7).

Hmmm, I think I'll maybe try to file a bug or something or maybe send a message to the lkml.

Arch Linux

#1 2004-08-03 22:37:53

kernel26mm and the new extra/kernels

#2 2004-08-03 23:13:51

Re: kernel26mm and the new extra/kernels

#3 2004-08-03 23:43:31

Re: kernel26mm and the new extra/kernels

#4 2004-08-04 00:37:03

Re: kernel26mm and the new extra/kernels

#5 2004-08-04 01:14:45

Re: kernel26mm and the new extra/kernels

#6 2004-08-04 09:55:13

Re: kernel26mm and the new extra/kernels

#7 2004-08-04 13:17:00

Re: kernel26mm and the new extra/kernels

#8 2004-08-04 13:27:36

Re: kernel26mm and the new extra/kernels

#9 2004-08-04 17:36:56

Re: kernel26mm and the new extra/kernels

#10 2004-08-04 18:44:53

Re: kernel26mm and the new extra/kernels

#11 2004-08-04 19:36:54

Re: kernel26mm and the new extra/kernels

#12 2004-08-04 20:26:01

Re: kernel26mm and the new extra/kernels

#13 2004-08-04 22:22:31

Re: kernel26mm and the new extra/kernels

#14 2004-08-05 00:33:02

Re: kernel26mm and the new extra/kernels

#15 2004-08-05 19:45:01

Re: kernel26mm and the new extra/kernels

#16 2004-08-06 01:00:44

Re: kernel26mm and the new extra/kernels

#17 2004-08-06 20:13:13

Re: kernel26mm and the new extra/kernels

#18 2004-08-06 20:52:50

Re: kernel26mm and the new extra/kernels

#19 2004-08-06 21:33:25

Re: kernel26mm and the new extra/kernels

#20 2004-08-06 21:38:40

Re: kernel26mm and the new extra/kernels

#21 2004-08-06 21:48:28

Re: kernel26mm and the new extra/kernels

#22 2004-08-06 22:15:59

Re: kernel26mm and the new extra/kernels

#23 2004-08-06 23:50:39

Re: kernel26mm and the new extra/kernels

#24 2004-08-07 09:48:35

Re: kernel26mm and the new extra/kernels

#25 2004-08-07 09:54:27

Re: kernel26mm and the new extra/kernels

Board footer