Linux kernel 6.12 RT

Mapleleaf · 2025-01-03 17:59:25

Hi, I previously posted in a "solved" topic, so I thought I'd create a new topic instead.

My question is, is there some ETA for a 6.12-RT?

The compile option PREEMPT_RT is not enabled in the standard 6.12 kernel, only PREEMP_DYNAMIC, and that changes a lot the latency, which leads to problems especially when recording audio with audio interfaces etc.
To get PREEMPT_RT at compile time, we need to enable "Fully Preemptible Kernel (Real-Time)" in the "Preemption Model" menu (of "General Setup", in make menuconfig), but "Fully Preemptible" is not an option visible by default, for that we need to enable the "Configure standard kernel features (expert users) (NEW)" option of "General Setup".

Even if I figured out where the RT option is, I'd still be glad to have an official RT kernel, because there are many other different compile options between standard and RT kernel besides the PREEMPT stuff, and I've not yet looked into what is important and not.

Edit: the "solved" topic was here -> https://bbs.archlinux.org/viewtopic.php … 2#p2218062
and actually the discussion happened mainly over there.

Last edited by Mapleleaf (2025-01-03 18:23:40)

gromit · 2025-01-03 18:02:10

As I already commented on the other thread, there is a official real time kernel, although a bit out of date: https://archlinux.org/packages/?q=linux-rt

stevensbsd · 2025-01-07 07:02:53

I run archlinux as a kvm guest on a almalinux 8.10 host with preempt_rt patch. After some tuning(mainly using tuned, kvm realtime tuning), the cyclictest test result is flowing:

# /dev/cpu_dma_latency set to 0us
# Histogram
000002 068349 000000 000000 000000 000000 000000
000003 2183931194 279103572 37091103 208305689 9181267 76903485
000004 83360478 1953955502 2117098491 1976503101 2092015302 2079991264
000005 638721 34936000 113802162 83184022 166786078 111090274
000006 000470 003649 006943 005822 015859 013626
000007 000415 000480 000423 000454 000574 000463
000008 000265 000371 000309 000265 000279 000333
000009 000071 000280 000275 000326 000218 000311
000010 000019 000095 000210 000216 000214 000167
000011 000006 000029 000060 000077 000138 000048
000012 000003 000004 000009 000006 000049 000006
000013 000003 000006 000004 000005 000007 000006
000014 000003 000004 000005 000004 000003 000003
000015 000003 000000 000002 000002 000004 000006
000016 000000 000003 000001 000005 000004 000001
000017 000000 000004 000001 000001 000000 000001
000018 000000 000000 000001 000002 000001 000002
000019 000000 000001 000000 000001 000000 000000
# Min Latencies: 00002 00003 00003 00003 00003 00003
# Avg Latencies: 00003 00003 00004 00003 00004 00004
# Max Latencies: 00015 00019 00018 00019 00018 00018
# Histogram Overflows: 00000 00000 00000 00000 00000 00000
# Histogram Overflow at cycle number:
# Thread 0:
# Thread 1:
# Thread 2:
# Thread 3:
# Thread 4:
# Thread 5:

archlinux kernel version: default 6.12.6
test duration: 63 hours

sorry for my poor english

Mapleleaf · 2025-01-07 22:42:46

What is your tuning? I don't get these results at all.
Haven't you forgot to run hackbench -l 1000000 to load your system while doing the test, as advised in the Arch documentation?

When I do cyclictest with 6.10-rt I get a max latency of 65, whereas with 6.12.7 (default) I get a latency of 5340 in the first run (I redid the test because it seemed weird) and 753 in the second run (still very high).
I did a third test after setting the kernel boot command line parameter "preempt=full", and I still get a quite high max latency of 240.
When I record audio with my USB audio interface and the default 6.12.7 kernel, I encounter a lot of xruns, which ruin my recording.
I get no xruns with the 6.10-rt with the same audio settings.

As you can see, it is very much important to get a version of the kernel compiled with CONFIG_PREEMPT_RT=y properly set.

Last edited by Mapleleaf (2025-01-07 22:54:24)

stevensbsd · 2025-01-10 03:49:39

Mapleleaf wrote:

What is your tuning? I don't get these results at all.
Haven't you forgot to run hackbench -l 1000000 to load your system while doing the test, as advised in the Arch documentation?
When I do cyclictest with 6.10-rt I get a max latency of 65, whereas with 6.12.7 (default) I get a latency of 5340 in the first run (I redid the test because it seemed weird) and 753 in the second run (still very high).
I did a third test after setting the kernel boot command line parameter "preempt=full", and I still get a quite high max latency of 240.
When I record audio with my USB audio interface and the default 6.12.7 kernel, I encounter a lot of xruns, which ruin my recording.
I get no xruns with the 6.10-rt with the same audio settings.
As you can see, it is very much important to get a version of the kernel compiled with CONFIG_PREEMPT_RT=y properly set.

I used to run industrial applications (such as ethercat) with realtime kernel(4.18、5.14 and 6.1). In those cases, the cpu load is usually low, so we basically used cyclictest with no load. The previous result is the result of no load.

Today I ran a hackbench+cyclictest to compare kernels 6.12.8 (default) and 6.10.2-rt. With the same hackbench command, using the 6.12.8 kernel execution efficiency is significantly higher, the load average will exceed 200(max latency 301,test duration 90 minutes), and the same number of tasks finished faster; With 6.10.2-rt, the maximum load average is less than 25（max latency 104,same test duration), and the same number of tasks take longer to complete.

To make the results fairer, I tweaked the hackbench command on a 6.12.8 kernel to keep the load average around 45 and run cyclictest, which gives the following results:

# /dev/cpu_dma_latency set to 0us
# Histogram
000003 000006 000000 065722 000000 000001 000000
000004 001309 003287 36048002 001078 000374 000366
000005 4494028 29294936 6841106 4689919 1403615 7446134
000006 2492363 20676402 3620302 9192447 1170768 21353546
000007 509985 1227347 5241169 8208477 4196739 19310050
000008 24448473 325045 1531726 18357218 25091827 4288777
000009 17237514 1258893 228823 9581714 18303699 1090879
000010 3709247 665523 081405 3172645 2783075 218047
000011 724680 207248 043077 509340 644739 113166
000012 184774 135234 022833 126799 198593 075745
000013 086532 079240 012827 074708 086827 044127
000014 045654 046497 009975 039565 046307 025109
000015 025914 024696 018181 018571 028005 014556
000016 015661 017282 009322 010936 017918 008090
000017 009985 013395 008255 006677 011318 004536
000018 005817 009126 047108 003891 006877 002584
000019 003346 006344 093821 002357 003926 001540
000020 001808 004018 037629 001375 002254 000861
000021 000947 002266 018196 000829 001215 000577
000022 000494 001355 011613 000493 000694 000409
000023 000283 000820 004480 000336 000444 000278
000024 000176 000415 002061 000194 000256 000196
000025 000100 000247 001017 000125 000166 000136
000026 000053 000134 000569 000105 000120 000100
000027 000050 000084 000299 000053 000059 000054
000028 000035 000055 000170 000036 000046 000044
000029 000029 000026 000107 000026 000034 000028
000030 000018 000026 000073 000019 000032 000019
000031 000008 000019 000031 000019 000018 000005
000032 000003 000018 000026 000017 000010 000009
000033 000010 000006 000019 000007 000012 000009
000034 000004 000005 000016 000004 000008 000005
000035 000006 000002 000016 000000 000006 000006
000036 000004 000002 000005 000005 000007 000003
000037 000005 000002 000006 000003 000001 000003
000038 000000 000000 000003 000002 000002 000000
000039 000000 000002 000004 000003 000001 000001
000040 000000 000001 000003 000000 000001 000000
000041 000000 000000 000001 000001 000000 000002
000042 000001 000001 000000 000002 000000 000000
000043 000000 000000 000001 000000 000001 000000
000044 000000 000001 000000 000001 000002 000000
000045 000000 000000 000001 000000 000000 000000
000048 000000 000000 000000 000001 000000 000000
000049 000000 000000 000000 000001 000001 000000
# Min Latencies: 00003 00004 00003 00004 00003 00004
# Avg Latencies: 00008 00005 00004 00007 00008 00006
# Max Latencies: 00042 00044 00045 00049 00049 00041
# Histogram Overflows: 00000 00000 00000 00000 00000 00000
# Histogram Overflow at cycle number:
# Thread 0:
# Thread 1:
# Thread 2:
# Thread 3:
# Thread 4:
# Thread 5:

my tuning mainly include:

host :
BIOS: disable hyper-thread; disable power manager
install tuned and use realtime-virtual-host profile(total 10 cores, 2-9 cores isolated)
kernel arguments: intel_idle.max_cstate=0 processor.max_cstate=1 iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G hugepages=24 skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 isolcpus=managed_irq,domain,2-9 intel_pstate=disable nosoftlockup nohz=on nohz_full=2-9 rcu_nocbs=2-9 irqaffinity=0,1

kvm guest :
kvm tuning：https://www.libvirt.org/kbase/kvm-realtime.html
install tuned and use realtime-virtual-guest profile（total 8 vcpus，2-7 vcpus isolated）
kernel arguments：intel_idle.max_cstate=0 processor.max_cstate=1 skew_tick=1 tsc=reliable rcupdate.rcu_normal_after_boot=1 isolcpus=managed_irq,domain,2-7 intel_pstate=disable nosoftlockup nohz=on nohz_full=2-7 rcu_nocbs=2-7 irqaffinity=0,1

test commands:
sudo taskset -c 2-7 sudo chrt -f 80 hackbench -P -f 10 -g 3 -l 50000000 -s 1000
execution duration ：6882.670s
sudo cyclictest -a2-7 -t6 -qm -i100 -p98 -h1500 -D90m

Last edited by stevensbsd (2025-01-10 05:49:28)

OpusOne · 2025-01-12 04:55:12

gromit wrote:

As I already commented on the other thread, there is a official real time kernel, although a bit out of date: https://archlinux.org/packages/?q=linux-rt

Yep, been using the linux-rt package for a longish time for one project. But yes, it always lagged a bit in terms of version, which was understandable as the preempt patches were outside of mainline linux. Now that they are included in Linux 6.12+, do you think there's a chance that this linux-rt package is going to follow the vanilla linux one more closely, as it's mainly a matter of enabling the right option when building it AFAIK?

That would be cool.

Arch Linux

#1 2025-01-03 17:59:25

Linux kernel 6.12 RT

#2 2025-01-03 18:02:10

Re: Linux kernel 6.12 RT

#3 2025-01-07 07:02:53

Re: Linux kernel 6.12 RT

#4 2025-01-07 22:42:46

Re: Linux kernel 6.12 RT

#5 2025-01-10 03:49:39

Re: Linux kernel 6.12 RT

#6 2025-01-12 04:55:12

Re: Linux kernel 6.12 RT

Board footer