You are not logged in.

#1 2018-02-13 19:27:32

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

[Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

@inglor what was the last version that did work and the first version you encountered that did not?
Have you tried changing the boot options such as removing quiet if present adding loglevel=7 earlyprintk=efi/earlyprintk=vga (vga for bios boot/efi for efi)

Last working version for linux-ck was with i7. From the point I've changed platform to AMD Threadripper I can't boot -ck kernel.
My kernel params:

menuentry 'Arch Linux, with Linux linux-ck' --class arch --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-ck-advanced-95560524-3ea7-4121-9859-4e9f763ae540' {
        load_video
        set gfxpayload=keep
        insmod gzio
        insmod part_gpt
        insmod fat
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root  305D-B889
        else
          search --no-floppy --fs-uuid --set=root 305D-B889
        fi
        echo    'Loading Linux linux-ck ...'
        linux   /vmlinuz-linux-ck root=UUID=95560524-3ea7-4121-9859-4e9f763ae540 rw  scsi_mod.use_blk_mq=1 nvidia-drm.modeset=1
        echo    'Loading initial ramdisk ...'
        initrd  /initramfs-linux-ck.img
}

Last edited by inglor (2018-02-15 11:38:45)

Offline

#2 2018-02-13 19:54:01

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

remove scsi_mod.use_blk_mq=1 nvidia-drm.modeset=1 and add loglevel=7 earlyprintk=efi/earlyprintk=vga (vga for bios boot/efi for efi) is any more output generated?

Offline

#3 2018-02-14 00:42:04

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

remove scsi_mod.use_blk_mq=1 nvidia-drm.modeset=1 and add loglevel=7 earlyprintk=efi/earlyprintk=vga (vga for bios boot/efi for efi) is any more output generated?

Got some stuff back but very weird. It was printing a line every second - was like the CPU was really slow. Finally after ~1 minute it has done it stuck with the following on the screen:
Final output: https://photos.app.goo.gl/hsAXeaBC5L893ZLO2
Video: https://photos.app.goo.gl/fGiTKGBW4HXolmFo2

My compilation flags are:

CFLAGS="-march=native -O2 -pipe -fstack-protector-strong -fno-plt"

and make flags

MAKEFLAGS="-j30"

the config I used:
https://ptpb.pw/YtK-

Last edited by inglor (2018-02-14 07:14:48)

Offline

#4 2018-02-14 09:05:03

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

@graysky just to be aware of for 4.15 https://bugs.archlinux.org/task/57496
@inglor when you used the config from the arch kernel did you keep MuQSS disabled?

Offline

#5 2018-02-14 10:42:07

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

@graysky just to be aware of for 4.15 https://bugs.archlinux.org/task/57496

Meh, I don't like the resolution of the bug as won't fix simply because it has too many issues.. but I guess that's a completly different issue and purely down to the maintainer.

loqs wrote:

@inglor when you used the config from the arch kernel did you keep MuQSS disabled?

I think it's enabled by default on the linux-ck packaged configuration. The only change I've make on the PKGBUILD was to enable NUMA. So I assume the config has MuQSS enabled from the "unfuck_MuQSS_for_4.14.15+.patch". I've edited my post above to include the config and I can see the:

CONFIG_SCHED_MUQSS=y

Offline

#6 2018-02-14 15:13:44

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

@inglor try the boot option rcu_nocbs=0-# (where # is the number of logical CPUs -1)
If that fails try building with a kernel with CONFIG_SCHED_MUQSS=n

Offline

#7 2018-02-14 20:22:06

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

@inglor try the boot option rcu_nocbs=0-# (where # is the number of logical CPUs -1)
If that fails try building with a kernel with CONFIG_SCHED_MUQSS=n

Update:
Kernel param didn't help.

rcu_nocbs=0-31

Recompiling with enabled NUMA and MuQSS helped:

~$ zcat /proc/config.gz | grep NUMA
CONFIG_ARCH_SUPPORTS_NUMA_BALANCING=y
# CONFIG_NUMA_BALANCING is not set
CONFIG_NUMA=y
CONFIG_AMD_NUMA=y
CONFIG_X86_64_ACPI_NUMA=y
# CONFIG_NUMA_EMU is not set
CONFIG_USE_PERCPU_NUMA_NODE_ID=y
CONFIG_ACPI_NUMA=y
~$ zcat /proc/config.gz| grep MUQSS
# CONFIG_SCHED_MUQSS is not set

I am able to boot now with linux-ck but the initial kernel boot is as described before (slow and printing 1 line per second, see the video I posted previously). Then it speeds up especially when it reaches the systemd parts. I will try removing the debug params in case this is what's slowing it down so much.

Update2:
It was indeed the debugging params slowing it so much (wow!). Now that I removed it it's booting OK.

Summary:
I'm only able to boot with enabled NUMA and disabled MuQSS on AMD Threadripper 1950x. That kind defeats the purpose of the -ck patchset, I'll see if I can find time to isolate which patch is responsible for this.

Last edited by inglor (2018-02-14 20:29:15)

Offline

#8 2018-02-14 21:06:12

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

With NUMA enabled and MuQSS disabled it will be the 0001-MuQSS-version-0.162-CPU-scheduler.patch  from the broken out patch set http://ck.kolivas.org/patches/4.0/4.14/ … 1/patches/

Offline

#9 2018-02-14 21:35:04

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

Not sure how many new variables your very new AMD chip adds to the mix since many other users are running fine as-is... just FYI, at last check, CK recommended disabling NUMA for single socket PCs under normal or MuQSS.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#10 2018-02-14 22:07:15

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

@graysky I thought threadripper was native NUMA but had a firmware switch/mode to emulate UMA?
Edit:
@graysky maybes you can figure out https://bbs.archlinux.org/viewtopic.php?id=234278
@inglor how many NUMA nodes does lscpu list?

Last edited by loqs (2018-02-14 22:19:03)

Offline

#11 2018-02-14 23:02:07

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

graysky wrote:

Not sure how many new variables your very new AMD chip adds to the mix since many other users are running fine as-is... just FYI, at last check, CK recommended disabling NUMA for single socket PCs under normal or MuQSS.

I rebuild once more with NUMA disabled and MuQSS disabled and I can boot fine.

loqs wrote:

@inglor how many NUMA nodes does lscpu list?

Enabled NUMA:

~$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD Ryzen Threadripper 1950X 16-Core Processor
Stepping:            1
CPU MHz:             3386.016
CPU max MHz:         3400.0000
CPU min MHz:         2200.0000
BogoMIPS:            6786.46
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           64K
L2 cache:            512K
L3 cache:            8192K
NUMA node0 CPU(s):   0-31
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2
ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

Disabled NUMA:

~$ cat disabled_numa.log
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               1
Model name:          AMD Ryzen Threadripper 1950X 16-Core Processor
Stepping:            1
CPU MHz:             1875.160
CPU max MHz:         3400.0000
CPU min MHz:         2200.0000
BogoMIPS:            6787.22
Virtualization:      AMD-V
L1d cache:           32K
L1i cache:           64K
L2 cache:            512K
L3 cache:            8192K
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2
ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb hw_pstate sme vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca

Offline

#12 2018-02-15 07:52:05

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

@graysky I thought threadripper was native NUMA but had a firmware switch/mode to emulate UMA?

I missed that, thanks for reminding me. There's a UEFI setting to make the CPU NUMA but for now it's considered a UMA.

UEFI setting wrote:

Memory interleaving
Controls fabric level memory interleaving (AUTO, none, channel, die, socket). Note that
channel, die, and socket has requirements on memory populations and it will be ignored if
the memory doesn't support the selected option.

There's a nice description in [1] page 3
[1]: https://www.anandtech.com/show/11697/th … x-review/3

Offline

#13 2018-02-15 08:43:37

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

With Memory interleaving set to Channel which assigns 4 banks for DDRAM to a die (left and right), forcing this CPU to be seen as NUMA this comes as:
Kernel with NUMA enabled (-ARCH)

~$ diff enabled_numa.log enabled_numa_and_uefi.log
9c9
< NUMA node(s):        1
---
> NUMA node(s):        2
15c15
< CPU MHz:             3174.481
---
> CPU MHz:             3440.128
18c18
< BogoMIPS:            6786.46
---
> BogoMIPS:            6786.60
24c24,25
< NUMA node0 CPU(s):   0-31
---
> NUMA node0 CPU(s):   0-7,16-23
> NUMA node1 CPU(s):   8-15,24-31

Kernel with NUMA disabled (-ck)

 ~$ diff disabled_numa.log disabled_numa_and_uefi.log
14c14
< CPU MHz:             1875.160
---
> CPU MHz:             2146.374
17c17
< BogoMIPS:            6787.22
---
> BogoMIPS:            6786.87

Offline

#14 2018-02-15 09:45:48

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

I’m thinking you should start a new thread since your issue is not related to [repo-ck].  Maybe ask a mod to split out your previous posts to keep this one cleaner?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#15 2018-02-15 11:29:58

V1del
Forum Moderator
Registered: 2012-10-16
Posts: 21,427

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

Split off off-topic discussion from https://bbs.archlinux.org/viewtopic.php?id=111715

Offline

#16 2018-02-15 11:40:43

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

I should probably raise this with ck on his blog as this is related to MuQSS and the only way to boot this is with disable MuQSS.

Offline

#17 2018-02-15 12:43:55

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

Have you tried building 4.14.0 with just the MuQSS patch applied and nothing else from linux-ck?
I might wait until ck releases MuQSS for 4.15 he has not responded to MuQSS being incompatible with 4.14.15 3 weeks ago but does seem to check his blog for a short period of time after new releases.
I have also had no success with emailing CK but he does seem to respond to graysky's emails.
As an alternative to MuQSS https://github.com/cchalpha/linux-gc/co … 4.14.y-pds there is also a 4.15 branch.

Last edited by loqs (2018-02-15 13:33:40)

Offline

#18 2018-02-15 14:11:55

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

Have you tried building 4.14.0 with just the MuQSS patch applied and nothing else from linux-ck?

I'll give it a go when got home today.

loqs wrote:

I might wait until ck releases MuQSS for 4.15 he has not responded to MuQSS being incompatible with 4.14.15 3 weeks ago but does seem to check his blog for a short period of time after new releases.
I have also had no success with emailing CK but he does seem to respond to graysky's emails.

I think he's busy Cryptomining.. big_smile

As an alternative to MuQSS https://github.com/cchalpha/linux-gc/co … 4.14.y-pds there is also a 4.15 branch.

Any more information on PDS{-MQ} (or VRQ)? Is it a spin off Brain Fuck Scheduler ?

Offline

#19 2018-02-15 15:45:46

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

Offline

#20 2018-02-15 20:10:03

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

V1del wrote:

Split off off-topic discussion from https://bbs.archlinux.org/viewtopic.php?id=111715

Thanks!


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#21 2018-02-15 20:19:12

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

Have you tried building 4.14.0 with just the MuQSS patch applied and nothing else from linux-ck?

I built the 4.14-1 from linux-ck and same results - not booting. I used commit '0c559ba9d5f531db022615f73181db24dd0293a2'

$ git log --oneline | grep " 4.14-1"
0c559ba Update to 4.14-1

Patches included:

  "https://raw.githubusercontent.com/graysky2/kernel_gcc_patch/master/enable_additional_cpu_optimizations_for_gcc_v4.9%2B_kernel_v4.13%2B.patch"
  "http://ck.kolivas.org/patches/4.0/4.14/4.14-ck1/patch-4.14-ck1.xz"
  "0001-platform-x86-hp-wmi-Fix-tablet-mode-detection-for-co.patch"

Offline

#22 2018-02-15 20:53:30

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

I would have used just http://ck.kolivas.org/patches/4.0/4.14/ … uler.patch to rule out any interaction from the other patches.
Would suggest trying that or PDS for 4.14 or 4.15.
Edit:
@graysky do you know of any linux-ck / MuQSS users using an AMD Threadripper / EPYC system?

Last edited by loqs (2018-02-15 21:45:43)

Offline

#23 2018-02-16 07:18:23

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

I would have used just http://ck.kolivas.org/patches/4.0/4.14/ … uler.patch to rule out any interaction from the other patches.
Would suggest trying that or PDS for 4.14 or 4.15.

Linux 4.14 with ck1 patch - unable to boot also.

I've build -ARCH (4.15.3) with PDS patch and works fine so I will stick with that for now.

~$ dmesg| grep PDS
[    0.000000] pds: PDS-mq CPU Scheduler 0.98k by Alfred Chen.

Edit: AUR package
https://aur.archlinux.org/packages/linux-gc/

Last edited by inglor (2018-02-17 10:15:56)

Offline

#24 2018-02-18 17:56:16

loqs
Member
Registered: 2014-03-06
Posts: 17,195

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

linux-ck 4.15 is out if you want to see if the issue continues.

Offline

#25 2018-02-18 22:03:43

inglor
Package Maintainer (PM)
Registered: 2008-07-22
Posts: 88

Re: [Solved] AMD Threadripper 1950x and MuQSS not booting (-ck patchset)

loqs wrote:

linux-ck 4.15 is out if you want to see if the issue continues.

linux-ck 4.15.x boot successfully smile

$ dmesg | grep MuQSS
[    0.170159] MuQSS runqueue share type MC total runqueues: 97
[    0.790228] MuQSS CPU scheduler v0.170 by Con Kolivas.

I'll mark this as SOLVED.

Offline

Board footer

Powered by FluxBB