unofficial linux-ck repo with cpu optimized packages is ONLINE

loqs · 2017-10-31 19:50:39

I would probably switch CONFIG_SCSI_MQ_DEFAULT back to Y but as already noted that can be done with the config option scsi_mod.use_blk_mq=1 as boot option
This covers as much as any udev rule can either way bfq must be available as a scheduler option for that queue meaning the queue must be using mq.
The problem is that 4.12+ has two sets of elevators cfq,deadline,noop which can only be accessed when the queue is not mq
and another set bfq,kyber,mq-deadline,none that can only be accessed when the queue is mq.
Perhaps numerous kernels later every queue will be mq and the choice will be down to whichever elevators have then been ported to mq.
Any word from CK if he has started on MuQSS for 4.14? Looks like the scheduler system had a fair amount of changes with 4.14.
Edit:
With 4.14 would switch to a bigger patch that includes upstreams fix for the oops as well as changing the deafault mq scheduler to bfq
but for 4.13 either leave as or just switch CONFIG_SCSI_MQ_DEFAULT unless users report issues from caused by the change.
Edit2:
Linux-ck#How_to_enable_the_BFQ_I.2FO_Scheduler probably needs updating with whatever solution you decide upon.

Last edited by loqs (2017-10-31 22:36:06)

graysky · 2017-11-01 07:38:58

@loqs - CK usually waits for the release before porting MuQSS. I will leave things as they are with 4.13.10-3. Please feel free to update the wiki you linked.

inglor · 2017-11-01 08:40:22

loqs wrote:

The problem is that 4.12+ has two sets of elevators cfq,deadline,noop which can only be accessed when the queue is not mq
and another set bfq,kyber,mq-deadline,none that can only be accessed when the queue is mq.
Perhaps numerous kernels later every queue will be mq and the choice will be down to whichever elevators have then been ported to mq.
Any word from CK if he has started on MuQSS for 4.14? Looks like the scheduler system had a fair amount of changes with 4.14.

@loqs how does the two different sets of elevators assigned to a block device? I understand your patch just overriding any rules and assign bfq but I'd be interested to know the rules of the default behaviour if you know.

It seems though all this discussion is upstream issue, the bfq elevator should have a way to enable it as it's merged already in kernel, just need to make sure the -ck patchset doesn't disable this.

Last edited by inglor (2017-11-01 11:18:43)

loqs · 2017-11-01 12:01:41

From block/elevator.c from linux 4.13 unpatched

	/*
	 * Use the default elevator specified by config boot param for
	 * non-mq devices, or by config option. Don't try to load modules
	 * as we could be running off async and request_module() isn't
	 * allowed from async.
	 */
	if (!e && !q->mq_ops && *chosen_elevator) {
		e = elevator_get(chosen_elevator, false);
		if (!e)
			printk(KERN_ERR "I/O scheduler %s not found\n",
							chosen_elevator);
	}

As the comment says block-mq devices can not use the elevator boot parameter
The OOPS case is that you can try and set the elevator to bfq a mq elevator when the device is not block-mq

	if (!e) {
		/*
		 * For blk-mq devices, we default to using mq-deadline,
		 * if available, for single queue devices. If deadline
		 * isn't available OR we have multiple queues, default
		 * to "none".
		 */
		if (q->mq_ops) {
			if (q->nr_hw_queues == 1)
				e = elevator_get("mq-deadline", false);
			if (!e)
				return 0;
		} else
			e = elevator_get(CONFIG_DEFAULT_IOSCHED, false);

		if (!e) {
			printk(KERN_ERR
				"Default I/O scheduler not found. " \
				"Using noop.\n");
			e = elevator_get("noop", false);
		}
	}

As the comment says for block-mq devices:
the default scheduler from the .config is not used this is why CK's patch failed
mq-deadline if available is used for single queue devices this in practise means devices using scsi_mod.use_blk_mq=1
If deadline isn't available OR we have multiple queues, default to "none" this means NVME devices get none.
for not block-mq devices:
use the defeault scheduler from the .config in linux-ck that is cfq
the OOPS case with CK's patch was the .config specified bfq a mq elevator but the device is not block-mq
What the patch does is for block-mq devices:
instead of picking none or deadline use bfq
So what is a block-mq device NVME's and devices that use the scsi subsystem when scsi_mod.use_blk_mq=1
What is not a block-mq device floppy disk, sd-card and probably various others.
Edit:
So how could everything be able to use bfq?
Coverage is already supplied for block-mq devices but those devices that do not use block-mq only have a choice of cfq,deadline or noop.
The old version of bfq which only supports not block-mq devices would need to be added back under a different name (schedulers can only be block-mq supporting or not they can not support both types in one)
Edit2:
For 4.14 potential patches https://github.com/loqs/linux-4.14-patches this would add support for bfq even on devices that do not use block-mq.

Last edited by loqs (2017-11-03 18:29:39)

kazimir · 2017-11-04 18:07:06

halocaridina wrote:

@graysky and others,
I've run into a reproducible kernel oops with linux-ck-sandybridge-4.13.10-1 upon insertion of an SD card into the onboard reader. Machine is a Lenovo X220, which continues to run following the oops but cannot mount the SD card and the system load across 1, 5 and 15 mins climbs to, and remains at, ~2.0. The machine immediately hangs on shutdown once this happens, requiring a hard power off to go completely down.

I had a similar issue with a thinkpad x230, the machine would fail to boot with a SD card in the onboard reader. Removing the card would cause kernel oops. I went back the vanilla kernel to be able to use my computer as none of the 4.12 ck kernels worked on my x230.

I just upgraded to the latest 4.13.11-1 and it hangs during boot at Starting TLP system startup/shutdown... not sure what I can do to provide more information here.

loqs · 2017-11-04 19:02:15

kazimir wrote:

I just upgraded to the latest 4.13.11-1 and it hangs during boot at Starting TLP system startup/shutdown...

The issue is present on the linux-ck 4.13.11-1 but not not linux 4.13.11-1?

cowlick · 2017-11-04 19:10:57

Boot failure. Really screwed the pooch this time.

No more boot failure. Thanks to whoever fixed it.

Last edited by cowlick (2017-11-08 22:05:39)

kazimir · 2017-11-04 19:26:44

loqs wrote:

kazimir wrote:
I just upgraded to the latest 4.13.11-1 and it hangs during boot at Starting TLP system startup/shutdown...
The issue is present on the linux-ck 4.13.11-1 but not not linux 4.13.11-1?

Good question !
I installed linux 4.13.11-1 to try and yes it has the same error with the exact same message.
So probably unrelated to linux-ck, I assumed it was as almost none of 4.12 linux-ck worked with x230.

DamDam · 2017-11-08 19:28:48

Hi,

I'm using the piledriver packages of linux-ck but the last working version for me is 4.13.10-1 with 4.13.10-2 and later i also got

BUG: unable to handle kernel NULL pointer dereference at 0000000000000014

loqs · 2017-11-08 19:35:43

@DamDam what kernel boot options does that system use, in particular does it use the elevator parameter?
Edit:
Oh and welcome to the arch linux forums.

Last edited by loqs (2017-11-08 19:36:03)

DamDam · 2017-11-08 19:51:12

@loqs

Thanks for this fast reply and sorry that I did not introduce myself. I'm a silent read since years and also using the linux-ck packages all the time without any problems. Thanks to all who work on it!

I'm using the following kernel parameters:

GRUB_CMDLINE_LINUX_DEFAULT="quiet elevator=bfq"

But to be honest, I never tried to remove the elevator parameter ...

loqs · 2017-11-08 19:53:56

That will be the cause it is intended to be fixed in mainline in 4.15, it may be fixed in linux-ck in 4.14 depending on what patches CK or graysky will accept.
Edit:
If you want bfq and your devices are SATA/SCSI you can use scsi_mod.use_blk_mq=1 instead of the elevator parameter.

Last edited by loqs (2017-11-08 19:56:21)

graysky · 2017-11-08 20:45:50

Please be aware of: https://www.archlinux.org/news/the-end-of-i686-support

Since linux-lts did it with 4.9.61-1, I did as well with 4.13.12-2.

Last edited by graysky (2017-11-08 20:46:04)

kazimir · 2017-11-09 10:34:14

Just reporting that I upgraded to the latest 4.13.12-2 which fixed the boot hanging at 'Starting TLP system startup/shutdown' on my thinkpad x230.

agm28011997 · 2017-11-10 21:17:53

inglor wrote:

agm28011997 wrote:
[..] are there any plans to support the architecture of btver2, ryzen and canonlake?[..]
Currently btver2 is supported, see Repo-ck. The ryzen depends on GCC version (older versions don't support it) but for GCC 6.3+ you could export these prior to makepkg.
export CFLAGS="-march=znver1 -O2"
export CXXFLAGS="${CFLAGS}"
There are some other details for older GCC versions in Gentoo wiki. Overall it seems Ryzen still has some issues on Linux. Gentoo Troubleshoot section of Ryzen (above link) suggests there's a Gentoo and Ryzen config and stability questionnaire which has some very recent answers records so I'd be careful with Ryzen.
As for cannonlake I don't think GCC supports optimizations for them, yet (I'd probably use native if I had one.

I don't see btver2, only bdver2 that is different, and the same for other like btver3 or i don't remember

agm28011997 · 2017-11-10 21:20:03

graysky wrote:

@inglor - Thanks for the reply to agm28011997's questions. To summarize:
1) I just added a new package group, ck-zen, that will build with the gcc's znver1 tunings. Should be online soon.
2) GCC 7.2 does not yet have anything unique to canonlake. If you have one of these systems, please post the output of this command: gcc -c -Q -march=native --help=target | grep march

i've seen the zen packages, I don't have cannonlake or zen cpu's they were questions only, but we(friends and me) have laptops and desktops pc with amd kaveri 7850K and laptops with amd apu's btver2 and 3, it is the cause of my question, not for me only but for them

inglor · 2017-11-11 11:40:54

agm28011997 wrote:

[..] but we(friends and me) have laptops and desktops pc with amd kaveri 7850K and laptops with amd apu's btver2 and 3, it is the cause of my question, not for me only but for them

For each CPU you mention can you post the outputs of:

$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
$ gcc -c -Q -march=native --help=target | grep march

For example mine would be:

~$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id : GenuineIntel
cpu family : 6
model : 94
model name : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
~$ gcc -c -Q -march=native --help=target | grep march
-march= skylake

Last edited by inglor (2017-11-11 11:43:01)

eschwartz · 2017-11-11 23:19:31

graysky wrote:

Please be aware of: https://www.archlinux.org/news/the-end-of-i686-support
Since linux-lts did it with 4.9.61-1, I did as well with 4.13.12-2.

A part of me wants to say "please keep i686 in at least until you actually have to deal with wrangling a new config.i686 and don't have any i686 machines to test".
https://archlinux32.org is still here, and it would be nice if the vast majority of AUR packages which work fine on i686 did not get dropped, especially in the case of a kernel where kernel configs are not intuitive to create from scratch.

agm28011997 · 2017-11-12 11:16:57

inglor wrote:

agm28011997 wrote:
[..] but we(friends and me) have laptops and desktops pc with amd kaveri 7850K and laptops with amd apu's btver2 and 3, it is the cause of my question, not for me only but for them
For each CPU you mention can you post the outputs of:
$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
$ gcc -c -Q -march=native --help=target | grep march
For example mine would be:
~$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id : GenuineIntel
cpu family : 6
model : 94
model name : Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz
~$ gcc -c -Q -march=native --help=target | grep march
-march= skylake

there you can view that the outputs are others than he has in the forum page and repo:
there the a10-7850K
[ccm@ccm-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 48
model name      : AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G
[ccm@ccm-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march=        bdver3

and there the laptop apu E-1 6010:
[marcela@marcela-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id : AuthenticAMD
cpu family : 22
model : 48
model name : AMD E1-6010 APU with AMD Radeon R2 Graphics
[marcela@marcela-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march= btver2

two cpu's are one version more than the versions are in the repo: btver1 and bdver2, they are previous versions in the repo, but not these versions

Last edited by agm28011997 (2017-11-12 11:25:12)

inglor · 2017-11-12 11:34:43

agm28011997 wrote:

there you can view that the outputs are others than he has in the forum page and repo:
there the a10-7850K
[ccm@ccm-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 48
model name      : AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G
[ccm@ccm-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march=        bdver3
and there the laptop apu E-1 6010:
[marcela@marcela-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id : AuthenticAMD
cpu family : 22
model : 48
model name : AMD E1-6010 APU with AMD Radeon R2 Graphics
[marcela@marcela-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march= btver2
two cpu's are one version more than the versions are in the repo: btver1 and bdver2, they are previous versions in the repo, but not these versions

Seems like the versions for Excavator,Steamroller and Jaguar/Puma CPUs/APUs are missing (march optimizations bdver4,bdver3 and btver2 respectively).

agm28011997 · 2017-11-12 11:44:28

inglor wrote:

agm28011997 wrote:
there you can view that the outputs are others than he has in the forum page and repo:
there the a10-7850K
[ccm@ccm-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 48
model name      : AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G
[ccm@ccm-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march=        bdver3
and there the laptop apu E-1 6010:
[marcela@marcela-pc ~]$ grep -m1 -A3 "vendor_id" /proc/cpuinfo
vendor_id : AuthenticAMD
cpu family : 22
model : 48
model name : AMD E1-6010 APU with AMD Radeon R2 Graphics
[marcela@marcela-pc ~]$ gcc -c -Q -march=native --help=target | grep march
-march= btver2
two cpu's are one version more than the versions are in the repo: btver1 and bdver2, they are previous versions in the repo, but not these versions
Seems like the versions for Excavator,Steamroller and Jaguar/Puma CPUs/APUs are missing (march optimizations bdver4,bdver3 and btver2 respectively).

A4/A6/A8/A10-7XXX/8XXX (Steamroller)

Kaveri and Godavari
vendor_id : AuthenticAMD
cpu family : 21
model : 48 or 56
model name : AMD A10-7850K Radeon R7, 12 Compute Cores 4C+8G
Important
Various lowpower AMD APUs branded as AX-7XXX (eg. A4-7210) don't belong to cpu family 21!
FILE /etc/portage/make.conf
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=bdver3 -pipe"
CXXFLAGS="${CFLAGS}"
E1/E2-XXXX, A4/A6/A8/A10-XXXX (Jaguar, Puma)

Kabini, Temash, Beema, Mullins, and Carrizo-L
vendor_id : AuthenticAMD
cpu family : 22
model : 0 or 48
model name : AMD A4-5000 APU with Radeon(TM) HD Graphics
Important
Majority of AMD APUs branded as A4/A6/A8-XXXX (eg. A4-5300, A4-4000 or A6-5345M) don't belong to cpu family 22! They should use settings listed for AMD APU model given by the cpu family and model.
FILE /etc/portage/make.conf
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=btver2 -pipe"
CXXFLAGS="${CFLAGS}"

this is writen in gentoo wiki
and yes, it is the cause for ask why are not in the repos too, if it is technical or simply statics make him to leave its support

Last edited by agm28011997 (2017-11-12 11:45:42)

DamDam · 2017-11-12 21:43:41

Now I'm using the kernel boot parameter mention by loqs and linux-ck 4.13.12-2 is working fine now! Thanks for it!

agm28011997 · 2017-11-14 10:15:35

will you make a version with that architectures in the future ? I think it is better to lose older versions than newer ones.. but it is my opinion and it is your work, and much thanks for this again, I apreciate so much this repo

loqs · 2017-11-15 17:03:35

@graysky emailed my port of MuQSS to 4.14 to CK incase its of any help to him it compiles boot and runs without error in my limited testing.

halocaridina · 2017-11-16 17:49:41

@graysky and loqs,

Apologies in advance for the late report. Currently on

4.13.13-1-ck-sandybridge

with kernel option

scsi_mod.use_blk_mq=1

to enable BFQ and can not reproduce the kernel oops that I experienced previously (post #3139).

Thanks again,

Halocaridina

Last edited by halocaridina (2017-11-16 17:50:49)

Arch Linux

#3151 2017-10-31 19:50:39

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3152 2017-11-01 07:38:58

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3153 2017-11-01 08:40:22

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3154 2017-11-01 12:01:41

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3155 2017-11-04 18:07:06

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3156 2017-11-04 19:02:15

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3157 2017-11-04 19:10:57

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3158 2017-11-04 19:26:44

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3159 2017-11-08 19:28:48

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3160 2017-11-08 19:35:43

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3161 2017-11-08 19:51:12

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3162 2017-11-08 19:53:56

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3163 2017-11-08 20:45:50

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3164 2017-11-09 10:34:14

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3165 2017-11-10 21:17:53

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3166 2017-11-10 21:20:03

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3167 2017-11-11 11:40:54

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3168 2017-11-11 23:19:31

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3169 2017-11-12 11:16:57

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3170 2017-11-12 11:34:43

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3171 2017-11-12 11:44:28

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3172 2017-11-12 21:43:41

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3173 2017-11-14 10:15:35

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3174 2017-11-15 17:03:35

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

#3175 2017-11-16 17:49:41

Re: unofficial linux-ck repo with cpu optimized packages is ONLINE

Board footer