You are not logged in.

#1 2013-07-12 12:52:17

BahbGTR
Member
From: Kentucky
Registered: 2013-07-12
Posts: 19

[Solved] Haswell-ck and Intel P State

Yesterday when I was doing my usual update, my kernel updated from 3.9.9-1-ck to 3.10.0-1-ck.  For 3.9.9-1-ck I was using the ondemand governor.  For 3.10, I figured I would try out the intel P-state driver.  I might be wrong about this, but I thought the intel pstate driver supported Haswell as of 3.10.  Ondemand worked fine, but I didn't have turbo mode with it.

In 3.10.0-1-ck with intel pstate, frequency scaling and turbo both work.  The problem is that the CPU never goes below its rated clock speed (3.4 GHz).  It's like the min_perf_pct file under /sys/devices/system/cpu/intel_pstate/ is unrecognized or being ignored.  Just for fun, I changed the value in max_perf_pct to 80 and my CPU's frequency dropped to 80% of its rated speed.  The default value in the min_perf_pct file is 21.  The minimum rated speed for this processor is 800 MHz.  I've tried setting the file to 95, 80, 24 and none of them work.  It just stays pegged at 3.4 GHz unless turbo kicks in.  I do not have any CPU intensive tasks running when I play around with this.

The motherboard and CPU are relatively new, so I'm working out the bugs.  For what it's worth, this is a UEFI system

Relevant specs:
Intel Xeon E3 1245v3
ASRock Z87 Xtreme6
20 GB DDR3-1666
Nvidia 560 Ti  (update also brought in 3.10 compatible 319 drivers)

Any help would be greatly appreciated!  Recent convert from Gentoo/Funtoo and loving Arch so far!

Last edited by BahbGTR (2013-07-14 02:05:14)


Xeon E3 1245v3  |  ASRock Z87 Extreme6  |  20 GB DDR3-1666  |  EVGA Nvidia GTX 560 ti |  MSI Nvidia GTX 760 (passthrough)  |  256 GB Samsung 850 Pro | 120 GB Samsung 830  |  300 GB WD Velociraptor  |  2 TB WD Black

Offline

#2 2013-07-12 13:39:31

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

The question is not where the frequency sits, but rather how much power it is consuming.  The processor can change frequencies in the range of microseconds, so the frequency at which it sits is not really "sitting" at all.  Can you use i7z to see what c-state the processors are sitting in most of the time?

My processor (Core i5-3210M) says that it is at 25500 MHz all the time, which is funny because the advertised non-turbo max is 25000 MHz.  But if I look at i7z while it is not under load, it tells me that the processors are actually sitting in C7 for >95% of the time.

So in reality, my system is using a bit less power than before simple because it is better managed with intel_pstate than ondemand ever could.

Offline

#3 2013-07-12 13:40:32

fsckd
Forum Moderator
Registered: 2009-06-15
Posts: 3,538

Re: [Solved] Haswell-ck and Intel P State

Hi BahbGTR, the -ck kernels are unofficial. I would merge this with the thread unofficial linux-ck repo with cpu optimized packages is ONLINE but first I need a question answered. Have you tried the kernel from [testing] and does the problem exist with that too?

P.S. welcome to the forums. smile

Edit: Merged. Title was "Haswell-ck and Intel P State". Good luck.

Edit2: After discussing with graysky, I've decided to "unmerge" the thread. "unofficial linux-ck repo with cpu optimized packages is ONLINE" is for issues regarding the repo itself. The problem regarding haswell processors is better suited for its own thread. Sorry for any confusion or inconveniences caused.


aur S & M :: forum rules :: Community Ethos
Resources for Women, POC, LGBT*, and allies

Offline

#4 2013-07-12 14:00:08

BahbGTR
Member
From: Kentucky
Registered: 2013-07-12
Posts: 19

Re: [Solved] Haswell-ck and Intel P State

fsckd,  thanks smile  Feel free to merge this.  I have not tried the testing kernel.

WonderWoofy wrote:

  Can you use i7z to see what c-state the processors are sitting in most of the time?

Here is the output I get at idle

Socket [0] - [physical cores=4, logical cores=8, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading ON
  Max Frequency without considering Turbo 3497.94 MHz (99.94 x [35])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  38x/38x/37x/36x
  Real Current Frequency 3400.02 MHz [99.94 x 34.02] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %
        Core 1 [0]:       3398.94 (34.01x)         1         98.4       1       0
        Core 2 [1]:       3399.47 (34.01x)         1         97.6       1       1
        Core 3 [2]:       3400.02 (34.02x)         1         99          1       0
        Core 4 [3]:       3322.61 (33.25x)         1         100        0       0

C0 = Processor running without halting
C1 = Processor running with halts (States >C0 are power saver)
C3 = Cores running with PLL turned off and core cache turned off
C6 = Everything in C3 + core state saved to last level cache

Last edited by BahbGTR (2013-07-12 14:03:35)


Xeon E3 1245v3  |  ASRock Z87 Extreme6  |  20 GB DDR3-1666  |  EVGA Nvidia GTX 560 ti |  MSI Nvidia GTX 760 (passthrough)  |  256 GB Samsung 850 Pro | 120 GB Samsung 830  |  300 GB WD Velociraptor  |  2 TB WD Black

Offline

#5 2013-07-12 14:16:38

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

Oh... that is probably not cool.  The addition of processors to the supported list of intel_ptate comes as intel develops the support for these given processors.  With 3.9, we were able to (mostly) successfully use it for IVB, but I would guess that because the haswell processors are so different than the previous two generations, it might be with good reason that it is not yet included as supported.  The haswell processors actually have the ability to go into two(?) lower c states than the previous generations, amongst other features.  So maybe things are just not yet ready...

(Of course this is just a total guess and I have not had the change/opportunity to play with a haswell processor yet)

Offline

#6 2013-07-12 14:25:03

BahbGTR
Member
From: Kentucky
Registered: 2013-07-12
Posts: 19

Re: [Solved] Haswell-ck and Intel P State

WonderWoofy,  I appreciate the effort!  I'll give the official testing kernel a shot and see if there is any difference.  I'm not really clear either if these CPUs are supported yet either.  I want to get away from the ondemand driver though.  The system as it is with intel_pstate idles at 30 C (~26 C ambient) and electricity is cheap around here.  I'll stick with the intel_pstate driver for now since I prefer to have access to the turbo modes that ondemand wasn't handling well.


Xeon E3 1245v3  |  ASRock Z87 Extreme6  |  20 GB DDR3-1666  |  EVGA Nvidia GTX 560 ti |  MSI Nvidia GTX 760 (passthrough)  |  256 GB Samsung 850 Pro | 120 GB Samsung 830  |  300 GB WD Velociraptor  |  2 TB WD Black

Offline

#7 2013-07-12 15:01:12

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

Alternatively, you could try using the performance governor.  Modern processors are able to regulate themselves, so gone are the days when something like ondemand is really required for the processor to be effective in power consumption.  You should probably check out this thread and follow the links included there.

BTW, is the CONFIG_NO_HZ_FULL also set in the kernel? There is some evidence that the combination of intel_pstate and the full tickless mode together can cause some issues (not unlike what you are describing).

Offline

#8 2013-07-12 15:14:16

BahbGTR
Member
From: Kentucky
Registered: 2013-07-12
Posts: 19

Re: [Solved] Haswell-ck and Intel P State

WonderWoofy wrote:

BTW, is the CONFIG_NO_HZ_FULL also set in the kernel? There is some evidence that the combination of intel_pstate and the full tickless mode together can cause some issues (not unlike what you are describing).

I don't think so.  But I can't find that option in the config.  Here's the pertinent snip of the config file

# Timers subsystem
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y

Xeon E3 1245v3  |  ASRock Z87 Extreme6  |  20 GB DDR3-1666  |  EVGA Nvidia GTX 560 ti |  MSI Nvidia GTX 760 (passthrough)  |  256 GB Samsung 850 Pro | 120 GB Samsung 830  |  300 GB WD Velociraptor  |  2 TB WD Black

Offline

#9 2013-07-12 16:29:07

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,450
Website

Re: [Solved] Haswell-ck and Intel P State

Note that the ck patchset disables some options.  See CK's blog for more.  To the haswell pstates issue... I will post the output of i7z when I return home.  I don't have a haswell, but I do have an ivy for comparision.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#10 2013-07-12 18:48:24

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,450
Website

Re: [Solved] Haswell-ck and Intel P State

Cpu speed from cpuinfo 3500.00Mhz
cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc
Linux's inbuilt cpu_khz code emulated now
True Frequency (without accounting Turbo) 3500 MHz
  CPU Multiplier 35x || Bus clock frequency (BCLK) 100.00 MHz

Socket [0] - [physical cores=4, logical cores=8, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading ON
  Max Frequency without considering Turbo 3600.00 MHz (100.00 x [36])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  45x/45x/45x/45x
  Real Current Frequency 3138.23 MHz [100.00 x 31.38] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %   C7 %  Temp
        Core 1 [0]:       2466.73 (24.67x)         1    99.9       0       0       0    27
        Core 2 [1]:       2620.46 (26.20x)         1    99.9       0       0       0    26
        Core 3 [2]:       3138.23 (31.38x)         1    99.8       0       0       0    30
        Core 4 [3]:       2609.41 (26.09x)         1    99.8       0       0       0    24



C0 = Processor running without halting
C1 = Processor running with halts (States >C0 are power saver)
C3 = Cores running with PLL turned off and core cache turned off
C6 = Everything in C3 + core state saved to last level cache
  Above values in table are in percentage over the last 1 sec
[core-id] refers to core-id number in /proc/cpuinfo
'Garbage Values' message printed when garbage values are read
  Ctrl+C to exit

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#11 2013-07-12 19:34:37

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,450
Website

Re: [Solved] Haswell-ck and Intel P State

BahbGTR wrote:

I'll give the official testing kernel a shot and see if there is any difference.

I am pretty sure that haswell support is not in upstream for pstates.  It is in linux-ck due to the little patch making drivers/cpufreq/intel_pstate.c  aware of haswell processors.  FYI - you may disable this patch in the PKGBUILD if you wish to revert back to the older governors.

As I read the output of your `i7z` command, your turbo mode multiplier are indeed available to you, but you or your BIOS has them set on a per-CPU basis asymmetrically.  On my i7-3660k, I have all 4 cores manually set to 45x.  You appear to have cores 0-1 set to 38x, 2 set to 37x, and 3 set to 36x:

Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  38x/38x/37x/36x

Here is mine:

Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  45x/45x/45x/45x

Does the output change if you start compiling or start mprime?  Mine does:

Idle snapshot (note that the current freq bounces around from 2.4 GHz - 3.8 GHz at idle):

Real Current Frequency 2474.72 MHz [100.00 x 24.75] (Max of below)

Load:

Real Current Frequency 4500.01 MHz [100.00 x 45.00] (Max of below)

Last edited by graysky (2013-07-12 19:39:33)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#12 2013-07-12 20:45:00

big_bum
Member
From: Bucharest,Romania,Europe,World
Registered: 2011-08-29
Posts: 47

Re: [Solved] Haswell-ck and Intel P State

I have a Haswell precessor and yes, I can confirm that the minimum freq is the one stated on the processor's spec: 3.2GHz. My guess is that it's an upstream bug?

Offline

#13 2013-07-12 20:58:40

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

It would be an upstream bug if this were actually implemented by upstream.  But in this case, graysky has simply added a patch to the kernel so that those interested in trying it can do so. 

You should just disable the intel_pstate drive if you are having problems with it, and your system should fall back to the old ondemand method.  Then you'll just have to wait until it is actually ready for haswell.

Offline

#14 2013-07-12 22:39:09

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,450
Website

Re: [Solved] Haswell-ck and Intel P State

WonderWoofy wrote:

It would be an upstream bug if this were actually implemented by upstream.  But in this case, graysky has simply added a patch to the kernel so that those interested in trying it can do so. 

You should just disable the intel_pstate drive if you are having problems with it, and your system should fall back to the old ondemand method.  Then you'll just have to wait until it is actually ready for haswell.

+1.  It seems that 3.10.1 will be released tomorrow after 3:45 PM ES.  I will be disabling the auto compilation of the Haswell patch there.  I just disabled it by default in 3.10-2 which I will NOT be rebuilding for the repo since 3.10.1 is just around the corner.

Last edited by graysky (2013-07-13 09:29:52)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#15 2013-07-13 14:49:39

DougQuaid
Member
Registered: 2013-07-13
Posts: 3

Re: [Solved] Haswell-ck and Intel P State

I have similar problem, my haswell is running on max turbo frequency all the time with intel_pstate, current version of i7z doesn't support haswell and falls back to nehalem settings (hence the missing C7 state). I made small modification to the source, so it will use the same settings as for ivy bridge for haswell and the cpu is mostly in C7 when idle

Cpu speed from cpuinfo 3499.00Mhz
cpuinfo might be wrong if cpufreq is enabled. To guess correctly try estimating via tsc
Linux's inbuilt cpu_khz code emulated now
True Frequency (without accounting Turbo) 3499 MHz
  CPU Multiplier 35x || Bus clock frequency (BCLK) 99.97 MHz

Socket [0] - [physical cores=4, logical cores=8, max online cores ever=4]
  TURBO ENABLED on 4 Cores, Hyper Threading ON
  Max Frequency without considering Turbo 3598.97 MHz (99.97 x [36])
  Max TURBO Multiplier (if Enabled) with 1/2/3/4 Cores is  41x/41x/41x/41x
  Real Current Frequency 4099.74 MHz [99.97 x 41.01] (Max of below)
        Core [core-id]  :Actual Freq (Mult.)      C0%   Halt(C1)%  C3 %   C6 %   C7 %  Temp
        Core 1 [0]:       4099.11 (41.00x)      2.27       0       1       1      96    34
        Core 2 [1]:       4098.57 (41.00x)      1.57       0    4.01       1    93.9    37
        Core 3 [2]:       4098.48 (41.00x)         1    0.0336     0       0    99.9    33
        Core 4 [3]:       4099.74 (41.01x)         0    0.00636    0       0     100    35

Last edited by DougQuaid (2013-07-13 15:56:52)

Offline

#16 2013-07-13 15:47:50

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

@DougQuaid, first of all, I need to point out that your picture is way too big for these forums.  It is going to be removed.  So if you want to give potential readers the ability to reference it, you should probably post it elsewhere and link to it (possibly with a thumbnail using some nesting magic).  Though for something like this, it might be easier to simply copy the content of i7z and paste it into a code block.

Anyway, I also wanted to point out that even though you modified the source to use the IVB settings (which is pretty cool BTW), unfortunately the differences between IVB and Haswell are pretty significant.  In fact, Haswell has even more (deeper) sleep states that the cores can go into.  So I would be afraid that this is not giving you an even remotely accurate representation of what your machine is actually doing.  Unfortunately, the tools have not been able to keep up with the technology...

Offline

#17 2013-07-13 15:52:46

DougQuaid
Member
Registered: 2013-07-13
Posts: 3

Re: [Solved] Haswell-ck and Intel P State

as far as I know, only mobile haswells have deeper C-states, desktop ends at C7 (and this is desktop cpu)

Offline

#18 2013-07-13 16:05:35

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

DougQuaid wrote:

as far as I know, only mobile haswells have deeper C-states, desktop ends at C7 (and this is desktop cpu)

This I did not know... interesting.

Offline

#19 2013-07-14 02:02:16

BahbGTR
Member
From: Kentucky
Registered: 2013-07-12
Posts: 19

Re: [Solved] Haswell-ck and Intel P State

Just wanted to say that the latest ck release fixed the issues I have having.  Thanks everyone!


Xeon E3 1245v3  |  ASRock Z87 Extreme6  |  20 GB DDR3-1666  |  EVGA Nvidia GTX 560 ti |  MSI Nvidia GTX 760 (passthrough)  |  256 GB Samsung 850 Pro | 120 GB Samsung 830  |  300 GB WD Velociraptor  |  2 TB WD Black

Offline

#20 2013-07-14 02:35:04

WonderWoofy
Member
From: Los Gatos, CA
Registered: 2012-05-19
Posts: 8,412

Re: [Solved] Haswell-ck and Intel P State

So BahbGTR, just to clarify, you mean that you are back to ondemand now, yes?

Offline

#21 2013-07-30 13:17:45

big_bum
Member
From: Bucharest,Romania,Europe,World
Registered: 2011-08-29
Posts: 47

Re: [Solved] Haswell-ck and Intel P State

DougQuaid wrote:

I made small modification to the source, so it will use the same settings as for ivy bridge for haswell and the cpu is mostly in C7 when idle

Would you lke to share it wth us?

Offline

#22 2013-07-30 18:47:26

graysky
Member
From: /run/user/1000
Registered: 2008-12-01
Posts: 8,450
Website

Re: [Solved] Haswell-ck and Intel P State

BahbGTR wrote:

Just wanted to say that the latest ck release fixed the issues I have having.  Thanks everyone!

Just FYI, I haven't had the haswell patch enabled by default since you or someone reported problems... so if you're using the packages from the repo, Haswell pstates are not enabled.

Last edited by graysky (2013-07-30 18:47:52)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#23 2013-08-02 07:34:11

DougQuaid
Member
Registered: 2013-07-13
Posts: 3

Re: [Solved] Haswell-ck and Intel P State

big_bum wrote:

Would you lke to share it wth us?

I just added another case branch to helper_functions.c from

else if (proc_info.extended_model == 0x3) {
            switch (proc_info.model)
            {
            case 0xA:
                printf ("i7z DEBUG: Detected an ivy bridge processor\n");
                *nehalem = false;
                *sandy_bridge = true;
                break;

to

else if (proc_info.extended_model == 0x3) {
            switch (proc_info.model)
            {
            case 0xA:
                printf ("i7z DEBUG: Detected an ivy bridge processor\n");
                *nehalem = false;
                *sandy_bridge = true;
                break;
            case 0xC:
                printf ("i7z DEBUG: Detected a haswell processor\n");
                *nehalem = false;
                *sandy_bridge = true;
                break;

so it will detect desktop haswell cpu by model number (desktop haswells have 0x3c, mobile have 0x3f and low power models have 0x45 and 0x46, I suppose that 0x4x models support additional C-states)

Offline

#24 2013-08-07 12:51:53

big_bum
Member
From: Bucharest,Romania,Europe,World
Registered: 2011-08-29
Posts: 47

Re: [Solved] Haswell-ck and Intel P State

Thank you!

Offline

Board footer

Powered by FluxBB