You are not logged in.

#1 2015-02-21 17:16:37

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

EDIT: The problem seems to be with the Arch config not with the video driver or with Xorg.  Skip to post #7 where I made this discovery.

EDIT2: You can measure the mean and median frequency for your processor using a simple bash script I show in post #37.


Problem: My i7-4790k with onboard GPU on a fresh install of Arch idles at much higher CPU frequencies compared to the very same hardware booted into either Ubuntu 15.04 (vivid-desktop-amd64.iso 21-Feb-2015 08:35) or into Fedora 21.

Data: In order to support the new CPU, I ran i7z-git while idle in Xorg collecting about 10 min of CPU freq (i7z -w a to write log files).  I repeated this for Arch, Fedora, and Ubuntu.  When I plot them in simple histograms, there is a clear bias for higher frequencies under Arch (dark blue) compared to Ubuntu (brown) and Fedora (green).  The median idle value is ~4x higher under Arch in Xorg.  What's more interesting is dropping Arch out of Xorg to a text TTY (light blue) and repeating this collection of CPU freqs brings Arch in line with the Ubuntu/Xorg values.  Video driver related?  Something different in our Xorg packages?

Median idle freq under Arch in Xorg: 3,316 MHz
Median idle freq under Fedora in Xorg: 1,123 MHz
Median idle freq under Ubuntu in Xorg:  867 MHz
Median idle freq under Arch in text TTY: 951 MHz

Questions: Why is there such a dramatic difference under Arch? Perhaps related to the video drivers since dropping out of Xorg reverses the high CPU usage?  Which Arch package would I open a bug report against?

Arch vs Ubuntu:
arch_ubuntu.jpg
Arch vs Fedora:
arch_fedora.jpg
Arch (Xorg) vs Arch (tty):
arch_arch.jpg

All distros have the same settings for the scaling driver/gov:

% cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor 
powersave

% cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_driver 
intel_pstate

Here are the installed packages on the Arch install: https://gist.github.com/graysky2/c1104b3820b64bca021b
Here are the installed packages on the Fedora install: https://gist.github.com/graysky2/574ab3b231eac6eb039f
Here are the installed packages on the Ubuntu install: https://gist.github.com/graysky2/975e485b4523e8f8104a

EDIT: FS#43910.
EDIT2: Some believe there is a real bug with intel_pstates for newer hardware; see this bugzilla.

Last edited by graysky (2015-11-18 14:49:23)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#2 2015-02-21 18:31:46

dragonn
Member
Registered: 2012-05-04
Posts: 65

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Do you experience higher temp on Arch? I have i7-4790k too and don't see any problem, I think you should more compare the C1 state and not the frequencies.

Offline

#3 2015-02-21 19:25:56

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

In C7 most of the time under each one.  You mind trying the same xorg and tty running i7z logging to confirm behavior in your machine?  Just post the two log files somewhere, I can plot.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#4 2015-02-22 23:16:03

dragonn
Member
Registered: 2012-05-04
Posts: 65

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

You mean C6? How can I create a log from i7z? Because using > log.txt gives a litte garbage file.

Offline

#5 2015-02-22 23:31:44

Gusar
Member
Registered: 2009-08-25
Posts: 3,605

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

No, he means C7. If your i7z doesn't show C7, it's outdated. Use i7z-git from AUR. Use "i7z --write l" to create a log or "i7z --write a" to append to an existing log.

BTW, another very nice tool is turbostat, also available in AUR, that one even shows PC state residency.

Offline

#6 2015-02-23 10:20:27

bastardo
Member
Registered: 2008-11-29
Posts: 15

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

In my opinion something fishy is going on with the pstate driver in general.
Even though I have a Ivy bridge, my PC is a lot hotter than before, since some time and I also only rarely see it idling at low frequencies.

There have been three threads this week and one older about pstate related inquiries in this forum:
https://bbs.archlinux.org/viewtopic.php?id=193882
https://bbs.archlinux.org/viewtopic.php?id=193887 (this here is my thread, I can't disable turbo currently).
https://bbs.archlinux.org/viewtopic.php?id=184817 (older)

My workaround was to just disable pstate for the moment and everything works well again.

Offline

#7 2015-02-23 20:39:19

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Eureka! The problem is not with xorg or with the video driver, it is with the kernel itself. I used the attached fedora config renaming it to config.x86_64 and compiled our 3.18.6 kernel against it. Upon booting into it, I am able to enjoy the same low idle frequencies as I have measured under Fedora and under Ubuntu. See the attached histogram where stock Arch is blue and modified is orange.

arch_with_fedora_config.jpg

3.18.6-1-ARCH with stock Arch config median CPU freq on idle in Xorg: 3,316 MHz
3.18.6-1-ARCH with Fedora's config median CPU freq on idle in Xorg: 925 MHz

Perhaps someone with more experience and knowledge than I can determine which of the settings would affect the intel_pstate driver this drastically. I am very glad to test out some settings and report back.

Link to Fedora config I used: https://gist.github.com/graysky2/a21dc75ea0803d7a8e8d
Link to Arch config: https://gist.github.com/graysky2/ca3789e6237a54892da1

Last edited by graysky (2015-02-23 20:40:06)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#8 2015-02-23 20:48:33

Gusar
Member
Registered: 2009-08-25
Posts: 3,605

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

I only just started looking at the diff of the configs, however I immediately saw something that could have such an effect - Arch uses CONFIG_NO_HZ_IDLE, Fedora instead uses CONFIG_NO_HZ_FULL. I'll look through the rest of the diff, but you could already test just this - use the Arch config and change just this NO_HZ setting.

In the past it was said that NO_HZ_FULL and intel_pstate don't like each other, maybe this has changed now?

Edit: Another change, I think it goes hand in hand with the above one, Arch uses CONFIG_TICK_CPU_ACCOUNTING (Simple tick based cputime accounting), Fedora uses CONFIG_VIRT_CPU_ACCOUNTING_GEN (Full dynticks CPU time accounting).
What I find interesting is that for my own kernels, I use neither of these smile, I use CONFIG_IRQ_TIME_ACCOUNTING. I think it was recommended (or even required) to use this one with the BFS scheduler (I no longer use it), though it's not even available if NO_HZ_FULL is set.

Last edited by Gusar (2015-02-23 20:55:46)

Offline

#9 2015-02-23 21:01:47

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

@Gusar - Yes, I too keyed in on the tickless settings... I am building the Arch kernel as we speak with these modifications:

% grep NO_HZ config.x86_64.last
CONFIG_NO_HZ_COMMON=y
# CONFIG_NO_HZ_IDLE is not set
CONFIG_NO_HZ_FULL=y
# CONFIG_NO_HZ_FULL_ALL is not set
# CONFIG_NO_HZ_FULL_SYSIDLE is not set
CONFIG_NO_HZ=y
CONFIG_RCU_FAST_NO_HZ=y

% grep ACCOUNTING config.x86_64.last
CONFIG_VIRT_CPU_ACCOUNTING=y
CONFIG_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_PARAVIRT_TIME_ACCOUNTING=y

Will post back with the results.

EDIT:  I read the NO_HZ.txt docs but most of it went over my head smile

Last edited by graysky (2015-02-23 21:07:27)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#10 2015-02-23 21:14:05

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Testing now...

Last edited by graysky (2015-02-23 21:40:47)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#11 2015-02-23 21:42:52

post-factum
Member
From: /cz
Registered: 2008-09-12
Posts: 149
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

graysky, I've got your letter, but unfortunately I have not so much to say.

First of all, I've stuck with BFS for very long time (even don't remember when I tried CFS last time). The only thing I remember precisely is that using CONFIG_NO_HZ_FULL=y with CFS screws my CPU's frequencies (according to powertop), so I gave up trying that. Surprisingly, it seems you have got the opposite behavior, and that baffles.

Another claim of mine is that the following config:

[~]$ zgrep NO_HZ /proc/config.gz 
CONFIG_NO_HZ_COMMON=y
CONFIG_NO_HZ_IDLE=y
CONFIG_NO_HZ=y
CONFIG_RCU_FAST_NO_HZ=y
[~]$ zgrep ACCOUNTING /proc/config.gz
CONFIG_IRQ_TIME_ACCOUNTING=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_HAVE_VIRT_CPU_ACCOUNTING_GEN=y
CONFIG_HAVE_IRQ_TIME_ACCOUNTING=y
CONFIG_PARAVIRT_TIME_ACCOUNTING=y

also works OK when replacing BFS with CFS (with intel_pstate enabled). But I have Ivy Bridge and not Haswell, and that may make the difference.

The only thing I'd suggest is to relax CPU time accounting (it really eats CPU resources, and simple accounting is enough for both desktop and server unless you are doing some profiling). Also I'd remind you that NO_HZ_FULL is *not* recommended for everyday use not because it's somehow experimental or unstable but because it's designed to be used with heavily-loaded single-tasking environment.

Hope my thoughts were somehow useful.

Nevertheless, I'll follow this topic smile.

Last edited by post-factum (2015-02-23 21:49:56)


uname == latest pf-kernel

Offline

#12 2015-02-24 02:01:25

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

OK! How crazy is this:  the problem is with the 300 Hz tick rate and this CPU.  Setting to either250 Hz or 1000 Hz both give idle rates that are comparable to either Ubuntu or Fedora. A setting of 300 Hz is to blame for the odd behavior I originally posted. Nothing else.

Original Arch config (300 Hz) median value on idle in Xorg = 3,316 MHz

Changing only the tickrate from the original Arch config...
To 250 Hz tick rate median value on idle in Xorg = 889 MHz
To 1000 Hz tick rate median value on idle in Xorg = 1,012 MHz

What's going on with the 300 Hz value and this particular processor?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#13 2015-02-24 13:25:26

Gusar
Member
Registered: 2009-08-25
Posts: 3,605

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Ok, so you've figured it out. I meant to research further yesterday, I saw other significant differences in the kernel config, and I must say I'm quite surprised by how different the kernels between Fedora and Arch are, but I got distracted by my roommate and beer smile

Anyway, really interesting that the Hz setting causes this. The only thing I can think of, regardless of this issue, is this: Why the heck isn't Arch using 1000 Hz?

Offline

#14 2015-02-24 15:59:27

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Gusar wrote:

Ok, so you've figured it out. I meant to research further yesterday, I saw other significant differences in the kernel config, and I must say I'm quite surprised by how different the kernels between Fedora and Arch are, but I got distracted by my roommate and beer smile

Anyway, really interesting that the Hz setting causes this. The only thing I can think of, regardless of this issue, is this: Why the heck isn't Arch using 1000 Hz?

Ha, no problem.

Not sure why we are using 300 Hz.... perhaps to try to bridge between pure servers and workstations?

linux documentation wrote:

CONFIG_HZ_1000
1000 Hz is the preferred choice for desktop systems and other systems requiring fast interactive responses to events.

CONFIG_HZ_300
300 Hz is a good compromise choice allowing server performance while also showing good interactive responsiveness even on SMP and NUMA systems and exactly dividing by both PAL and NTSC frame rates for video and multimedia work.

CONFIG_HZ_250
250 Hz is a good compromise choice allowing server performance while also showing good interactive responsiveness even on SMP and NUMA systems. If you are going to be using NTSC video or multimedia, selected 300Hz instead.

CONFIG_HZ_100
100 Hz is a typical choice for servers, SMP and NUMA systems with lots of processors that may show reduced performance if too many timer interrupts are occurring.


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#15 2015-02-24 17:24:55

biergaizi
Member
Registered: 2015-02-19
Posts: 13

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

It seems I also hit by the issue https://bbs.archlinux.org/viewtopic.php?id=193882.

Offline

#16 2015-02-24 17:25:58

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

biergaizi wrote:

It seems I also hit by the issue https://bbs.archlinux.org/viewtopic.php?id=193882.

And if you compile the kernel from ABS changing the tick rate to 1000?


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#17 2015-02-24 21:09:31

mar04
Member
From: Poland
Registered: 2010-02-08
Posts: 117

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

graysky wrote:
Gusar wrote:

Ok, so you've figured it out. I meant to research further yesterday, I saw other significant differences in the kernel config, and I must say I'm quite surprised by how different the kernels between Fedora and Arch are, but I got distracted by my roommate and beer smile

Anyway, really interesting that the Hz setting causes this. The only thing I can think of, regardless of this issue, is this: Why the heck isn't Arch using 1000 Hz?

Ha, no problem.

Not sure why we are using 300 Hz.... perhaps to try to bridge between pure servers and workstations?

linux documentation wrote:

CONFIG_HZ_1000
1000 Hz is the preferred choice for desktop systems and other systems requiring fast interactive responses to events.

CONFIG_HZ_300
300 Hz is a good compromise choice allowing server performance while also showing good interactive responsiveness even on SMP and NUMA systems and exactly dividing by both PAL and NTSC frame rates for video and multimedia work.

CONFIG_HZ_250
250 Hz is a good compromise choice allowing server performance while also showing good interactive responsiveness even on SMP and NUMA systems. If you are going to be using NTSC video or multimedia, selected 300Hz instead.

CONFIG_HZ_100
100 Hz is a typical choice for servers, SMP and NUMA systems with lots of processors that may show reduced performance if too many timer interrupts are occurring.

I'm using kernel from linux-lts package and I only now realized that it has different config with CONFIG_HZ set to 100Hz.

Offline

#18 2015-02-24 23:11:20

dragonn
Member
Registered: 2012-05-04
Posts: 65

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Well that is strange, I use linux-bld with have HZ_1000 and my freq still doesn't goes down? Here is my log wklej.to/9tJTIa and my config:

zcat /proc/config.gz | grep HZ
CONFIG_NO_HZ_COMMON=y
# CONFIG_HZ_PERIODIC is not set
CONFIG_NO_HZ_IDLE=y
# CONFIG_NO_HZ_FULL is not set
# CONFIG_NO_HZ is not set
CONFIG_RCU_FAST_NO_HZ=y
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_MACHZ_WDT=m

Offline

#19 2015-02-24 23:36:27

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Dragon - Can you build i7z-git from the AUR and just leave the machine sit idle in X (make sure you disable the screensaver).  Run it like this and email me or post the cpifreq log it creates somewhere and I will plot for you and inspect the median as I have done.

sudo i7z- w a

Let it collect and log 10 min of data.

Also, that link you posted doesn't work "wklej.to/9tJTIa"

Last edited by graysky (2015-02-24 23:47:07)


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#20 2015-02-24 23:53:08

bmanuel
Member
Registered: 2011-10-06
Posts: 106

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Pretty sure i'm experiencing the same: on my freshly built i7-4790k i've set the powersave governor but it idle at ~4Ghz, do i just need to download the PKGBUILD from abs and tweak the config and set it to 1000Hz?

Offline

#21 2015-02-25 00:10:56

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

@bmanuel - Yes.  You don't even have to tweak the PKGBUILD.  Just run this simple sed line to change it in the config.x86_64 itself and then regenerate the md5sums.

cd /path/to/PKGBUILD_and_related_files

sed -i -e 's/^CONFIG_HZ_300=y/# CONFIG_HZ_300 is not set/' \
      -i -e 's/^# CONFIG_HZ_1000 is not set/CONFIG_HZ_1000=y/' \
      -i -e 's/^CONFIG_HZ=300/CONFIG_HZ=1000/' config.x86_64

updpkgsums

CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#22 2015-02-25 00:36:13

bmanuel
Member
Registered: 2011-10-06
Posts: 106

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

Thank you much!

Offline

#23 2015-02-25 08:54:38

bmanuel
Member
Registered: 2011-10-06
Posts: 106

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

@graysky, is it normal for this CPU to reach 93°C (gkrellm/lmsensors) while compiling the linux kernel? I've the stock cooler and the CPU is not overclocked o_O

Offline

#24 2015-02-25 09:49:54

graysky
Wiki Maintainer
From: :wq
Registered: 2008-12-01
Posts: 10,595
Website

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

I forget what the max operating temp is for this chip... For stock cooling it is not unreasonable.  You can probably lower it by identifying the minimum vcore it takes to run under several loads.  See: https://wiki.archlinux.org/index.php/Stress_Test


CPU-optimized Linux-ck packages @ Repo-ck  • AUR packagesZsh and other configs

Offline

#25 2015-02-25 13:21:20

Gusar
Member
Registered: 2009-08-25
Posts: 3,605

Re: Arch specific bug-high idle CPU freq for Haswell Refresh CPUs

graysky wrote:

Not sure why we are using 300 Hz.... perhaps to try to bridge between pure servers and workstations?

Probably. Though I recall Con Kolivas greatly discourages 300Hz.

graysky wrote:

I forget what the max operating temp is for this chip... For stock cooling it is not unreasonable.

Really? Now granted I have an i5-4570S (65W TDP), but still, with stock cooling the temp never went above 70°C during heavy compiles. With the cooler I have now, it doesn't go above 54°C. So even for an i7, 90°C sounds too much to me.

Last edited by Gusar (2015-02-25 13:26:31)

Offline

Board footer

Powered by FluxBB