You are not logged in.

#1 2011-06-11 08:13:38

useradded
Member
From: Edinburgh, UK
Registered: 2010-05-15
Posts: 77

pcie_aspm control not working since upgrade to 2.6.39.1

Hi


Since upgrading to 2.6.39.1 it seems no longer possible to control power saving of pcie devices by echoing (default|performance|powersave) to /sys/module/pcie_aspm/parameters/policy.

I am doing this as root, obviously.  The file is still there and can be read, root also has read and write permission, yet it seems I can no longer switch power levels by echoing like I used to.

Has this been somehow disabled, or have the acceptable values been changed to something other than (default|performance|powersave)?

Thanks

Offline

#2 2011-06-11 13:09:12

useradded
Member
From: Edinburgh, UK
Registered: 2010-05-15
Posts: 77

Re: pcie_aspm control not working since upgrade to 2.6.39.1

OK. This same problem occurs on three different machines. 

Before upgrading to 2.6.39.1 I can change the value of /sys/module/pcie_aspm/parameters/policy, after upgrading I cannot - I just get "bash: echo: write error: Operation not permitted"


A look in /var/log/kernel.log finds these entries for pcie_aspm

Jun  6 14:22:18 localhost kernel: [    0.441788] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  6 21:00:12 localhost kernel: [    0.441160] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  7 22:17:28 localhost kernel: [    0.441171] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  8 07:23:00 localhost kernel: [    0.441195] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  8 17:58:05 localhost kernel: [    0.448257] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  8 18:08:47 localhost kernel: [    0.448229] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun  9 09:02:31 localhost kernel: [    0.444868] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun 10 13:35:44 localhost kernel: [    0.448193] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun 11 09:48:17 localhost kernel: [    0.448820] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
Jun 11 09:51:21 localhost kernel: [    0.490917] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'

But there is no change here between today, when I upgraded and the problem began, and a few days ago when everything was working well.


I also found this in the changelog for 2.6.39 with regard to pcie aspm.  But it's not clear, at least to me, if this might have any bearing on my issue.  I would appreciate more knowledgeable eyes taking a quick look.

commit bbfa306a1e5d9618231aa0de3d52a8eb1219d0c3
Author: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
Date:   Mon Mar 21 03:29:14 2011 +0000

    PCI: Changing ASPM policy, via /sys, to POWERSAVE could cause NMIs
    
    v3 -> v2: Modified the text that describes the problem
    v2 -> v1: Returned -EPERM
    v1      : http://marc.info/?l=linux-pci&m=130013194803727&w=2
    
    For servers whose hardware cannot handle ASPM the BIOS ought to set the
    FADT bit shown below:
    In Sec 5.2.9.3 (IA-PC Boot Arch. Flags) of ACPI4.0a Specification, please
    see Table 5-11:
    PCIe ASPM Controls: If set, indicates to OSPM that it must not enable
    OPSM ASPM control on this platform.
    
    However there are shipping servers whose BIOS did not set this bit. (An
    example is the HP ProLiant DL385 G6. A Maintenance BIOS will fix that).
    For such servers even if a call is made via pci_no_aspm(), based on _OSC
    support in the BIOS, it may be too late because the ASPM code may have
    already allocated and filled its "link_list".
    
    So if a user sets the ASPM "policy" to "powersave" via /sys then
    pcie_aspm_set_policy() will run through the "link_list" and re-configure
    ASPM policy on devices that advertise ASPM L0s/L1 capability:
    # echo powersave > /sys/module/pcie_aspm/parameters/policy
    # cat /sys/module/pcie_aspm/parameters/policy
    default performance [powersave]
    
    That can cause NMIs since the hardware doesn't play well with ASPM:
    [ 1651.906015] NMI: PCI system error (SERR) for reason b1 on CPU 0.
    [ 1651.906015] Dazed and confused, but trying to continue
    
    Ideally, the BIOS should have set that FADT bit in the first place but we
    could be more robust - especially given the fact that Windows doesn't
    cause NMIs in the above scenario.
    
    There should be a sanity check to not allow a user to modify ASPM policy
    when aspm_disabled is set.
    
    Signed-off-by: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
    Acked-by: Rafael J. Wysocki <rjw@sisk.pl>
    Cc: Matthew Garrett <mjg59@srcf.ucam.org>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

commit 1a680b7c325882188865f05b9a88d32f75f26495
Author: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
Date:   Mon Mar 21 03:29:08 2011 +0000

    PCI: PCIe links may not get configured for ASPM under POWERSAVE mode
    
    v3 -> v2: Moved ASPM enabling logic to pci_set_power_state()
    v2 -> v1: Preserved the logic in pci_raw_set_power_state()
        : Added ASPM enabling logic after scanning Root Bridge
        : http://marc.info/?l=linux-pci&m=130046996216391&w=2
    v1    : http://marc.info/?l=linux-pci&m=130013164703283&w=2
    
    The assumption made in commit 41cd766b065970ff6f6c89dd1cf55fa706c84a3d
    (PCI: Don't enable aspm before drivers have had a chance to veto it) that
    pci_enable_device() will result in re-configuring ASPM when aspm_policy is
    POWERSAVE is no longer valid.  This is due to commit
    97c145f7c87453cec90e91238fba5fe2c1561b32 (PCI: read current power state
    at enable time) which resets dev->current_state to D0. Due to this the
    call to pcie_aspm_pm_state_change() is never made. Note the equality check
    (below) that returns early:
    ./drivers/pci/pci.c: pci_raw_set_pci_power_state()
    546         /* Check if we're already there */
    547         if (dev->current_state == state)
    548                 return 0;
    
    Therefore OSPM never configures the PCIe links for ASPM to turn them "on".
    
    Fix it by configuring ASPM from the pci_enable_device() code path. This
    also allows a driver such as the e1000e networking driver a chance to
    disable ASPM (L0s, L1), if need be, prior to enabling the device. A
    driver may perform this action if the device is known to mis-behave
    wrt ASPM.
    
    Signed-off-by: Naga Chumbalkar <nagananda.chumbalkar@hp.com>
    Acked-by: Rafael J. Wysocki <rjw@sisk.pl>
    Cc: Matthew Garrett <mjg59@srcf.ucam.org>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

commit 8b8bae901ce23addbdcdb54fa1696fb2d049feb5
Author: Rafael J. Wysocki <rjw@sisk.pl>
Date:   Sat Mar 5 13:21:51 2011 +0100

    PCI/ACPI: Report ASPM support to BIOS if not disabled from command line
    
    We need to distinguish the situation in which ASPM support is
    disabled from the command line or through .config from the situation
    in which it is disabled, because the hardware or BIOS can't handle
    it.  In the former case we should not report ASPM support to the BIOS
    through ACPI _OSC, but in the latter case we should do that.
    
    Introduce pcie_aspm_support_enabled() that can be used by
    acpi_pci_root_add() to determine whether or not it should report ASPM
    support to the BIOS through _OSC.
    
    Cc: stable@kernel.org
    References: https://bugzilla.kernel.org/show_bug.cgi?id=29722
    References: https://bugzilla.kernel.org/show_bug.cgi?id=20232
    Reported-and-tested-by: Ortwin Glück <odi@odi.ch>
    Reviewed-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
    Tested-by: Kenji Kaneshige <kaneshige.kenji@jp.fujitsu.com>
    Signed-off-by: Rafael J. Wysocki <rjw@sisk.pl>
    Signed-off-by: Jesse Barnes <jbarnes@virtuousgeek.org>

Thanks

Offline

#3 2011-06-11 16:53:51

fphillips
Member
From: Austin, TX
Registered: 2009-01-24
Posts: 202

Re: pcie_aspm control not working since upgrade to 2.6.39.1

It looks like what is happening is that the kernel has been preventing aspm usage on your hardware (see kernel.log), but you were able to override that by setting the policy manually in sysfs. Now in the latest kernel, you can't set the policy unless you force it on with pcie_aspm=force.

Last edited by fphillips (2011-06-11 16:54:34)

Offline

Board footer

Powered by FluxBB