[6.10.0-RC1] Unable to enable ASPM: it stays disabled even with policy and boot option (Ubuntu works, Unraid not)

Do you need the "pcie_aspm=force" kernel parameter?  There is this warning:

WARNING: Forcing ASPM on may cause system lockups.

 

Also it would be helpful to post the kernel .config file of your Ubuntu install.

Thank you for the prompt response.

 

No, the parameter is not needed. Ubuntu does enable ASPM without the parameter.

 

Unraid does not enable ASPM with or without parameter.

 

I was only comfortable trying this parameter because I know that Ubuntu did enable ASPM on all devices by default without setting anything. But force on Unraid didn't help at all. There are some more tricks, which I documented and tested in the old 6.9 topics. For the 6.10 I did only test the force parameter.

 

By the way, this issue is related to my chipset. If I connect PCI-E Devices direct to the CPU, they get ASPM enabled on unraid like they should, even without force. So maybe something related to the chipset is causing the disabling.

 

@limetechWhat else do you need from Ubuntu?
Do you like to get exports from all 3 Ubuntu Versions, or only the 21.10 Alpha?

Edited August 11, 2021 by Falcosc

 

2 hours ago, Falcosc said:

Do you like to get exports from all 3 Ubuntu Versions, or only the 21.10 Alpha?

 

Just the kernel .config file from whichever one works for you. I want to see how hey defined acpm-related kernel config settings.

Your syslog from diags report that ACPM is enabled...

6 hours ago, limetech said:

Your syslog from diags report that ACPM is enabled...

Yes, I did confirm that the force command was correctly put into boot parameters by looking into syslog as well. And ASPM does work on devices which are directly connected to CPU.

So, there is something which does ignore the enablement or something which disables ASPM for selected devices.

I did also check the power consumption in case lspci is wrong, but ASPM is really disabled on my chipset and nic.

 

I will export the ubuntu syslog as well, maybe it is the other way around and something inside ubuntu does manually enable ASPM after the kernel fails to do it? There are so many possibilities. The correct way would be to debug the boot process to find out which driver/module/script does detect and enable ASPM on the chipset. But I don't know how to do this.

Edited August 12, 2021 by Falcosc

I did expect an 5.13 or 5.14 kernel on the Ubuntu daily build, but was only 5.11

 

syslogconfig-5.11.0-20-generic

 

@TraumG may can tell you about his devices because he has the same issue to collect more data.

 

@limetech Do you like to get an RC2 retest from me and maybe from @TraumG as well?

Edited November 2, 2021 by Falcosc

On 8/12/2021 at 1:15 AM, limetech said:

Your syslog from diags report that ACPM is enabled...

I think around this lies the problem.

 

One of my servers has "perfectly" working ASPM (without setting any additional Kernel options):

lspci -vv | awk '/ASPM.*?abled/{print $0}' RS= | grep --color -P '(^[a-z0-9:.]+|ASPM.*?abled)'
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 07) (prog-if 00 [Normal decode])
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0) (prog-if 00 [Normal decode])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0) (prog-if 00 [Normal decode])
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.5 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #6 (rev f0) (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0) (prog-if 00 [Normal decode])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
04:00.0 Ethernet controller: Intel Corporation I210 Gigabit Network Connection (rev 03)
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 (prog-if 02 [NVM Express])
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

 

But its syslog claims the opposite?! Why does FADT return the wrong ASPM capabilities?

ACPI FADT declares the system doesn't support PCIe ASPM, so disable it

 

And I have a different server which does not return this message in the syslog, but all its devices return "ASPM Disabled", although it's enabled in the BIOS (and enabled, when using Ubuntu). Even "pcie_aspm=force" doesn't change anything. Only setpci works.

 

For me it sounds like a driver is missing?!

 

@Falcosc

Maybe this script helps to enable ASPM automatically:

https://web.archive.org/web/20190301120327/http://drvbp1.linux-foundation.org/~mcgrof/scripts/enable-aspm

I do already use this script, it was mentioned on the documentation about manually enabling ASPM

yes, I just notice the same thing,  I'm quite sure before 6.92,  it is work as I had to manually add pcie_asmp=off to get rid of some PCIe Bus Error message