Reduce power consumption with powertop

[genesisdoeswhatnintendont]

@chchiyan as far as I know there is no such command. But if your motherboard has AMI BIOS, names should be similar at least.

You also can reveal hidden setings with UEFITool (see guide I mentioned) but it's more complicated way IMO.

[Lucict]

Hello--

 

Does anyone have issues with tick_sched_timer and excessive wake up events?  I am getting between 750-5000 events at some points.  A low number for me would be 400.

 

I've attached a diagnostic as well as a powertop report.  If anyone has insight, I would greatly appreciate it.

 

Thanks!

rosellis-diagnostics-20240117-1103.zip powertop.html

[firstTimer]

Hi guys, I just run this command and I just saw that a lot of stuff is disabled (mostly intel stuff) and I don't understand why. I enabled every possible ASPM key in the bios but nothing. I know I have a bad sata SSD, but other than that I can't reach a status lower than C3

 

[mich2k]

Hello

how did you all manage to install powertop on the lastest unraid version? Thanks

[Limeth]

So I was building a brand new Unraid box and ended up getting a 14-series Intel processor. It turns out that C-states higher than C3 are currently unsupported in the Linux kernel. For details, see this support thread I made.

[KarlMeyer]

21 hours ago, mich2k said:

Hello

how did you all manage to install powertop on the lastest unraid version? Thanks

Install nerdtools from the app store, scroll down until you find powertop & toggle it to on & click save at the bottom of the page. This will install powertop every time you start up your unraid machine.

[xreyuk]

Hi all, 

 

I have previously managed to acheive C10 state on my machine with BIOS settings and using --auto-tune, however I needed to expand my storage earlier than expected and thus have had to install a PCI to SATA card with the JMicron JBM585 controller.

 

From my understanding using --auto-tune is now not possible as it stops the drives connected to this card spinning up. Therefore I need to use the CLI commands and a startup script to achieve any power savings I can (I realise the PCI card may stop me reaching low power states too).

 

My output to see whether ASPM is enabled is as follows.

 

00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
                LnkCap: Port #2, Speed 32GT/s, Width x16, ASPM L1, Exit Latency L1 <16us
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 SATA controller: JMicron Technology Corp. JMB58x AHCI SATA controller (prog-if 01 [AHCI 1.0])
                LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM not supported
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk-
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P2 [Nick P2] / P3 / P3 Plus NVMe PCIe SSD (DRAM-less) (rev 01) (prog-if 02 [NVM Express])
                LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

 

All ASPM/Power options are enabled in BIOS (as previously mentioned but double checked), and the PCI to SATA controller is plugged into the PCI x16 slot. I can see that ASPM is disabled on the x16 controller and the PCI to SATA card - do I need to get this enabled somehow or is this expected? (there are no explicit BIOS options to enable this).

 

Furthermore, here is the output of 'bad' devices from powertop before the card was installed. All of these auto-tuned correctly.

 

>> Bad           Enable SATA link power management for host6                        
   Bad           Enable SATA link power management for host7
   Bad           Enable SATA link power management for host5
   Bad           Enable SATA link power management for host3
   Bad           Enable SATA link power management for host1
   Bad           Enable SATA link power management for host8
   Bad           Enable SATA link power management for host4
   Bad           Enable SATA link power management for host2
   Bad           VM writeback timeout
   Bad           Autosuspend for USB device Flash Drive [Samsung]
   Bad           Runtime PM for disk sdd
   Bad           Runtime PM for disk sda
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH SATA C
   Bad           Runtime PM for disk sdc
   Bad           Runtime PM for port ata1 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata2 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata3 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata4 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata5 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata6 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for port ata7 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for disk sde
   Bad           Runtime PM for port ata8 of PCI device: Intel Corporation Alder Lak
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Ethernet Connection (17
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH PCI Ex
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Shared
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Device 4630
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH SPI Co
   Bad           Runtime PM for PCI Device Micron/Crucial Technology P2 [Nick P2] /
   Bad           Runtime PM for PCI Device Intel Corporation Device 7a87
   Bad           Runtime PM for disk sdb

 

Here is the output from after I've installed the card.

 

   Bad           Enable SATA link power management for host6                                                            
   Bad           Enable SATA link power management for host7
   Bad           Enable SATA link power management for host8
   Bad           Enable SATA link power management for host13
   Bad           Enable SATA link power management for host11
   Bad           Enable SATA link power management for host5
   Bad           Enable SATA link power management for host3
   Bad           Enable SATA link power management for host1
   Bad           Enable SATA link power management for host12
   Bad           Enable SATA link power management for host10
   Bad           Enable SATA link power management for host4
   Bad           Enable SATA link power management for host2
   Bad           VM writeback timeout
   Bad           Enable SATA link power management for host9
   Bad           Autosuspend for USB device Flash Drive [Samsung]
   Bad           Runtime PM for PCI Device Micron/Crucial Technology P2 [Nick P2] / P3 / P3 Plus NVMe PCIe SSD (DRAM-less)
   Bad           Runtime PM for port ata6 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #3
   Bad           Runtime PM for port ata5 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for disk sda
   Bad           Runtime PM for disk sdb
   Bad           Runtime PM for disk sdc
   Bad           Runtime PM for disk sdd
   Bad           Runtime PM for disk sde
   Bad           Runtime PM for disk sdf
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for port ata2 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for port ata3 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for PCI Device JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for port ata1 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for port ata4 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for port ata7 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for PCI Device Intel Corporation Ethernet Connection (17) I219-V
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #5
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH PCI Express Root Port #1
   Bad           Runtime PM for port ata8 of PCI device: Intel Corporation Alder Lake-S PCH SATA Controller [AHCI Mode]
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #1
   Bad           Runtime PM for PCI Device Intel Corporation Device 7a87
   Bad           Runtime PM for PCI Device Intel Corporation Device 4630
   Bad           Runtime PM for port ata10 of PCI device: JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for port ata11 of PCI device: JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for port ata12 of PCI device: JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for port ata13 of PCI device: JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH SPI Controller
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #4
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #2
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Serial IO I2C Controller #0
   Bad           Runtime PM for port ata9 of PCI device: JMicron Technology Corp. JMB58x AHCI SATA controller
   Bad           Runtime PM for PCI Device Intel Corporation Alder Lake-S PCH Shared SRAM

 

So from what I can see I would need to do the commands for all disks except sdf, as that did not exist before and will be the new one connected to the controller (confirmed from main tab as well).

 

I would need to do the sata link management for hosts 1 to 8, and the PM ATA port for ata1 to ata8.

 

However after that I am unable to identify which other tweaks to do and which devices they need doing on listed in the /sys/ folder directories. 

 

I would basically like to be able to do as much power tuning as possible without messing up the controller and the disk currently connected to it (and future disks).

 

Is anyone able to advise how to find these devices and then use that information to help me do the commands manually?

 

Edited January 22 by xreyuk

[mgutt]

48 minutes ago, xreyuk said:

I would need to do the sata link management for hosts 1 to 8, and the PM ATA port for ata1 to ata8.

 

However after that I am unable to identify which other tweaks to do and which devices they need doing on listed in the /sys/ folder directories. 

All commands are mentioned in the first post. Even an example how to set sleep for specific SATA hosts:

echo med_power_with_dipm | tee /sys/class/scsi_host/host1/link_power_management_policy

 

But finally you should simply buy an ASM1166 card which supports those sleep states.

[xreyuk]

31 minutes ago, mgutt said:

All commands are mentioned in the first post. Even an example how to set sleep for specific SATA hosts:

echo med_power_with_dipm | tee /sys/class/scsi_host/host1/link_power_management_policy

 

But finally you should simply buy an ASM1166 card which supports those sleep states.

Could I just use the --auto-tune settings if I get an ASM1166 card and make sure firmware is upgraded?

 

*EDIT*

 

I saw another post, I can. I'll return the JBM one and get an ASM1166 card. Thanks.

Edited January 22 by xreyuk

[KarlMeyer]

For anyone curious, I got my Mellanox Connect-x4 MCX4121A-ACAT in the mail today from ebay. I updated it to the latest firmware using the Mellanox plugin in the app store & as you can see, I have ASPM working on the card running at 10gbe DAC connected to my UDM Pro SE SFP+ port. I can not get below C3 now, whereas without it, I can get to C6. But if I understand correctly, all those 10gbe cards that don't support ASPM (Mellanox Connect-x3) simply block you from using any C states. My system idles at around 35 watts +-2 (spin down enabled) which I think isn't terrible at all, just not great, not special. I wanted to be special :(. Devices are: PRO Z790-A WIFI DDR4, 2x32gbe nemix ecc RAM (don't ask, long story), intel 12500, MZHOU PCIe 6 portSATA Card (3.0x1), 2x1tb WD sn700, 2x1tb WD blue sata SSDs, 6x18tb WD red pro big bois. I might try putting my optane m.2s back in to see if I can still idle at C3 or not. I've also got an m.2 to 6 sata (3.0x2) adapter & a wifi e key to 2 sata card arriving soon that I want to play with, see if what combos I can do to keep C3. I don't think I can give up my Mellanox baby now that I have it working in low power (compared to 10gbe RJ45) SFP+ especially since it's actually a 25gbe card. I'm thinking maybe in the future, ubiquiti will release a router/switch with one or two 25gbe SFP ports which would be plug & play for me. I run autotune on boot using userscripts & have everything I can find in the bios turned off. I hope this helps someone. Thank you @mgutt.

[xreyuk]

So I have changed my card to an ASM1166 card which is running a high enough firmware version to support ASPM.

 

This is the output of lspci

 

00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
                LnkCap: Port #2, Speed 32GT/s, Width x16, ASPM L1, Exit Latency L1 <16us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
                LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P2 [Nick P2] / P3 / P3 Plus NVMe PCIe SSD (DRAM-less) (rev 01) (prog-if 02 [NVM Express])
                LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

 

So I am using the same settings as before installing the card where I could reach package C10 states. I have applied --auto-tune and the disks seem to spin up and down as expected, and all tuneables are listed as 'good'. However, now I can only reach package state of C2, and core state of C10. Does anyone know how I can troubleshoot why the package is no longer going to C10 with the PCI card installed?

 

I have tried disabling all dockers and VM's I have installed, I am still running the same plugins (no new ones) and I am still not able to reach C10 with these disabled.

 

[dopeytree]

Do you have an intel i226v type network card? i.e 2.5Gb

[xreyuk]

25 minutes ago, dopeytree said:

Do you have an intel i226v type network card? i.e 2.5Gb

No, it's GbE - and without the PCI card plugged in I am able to reach package state C10, so I think this is somehow related to the card/PCI slot.

 

All the tuneables are listed as 'good' in powertop after the --auto-tune

Edited January 24 by xreyuk

[KarlMeyer]

I've got an update. I can confirm that my optane P1600X 118GB nvme kicks me out of C3 down to C2. This shouldn't come as a surprise since the drive's compatibility tab in unraid says it's lowest power consumption in 3.3 watts which I double-checked with my eve smart plug. For comparison I've im sharing the same infor page for my WD Red SN700 1tb which is clearly significantly better in terms of power consumption. I also swapped the motherboard out for the Asrock z790 Pro RS/D4 & was able to reduce my idle down to 32 watts at idle which is better. I was able to pull out the MZHOU 6 port SATA card which is where most of that power came back from. I think i'll stick to the asrock board for now. I do think z790 is the right way to go, simply for the extra chipset NVME. I'm now curious to see if the 960GB 905p optane drive has different power consumption profile than the P1600X 118GB. I'm guessing no, but I would still like to know, because if so, that would make it the only optane worth getting IMO. I think i've reached the end of my powertop journey for the time being because there's simply nothing else I can tweak at the moment. Future versions of unraid might opens some new doors, i'll have to wait until then.

[KarlMeyer]

I should also mention I tried this i226v 2.5gbe card.

https://www.amazon.com/dp/B0C2PS2P5W?psc=1&ref=ppx_yo2ov_dt_b_product_details

Though it worked at C3 I believe, it would crash my network if used too intensely for too long (Qbittorrent seeding linux isos). Bizzare I know. This also happened with integrated realtek 2.5gbe 8125 but was fixed when I installed the app store driver for it. All got down to C3 for me.

[_cjd_]

19 hours ago, KarlMeyer said:

For anyone curious, I got my Mellanox Connect-x4 MCX4121A-ACAT in the mail today from ebay. <snip>

 

I believe it has been noted elsewhere that the intel x710 is the one to get; I snagged a dual-port x710 SFP+ for only a tiny bit more than I paid for my dual port ConnectX-3 - both ports DAC > Unifi Aggregation. On my AMD system it had no real change in power efficiency (idle mean is ~48-49W with lowest I've seen ~43W) but I was curious, it was cheap enough, so I tried it out. Now I have this ConnectX-3 to use...
Also, just in case you have cash burning a hole in your pocket, the Unifi "Hi-Capacity Aggregation" has 4 25G SFP28 ports (and 28 10G SFP+)... a few other ethernet switches also have the SFP28 for uplink etc. All beyond my price range. My 8 port SFP+ switch is fully stuffed though... 10Gb from my desktops and I had to start chasing higher performance on the Unraid server for certain things. Backup server is 10G too.

[vmasip]

On 12/15/2023 at 4:13 PM, mgutt said:

NVMe or you won't be able to exclude USB as the source of the problem 😉

Hello

 

@mgutt, finally I've been able to install latest ubuntu server (release normal version)  at the nvme. I have had same results and questions explained here. Please, could you or anyone address them?:
 

 

Of course, because of realtek driver bug, I need to disconnect ethernet to reach higher states. But still appearing other pci device with aspm disabled. But, works ... not high percentages as I question here:

 

On 1/12/2024 at 1:54 PM, vmasip said:

Doing nothing, is 48% of the time desirable or should be ~99% if user is doing literally nothing?

 

I've researched this, running powertop -t 60, even -t 1000 and still not going above 48%. I've make it run without usb or hdmi, using sleep 10 && powertop --auto-tune && sleep 10 && powertop -C -t 100 , then disconnecting keyboard and hdmi and waiting for 100 seconds, but same results at the csv.

 

Hope it helps and someone can help.

Edited February 1 by vmasip

[KarlMeyer]

3 hours ago, _cjd_ said:

I believe it has been noted elsewhere that the intel x710 is the one to get; I snagged a dual-port x710 SFP+ for only a tiny bit more than I paid for my dual port ConnectX-3 - both ports DAC > Unifi Aggregation. On my AMD system it had no real change in power efficiency (idle mean is ~48-49W with lowest I've seen ~43W) but I was curious, it was cheap enough, so I tried it out. Now I have this ConnectX-3 to use...
Also, just in case you have cash burning a hole in your pocket, the Unifi "Hi-Capacity Aggregation" has 4 25G SFP28 ports (and 28 10G SFP+)... a few other ethernet switches also have the SFP28 for uplink etc. All beyond my price range. My 8 port SFP+ switch is fully stuffed though... 10Gb from my desktops and I had to start chasing higher performance on the Unraid server for certain things. Backup server is 10G too.

I was aware of this but was kinda set on a mellanox card because of the plugin available in the app store. It's very helpful. I'll have to give the intel card a shot at some point. I didn't realize they're 2x the price of my mellanox on ebay.

[KarlMeyer]

I have friends that I like less than my new COMeap M.2 to SATA Adapter A+E Key to Dual Ports SATA 3.0 Converter Hard Drive Expansion Card with 6Gbps JMB582 2230 Got it in the mail from Amazon, thing works very well for $14. Idle consumption goes up just .5 to 1 watts & I can stay in HW C3. Seems like it struggles just a bit with 2 sata SSDs at full speed (see attached benchmark) but the low idle make it well worth it IMO. It would obviously work best with 2 hard drives because you couldn't saturate the 985mb pcie 3x1 link that motherboard wifi e keys use. If you need a few more drives, these kinds of e key to sata adapters are probably the first thing you should try. Just remember to turn the wifi option back on in the bios. It performed only about 20MBps less than my MZHOU PCIe 6 portSATA Card (3.0x1).

Edited January 25 by KarlMeyer
Adding more data

[infiltrazion]

Finally got c-states higher than c3 on my asrock lga 1700 motherboard, what pushed me over after doing all that BIOS stuff and everything else was installing realtek drivers from the app store, so yeah if you're stuck with c3 states on one of those and you got a realtek nic maybe try it.

Edited January 26 by infiltrazion

[_cjd_]

On 1/24/2024 at 9:06 PM, KarlMeyer said:

I was aware of this but was kinda set on a mellanox card because of the plugin available in the app store. It's very helpful. I'll have to give the intel card a shot at some point. I didn't realize they're 2x the price of my mellanox on ebay.

 

Cheapest ConnectX-4 I see right now is what I paid for the x710 card. May just be luck of timing.

[Alistar]

Hi all,
So far I have managed to go down from ~80W (fresh build) to ~40 (lowest I have noticed was 38, but usually its around 40-45) - with 2 disks active (+2 standby) and some dockers active and qbit running. At first I was quite happy with the result, and then a bit dissapointed after seeing results on forum and users achieveing sub 20W. So in general been hoping to reduce even more with powertop, but got stuck at C3. Let me share more details:

Hardware:
Unraid 6.12.6
CPU: i3-12100
MB: Biostar B760MXC PRO 2.0
RAM: Lexar Thor, DDR4, 16 GB, 3200MHz, CL16 (2 x 8GB)
PSU:  be quiet! Pure Power 11 400W (80 Plus Gold)

HDD: 4x Seagate Exos 10TB ST10000NM018G

SSD (cache): Lexar NM620 1TB
NIC: Additional 2.5gbps Intel i226
APC UPS and tp-link tapo p110 smart plug with power meter

 

What I did so far in general:
- Enabled drives to spin down when not used

- 'power save' mode in tips and tricks, +other config and CA plugins to utilize ssd cache and optimise hdd's

#Powertop (installed using nerdtools):

- enabled c-states in bios;  disabled hdaudio, led controller, builtin 1gb nic, undervolting by setting offset at -0.100

- with powertop auto-tune all tunables are 'good'

- CPU reaching C7, while pkg max C3

Now:
It was advised to enable ASPM in bios. However such option seem to not exist in biostar uefi BIOS Manual
I run below command as per guide:

ASPM is disabled for listed devices as theres's no such option to enable anything in bios. 
More over, ASPM is NOT supported for few devices, which concerns me the most (i guess theres no point in even trying to force enable them somehow as it is not supported?).

Questions:
1. Is 'ASPM not supported' permanently blocking me from further actions and reaching lower c-states, and all I can do is just buy another motherboard?
- If yes, are there any 'whitelisted' or considered as 'good' consumer grade matx boards (socket 1700)?
- if not - what would be next steps here?
2. 2.5gbps Intel i226 pcie card  - Ive seen someone asking about it - is it known to cause potential issues with reaching lower c-states and potential blocker? 
3. Do you see any other risks related to my current hardware setup?

Sorry for long post, any help and explanation appreciated.

[mgutt]

9 hours ago, Alistar said:

with 2 disks active

Two active disks cost you 10 to 16W. Why are they active?

 

9 hours ago, Alistar said:

be quiet! Pure Power 11 400W

Not the best for super low power consumptions, but at the moment not your main problem.

 

9 hours ago, Alistar said:

undervolting by setting offset at -0.100

Undervolting is nearly useless for idle power consumption. It influences only high CPU usage.

 

9 hours ago, Alistar said:

Lexar NM620 1TB

2.5gbps Intel i226 pcie card

Disable array auto start, remove both components, spindown your HDDs, execute powertop and check if this influences your PKG states. If yes, find out, which of both is your problem.

 

9 hours ago, Alistar said:

However such option seem to not exist in biostar uefi BIOS Manual

Ask the biostar support. If their answer doesn't make you happy: Throw it in the trash.

[tgiannak]

hello, 

 

sys specs

Asrock B460M

i5-10400

32GB Crucial Pro

3x 12tb disks

2x 980 pro nvme cache

3 dockers

1 vm

 

 

with everything spun up i was running 55-60w, thanks to mgutt i made all necessary bios settings: no turbo, cpu page: max c states all enabled (LO , L1), pcie page: all enabled ASPM etc etc,  installed powertop and spun down disks and im getting about 35w but i have some problems not going below c3. plus i have a problem with my TBS 6281SE it supports L0 but it doesnt activate at all...plus i tested without it and getting the same c3..one last thing if i run a simple win vm then no c states!!???

 

 

Edited January 29 by tgiannak

[xreyuk]

On 1/24/2024 at 6:01 PM, xreyuk said:

So I have changed my card to an ASM1166 card which is running a high enough firmware version to support ASPM.

 

This is the output of lspci

 

00:01.0 PCI bridge: Intel Corporation 12th Gen Core Processor PCI Express x16 Controller #1 (rev 05) (prog-if 00 [Normal decode])
                LnkCap: Port #2, Speed 32GT/s, Width x16, ASPM L1, Exit Latency L1 <16us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
00:1c.0 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #1 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #1, Speed 8GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
                LnkCtl: ASPM L0s L1 Enabled; RCB 64 bytes, Disabled- CommClk-
00:1c.4 PCI bridge: Intel Corporation Alder Lake-S PCH PCI Express Root Port #5 (rev 11) (prog-if 00 [Normal decode])
                LnkCap: Port #5, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
01:00.0 SATA controller: ASMedia Technology Inc. ASM1166 Serial ATA Controller (rev 02) (prog-if 01 [AHCI 1.0])
                LnkCap: Port #0, Speed 8GT/s, Width x2, ASPM L0s L1, Exit Latency L0s <4us, L1 <64us
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology P2 [Nick P2] / P3 / P3 Plus NVMe PCIe SSD (DRAM-less) (rev 01) (prog-if 02 [NVM Express])
                LnkCap: Port #1, Speed 16GT/s, Width x4, ASPM L1, Exit Latency L1 unlimited
                LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+

 

So I am using the same settings as before installing the card where I could reach package C10 states. I have applied --auto-tune and the disks seem to spin up and down as expected, and all tuneables are listed as 'good'. However, now I can only reach package state of C2, and core state of C10. Does anyone know how I can troubleshoot why the package is no longer going to C10 with the PCI card installed?

 

I have tried disabling all dockers and VM's I have installed, I am still running the same plugins (no new ones) and I am still not able to reach C10 with these disabled.

 

 

Confirmed it is definitely the card causing this. I started the server with the PCI E card removed and was able to instantly hit C10 on the package as a whole.

 

I'm using the same ASM1166 card everyone else is using and it's on a later firmware than the one that fixes the power states issue and as you can see from the quoted post, it does support ASPM.

 

The only other thing I've seen is that someone had to compile their own kernel to get it to go lower.

 

Any ideas on anything I can do or am I just stuck?