Running an Energy Efficient Unraid Server


Recommended Posts

PowerEfficiencyForum.png

 

With the rise of global electricity prices, (especially in Europe), running an energy-efficient Unraid server is more relevant than ever.

In this blog, we go over some best practices, tips + tricks, and build basics for users to run their servers in the most energy-efficient way possible!

 

https://unraid.net/blog/energy-efficient-server

 

How many watts does your Unraid server run at idle?

 

Do you have any other energy efficiency tips?

  • Like 10
Link to comment

Nice Job @SpencerJ. Keep in mind everybody he has been working on this for a while and its always been a touchy subject of performance vs economy. Why can't we have a little bit of both? AM I right? ;)

 

With my Current build I'm idling at 36watts and keep in mind I'm using the UPS readings from the dashboard. 

 

Here are a few things I use. 

  • I've always been a fan of oversized coolers and big slow spinning 120mm fans vs the pancake style. Much Quieter, moves more CFM and has to be way more energy efficient. 
  • Also I like the PSU with the fans that never seem to come on unless the PSU starts to warm up. 
  • I always recommend as much medal on my case as possible vs plastic because medal transfers heat way more efficiently than plastic. 
  • I also use the Dynamix Fan Plugin so my Cage fans are always off and only my exhaust fan is pulling in air unless the drives spool up and then the cage fans come on. 
  • I have my Parity drive set to spin down after 30min and my drives at 45min. Your times my vary, but I figure I read way more than I write on a daily basis. 

 

  • Thanks 1
Link to comment

my 5th UNRAID server ran at 250 watts for 24 hard drives and dual opteron processors with 24 cores i think.   

 

my 6th UNRAID server runs at 250-350 watts specs below. 

 

EST006 - CURRENT - MAIN - 

  • CASE: Norco-4224
  • MB: Supermicro X10SLL-F
  • CPU: Intel® Xeon® CPU E3-1246 v3 @ 3.50GHz
  • RAM: 32 GiB DDR3 Single-bit ECC (max. installable capacity 32 GiB)
  • GPU: NVIDIA P2000
  • ARRAY:
    • 2022-10: 136 TB  
  • CONNECTION: 
    • 1gbps symmetric fiber

 

I do not spin down hard drives ever on either

 

EST005 had 3x 120mm fans and big passive heat sinks on processors with 2x 80mm on the rear

 

EST006 has 6x 120mm magnetic lev fans and a corsair water cooler on the processor with 2x 80mm fans on the rear 

 

Both have additional copper on other components to keep heat down, hard drives on EST006 sit around 35c not used and 40c used and the processor sits around 55-65c

 

Both are relatively silent and under 50 db all the time.  

 

My suggestion to you is that the processors take the most and that E3-1246v3 is like under 40 watts power pull.   You want to focus on ultra low power processors with more cores and offload harder work to a NVIDIA P2000 or better.   

  • Thanks 1
Link to comment

I have a similar built server to bing's and idles at about ~200 watts from my observation, the smart plug its on reports a 4.95kwh daily average, which puts it closer to a 206watt average over 30 days.

 

Case: Norco 4224

MB: Supermicro X10SLR-F

CPU: Xeon E5-2630L v3 @ 1.8ghz (8c/8t)

RAM: 32gb DDR4 ECC

GPU: none (just onboard vga)

ARRAY: 23 drives for 131TB (most are 5400rpm), 2x 512gb NVME in raid-1 cache (supermicro pcie addon board)

NETWORK: Mellanox 10gbe fiber pcie

 

3x 120mm Noctua fans, plus the CPU fan (standard kit it came with) are the only fans in it.  Runs 27c in the case, 40c for CPU.  Drives are usually around 32-35c idle, upwards of 40-42 if under heavy access.

 

I do let disks spin-down, however only a handful really ever do because of dockers, VMs, other activity.  As of this writing only 6 of 23 drives are spun down.  And it's reporting 195watts as of this writing.  And usually is running about a dozen dockers and 2 VMs at all times for reference. Mover just runs once daily to migrate any cached data to the array (and thus spinning up all drives).

 

Hope that is useful for folks!

 

  • Thanks 1
Link to comment

Hello

 

My first post! Long time lurker and reader but i decided to write my first post here :)

 

My current setup:

 

MB: Supermicro X9SCL-F

CPU: G1610T

RAM: 4gb DDR3 ECC

GPU: None

Array: 5 x 2,5 hdd drives

Cooling: Arctic Alpine 11 passive on CPU, 2 x Scythe Gentle Typhoon in the case

PSU: 80+ Gold from Dell

 

Idling at 25W with drives spun down, 45W during parity check. 

 

It's cool and quiet :) I don't think it's worth for me to try and swap the server for newer items - the cost would have a very long return time. Did i mention i have solar panels? 

 

 

Edited by Horgrim
  • Upvote 1
Link to comment

My setup:

 

MB: Gigabyte C246N-WU2

CPU: Xeon E-2146G

RAM: 64GB DDR4 ECC

GPU: None

Array: 8x 18TB WD Ultrastar and White Label (shucked), spun down most of the time

Pool: 2TB Samsung Evo Plus NVMe

Cooling: 3x Noctua

Network: 10G Intel X550T RJ45 (a SFP+ DAC would be more efficient)

USB: Sonoff Zigbee USB Dongle

PSU: Corsair SF450 Platinum

Docker: Home Assistant, Nextcloud, Plex, HTTP-Fileserver, rsync-server, Nginx Proxy Manager

 

Consumption: 24W in idle (-6W without 10G card, -3W with my previous 12TB HDDs, -2W with Corsair RM550x 2021)

 

It raises by 10W if my son starts a Minecraft container. That's why I use a script which automatically stops it, if no players are active for 2 hours:

 

Plans for the future:

- SSD only built (collecting super expensive 8TB QVO SSDs at the moment 🤪)

- SFP+ card with active DAC cables

- Seasonic Titanium power supply

- remove the 10G RJ45 switch (instead my PC get's a direct connection to the SFP+ card, while the other household users access the server through 1G)

 

P.S. best power saving tip: 😅

https://www.reddit.com/r/unRAID/comments/ye3z55/comment/itw2cgz/?utm_source=share&utm_medium=web2x&context=3

Quote

step 1) live in an area where electricity is reasonably priced

 

 

  • Like 3
  • Upvote 1
Link to comment
On 11/2/2022 at 11:50 PM, mgutt said:

My setup:

 

MB: Gigabyte C246N-WU2

CPU: Xeon E-2146G

RAM: 64GB DDR4 ECC

GPU: None

Array: 8x 18TB WD Ultrastar and White Label (shucked), spun down most of the time

Pool: 2TB Samsung Evo Plus NVMe

Cooling: 3x Noctua

Network: 10G Intel X550T RJ45 (a SFP+ DAC would be more efficient)

USB: Sonoff Zigbee USB Dongle

PSU: Corsair SF450 Platinum

Docker: Home Assistant, Nextcloud, Plex, HTTP-Fileserver, rsync-server, Nginx Proxy Manager

 

Consumption: 24W in idle (-6W without 10G card, -3W with my previous 12TB HDDs, -2W with Corsair RM550x 2021)

 

 


I'm planning to build a quite similar server build, due to the incredible efficiency of your's.
What ECC RAM have you installed exactly ?

Link to comment
1 hour ago, websl said:

What ECC RAM have you installed exactly ?

Is not important. They are all equal in efficiency. But those are mine:

Screenshot_20221104-203448.png.3be7cca4df91a974f5451d59f57015dc.png

 

List of compatible modules:

https://geizhals.de/?cat=ramddr3&xf=15903_DDR4~15903_keinSO~15903_mitECC~15903_ohneLR~15903_ohneREG&sort=r&hloc=at&hloc=de&v=e

 

You can even buy faster modules as they run automatically slower if the CPU does not support the higher frequency. So focus only in the price.

  • Like 1
Link to comment
  • 3 weeks later...

I put 9KW of solar on the roof and export more than I use here in sunny north Queensland (Australia), in any given 24 hour period.  I no longer care how much power I use :)

Sure, we use about 7kwh overnight including the aircons and stuff, but once I evaluate the best battery solution I won't have that problem either.

 

The unRAID machine I have idles at about 200W and during some hefty workloads gets up to about 450W. 

Edited by DarthKegRaider
  • Like 1
  • Upvote 1
Link to comment

My server runs at 99 watts most of the time.

Ryzen 7 1600

32GB RAM

5 - 14TB 7200RPM drives

2 - 500GB SSD

P200 and AMD 6450 GPUs

 

I re-built the system about 3 years ago because I had a AMD FX8350 (I think) that ran over 200 watts and wanted to save energy and money.  I also got double the threads from 8 to 16.  I am thinking of upgrading just the chip to a later gen (G) with video to give me more options with VMS and decoding video.

Link to comment
On 11/2/2022 at 10:50 PM, mgutt said:

My setup:

 

MB: Gigabyte C246N-WU2

CPU: Xeon E-2146G

RAM: 64GB DDR4 ECC

GPU: None

Array: 8x 18TB WD Ultrastar and White Label (shucked), spun down most of the time

Pool: 2TB Samsung Evo Plus NVMe

Cooling: 3x Noctua

Network: 10G Intel X550T RJ45 (a SFP+ DAC would be more efficient)

USB: Sonoff Zigbee USB Dongle

PSU: Corsair SF450 Platinum

Docker: Home Assistant, Nextcloud, Plex, HTTP-Fileserver, rsync-server, Nginx Proxy Manager

 

Consumption: 24W in idle (-6W without 10G card, -3W with my previous 12TB HDDs, -2W with Corsair RM550x 2021)

 

It raises by 10W if my son starts a Minecraft container. That's why I use a script which automatically stops it, if no players are active for 2 hours:

 

Plans for the future:

- SSD only built (collecting super expensive 8TB QVO SSDs at the moment 🤪)

- SFP+ card with active DAC cables

- Seasonic Titanium power supply

- remove the 10G RJ45 switch (instead my PC get's a direct connection to the SFP+ card, while the other household users access the server through 1G)

 

P.S. best power saving tip: 😅

https://www.reddit.com/r/unRAID/comments/ye3z55/comment/itw2cgz/?utm_source=share&utm_medium=web2x&context=3

 

 


 

How are you measuring those numbers? Im just curious as those numbers are less than mine and I don’t see how or why.


The following draws 29W at the absolute minimum (measured via three methods) but typically power meter shows 30-31W when all drives spun down.


i5-8600T (under-volted), saved 1W at idle and gains me 5% performance when hitting 35W power limit!

16GB DDR4 (non-ECC)

H310CM-HDV/M.2

1TB M2

Intel Quad-Port NIC (adds 4-5W)

Four 14TB WD (Shucked)

Seasonic 550W Platinum


I have less disks (spun down call that 0.4W a disk based on data sheets) means your 1.6W higher nominally.

 

48GB more memory, difficult to put numbers on it but 0.1W/GB seems not far off so that’s another 4.8W. Plus you have ECC which adds another chunk too due to the IO overheads and chip on the PCB.

 

Plus im running a T CPU which typically idles a tad better than the higher TDP CPUs due to being less leaky silicon (lower idle voltage).

 

Taking the NICs out of the equation:

 

Your machine: 18W

My machine: 25W

 

So unless my motherboard/PSU at these low loads are totally rubbish efficiency wise you should be in the 35W region, not 18…!

 

A little HP i3-8100T SFF desktop I tested drew 11W at idle, and that had 8GB RAM, no HDDs other than a small SSD and a 180W 12V only PSU in it.

 

When we are talking about these low numbers, we need to be taking accurate measurements 👍
 

Next upgrade will be when the ATX12VO stuff comes along, then I’ll upgrade to a 13/14th Gen T CPU, 300W PSU, 32GB DDR5, etc.

 

Hopefully bring system power down below 20W.

Edited by Interstellar
Link to comment
4 hours ago, Interstellar said:

When we are talking about these low numbers, we need to be taking accurate measurements 👍
 

I'm using a Fritz!DECT 200. Winner in multiple tests because of its accuracy 😉

 

4 hours ago, Interstellar said:

H310CM-HDV/M.2

Maybe not a good board regarding efficiency. ASRock often disallows very low C-States.

 

 

4 hours ago, Interstellar said:

Intel Quad-Port NIC (adds 4-5W)

Bad and kills your C-States

 

4 hours ago, Interstellar said:

Seasonic 550W Platinum

Bad for low loads. Use the Corsair RM550x (2021). Should save additional 3W.

 

4 hours ago, Interstellar said:

i5-8600T (under-volted), saved 1W

I tested this with several Intel CPUs, too, and it saved only 1W while my server crashed two times while unzipping a huge file. That's because I don't undervolt anymore.

 

PS: A T-CPU does not save energy. Instead it costs you more energy as processes like a parity check takes much longer. If you need a small TDP because your want to use a small or passive cooler: Set the TDP as you need it through the BIOS. No need to buy an expensive T-CPU.

 

4 hours ago, Interstellar said:

48GB more memory, difficult to put numbers on it but 0.1W/GB seems not far off

One module needs <1W. There is no real difference between 2x 4 or 3x 32GB.

 

4 hours ago, Interstellar said:

A little HP i3-8100T SFF desktop I tested drew 11W at idle, and that had 8GB RAM, no HDDs other than a small SSD and a 180W 12V only PSU in it.

Bad. ^^

 

Check this thread:

https://www.hardwareluxx.de/community/threads/die-sparsamsten-systeme-30w-idle.1007101/

 

And: You didn't write anything about C-States, ASPM, AHCI Link Power Management, CPU Governor, etc. There are many settings available to save energy:

 

 

 

  • Like 1
  • Thanks 1
Link to comment
9 hours ago, mgutt said:

I'm using a Fritz!DECT 200. Winner in multiple tests because of its accuracy 😉

 

2% above 5W, seems solid enough :D

 

9 hours ago, mgutt said:

Maybe not a good board regarding efficiency. ASRock often disallows very low C-States.

 

Will check, cores certainly spend a lot of time in C7, will restart my server tomorrow to see what powertop says.

 

9 hours ago, mgutt said:

Bad and kills your C-States

 

Maybe so, but I've run it with and without the card in, and it was a 4-5W change, so technically the contribution of the card is removed anyway.

 

9 hours ago, mgutt said:

 

Bad for low loads. Use the Corsair RM550x (2021). Should save additional 3W.

 

Isn't that a Gold rated PSU? Mine is Platinum so I'd hope its better :D

 

 

9 hours ago, mgutt said:

 

I tested this with several Intel CPUs, too, and it saved only 1W while my server crashed two times while unzipping a huge file. That's because I don't undervolt anymore.

 

Heavily stress tested this CPU, prime95 for 48 hours, various memory tests, and 90 days uptime all seems good at -50mV!

 

 

9 hours ago, mgutt said:

 

PS: A T-CPU does not save energy. Instead it costs you more energy as processes like a parity check takes much longer. If you need a small TDP because your want to use a small or passive cooler: Set the TDP as you need it through the BIOS. No need to buy an expensive T-CPU.

 

Doesn't make my parity check take longer and I don't believe this is the case. There are low leakage CPUs (Mobile) and high leakage CPUs (Tend to go into high end desktop, i.e. K). The 8600T has an idle voltage lower than the 8600K I used to have (600mV against 700mV).

 

 

9 hours ago, mgutt said:

 

One module needs <1W. There is no real difference between 2x 4 or 3x 32GB.

 

It's very difficult to find idle values for memory power usage. Websites such as this show a marked change in power usage when you increase capacity: https://www.tomshardware.com/reviews/intel-core-i7-5960x-haswell-e-cpu,3918-13.html but likely that is under load.

 

Ultimately you have more RAM, that is more memory that needs to be refreshed every x number of cycles,  plus ECC which requires an additional die to store the parity information IIRC and then the memory controller in theory is working harder (albeit not at idle). How much those things add is the question...

 

 

9 hours ago, mgutt said:

 

Bad. ^^

 

11W seemed pretty damn good at the time :D

 

9 hours ago, mgutt said:

 

Check this thread:

https://www.hardwareluxx.de/community/threads/die-sparsamsten-systeme-30w-idle.1007101/

 

And: You didn't write anything about C-States, ASPM, AHCI Link Power Management, CPU Governor, etc. There are many settings available to save energy:

 

Will have to check these details...! I still don't see how I'd get anywhere near your power draw though.

 

9 hours ago, mgutt said:

 

 

 

 

Link to comment

I got a corsair PSU & the plugin shows on idle when HDDs are spun down, I'm running at 100 WATTS approx, my Ryzen 1700x is also OC to 3.8Ghz. In addition to a pfSense box, 24 port switch and the overhead of the UPS my power meter shows around 200 WATTS

 

My second Unraid server is almost always offline & only used for weekly or monthly backups, but also runs under 100 watts when powered on.

 

Thankfully, power where I live is pretty cheap at 0.05/kWh

Link to comment

Excellent blog-post. I live in Europe and my energy-bill is twice my mortgage which is ridiculous. Last month my combined energy-bill (power/gas) got estimated at 1500,-- euro's which is insane. Is half my salary.

 

My current action to reduce power-consumption:

  1. CPU governor to power saving.
  2. Stop letting unraid decide where and how to scatter the shares: my static and rarely used data is on 3, 3TB drives. They hardly spin up anymore where the family shares are on two 4TB share - they are WD Red but it think they should be replaced with WD Green or some other low-power disks. I got now an old 1.5T and a couple of obsolete 2TB drives that should be removed.
  3. Mover scheduled to move once a day (don't want to set that at a lower frequency due to possible data-loss - I got no clue how much power ssd's draws so I might need to set up the cache as a raid 1 cache-pool)

1) and 2) greatly reduced power-consumption.

 

Further actions I want to take is removing disks (and should also make one of the two lsi 9211-8i adapters obsolete)

  • Shrink the array by:
    • Remove one of the two parity drives and use a spare data-disks
    • Remove the unassigned disks and empty disks from the array
  • Further
    • Move the the vm's from 2 unassigned drives to the cache pool

When that is done I need to look at the hardware itself - currently on a quite old i7 - although it's TDP is rated a 35Watt's unraid does not need an i7 and nor do the vm's or dockers (nginx proxy manager and gitlab-ce). Also want the server to shutdown at night but my home-assistant and personal website are also on the server so not entirely sure how to deal with that a.t.m. It's just that you cannot by mobile cpu's rated at 10/15Watt's to put into a mitx board..

 

 

Link to comment
  • 5 weeks later...

I am also trying to get my idle consumption as low as I can and maybe someone has some helpful info to share.

 

My system:

i5 8500

Asus Prime H370-A

2x 8 GB RAM

1x HBA LSI 9200-16e

1x 500GB Cache NVME

11x HDD

PSU: Corsair HX750i

CPU Cooler: Artic Freezer 11 LP

3x 120mm fan Noctua Low Noise Adapter

1x 80mm fan with Noctua Low Noise Adapter

 

powertop shows, that the maximum c-state reached is c3 in Pkg(HW) ...if powertop is correct

 

I am idleing at around 45W with drives spun down

  • Like 1
Link to comment
On 11/2/2022 at 11:50 PM, mgutt said:

My setup:

 

MB: Gigabyte C246N-WU2

Is there any possibility to get this board in Germany/Switzerland? I am looking for it for a while now but it seems Gigabyte does not produce it anymore?

It would be the perfect fit to upgrade my current setup.

Link to comment
8 hours ago, pille said:

 

1x HBA LSI 9200-16e

PSU: Corsair HX750i

That's your problem. Buy the Corsair RM550x (2021) and swap the HBA card against a SATA card with ASM1166. But it will still kill your C-States. That's the downside of a pcie card. But with the changes you should safe 15 to 20W.

 

7 hours ago, UnKwicks said:

Is there any possibility to get this board in Germany/Switzerland?

No. Sadly sold out worldwide for a very long time. And there is no alternative. The next board I will try this one:

https://forums.unraid.net/topic/131537-asrock-rack-w680-w680d4id-2t-mit-i7-13700/

Link to comment
5 hours ago, mgutt said:

That's your problem. Buy the Corsair RM550x (2021) and swap the HBA card against a SATA card with ASM1166. But it will still kill your C-States. That's the downside of a pcie card. But with the changes you should safe 15 to 20W.

Sounsd great. Thanks a lot, will give it a try. Can I just interchange the HBA with a card like this? SATA Card Or might that cause any problems to my array? Or are there any recommendations? What might be a downside, since I always read that one should use HBA instead of SATA cards (not consumption wise).

Do you know how much the power supply change will lower the W approximately? Asking because if it is just some W I might need to run it a couple of years to have a benefit. Guess in that period I am going to buy a new PSU anyway. Also saw, that the RM550x has only 3 SATA plugs, which means, that one can not add more then 12 drives, if I am correct?

Edited by pille
Link to comment
10 hours ago, pille said:

Do you know how much the power supply change will lower the W approximately?

5 to 10W. This is extremely different.

 

10 hours ago, pille said:

with a card like this? SATA Card

Bad card. The ASM1166 supports only 6 SATA ports. 

 

10 hours ago, pille said:

the RM550x has only 3 SATA plugs, which means, that one can not add more then 12 drives, if I am correct?

Simply buy more cables. You can even adapt molex to power SATA. Maybe you can reuse those from the old one as it is the same brand.

 

Link to comment
1 hour ago, mgutt said:

Bad card. The ASM1166 supports only 6 SATA ports.

Thanks for the info. So one just need to know that or how to determine that?

 

1 hour ago, mgutt said:

Simply buy more cables.

Am I missing something here? There are 3 of these plugins.

Screenshot_20230104_110251.png.7711ac2175f84335ade62ce526c4b3d5.png

 

which are connected with these kind of cables

Screenshot_20230104_110801.png.d77a2a31f18e10110bce03a88a125c2f.png

So this is 3 cables with 4 connectors. Adding up to 12drives being able to be powered

 

 

Or do you mean buying these kind of cables

Screenshot_20230104_110546.png.393e80cebfb9371a166d5ab2bd938bdb.png

and connect each of them to a cable like this one?

Screenshot_20230104_111758.png.554bb230446c423a374d5db1b809c780.png

That would add up in 24 drives, if that works voltage wise.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.