So, reading through this 14 pages; seems as if there were some kinks to work out in order to get this board fully working.  With all that effort that you all had to do; would you do it again?  Asking as my 13 year old dual xeon box is starting to get a little long in the tooth and the 170W idle is not the best way to spend my money.  I like the idea of ECC ram and this seems to be the only "good" option for someone using an intel i7-12700.

 

Anyways, hoping to get some feedback if it's worth it to go this route vs just getting a consumer 790 board...  For reference my current box runs all the various Arr's, 3 vm's (windows server, windows 11 for light gaming, home assistant), plex, and another 15 or so dockers.  I have 256gb of memory in my current workstation build on an older Supermicro X9DAI board and a pair of 12 core xeons. Currently I have 17 drives across 13 spinners (1 parity, 12 pool) and 4 SSD's in various cache roles and a Bluray drive for rips.  I use some older perc310's flashed into IT mode but have already bought a 9400-16i for whatever new setup I get and I picked up a pair of 2tb NVME samsung 990 pro's to be my new VM/Docker cache pool. 

 

Thanks in advance for any Yay's or Nea's out there.  

Edited May 1, 20251 yr by dnoyeb

On 4/6/2025 at 2:48 PM, Daniel15 said:

There's an option in the BIOS called something like "iGPU multi monitor support". I think that has to be enabled.

 

As far as I know, HDD temperature isn't supported as a fan temperature source. It's not in the list in the IPMI web UI:

 

 

and `ipmisensors` doesn't show any HDD temperature sensor attached to the IPMI.

 

I've never tried it, but the "Dynamix System Autofan" plugin might be useful? AFAIK it can control fan speed based on HDD temperature sensors.

 

To be honest, I'd just get quiet fans (like Noctuas) and run them at a fast speed all the time. There's not really such a thing as too much cooling

Thanks for that.  I tried adjusting the BIOS setting you mentioned and it still doesn't seem to work.  Maybe I'll need to go through it again... it's not critical, I suppose, since accessing the BIOS is the key feature I really needed, and I can also still set what mode Unraid loads into.  It would be nice to have it working fully in the event of a bigger issue, and it's just puzzling as to why it's having an issue.

 

1 hour ago, dnoyeb said:

So, reading through this 14 pages; seems as if there were some kinks to work out in order to get this board fully working.  With all that effort that you all had to do; would you do it again?  Asking as my 13 year old dual xeon box is starting to get a little long in the tooth and the 170W idle is not the best way to spend my money.  I like the idea of ECC ram and this seems to be the only "good" option for someone using an intel i7-12700.

 

Anyways, hoping to get some feedback if it's worth it to go this route vs just getting a consumer 790 board...  For reference my current box runs all the various Arr's, 3 vm's (windows server, windows 11 for light gaming, home assistant), plex, and another 15 or so dockers.  I have 256gb of memory in my current workstation build on an older Supermicro X9DAI board and a pair of 12 core xeons. Currently I have 17 drives across 13 spinners (1 parity, 12 pool) and 4 SSD's in various cache roles and a Bluray drive for rips.  I use some older perc310's flashed into IT mode but have already bought a 9400-16i for whatever new setup I get and I picked up a pair of 2tb NVME samsung 990 pro's to be my new VM/Docker cache pool. 

 

Thanks in advance for any Yay's or Nea's out there.  

Other than my IPMI KVM issue (where the IPMI card is optional, anyway), yes, I'd get it again.  All of my issues have been with the IPMI card; the motherboard, itself, seems to work just fine.  There aren't too many choices for ECC-supporting motherboards on the Intel side, and I think this is one of the better ones.

 

If I were buying again, the only thing I might change would be getting an AMD processor instead of Intel.  I went Intel because I wanted to have transcoding capabilities without having to buy a dedicated GPU.  But I ended up buying a GPU anyway (Intel A310 - low profile, and quite cheap).  There seems to be wider ECC support on the AMD side, and when I briefly looked it over, I think the IPMI functionality might be baked into the motherboard instead of requiring a card that takes up a PCI slot.

1 hour ago, dnoyeb said:

With all that effort that you all had to do; would you do it again?

I'd definitely get it again. The quirks don't really matter too much once it's up and running.

The price may seem expensive, but you need to consider the fact that it has IPMI built-in. You'd need to pair a cheaper motherboard with something like a PiKVM / NanoKVM to get similar functionality.

 

18 minutes ago, Ledgem said:

 There seems to be wider ECC support on the AMD side

The difference with AMD is that while many motherboards support it with consumer-grade Ryzen CPUs, it's not "officially" supported by AMD. They don't advertise it as a feature, don't validate it (meaning they don't test and certify it to ensure it works properly), and won't provide support if something goes wrong. From what I've seen online, the ECC support on Ryzen is incomplete. For "official" ECC support with AMD, you need to go with Threadripper or EPYC.

 

On the other hand, Intel does officially support ECC on their consumer-grade CPUs, but artificially limit it to workstation (like W680) and server motherboard chipsets instead. I say "artificially limit" because the memory controller is on the CPU, so there's no technical reason that consumer-grade motherboards can't support it - it's purely to differentiate consumer-grade vs workstation vs server product lines.

Edited May 1, 20251 yr by Daniel15

Thanks for the feedback to you both!  Looks like I have finally found my motherboard; now to figure out the memory to get to max this thing out .  Searching for ECC DDR5 is a bit tricky for sure... (so if there are any model number recommendations, please feel free to post them, i'd ideally max it out from the get go).

 

thanks!

1 hour ago, dnoyeb said:

 if there are any model number recommendations, please feel free to post them

 

I've got 2 x Kingston server KSM48E40BD8KM-32HM (32GB DDR5 ECC 4800MT/s) and 2 x KSM56E46BD8KM-32HA (5600MT/s) for a total of 128GB. They're both listed in Asus' compatibility list. It's all running at 4800MT/s with no issues. The only reason the speeds differ is that the 5600MT/s sticks weren't available when I first set up the server with 64GB RAM. When I went to upgrade it, the 5600MT/s sticks were the same price as the 4800MT/s used to be, so it didn't make sense to buy the slower ones. I'll have to upgrade the slower ones if I ever want to use the faster speeds though.

 

RAM has gone up a LOT since I got mine. I got the 4800MT/s sticks in August 2023 and the 5600MT/s sticks in July 2024. They were $101 each through a supplier we use at work (we get a discount for personal purchases). As of a few months ago, they were double the price, and if you're in the USA then they may go up even more thanks to tariffs.

The model numbers ending with "M" are Hynix M-die while the ones ending in "A" are Hynix A-die. The part numbers may be different now - you can search Kingston's site.

 

If you want to max it out, you'll need 4 x 48GB for 192GB total. The Kingston part number is KSM56E46BD8KM-48HM.

 

Do not overclock ECC RAM, as ECC handling is only verified at the base speed.

Edited May 1, 20251 yr by Daniel15

Unraid lists this motherboard as being able to max out at 256 GB.  I thought that was a glitch, since all of the documentation that I saw indicated 192 GB as the maximum, but when I looked it up a few weeks ago it seems that 256 GB is indeed the maximum.  I'm not sure if that was enabled with a firmware update, or if that capacity wasn't available when Asus did the initial testing with this motherboard.

 

I also have 128 GB, and funny enough, also have mismatched speeds (4800 MHz and 5600 MHz).  In my case it's because I went with OWC (purchased from Amazon) and had to do an exchange.  I initially bought 64 GB (2x32 GB) and then decided to just max out all RAM slots, buying a second pair of 32 GB sticks.  My system wouldn't boot at all if a specific two of the RAM sticks were installed.  I did an exchange through Amazon and the two sticks that they sent me as replacements were the 5600.  It works fine, ECC seems to be registering, so we're good.  Two of the RAM sticks are Samsung and two are Micron Technology.  My IPMI card reports that they're operating at 4200 MHz... meh.

 

I wanted to go with Kingston RAM, as that's what most people remarked upon, but the only ECC options I could easily find were the OWC or Nemix options.  I've used Nemix ECC RAM in my Synology system without problems, but that was my only experience with them.  As a primary Mac user, OWC sort of has legendary status and so I went with them, but given my experience (or if I were buying again to upgrade further), I'd probably go with Nemix unless OWC were significantly cheaper.

 

128 GB is probably more than I need.  I have a ZFS pool and the arc cache typically uses 1/8 of RAM; I manually adjusted it to use 50% of the RAM, and there was a very noticeable improvement in performance.  (Copying data from my Synology, with both systems running on 10 Gbps SFP+ connections, the speed barely went above 2.5 Gbps burst speeds with only 16 GB of cache, but with 64 GB I'd burst to close to 8 Gbps and the sustained rates were also higher.)

Yeah I found the press release from March 24 where they said this board and many others in the 700/600 range had been upgraded to 256gb max.  I cannot find any 64gb sticks with ecc though, perhaps I’m just not searching right…  If any of you have seen them, please let me know!  

Edited May 2, 20251 yr by dnoyeb

Thanks again for everyone's responses; went ahead and ordered the board, a pair of 48gig sticks, an i7-12700k, and a cpu cooler; should be up and running next week. 

 

 

@dnoyeb I forgot to mention... When you're setting up the system, make sure you go into the BIOS and disable fast boot, otherwise it'll just hang on boot. It's a common issue with Unraid, but it's easy to forget.

On 5/1/2025 at 11:51 PM, dnoyeb said:

Yeah I found the press release from March 24 where they said this board and many others in the 700/600 range had been upgraded to 256gb max.  I cannot find any 64gb sticks with ecc though, perhaps I’m just not searching right…  If any of you have seen them, please let me know!  

Nemix 4800 MHz ECC DDR5 RAM (link is to a 256 GB set of 4x64 GB sticks currently listed for $1,356, but the same page also lists other multiples of 64 GB sticks)

OWC 5600 MHz ECC DDR5 RAM (Nemix also lists a 5600 MHz variant on a different search page, but it's the most expensive of the lot so I'm not linking it)

 

EDIT: these links are for registered (buffered) DIMMs, but the W680 requires unregistered (unbuffered) DIMMs - my mistake, they're not going to be compatible.

Edited May 6, 20251 yr by Ledgem

5 hours ago, Ledgem said:

Nemix 4800 MHz ECC DDR5 RAM (link is to a 256 GB set of 4x64 GB sticks currently listed for $1,356, but the same page also lists other multiples of 64 GB sticks)

OWC 5600 MHz ECC DDR5 RAM (Nemix also lists a 5600 MHz variant on a different search page, but it's the most expensive of the lot so I'm not linking it)

Those are rdimms though, pretty sure we can only use udimm's?  

 

13 hours ago, Daniel15 said:

@dnoyeb I forgot to mention... When you're setting up the system, make sure you go into the BIOS and disable fast boot, otherwise it'll just hang on boot. It's a common issue with Unraid, but it's easy to forget.

Thanks!  

 

Got the system swapped out this weekend, ended up in replacing the PSU as well with a corsair rm850 as they seem to be much more efficient than the older evga one I had from 7 years ago.  This thing is a beast!  stress tested with downloading at full gig, unraring/repairing a file, while running 2 vm's, transcoding (don't ever transcode but wanted to try to make sure I had the intel setup right) with plex, and moving 100 of gigs via MC from one disk to another to balance stuff out after adding a new data drive.  Didn't break a sweat!  

 

Now the bad... So, one thing I thought i'd skimp on was the IPMI version and go with a non as I found an open box on bhphoto for a great deal.  Now after going back through this thread and seeing that I cannot actually get all that fun raw data off the normal non ipmi version; i'm jaded... I really like data! Thankfully amazon had an openbox ipmi version on a great price so I grabbed it.  Here's hoping it doesn't have any issues.  Will swap it out and return the non ipmi version back to bhphoto.  

Edited May 5, 20251 yr by dnoyeb

10 hours ago, dnoyeb said:

Those are rdimms though, pretty sure we can only use udimm's?  

 

Thanks!  

 

Got the system swapped out this weekend, ended up in replacing the PSU as well with a corsair rm850 as they seem to be much more efficient than the older evga one I had from 7 years ago.  This thing is a beast!  stress tested with downloading at full gig, unraring/repairing a file, while running 2 vm's, transcoding (don't ever transcode but wanted to try to make sure I had the intel setup right) with plex, and moving 100 of gigs via MC from one disk to another to balance stuff out after adding a new data drive.  Didn't break a sweat!  

 

Now the bad... So, one thing I thought i'd skimp on was the IPMI version and go with a non as I found an open box on bhphoto for a great deal.  Now after going back through this thread and seeing that I cannot actually get all that fun raw data off the normal non ipmi version; i'm jaded... I really like data! Thankfully amazon had an openbox ipmi version on a great price so I grabbed it.  Here's hoping it doesn't have any issues.  Will swap it out and return the non ipmi version back to bhphoto.  

You know, I actually can't find anything specific on whether the W680 (and Intel's processors) support registered DIMMs or only unregistered for ECC.  The Asus website does mention unbuffered... you're probably right not to experiment, sorry for that.

 

The IPMI card does give some interesting data but the Asus IPMI interface isn't great.  The IPMI data (including fan control) does not carry over to Unraid by default.  Maybe there's a way to tweak it.

 

I don't regret getting the IPMI card - it's been useful for me.  I just wish it was built-in to the motherboard, as I think they did with the AMD version.

58 minutes ago, Ledgem said:

You know, I actually can't find anything specific on whether the W680 (and Intel's processors) support registered DIMMs or only unregistered for ECC.

This board only supports UDIMMs, as per the specs: https://www.asus.com/us/motherboards-components/motherboards/workstation/pro-ws-w680-ace/techspec/

 

Quote

4 x DIMM, Max. 192GB, DDR5  ECC and Non-ECC Un-buffered Memory*

As far as I know, all W680 boards only support UDIMMs. I checked ASRock Rack and theirs are only UDIMM too: https://www.asrockrack.com/general/products.asp?Model=w680

 

Generally, only server-grade boards with server CPUs (Intel Xeon, AMD EPYC, etc) support RDIMMs.

Edited May 6, 20251 yr by Daniel15

Has anyone done some testing on performance of doing some common tasks with different drives/shares on different locations on the motherboard? 

 

Here is my original setup that allowed me to maximize performance on my old system; no issues downloading at gigabit while unpacking files etc; all while using plex/vms etc.

4 SSDs:

1) Array-Cache

2) Docker-Cache

3) Download-Cache (downloads temporarily hit here and then unpack to the Array-Cache)

4) VM-Cache

 

Question 1:

Now with this new system having NVME support, I want to maximize my performance and I think I can potentially consolidate down some since NVME's perform so much faster. So, is option A or B better?  

 

A) Docker / VM cache on an NVME in m.2-1 (the slot attached to the CPU directly) to maximize performance on things like Plex's database / images etc and then put an NVME in one of the other m.2 slots to host downloads / array cache?

 

B) Download / Array Cache on an NVME in m.2-1 (the slot attached to the CPU directly) to maximize performance of downloads/unpacking; and then put the Docker / VM Cache on a second NVME in the other m.2 slot.

 

C) something else?

 

Question 2:

any thoughts on if there are any performance gains for an HBA card like the 9400-16i connected to the Gen5 slot that's directly connected to the CPU?  I'm just not sure if spinning drives and that card would benefit at all from being directly attached to the CPU.  

 

 

 

For most of what we do the IOPS are more important than raw speed, and IOPS shouldn't be affected by which slot the NVME is in.  So for your first question, A or B, I'm not sure that it really matters.  What may matter more would be considerations for airflow, since the first slot has a pretty nice heat sink and good exposure, while the second slot does not come with its own heat sink and it's more covered by any PCI cards.  For what it's worth, I added a passive heat sink to my NVME in that second slot, both NVMEs are in a mirrored pool, and their temperatures are about the same.

 

For placement of the HBA card, I don't think there's much of a difference in performance.  Here my consideration is again based around airflow.  In my chassis, the final PCI slot is right near the chassis wall, and there are a lot of cables that I routed down that way.  I originally had my HBA card in the third slot (counting up with #1 being closest to the CPU), but found the heat sink was painfully hot to the touch when powering down the system. I added a 20x10mm Noctua fan, which fit well but made me think I was just heating up the next card in line.  I moved the HBA card to the first slot and changed the fan for a 20x20mm Noctua, then moved the 20x10mm fan to the heat sink of my SFP+ card (which is in slot #2 - again, has the second most space between the next slot).  If not actively cooling the heat sinks then having the HBA card in slot 3 or 4 might result in better airflow from chassis fans, depending on your setup.

 

Adding the fans to the heat sinks wasn't too difficult.  For my HBA card (LSI 9200-8i I think) the grooves in the heat sink are not wide enough for the screw-in method I've seen others use, so I got some medium-length twist ties, looped them around the heat sink springs (two), and then looped them through two of the fan's screw holes.  The fan is oriented to suck air up from the heat sink and blow it away, so it's not like the securing mechanisms are fighting air flow.  For my SFP card (a dual-port Intel SFP+ card) the grooves in the heat sink were wide enough that you just put in a screw (I can't remember if I used a M3 or M4, nor what length) and just twist the screw as if you're screwing something in.  The screw gently wedges its way into the groove and secures the fan nicely.

one thing i'm noticing so far with the non ipmi board is that the default speeds it's doing my case fans is just too low... Need to adjust them all up as I have an ancient Lian Li tower server case that has compartments for things like the PSU, HDD's, motherboard, 5 1/4inch bays etc.  I'm seeing the temps of the area with the motherboard are just fine; but the bays with things like the HDD's have their fans running way too slow which is resulting in hitting excess of 50c on the drives during big writes.  

 

Here's hoping swapping over to the other version of the board (hopefully later today) will give me that extra functionality to edit the speeds and get things cooler like they used to be.  

 

I did go ahead and get another heatsink for the nvme; noticed the one without it was running almost 10c hotter!  Should be installing that today as well... 

 

One nice part about not needing to put that HBA card in those CPU direct slots is now I can save them for a dedicated GPU that I plan to get in the future so I can do some gaming in a VM with passthrough etc.  

I’ve recently upgraded my Unraid 7.1 server to a new motherboard and wanted to share my experience as well as ask for help with a strange issue I’m encountering.

 

New System Specs:

Motherboard: ASUSTeK COMPUTER INC. Pro WS W680-ACE IPMI (Rev 1.xx)

BIOS: American Megatrends Inc. Version 4101 (Dated 12/03/2024)

CPU: Intel® Core™ i9-12900K (12th Gen) @ 3.17 GHz

Memory: 96 GiB DDR5 Single-bit ECC (max supported: 256 GiB)

Cache: L1: 384 KiB / 256 KiB / 512 KiB, L2: 14 MiB total, L3: 30 MiB

Networking: eth0 – 2.5 Gbps, full duplex

Kernel: Linux 6.12.24-Unraid x86_64

HVM/IOMMU: Enabled
 

The system runs great overall, but I’ve noticed that the onboard USB ports seem extremely picky about what devices they accept:

A USB 2.0 flash drive with my Unraid OS – not recognized

USB 2.0 mouse and keyboard – not recognized

USB 2.0 UPS – not recognized

Plug in a USB 3.0 flash drive – recognized immediately

 

Strangely, if I plug those same USB 2.0 devices into the front USB 3.0 ports (connected to the internal USB 3 header on the motherboard), everything works as expected.

 

I’ve already set the recommended BIOS settings from this thread, but the issue persists.

Has anyone else experienced something similar with this board or chipset? Any ideas on how to get the rear I/O USB 2.0 ports to work reliably with legacy devices?

Edited May 8, 20251 yr by Cout99

13 hours ago, Cout99 said:

I’ve recently upgraded my Unraid 7.1 server to a new motherboard and wanted to share my experience as well as ask for help with a strange issue I’m encountering.

 

New System Specs:

Motherboard: ASUSTeK COMPUTER INC. Pro WS W680-ACE IPMI (Rev 1.xx)

BIOS: American Megatrends Inc. Version 4101 (Dated 12/03/2024)

CPU: Intel® Core™ i9-12900K (12th Gen) @ 3.17 GHz

Memory: 96 GiB DDR5 Single-bit ECC (max supported: 256 GiB)

Cache: L1: 384 KiB / 256 KiB / 512 KiB, L2: 14 MiB total, L3: 30 MiB

Networking: eth0 – 2.5 Gbps, full duplex

Kernel: Linux 6.12.24-Unraid x86_64

HVM/IOMMU: Enabled
 

The system runs great overall, but I’ve noticed that the onboard USB ports seem extremely picky about what devices they accept:

A USB 2.0 flash drive with my Unraid OS – not recognized

USB 2.0 mouse and keyboard – not recognized

USB 2.0 UPS – not recognized

Plug in a USB 3.0 flash drive – recognized immediately

 

Strangely, if I plug those same USB 2.0 devices into the front USB 3.0 ports (connected to the internal USB 3 header on the motherboard), everything works as expected.

 

I’ve already set the recommended BIOS settings from this thread, but the issue persists.

Has anyone else experienced something similar with this board or chipset? Any ideas on how to get the rear I/O USB 2.0 ports to work reliably with legacy devices?

Super weird... I'm not having any issues like that from my older units.  What happens when you put them in the other colored usb slots on the back?  Just to verify, you do have fast boot disabled right?  

 

I wonder if I wouldn't factory reset it, boot without anything and turn off fast boot; reboot and go back into bios.  While in there, plug in the usb stick with unraid and verify that it shows up in the usb menu area (I forget where exactly, but know mine showed up; I went in and changed mine to show up as a hard drive).  

 

One other weird thing with mine was I couldn't get the USB's fully seated in a couple of the slots unless I really pushed them.

 

 

Topic: Controlling Fans Based on HDD Temperatures with IPMI

 

After looking around here and in the support thread for the “IPMI Tools” without finding anything useful for fan control based on HDD temperatures – specifically for the ASUS Pro WS W680-ACE IPMI – and after the “Dynamix System Temp” plugin (including drivers) didn’t help either, I’ve been doing a bit of experimenting lately. Conclusion: Using a few packages and “user script” you can build your own fan controller for this board…

 

But first, maybe the most important bit upfront before I get into the details. At the beginning, I just couldn’t figure out the right IPMI command to change the fan/PWM settings. The manual wasn’t helpful, but eventually ASUS support came through and provided the info I needed. So for those of you who just want the quick answer on how to control things via IPMI, here’s the relevant bit straight from ASUS:

 

“Please note that a FAN source that doesn’t physically exist can’t be activated or ‘created’ via IPMI either. But if you want to manually control, for example, an HDD fan, that’s basically possible via IPMI.

Here are some basic commands to control fan modes:

• Set fan mode to ‘Generic’:

ipmitool raw 0x30 0x0e 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

• Set fan mode to ‘Full Speed’:

ipmitool raw 0x30 0x0e 0x03 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00

• Set fan mode to ‘Custom’ (manual):

Example for fan zone “0” with multiple temp/duty pairs – this would reach 100% fan speed at 80 °C:

ipmitool raw 0x30 0x0e 0x04 0x00 0x28 0x32 0x32 0x41 0x3c 0x4b 0x46 0x55 0x50 0x64

 

Explanation of values (each pair is temp and fan speed %):

• 0x28 = 40 °C → 0x32 = 50 %

• 0x32 = 50 °C → 0x41 = 65 %

• 0x3c = 60 °C → 0x4b = 75 %

• 0x46 = 70 °C → 0x55 = 85 %

• 0x50 = 80 °C → 0x64 = 100 %

 

If you want to target a different fan zone, change the fourth value:

• 0x00 = Zone 1

• 0x01 = Zone 2, etc.”

 

Important: What this means is that you can’t just set a single PWM value per fan – you always have to define an entire fan curve. Also, keep in mind that once the BMC receives a fan curve via IPMI, it still needs a temperature input to decide what PWM value to use. Unfortunately, the BMC only has access to the CPU temperature and external physical sensors – no HDD temps. So if you want to set a fixed PWM value safely, you need to create a fan curve where all the temperature points lead to the same fan speed, set the trigger temperature to 1°C, and select the CPU as the source. That way, since the CPU is always at least 1°C, the curve will reliably apply the PWM value you specified.

 

Now, I’ve got zero programming skills, so I couldn’t write a script myself to handle this (to control the fan PWM based on HDD temps). But hey, we live in great times – that’s what ChatGPT is for.

 

So I started by explaining my setup to ChatGPT – focusing on my hardware. I’ve got a Supermicro SuperChassis 743TQ-903B-SQ, and it was important to know that on the ASUS BMC, CHA-FAN1 through CHA-FAN4 are connected to fans that cool the HDDs. FAN1 and FAN2 are directly for the HDDs, while FAN3 and FAN4 also help cool five more HDDs and provide airflow for the CPU.

 

That led to this requirement: FAN1 and FAN2 should respond only to the HDD temperatures, while FAN3 and FAN4 should react to both HDD temps and the CPU package temp. I then defined fan curves for both the HDDs and the CPU, and each fan was assigned to the appropriate HDDs.

 

I wanted to keep the number of required packages for the script as low as possible, and decided to go with freeipmi instead of ipmitool, since that's what the “IPMI Tools” package actually uses under the hood. That’s also why the commands in the script have been adapted for freeipmi. I installed the IPMI Tools package so I could monitor fan speeds and temperatures directly in the UNRAID dashboard.

 

Required packages: ipmi-raw (freeipmi/IPMI Tools), smartctl, hdparm, sensors, iostat (from sysstat), awk, printf, head, wc, tail

 

Step by step, I had ChatGPT add more and more features to the script…

1. HDD Activity Detection using iostat

    Only queries SMART temperature if a disk is actually reading or writing data.

    Prevents unnecessary SMART access during idle time, allowing drives to spin down as expected.

2. Spin-down Protection for HDDs

    Uses hdparm -C to check if a disk is in standby mode.

    If so, it skips temperature checks to avoid waking the drive unnecessarily — maintaining UNRAID’s spin-down behavior.

3. Correct SMART Temperature Extraction

    Reads the temperature from column 10 (RAW_VALUE) of the Temperature_Celsius SMART attribute.

    Avoids misinterpretation by ensuring accurate temperature parsing across all drive vendors.

4. Fan Speed Control Based on Temperature

    Custom fan curves:

        HDD zones use a stepped curve: 10 % / 25 % / 45 % / 85 % at increasing thresholds.

        CPU zones (CHA_FAN3+4) use 10 % / 35 % / 50 % / 80 % / 100 %.

5. Independent Control per Fan Zone (CHA_FAN1–4)

    Each fan zone is assigned a group of HDDs.

    Zones C and D additionally factor in CPU temperature in the control logic.

6. Change Detection (Delta Update)

    The script only sends an ipmi-raw command if the PWM value has changed since the last loop.

    Reduces IPMI traffic and log spam.

7. Failsafe Mechanism (IPMI Failure)

    If the ipmi-raw command fails, the script sends a failsafe PWM value of 60%.

    Logs whether the fallback succeeded or failed (e.g., due to a hung BMC).

8. Compact Log with Auto-Rotation

    Logs are written to RAM

    If the log exceeds 100 lines, it's trimmed to the last 50 lines.

    Each log entry includes:

        HDD temperature (or (all sleeping or inactive))

        CPU temperature

        Target fan PWM

        Date and time

9. Optional Dry-Run Mode

    Set DRY_RUN=1 at the top of the script to simulate fan control without sending actual IPMI commands.

10. One-Time Dependency Check

    On startup, the script checks for required tools.

    If anything is missing, it logs the missing dependencies and exits gracefully.

11. Cooldown-Logik at idle/sleep

    After the last active access per fan zone, the last PWM value remains valid for 120s

    Gives the currently used HDDs some time to cool down

    Only then does the value fall back to DEFAULT_SLEEPING_HDD_SPEED

12. The script runs every 10 seconds. It is activated when the array is started via "User Scripts."

 

For anyone who missed this, have fun playing with this information.

 

The following script is my current version, which I've been testing casually for a week. So far, it's stable, and no errors have been detected. However, the script is for my setup and basically only works for me (fan and HDD/CPU allocation). But as I said, you can achieve a lot with ChatGPT...

 

I'm open to suggestions if anyone has any ideas for improvement/features or sees a bug.

 

#!/bin/bash

INTERVAL=10
LOGFILE="/tmp/user.scripts/tmpScripts/Fan_Control_HDDs/log.txt"
DRY_RUN=0

MIN_FAN_SPEED=10
DEFAULT_SLEEPING_HDD_SPEED=10
FAILSAFE_FAN_SPEED=60
AFTER_ACTIVITY_COOLDOWN=120

MAX_SAFE_HDD_TEMP=65
MAX_SAFE_CPU_TEMP=80

declare -A fan_disks
fan_disks[A]="/dev/sda /dev/sdb /dev/sdc /dev/sdd"
fan_disks[B]="/dev/sde /dev/sdf /dev/sdh /dev/sdg"
fan_disks[C]="/dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm"
fan_disks[D]="/dev/sdi /dev/sdj /dev/sdk /dev/sdl /dev/sdm"

declare -A fan_zone
fan_zone[A]=0x00
fan_zone[B]=0x01
fan_zone[C]=0x02
fan_zone[D]=0x03

declare -A last_pwm
declare -A last_active_time
declare -A last_active_pwm

log_echo() {
    echo "$1" >> "$LOGFILE"
    local line_count
    line_count=$(wc -l < "$LOGFILE" 2>/dev/null)
    if [[ "$line_count" -gt 100 ]]; then
        tail -n 50 "$LOGFILE" > "${LOGFILE}.tmp" && mv "${LOGFILE}.tmp" "$LOGFILE"
    fi
}

check_dependencies() {
    local missing=0
    for cmd in ipmi-raw smartctl hdparm sensors iostat; do
        if ! command -v "$cmd" >/dev/null 2>&1; then
            log_echo "$(date '+%F %T') ❌ Dependency missing: $cmd"
            missing=1
        fi
    done
    if (( missing )); then
        echo "Exiting due to missing dependencies."
        exit 1
    fi
}

get_temp() {
    local dev=$1
    [[ ! -e "$dev" ]] && echo "SKIP" && return

    local state
    state=$(hdparm -C "$dev" 2>/dev/null | awk '/drive state/ {print $4}')
    [[ "$state" == "standby" ]] && echo "SKIP" && return

    local io_activity
    io_activity=$(iostat -d -k -x "$dev" 1 2 | awk -v d=$(basename "$dev") '
        $1 == d { count++; if (count == 2) print ($5 + $6) }')

    local io_whole
    io_whole=$(printf "%.0f" "$io_activity")

    if [[ "$io_whole" -gt 0 ]]; then
        smartctl -A "$dev" 2>/dev/null | awk '
            /Temperature_Celsius|Temperature/ && NF >= 10 {print $10}' | head -n1
    else
        echo "SKIP"
    fi
}

get_max_temp_for_fan() {
    local devices=($1)
    local max_temp=0
    local any_awake=0

    for dev in "${devices[@]}"; do
        [[ ! -e "$dev" ]] && continue
        temp=$(get_temp "$dev")
        if [[ "$temp" != "SKIP" && -n "$temp" && "$temp" -gt $max_temp ]]; then
            max_temp=$temp
            any_awake=1
        fi
    done

    if (( any_awake == 0 )); then
        echo -1
    else
        echo "$max_temp"
    fi
}

get_cpu_temp() {
    sensors 2>/dev/null | awk '/Package id 0:/ {print int($4)}' | head -n1
}

cpu_fan_curve() {
    local t=$1
    if (( t < 60 )); then echo 10
    elif (( t < 70 )); then echo 35
    elif (( t < 80 )); then echo 50
    elif (( t < 90 )); then echo 80
    else echo 100
    fi
}

check_dependencies

while true; do
    for fan in A B C D; do
        now=$(date +%s)
        devices="${fan_disks[$fan]}"
        hdd_temp=$(get_max_temp_for_fan "$devices")

        if [[ $hdd_temp -eq -1 ]]; then
            if [[ -n "${last_active_time[$fan]}" && $(( now - last_active_time[$fan] )) -lt $AFTER_ACTIVITY_COOLDOWN ]]; then
                final_speed=${last_active_pwm[$fan]}
                hdd_temp_display="(recent activity, cooling)"
            else
                final_speed=$DEFAULT_SLEEPING_HDD_SPEED
                hdd_temp_display="(all sleeping or inactive)"
                log_cooldown=1
            fi
        else
            hdd_temp_display="${hdd_temp}°C"
            hdd_speed=$(( hdd_temp < 40 ? 10 :
                         hdd_temp < 45 ? 25 :
                         hdd_temp < 50 ? 45 :
                         hdd_temp < 55 ? 85 : 85 ))
            final_speed=$hdd_speed
            last_active_time[$fan]=$now
            last_active_pwm[$fan]=$hdd_speed
        fi

        if [[ $fan == C || $fan == D ]]; then
            cpu_temp=$(get_cpu_temp)
            cpu_speed=$(cpu_fan_curve "$cpu_temp")
            final_speed=$(( final_speed > cpu_speed ? final_speed : cpu_speed ))
            final_speed=$(( final_speed < MIN_FAN_SPEED ? MIN_FAN_SPEED : final_speed ))
        else
            final_speed=$(( final_speed < MIN_FAN_SPEED ? MIN_FAN_SPEED : final_speed ))
            cpu_temp="N/A"
        fi

        # Emergency override
        if [[ "$cpu_temp" =~ ^[0-9]+$ ]] && (( cpu_temp >= MAX_SAFE_CPU_TEMP )) || (( hdd_temp >= MAX_SAFE_HDD_TEMP )); then
            final_speed=100
        fi

        zone="${fan_zone[$fan]}"
        pwm_key="${zone}"
        prev="${last_pwm[$pwm_key]:--1}"

        if [[ "$final_speed" -ne "$prev" ]]; then
            last_pwm[$pwm_key]="$final_speed"
            timestamp=$(date '+%F | %T')
            log_msg="🚀 Set CHA_FAN$((fan_zone[$fan] + 1)) | HDD=$hdd_temp_display | CPU=$cpu_temp°C → Target=${final_speed}% | $timestamp"

            if [[ "$hdd_temp" -eq -1 && "${log_cooldown:-0}" -eq 1 ]]; then
                log_msg="⌛ Cooldown expired → $log_msg"
                unset log_cooldown
            fi

            log_echo "$log_msg"

            if (( DRY_RUN )); then
                log_echo "🧪 DRY-RUN: Would send to zone $zone → PWM $final_speed%"
            else
                hex_pwm=$(printf '0x%02x' "$final_speed")
                if ! ipmi-raw 0x00 0x30 0x0e 0x04 "$zone" \
                    0x01 "$hex_pwm" 0x01 "$hex_pwm" 0x01 "$hex_pwm" 0x01 "$hex_pwm" 0x01 "$hex_pwm" > /dev/null; then
                    log_echo "❌ IPMI failed → fallback to $FAILSAFE_FAN_SPEED%"
                    hex_pwm_failsafe=$(printf '0x%02x' "$FAILSAFE_FAN_SPEED")
                    ipmi-raw 0x00 0x30 0x0e 0x04 "$zone" \
                        0x01 "$hex_pwm_failsafe" 0x01 "$hex_pwm_failsafe" 0x01 "$hex_pwm_failsafe" 0x01 "$hex_pwm_failsafe" 0x01 "$hex_pwm_failsafe"
                    last_pwm[$pwm_key]="$FAILSAFE_FAN_SPEED"
                fi
            fi
        fi
    done
    sleep "$INTERVAL"
done

 

Really awesome work to the person before this post!  Comically I was trying to address the exact same thing the day before; but instead I spent the 10$ and got myself a TR sensor that I have attached to one of my parity drives that routinely runs the hottest in the hdd bay.   (Link to the one I bought)

 

Solved it and allowed me to just use the native IPMI web portal with its pretty graphs and such.

 

For example here is my chart for the front HDD bay on my system:

 

Works great and here's a shot from the IPMI dashboard in unraid:

 

It's gone so well that now i'm going to grab another TR to put in other locations of the case I have.  

Edited May 16, 20251 yr by dnoyeb

Big thanks to everyone that answered some questions and all the posts in this thread; with it I was able to really get this box humming along. I have been having a bit of fun with this build out! I still have a few things to do: 1/ swap out a few older 5400rpm 8TB drives for some 16TB SAS drives, 2/ redo my NVME setup (see concern below), 3/ rearrange some drives up into my 5in3 Supermicro SATA docks, 4/ add two more TR sensors for the 5in3 docks 5/ add in 92mm fans I can control from the IPMI for the docks.

My only concern will be the combining of the Array cache, Download, VM, Dockers all onto the one large NVME pool. In the old days on my old dual Xeon setup; having all that on a single ssd would result in performance issues during downloads and playing Plex. I currently have the array cache/download on one NVME and the VM/Dockers on a second NVME. However, this leaves me with no parity; so, I am adding a 3rd NVME and turning all three into a ZFS rz1 pool. These new nvme’s are so fast that I am going to try it out and see if I can make it work. I do have a spare 500GB SSD I could have as the download drive and then extract to the array cache on the NVME if things don't go as planned. This way I can keep the latest movies on the cache a little longer too; resulting in less spin ups of the drives.

Here’s where I’ll be by the end of the week if all goes to plan:

Machine:

Asus Pro WS W680 ACE IPMI motherboard

Intel i7-12700 processor

96GB (2x48) ECC DDR5 memory

LSI 9400-16i

Mellonox Connect3x 10GB fiber card

XFS Storage Array:

Dual Parity - Seagate Exos X18 16TB SAS drives (with working spin down)

Array – Mixture of 12-16TB drives for 110 TB of space

ZFS Pools:

3xSamsung 990 Pro 2TB NVMEs in ZFS RZ1 for Array cache/Download/VM/Dockers

3xSamsung 870 Evo SSDs (1TB each) in ZFS RZ1 for important files / photos

2xSeagate 5400RPM spinners in ZFS mirror for ZFS backups

Overall looking and running pretty well; been very happy.

Thanks again for everyone's help!

Edited May 20, 20251 yr by dnoyeb

New BIOS v4302 released

mentions further microcode fixes.

hopefully that fixes my issue of degrading 14900k's within 1-3 months 🫠

Hi all!

I have been using this MOBO for a month now and I love it.

I do not know you guys what you are doing to get that high idle power consumption but mine is idleing at 14W.

Of course if You use some PCIE cards which prevent lower C-states it is understandable.

But I think some potential buyer skipped this MOBO because this false information about high power consumption.

7 hours ago, melorin83 said:

mine is idleing at 14W

Hi... Then please describe exactly what hardware your system is running. I don't think 14W is possible with the IMPI version.

I do not use the IPMI card.

13600K

32GB ECC RAM

1x500GB WD RED NVMe SSD (cache)

3x18TB HDD (idle)

I think most people do not touch this setting in the BIOS, but it it the most important one:

Package C State Limit: Auto

You MUST set it to C8 or C10, otherwise it won't go lower than C3.