Pri

There is software which logs in to run commands for docker. When the server is working properly it only needs to login very seldom. So that isn't me, it's automated software doing that. All the disks have 100GB or more free, they're not at 100%, they're high capacity disks. I've uninstalled the UD plugin for the time being to see if the crash still occurs because I too have no indication that it is UD, I'm just going by what the guy said on reddit and trying it. When he said, reinstall UD and it would fix it, for me I was getting these lockups daily until I reinstalled UD. At that point I had 16 days of uptime without issues until today. So now I've uninstalled UD entirely and I'll leave it like that for at least 2 months or until I get a crash again which would rule out UD as the culprit entirely if that happens.

https://forums.unraid.net/topic/92462-unassigned-devices-managing-disk-drives-and-remote-shares-outside-of-the-unraid-array/page/399/#comment-1312939

Sorry this took so long, I had to wait for the problem to occur again. Which it just did:

Some of us are having lockups with out of memory issues, one user thinks that he has found the culprit and that it's the unassigned devices plugin. But there's nothing to discern whether that really is the case. Here is the thread on reddit about it incase you want to take a look @dlandon (I'm having the same Out of memory lockups as those in the thread describe and I am using Unassigned Devices with a USB based external hard drive but I have no idea if this issue is correlated or not).

Changed my USB stick yesterday and so far so good, no more errors, no more lockups or crashes. Thank you to the people on the unRAID discord who believed the issue was related to the USB key as I postulated in my initial post

unRAID Version in use: 6.12.3 (first crash) 6.12.4 (second and third crash). So I've had this issue three times recently (within the past month) where one of three VM's (Windows 2022 Server x 2 + pfSens) I'm running on the system crashes or locks up. Then when I access the unRAID UI, the "main" tab and the "vms" tab just hang and don't load. A reboot fixed it the first time so I thought it was a fluke. About a month later, it happens again, I reboot again, this time this only fixed it for about 3 hours then it happened a 3rd time. I have included the log below as I have the syslog enabled. This covers the most recent lockup and shutdown-bootup sequence. I have logs going back further all the way to August if that's helpful. Looking at the log myself, it seems that maybe it's the USB stick I'm using that is failing but I cannot be certain as I don't know if this kind of error related to the USB stick is a symptom of something else so I'll leave that determination to your experience. Thank you for any assistance syslog-hyepyc.log

Set the fan speed in the motherboards IPMI menus to 100% then use the IPMI Tools plugin in unRAID to set the RPM. If you set the motherboards IPMI fan system to anything other than full speed / 100%, it will cause conflicts with the unRAID IPMI plugin.

Yes you can access the IPMI KVM from the WebUI. That will allow you to enter the BIOS remotely over network etc No video card is necessary as there is one included in the BMC chip of the motherboard.

Ah yes, you're 100% correct, the Supermicro motherboard has it inverted compared to the manual. I remember now having to think that one out aswell.

To connect the front panel USB you will need to use a USB3 to USB2 adapter. I did buy one but didn't end up using it and just left the front disconnected. I'm unable to take the photo you requested as the server is all powered on and everything and I can't turn it off etc

The 9500-8i I purchased is a IT mode HBA yes. It lacks the RAM for raid modes. You would have to buy a 9560 to get RAID functionality (RAID5 / RAID6 etc).

I did try the powertop script months ago, it didn't have any effect for my system and by that I mean the power use never changed before or after. I think the lowest C state I saw was C2 but that's just going by my memory. I did notice though that the power use of the x86 cores are already very low, they can go sub 0.25 watts. But the I/O die which handles PCIe, Memory, Infinity Fabric to/from the x86 cores uses a lot of energy, I think the lowest I saw it was 19.5 Watts while the x86 cores were all sub 0.25 etc I suspect an Intel system would have better luck with testing what the 9500-8i can do sleep state wise.

My case came with two brackets from Gooxi, they allowed for single or dual-redundant power supplies to be fitted. I also had a 3rd bracket from Xcase, the retailer I bought mine from which was an ATX bracket so that I could use a normal power supply and that was the one I installed. So to be clear, that ATX bracket is not stock, but the single and dual server power supply brackets are standard from Gooxi and included in the cases accessory kit which ships with the case.

Usually only my completely idle cores are at 1.5GHz, active cores are between 3GHz and 3.8GHz. When there's low CPU utilisation and only say 3-4 threads are active I've seen them hit 4GHz. No issues with CPU utilisation here, frequencies quite high all the time, I'd say 3.2GHz is the most common frequency I see due to multithreaded loads. I'm using the OnDemand CPU scaler with Performance Boost enabled.

Essentially yes. I've recently switched my setup up, I put in an Intel X550-T2 (1Gb/2.5Gb/5Gb/10Gb capable) instead of the X540-T2 (1Gb/10Gb capable) into my unRAID box and I've hooked up a single cable from that card to my Modem for WAN (2.5Gb/s). For the LAN side, I'm using a network bridge created by my unRAID box and I pass through a VirtIO nic to my pfSense virtual machine. If I need my dedicated pfSense box for some reason I can just plug the WAN cable from that to my modem and turn it on. I left its LAN cable hooked up to my switch even though its powered off etc I've found pfSense runs really well virtualised on this server and the performance exceeds my dedicated box (I have 1.2Gb home internet). And the energy usage is a lot lower than when I had a physical box. I'm saving almost 50 watts idle by virtualising pfSense as a result. Hey cbapel, I'm glad you found the thread useful! So I can answer your questions. Firstly, I have a friend who is passing their HBA through to unRAID from TrueNAS where he runs unRAID in a VM. He's using the same motherboard that you and I are using except he has the SKU with the built in HBA and that is the one he's passing through. For him it works 100%, so we could totally pass through a 9300, 9400, 9500 etc to unRAID from proxmox and that would function properly. As for the SlimSAS cable. It comes with 2 x MiniSAS on one side and 1 x SlimSAS on the other. The SlimSAS side goes into the HBA. So basically the SlimSAS is a PCIe x8 or 8 x SATA/SAS. And the backplane on my case is an expander backplane. It only needs a single cable connection to enable all 24 drives but it has two inwards connections to increase bandwidth. The 3rd connection on my backplane is an output intended to allow you to connect this backplane to another backplane for daisy-chaining. So in my case I have a single 24 slot Backplane at the front. But if I had a 36 bay case which is a 24 bay at the front and 12 bays at the rear a cable could be used to connect those two backplanes enabling all 36 drives to be used from a single or dual-port HBA. In my case the HBA I'm using has a single port that acts like dual-ports when connecting to MiniSAS backplanes due to the breakout cable (SlimSAS to 2xMiniSAS) that is purchasable. The cable I went with was from Broadcom directly. If your backplane is not an expander type then you would need to purchase a HBA that can provide at-least three MiniSAS for your three columns of drives. The 16i models (9300, 9400, 9500, 9600 etc) are popular choices for this. If you do decide to go with a 9500 or 9600 HBA that uses the new SlimSAS connector, please make sure you get the right cable as there are two different varieties, one for PCIe backplanes only and one for SATA/SAS backplanes. If you buy the official Broadcom cables the PCIe only cable is green while the SAS version is black, the two types of cables are not pin compatible even though they'll fit the same receptacles at both ends so it's important to make sure the SlimSAS to MiniSAS cable you purchase is the correct one for your backplane type (I believe in your case you want the pure SAS version).

Looks awesome cbapel! - What is the noise and power consumption like?

Neat project, keep going!

Howdy, all the specs including the case are link on the first post of the build. The case is a Gooxi 24-bay Chassis. Here is a link to the store I bought it from: https://www.xcase.co.uk/products/gooxi-rmc4124-670-hse-no-psu-atx-psu-barcket The full model number is: RMC4124-670-HSE

When I first got my board the IPMI wouldn't even work on Chrome for me haha I could only use it in Firefox. I had to update all the software before it would work properly, since then no trouble

I've not had any issues with the SuperMicro IPMI features. The fan speeds are correct and all temperature sensors are showing correctly within the IPMI web interface provided by the BMC. I assume you've updated the firmware and BIOS to latest, if not I'd do that. To control my fans precisely I am not using the SuperMicro Fan controls found within their IPMI interface and instead I am using the IPMI Tools package found within the unRAID community apps repository, specifically the one from dmacias. To use that properly you need to set the fans in the SuperMicro IPMI interface to "Full Speed" as this makes the board only send a speed signal to the fans on boot and after that it wont send any more commands allowing the unRAID IPMI software to send its own speed commands without interference.

I look forward to seeing your build! To answer your question. I was able to remove the stock fans from those hotswap plastic shells and reuse the included rubber grommets that the cases use with the Noctua fans. So I maintained the hotswap nature in that I can slide the fans out like normal. However connecting the fans electrically I had to lose that. To do that I simply unscrewed the receiver side which is bolted to the case and fed a PWM cable through. The A12x25 Black's that I have come with a modular 30cm long PWM cable which is sleeved. The fans themselves only have a very short (2cm or so) PWM connector on them. So basically I can pull the fans up, unplug them from their extension cables and have essentially jerry-rigged my own hotswap system in this manner etc - Keep in mind only the black A12x25's have this very short PWM connector on the fans themselves. The brown fans have a normal length cable on them.

A lot of his argument in the video is around power consumption but I don't think he actually tested the power consumption with a wattage meter. But I have. The X540-T2 (dual-port 10GbE card) that I have at idle increased my server power draw from the wall socket from 165 watts to .. 165 watts. I suspect the card pulls milliwatts at idle while maintaining a 10G link. When it comes to load testing it's higher, I measured 15 watts when doing a 10Gb/s iPerf load for 45 seconds. But the CPU burden is also higher so it's hard to exactly quantify all of the power consumption to the network card itself since the iPerf software does produce some CPU load itself just trying to sling bits that quickly. I think though for most of us, probably 99.9% of the time our 10Gb connections will be idle. At most moving a few kilobytes. It's rare you're going to max out a 10Gb connection in your home network but when you do need it, it's nice to know its there. I'm using as I said the X540-T2 which is a 10 year old card now. If you were to instead get a X710-T2L that has a typical power draw per port of 4.1 Watts at 10Gb/s. That number is from Intel directly. That's really not much and that is only when actually using the port at its full rated speed, it's much much less idle.

Very cool case, I would have killed for something like this as an alternative to a QNap/Synology/Asustore type 5 bay system when I needed it. I like the low power draw too, thank you for including that in the build information

Great photos! looking forward to seeing more.