Silent but deadly - Replacing 3 machines with 1 [Update: 2021-08-29]


Recommended Posts

After my trusty old NAS started throwing errors (the drives did have a run time of around 7.5 years so they had done okay) I did some emergency backing up to a QNAP I acquired, but ran unRaid on it as a trial. It went so well that I brought Pro within days, and planned a new server to allow me to decommission that NAS and ~12 year old box (my last custom build) that did little more than create noise and draw power, and almost as a secondary function ran Plex) both of which were maxed out for storage space and lacked much transcoding ability.

 

The QNAP had 20TB with all the drives I cobbled into it (most of which were probably approaching failure also), and emptying both the old NAS and server left around 300GB, so that wasn't going to cut it, and it was a rack mount noisy beast that at night had a hum that could be heard in the other end of the house with no virtualisation support!

 

The plan was for a new build to not only expand the storage I had, and up the specs to handle more transcoding, but that also:

  • Was quiet enough to live in the office instead of the garage
  • Would run a Windows VM with GPU passthrough to be a new daily driver for some light gaming (I am otherwise typically a Mac laptop user)
  • Could run home automation bits, Zabbix, and some other dockers which would also allow me to decommission a 3rd computer (linux box) I had running as a bit of management server.
  • Would hopefully use less overall power than those 3 machines combined. This is yet to be tested, but when I started speccing options, my debate was either quiet enough to live in the office and replace all 3 machines, or low power oriented (and maybe noisy) in the garage but requiring another new daily driver; obviously quiet won out.

 

 

Build updated with additions up to 21-08-29

 

OS at time of building: unRAID 6.9.2 Pro

CPU: Intel Core i9-11900K 3.5GHz 8-Core [Intel Ark | PBTech | Benchmark]

CPU Cooler: Noctua NH-D15 82.5 CFM CPU Cooler [Noctua]

Motherboard: MSI MAG z590 Torpedo ATX LGA1200 MB [MSI]

RAM: Corsair Vengeance RGB Pro 48GB (2 x 16GB, 2 x 8GB) DDR4-3200 CL16 [Corsair]

GPU: Gigabyte GeForce GTX 950 2 GB WINDFORCE 2X Video Card (from parts on hand, while the bank balance recovers) [Gigabyte]

Case: Fractal Design Define 7 Dark ATX Mid Tower Case [Fractal Design]

Power Supply: Corsair HX Platinum 850W 80+ Platinum Certified Fully Modular ATX PSU [Corsair]

Fans: 3 stock fans with the case plus 2 on the Noctua.

 

Parity Drive: 1x Western Digital Red Pro 10TB 3.5" 7200RPM [WD]

Data Drives: 5x Western Digital Red Pro 10TB 3.5" 7200RPM [WD]

Cache Drives: 2x Crucial P2 1TB M.2-2280 NVME SSD [Crucial]

 

Other:

Fan Controller: iCue Commander Pro [Corsair] (Comments on my experience on this in a post below)

In-Case Lighting: iCUE LS100 Smart Lighting Strip Starter Kit [Corsair]

 

 

 

Primary Use: Plex and friends (Radarr, Sonarr, qBittorrent, SABnzbd, Overseerr, Gaps), general file store/backup, and Windows gaming (lite) VM

 

878433091_ScreenShot2021-08-29at10_11_52.png.34288e35b28d8ecc20bc858a6be0e188.png

(Windows VM shut down while I'm preclearing some disks that its storage is being moved to, but otherwise It's been running well)

 

 

 

Likes: Very quiet, should have plenty of power to experiment with. And I like the look.

     Also, I was worried that the z590 and 19-11900K might be too new to be well supported by unRaid, so I quite like that it actually works!! :D

Dislikes: Other than my cabling skills to work with the available space, not a thing! Actually, it's just that as I've added things, I've not shut down and cabled nicely, I've tried to keep downtime minimum at the cost of tidiness.

 

 

Future Plans:

  • [Done] Get hardware transcoding going
  • [Done] Trying to get a Win10 VM with GPU passthrough going (Result: It worked easily and well. I did have to set power settings to not turn off otherwise I'd have to use other machine to power it on from Unraid again which makes sense but I didn't consider first boot)
  • [Still todo] Also getting back to Home Automation
  • [Done] Need to see how the temp's go and determine if more cooling is required (Result: More not "needed" but looking to add some for redundancy and future proofing)
  • [Still todo] Probably move my website back to internal hosting
  • [In Progress] Lighting has never really been my thing, but this case was cheaper with the glass, so perhaps I'll get some more RGB fans or something to brighten it up (Update: Lighting Strips don't position particularly well around the edges of the case. Still playing with layouts that hide the lights but light the case. More to be done, possibly use more top-of-case RGB fans to assist)
  • [New/In Progress] Good authenticated/authorised external access portal to expose the likes of Overseer to specific friends.
  • [New/todo] Setup mail services so I can move all my accounts from GSuite/Office365 (I have a number of accounts) possibly based on Mailcow
  • [New/todo] Migrate my old Zabbix config and get it running in docker form
  • [New/todo] Start researching expansion cards. I'm filling space fast and don't have any more usable on-board SATA, so expansion will require a card and that's an area I'm just not familiar enough in yet so I expect more research than I'd really like 🤣

 

 

Power Consumption:

(Still needs to be both measured better, and updated for latest additions)

Boot (peak): 156W (very briefly. Mostly about 123W)

Idle (avg sample): 97W with drives running, 82W with 2 drives sleeping (3rd was too active to get a decent read with it down). Both with 13 various docker containers running.

Active (avg sample): 130W to 250W, average around 170W running a Windows 10 VM with GPU passthrough, playing Starcraft 2.

Light use (avg sample): Samples between 82W and 130W, mostly around 92W watching a movie with Plex.

 

Measured with a rather crappy TP-Link smart plug so subject to my poor sampling. :D

 

 

IMG_2575.jpeg

IMG_2582.jpeg

IMG_2585.jpeg

IMG_2588.jpeg

IMG_2589.jpeg

IMG_2624.jpeg

IMG_2625.jpeg

IMG_2626.jpeg

IMG_2628.jpeg

Edited by hoppers99
System updates and added links
  • Like 1
Link to comment

I am working on a similar system, with an i9-11900K and an MSI MEG z590 ACE.

 

> perhaps I'll get some more RGB fans or something to brighten it up

 

Let me encourage you to check out these thermaltake Riing fans in "Unraid Orange":
  https://www.thermaltakeusa.com/riing-12-led-orange.html
I've never used LEDs in a system before, but I love the way my build looks with these fans :) Plus they don't need a special controller or anything

 

BTW there is a "Nuvoton NCT6687" plugin in CA that should let you access temps and fan speeds on this board (assuming it is similar to mine) using the "Dynamix System Temperature" plugin.

Link to comment

Oh that is a nice motherboard. I think it was one I looked but but iirc price drove me to where I ended up.

 

2 hours ago, ljm42 said:

Let me encourage you to check out these thermaltake Riing fans in "Unraid Orange":
  https://www.thermaltakeusa.com/riing-12-led-orange.html
I've never used LEDs in a system before, but I love the way my build looks with these fans :) Plus they don't need a special controller or anything

 

Those do look very nice. I was debating something like https://lian-li.com/product/uni-fan-sl/ as I had seen a nice looking build with them but of course the trouble with RGB is being able to control them from unRaid! I haven't done any research yet but I was planning to see if there were some I could pass through the controller to a windows VM to program and then just let them do their thing from there. And volume/noise will be a factor in the end selection as I still want to keep the noise levels down. Single colour may end up being the practical choice at the end of the day.

 

Quote

BTW there is a "Nuvoton NCT6687" plugin in CA that should let you access temps and fan speeds on this board (assuming it is similar to mine) using the "Dynamix System Temperature" plugin.

 

Awesome thanks, I'll check that out!

 

Have you had any joy with stats from the iGPU? I've installed the gpu_top plugin and added the component to the dashboard but as yet it gives me nothing. Haven't spent any time trying to debug yet, but it's still on the list. Slightly behind getting Plex to use the iGPU for HW transcoding :D

 

 

 

Link to comment

Very cool build! I am in a similar boat, I use mac as my daily and would like to turn my old windows gaming PC into a new server. The idea of using a windows VM with GPU passthrough for occasional gaming would be great. I need a case with more 3.5 bays as well. How many 3.5 drives can you reasonably fit into the Fractal Define 7 mid-tower? Also interested to see your power draw stats!

Edited by CaliHeatx
Link to comment
3 hours ago, hoppers99 said:

but of course the trouble with RGB is being able to control them from unRaid! I haven't done any research yet but I was planning to see if there were some I could pass through the controller to a windows VM to program and then just let them do their thing from there.

 

Right! My board has MSI Mystic Lights, but so far I haven't been able to control it from the VM, even when I pass in the USB device. So I have a few things that run in default mode.

 

RGB control should be possible as long as it comes with a separate controller rather than plugging into the motherboard.  TBH I just ordered some Lian Li Strimer Plus cables with that expectation. Turns out I am liking these LEDs more than I ever expected too :) 

 

I do have Intel GPU TOP and I wasn't getting any stats either, but I think it was just that I wasn't doing anything with the iGPU yet. I haven't yet migrated Plex over from my other system.

Link to comment
5 hours ago, ljm42 said:

Right! My board has MSI Mystic Lights, but so far I haven't been able to control it from the VM, even when I pass in the USB device. So I have a few things that run in default mode.

 

I've come across https://github.com/P3R-CO/openrgb-container which seems promising for my RAM, but thee OpenRGB site suggests they no longer support Mystic Lights as there are different implementations and basically, you can brick things. It comes with lots of warnings in general so I'm hoping to try look at it and read up when I have a good chunk of time rather than rushing... I don't fancy breaking things just for a bit of colour control lol. If I get it working though, I'll probably aim for things listed as compatible when I eye up some colour fans.

 

Quote

I do have Intel GPU TOP and I wasn't getting any stats either, but I think it was just that I wasn't doing anything with the iGPU yet. I haven't yet migrated Plex over from my other system.

 

I did wonder that but I'd expect to at least see something for GPU clock or something. No doubt I've missed something but I'll spend some time RTFM/support threads to see if I've overlooked something obvious to start with.

  • Like 1
Link to comment
9 hours ago, CaliHeatx said:

How many 3.5 drives can you reasonably fit

Easily 11, however I'd need 5 more trays than comes with the case. And the 2x 2.5" in addition. I'm pretty sure I'd need to do some more work on cooling in a decent way if I was to fully load it though.

 

Quote

The idea of using a windows VM with GPU passthrough for occasional gaming would be great.

That's part of my plan. I got the VM and passthrough working great. However I came out the next morning to find the screen off and when I looked in unRaid the VM was paused. I assumed the VM had just tried to go to sleep or something so I may need to tweak the power settings, but when I tried to start it again it froze my entire unraid server so that all I could do was hard reset the entire server! And when it finally started again, the VM tab was disabled. I'm nervous to try again but by all accounts it's certainly doable so my journey continues!

Link to comment
 
I did wonder that but I'd expect to at least see something for GPU clock or something. No doubt I've missed something but I'll spend some time RTFM/support threads to see if I've overlooked something obvious to start with.

I have finally seen the dashboard show the iGPU as doing something, after I finally got hardware transcoding for Plex going. I did do what may have been an otherwise redundant modprobe, but after next reboot I’ll see if it’s still behaving or confirm if the modprobe played a part.
Link to comment
  • 4 weeks later...
On 7/19/2021 at 9:08 AM, Solid Rhino said:

@hoppers99 Why have you chosen an Intel processor over an AMD processor?

 

I did definitely look at the AMD side of things. My last build was AMD and price wise it would have likely come out cheaper. But I wanted this to be a flexible and generally lower-effort machine to run and I still see a lot of people having to work harder than ideal to get things working or performing on AMD at the moment. I can't remember where but I found someone on here summarised it basically along the lines of; you can get it to work either way, but AMD might require some effort to get it going where Intel will just work.

 

If I had more time and inclination to put more effort into things, I would happily go with AMD and save a few $ along the way, but I wanted something that would pretty much do the things I asked out of the box, and this build has definitely done just that.

 

The only things I'm planning to change on it is to swap the two Intel SSD's (2nd Cache) out for a couple more 10TB Red storage since I have almost filled my 30TB and trying to add capacity while keeping those will require an expansion card (I'll likely add 1 drive at a time, maybe going more than 10TB depending what deals I can get to up parity and  capacity). I'm also about to order some more RAM - while the 16GB is plenty for running Unraid and all the docker stuff, I've ended up probably running more (finding more to run) than I had expected, so when I load up a VM that I want to commit some memory to things get a little constrained.

 

A new GPU is still on the list but to be honest using the iGPU for transcoding has worked well (I have had the occasional issue with the green artefacts that are discussed in elsewhere on here, but it's only been a couple of movies I've seen it on so not looked too deep into it yet) and the old GTX 950 has worked perfectly fine for my Windows VM and running StarCraft 2, so I'll probably wait until availability picks up more and prices stabilise before I worry too much about that one. Unless I find a better game I want to play :D. No doubt once I get the new RAM and have the VM running more commonly I'll start to find drivers for improved GPU.

 

Other than that it's running great - CPU and MB temps sitting around 35C atm and the fans are about 40% and barely audible, so I don't have a need for more cooling yet, probably look/lighting will be more a driver for LED fans than the actual cooling requirement!

 

I'll try and do an update on the container setup I have operating this afternoon and detail a little more of how things are going etc. actually. Since it was one of the things I didn't know what I'd end up running as there are some great tools I hadn't come across yet!

Link to comment
  • 1 month later...

A few updates I thought I'd catch up on while I have a rainy weekend in lockdown and my new drives are pre-clearing.

Who would have thought, 30TB of storage would not be enough... When I consolidated the data I had I had already cracked 20TB, and the other 10TB filled up awfully fast just by getting some updated content made FAR too easy with Overseer, Radarr/Sonarr, etc. Also I decided to play a little with lightning, and upped the RAM.

 

So I've added:

2x 10TB WD Red Pro Drives (to make a total of 6 drives = 50TB storage).

Corsair Vengeance RGB Pro 32GB (2 x 16GB) DDR4-3200 CL16 (for a total of 48GB)

Corsair Midnight Commander Pro to run my fans

Corsair iCUE LS100 Smart Lighting Strip Starter Kit for some in-case lighting

 

I still plan to add:

3-4x Corsair iCUE SP140 RGB ELITE Performance 140mm - These will replace the 2 front of case stock fans, add an additional front of case fan (I want 3 to help cool my HDD's which are all front of case). Debating one as the exhaust fan also but considering static pressure, airflow volume, and impact on my CPU cooling fans also. Research in progress, feel free to give any thoughts you have. Also considering changing the case top to the vented version and adding some exhaust fans at the top. These fans also supposedly have a slightly lower noise level than the stock ones (the stock ones are pretty quiet and perfectly fine, it's mainly a drive to add some LED fans)

2x Fractal Design Drive trays - I have the 2 new drives mounted in the lower cage, but I'd like to bring them up so they are better in the airflow of the fans. Not a biggie, just a matter of stock/supply timeframes these days.

 

My experience of the Midnight Commander Pro so far

I thought I'd immortalise some thought/experience here. I've been running it for only about a week and a half now but it's been interesting.

 

Prior to the Pro, I had my fans running off the in-case Nexus+ 2 fan controller. Combine this with the Dynamix Auto Fan Control plugin and I'd managed to get things running marvellously including fan speed display, all nice, quiet and functional. For the record, if that's what you plan to run i think you'll be perfectly happy by the way, I'm certainly not trying to suggest it's not a good solution.

 

I chose to get the Pro hoping to get better control of the fans individually to manage airflow better; the Nexus+ 2 just "does it's thing" based upon effectively a single fan input from the motherboard (in my case the CPU fan header) so if one fan is going up, they all are. That worked fine but I'm a control freak some times so thought the Pro along with its 4 temperature probes would be good to try out.

 

I manage the Pro through my Windows VM as it runs through Corsair's iCue software. That works fine too, however iCue seems to not see the actual CPU temp. It see's my GPU temp, and the Pro's temp probes fine however. No problem, I have one of my temp probes buried into the fins of my CPU cooler so I get to know if the temp is going up or down at least, even if not the actual CPU temp.

 

Overall I can control the fans, set their performance profile/curve, and apply different profiles per fan, but even on quiet I feel it's all running a little louder than when I had it running from my Nexus, and monitoring the temps that Unraid see's I'm not sure it is actually keeping things cooler than the Nexus was.

 

This is all totally subjective, I did not make detailed measurements before to compare temp (although I could possibly get it out of Grafana stats if I cared enough), and the noise... well as I say it's just subjective. In theory the Pro could shut down a fan if the temp was cool enough but I haven't seen that happen even when everything is sitting down in the low 20degrees celsius so I'm not sure what temp would be required for that.

 

What I'm running these days

I have Overseer feeding Radarr and Sonarr, which in turn feed qBittorrent and sabnzbd (which uses jackett). All these feed my Plex content. So I can automatically request something on Overseer and it magically pops up in my Plex some time later. This took a little planning to get all the drive mappings etc. setup right, but now it works pretty damn well and is pretty much how I made my last 10TB magically disappear!

 

I've also got nextcloud+mariadb+redis going. It "works" but it's no dropbox at the end of the day. My intention was to replace my dropbox, I'm debating if I keep going with nextcloud, try something else, or just keep paying dropbox because nextcloud just isn't as smooth and reliable as dropbox by a long shot.

 

On the monitoring side I have telegraf+influxdb+grafana. These work pretty well, I'm yet to spend the time to get Ultimate Unraid Dashboard up and running to expand what grafana is monitoring. I've also got Zabbix on my "to-do list", I used to have it on one of the servers I shutdown and I'd like to get my backed-up config running with the docker version on Unraid.

 

Other than that:

  • Trillium which I played with but have not really gotten into to replace several other notes apps I use
  • Swag and Heimdall, working on trying to expose certain services off my network and make it easier for me to access all the things from ideally a nice shiny browser homescreen. I'd say this is still WIP in heimdall works pretty nicely overall especially compared to a number of similar things I've tried. Ideally, I'd have a portal page that utilised plex accounts for login and added an authorisation layer, but dreams are free, and I'm not yet motivated to go write one. Specifically I'd like to let key friends access overseer to make requests.
  • Wireguard provides nice reliable access for my laptop and another remote computer back to my server. I've considered this as an option for friends, but I still want more control than a "server or not server" and expecting them to run specific software is not the best user experience especially as many of them are not as geeky as I am.
  • Pi Hole is now running my network DNS. I'm not doing much filtering with it, my router does parental control filtering on the kids devices, but it is interesting to see the stats. The one thing I haven't messed to far with is the default upstreams for it are Cloudflare and something else (I can't remember) but I like them to be my ISP DNS as there is a measurable performance increase by querying the closer servers. I can update it at runtime but if it restarts it resets, so I need to go look at the appdata config - I just haven't looked that far yet.
  • Binhex's Preclear is of course a stock standard these days. I only run the docker container when I need to use preclear however so most to the time it's taking up a little space but zero resources.
  • Honourable mentions to Gaps, dupeGuru, Code-Server, P3R-OpenRGB that I still have installed but either run rarely, or have given up on. Happy to share experiences with them if anyone is curious.

 

 

Wow, that was more than I expected, and probably bloody hard to read so if you got here, well done and thanks for taking an interest.

 

The only other things I wanted to mention was that the old GPU I'm still using has actually been working really well for my GPU passthrough Windows VM and is running StarCraft II perfectly well. I'd still like to update some day, but it's working well enough to make me go on a saving spree. Also, I have some annoying boot anomalies when I reboot sometimes having to go into bios and exit (without changing a thing) for it to boot through to Unraid sometimes; I don't think it's every time but I don't reboot enough for me to have spent more time debugging - may be related to if I had something plugged into the on-board graphics for it to boot, and I'll get a dummy HDMI load at some point to try anyway, but one day I'll likely have to work out more exactly what's going on so it's reboot-safe.

 

Hope everyone is keeping well in the carnage that is life at present!

 

 

 

 

 

 

Link to comment
  • hoppers99 changed the title to Silent but deadly - Replacing 3 machines with 1 [Update: 2021-08-29]
  • 4 months later...

Hi! regarding the corsair commander pro, planning to get one just wondering, if ever i dont want a VM to control it can the "dynamix auto fan" detect it and control its pwm? planning to use it on noctua 3000 rpm fans so pwm control is a must i heard it sounds like a jet engine on 100%.

Thanks!

Regards,
Adam

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.