LoloNZ

Hi all, It is only my 3rd post (I believe), I thought I would post my experience with trying to setup an energy efficient server... sorry for the long post ! It all started a while back when I had a Synology NAS (DS918+) that became a bit short in terms of processing capacity. In addition, because "Synology Surveillance Station" is attached to relatively expensive licenses, and also because I would have had to fully reconfigure my existing NAS Array to keep 1 of 4 HDD just for recording video streams from cameras, I also decided to install Security Camera and have a dedicated BlueIris machine (an Optiplex 7040 micro PC, i7-6700T, 16GB, 250GB NVME, 2TB HDD) for that. Then my son was using an old computer for gaming (i7-3770 with a GTX 1060 nvidia GPU) which was starting to become a bit too slow for the type of games played... This meant that I had to look after 3 different pieces of equipment (1 NAS, 1 Tiny CCTV server, 1 gaming desktop), with an overall combined power consumption at least of just below 60-ish watts at the very least. Then I discovered Unraid 🙂... And it fully decided to give it a go, with the following goals for this new server: a Linux VM for HomeAssistant OS (previously I used the docker installation, but really the OS version is just much easier to manage and maintain) a Windows VM dedicated to BlueIris for the Security Camera (using a dedicated "Surveillance type" HDD passed through for all video recording) another dedicated VM for gaming (using parsec, a dedicated GPU passthrough, some NVME drives passthrough), that can be started remotely (from a web page on the LAN) by my son (using RWSOL-Server for this) at least the following containers apps (around 30 containers at the moment) : a proper authentication solution (with Authentik) allowing SSO (this was actually triggered with some kind of discovery project I had to do anyway for work) NginxProxyManager for most of the services exposed outside Omada Controller to manage my Access Points and Switch Nextcloud (with OnlyOfficeDocument Server, Redis, MariaDB) All the "Arrs" applications so that I have a fully automated solution with Overseer as the frontend... Plex media server with intel iGPU HW acceleration for transcoding needs with Plex Some other apps... Well, all of this was currently running on my new server composed of: an intel i5-13600K a MSI Pro Z690-A DDR4 motherboard (I had already access to some DDR4 sticks...) 48 GB of RAM 1 Array made of 4 x WD Red 3TB (WD30EFRX), 1 of which is used for parity 1 cache NVME SSD Samsung 970 Evo+ (500GB) 1 Western Digital 4TB WD Purple Surveillance for the video camera recordings => passed through to the BlueIris Windows 11 VM 1 Western Digital Blue SN570 1TB NVME drive => passed through to the Gaming Windows 11 VM (OS + games) 1 Samsung SSD 980 1TB NVME drive => passed through to the Gaming Windows 11 VM (more games storage) 1 Asus GeForce RTX 3060 Ti Dual OC 8GB Graphics Card => passed through to the Gaming Windows 11 VM (more games storage), else managed by Unraid Nvida-Driver plugin when the VM is not running. 5 x 120mm fans (2 for CPU cooler, 3 in the case) With CPU pinning configured to only use the (8) e-cores with Unraid, the containers and the BlueIris and HomeAssistant VMs, and dedicating the (6) p-cores only to the gaming VM, the following observations can be made: Total power at the socket was just around 77W with the (4) drives from the array spun down, from which 12W are from the GPU in P8 state after applying this script from fellow member @MeisterPilaf (Many thanks, it works great!). This is when the server is doing is BAU stuff, including recording (4) video camera streams on the "Purple" HDD and the gaming VM being not started. Measured from a Zigbee Xiaomi plug that monitors power. The CPU can really run efficiently and cool if the configuration is making use of the e-cores, but has enough sleeping capacity to have a great gaming experience whin using the gaming VM. In the end, my son has stopped using the VM, moved to an XBox Serie S, mainly because of various issues related to anti-cheat implementation with online gaming where VMs are not welcome 😞, so the GPU was sold to finance the console ! As of now, the server consumes around 64W, which is in my opinion very acceptable given to reserve of power accessible for future projects, and not far from the initial combined power consumption (when not gaming): I thought of changing the whole HW to something more conservative, but I am not sure I want to go through that pain... any thoughts on all this? I know that this project was a great learning opportunity, having everything centralised is great and easier to maintain, from a cost perspective it is difficult to fully justify it. ### Update 30 Nov 2023 ### I have now updated the bios on the mobo to the latest version, and also spent time to look at the bios settings: - disable the onboard audio - disable the LED thingy - changed all "auto" settings in the ASPM section to L1, L0sL1 and C10 where I could - disable Intel® Turbo Boost - changed CPU cooler type to "Boxed Cooler" => this changed the power and current limits. Also did the "powertop --auto-tune", and I saw that all the isolated p-cores ("Isolated CPUs") that I configured in the CPU pinning settings were never going below 3.5GHz..., and then I realised this was because these cores are not at all managed by Unraid OS any more, so they are not getting into a lower power mode even thought they are not in use (the VM using these p-cores is currently not started). So I did revert that setting to remove all CPU isolation, and once this was done, the P-cores where finally going into a C7 state with a very low frequency. Now, the server is chugging along at ~46W, the CPU is at 32°C, and mobo at 28°C 🙂

Many thanks, I was having issue to download files bigger than 1GB, this fixed it. FYI, I am using nextcloud behind NPM (Nginx Proxy Manager), and all I had to do is add those lines to the "Custom Nginx Configuration" in the "Advanced" tab of the Nextcloud Proxy Host configuration.

If you have Fastboot enabled in the bios try to disable it, I had issue myself where my machine would not reboot when selecting the unraid restart button in the UI.