NathanR Posted July 13, 2022 Share Posted July 13, 2022 Figured I could chronicle my journey a bit and help anyone in the future or just be a place for reference and input as I have no idea what I'm doing Build: Model: Custom M/B: ASRockRack X570D4U-2L2T Version - BIOS: American Megatrends International, LLC. Version P1.40. Dated: 05/19/2021 BMS: 1.2 CPU: AMD Ryzen 9 3900X 12-Core @ 3800 MHz HVM: Enabled IOMMU: Enabled Cache: 768 KiB, 6 MB, 64 MB Memory: 64 GiB DDR4 Multi-bit ECC (max. installable capacity 128 GiB) Network: bond0: fault-tolerance (active-backup), mtu 1500 eth0: interface down eth1: 1000 Mbps, full duplex, mtu 1500 eth2: interface down eth3: interface down Kernel: Linux 5.15.46-Unraid x86_64 OpenSSL: 1.1.1o Uptime: ~3hrs Previous (noob getting started) thread: --- Me trying to figure out how to pass through a NVMe Drive in 6.10.3 I might just give up and run xcp-ng with Unraid nested? Or I guess I don't really need NVMe drives passthru I could just install a new Win10...but I have a working image with all my files, boo Idk, will revisit slowly...we'll see what Ghost82 says. Then I went down the why USB bootable drive and 64GB drive and which drive hole... Ugh, please just make this bootable from a standard HDD and the USB nightmare can go away. I read that 64GB drive Fat32 formatted with something like GUI Format.exe will work still. I read that Samsung BAR Plus is the best drive (as of 2022-07-12) for this. I get consternation given the reviews of the BAR Plus's longevity and lack of USB 2.0 drives. I'd rather have a list of tested compatible drives or better-yet a USB drive sold by Lime Technology...but it's all good. --- Current plan is to trade-for/buy a 5900X and run that. Using my old 3900X until then. I want some cache drives... but I am seeing now that there are issues with even that. Me complaining: Ugh... WHY is everything so complicated?!?! I read that Unraid was simple and 'just worked' FFFFFFF's in chat!!! Seems this is the fix: nvme_core.default_ps_max_latency_us=1500 --- Next is figuring out drive structure, folder structure, cache drives, etc. --- I'd like some input on the most common first things y'all do on an install... I've heard of zeroing or wiping or cleaning drives...whats that? What dockers, apps, VM's do you typically run? What's your folder/drive structure and why do you like it? Etc. Quote Link to comment
NathanR Posted July 14, 2022 Author Share Posted July 14, 2022 @Arbadacarba was instrumental in solving my VM issues. Thank you \/ \/ \/ \/ See fixes here \/ \/ \/ \/ Having BMC issues with my X570D4U-2L2T Somehow it lost the 1.2 image --- On happier notes I ordered: -5900X -2x 980 1T On Prime Day I plan on using 5900X for H265 encoding and the 980's to store VM's and do write cache etc. Yes I plan on having 10G NIC's to utilize the throughput --- Here are all the app's I have installed so far: I have long wanted to run Home-Assistant!!! I CAN FINALLY DO IT YEASSSSSSS PLZZZZZZ THANK YOU Unraid! Plex is next I guess? Quote Link to comment
NathanR Posted July 18, 2022 Author Share Posted July 18, 2022 (edited) Plugins: Community Applications - https://forums.unraid.net/topic/38582-plug-in-community-applications/ Unassigned Devices (And Plus) - https://forums.unraid.net/topic/92462-unassigned-devices-managing-disk-drives-and-remote-shares-outside-of-the-unraid-array/ Unassigned Devices Preclear - https://forums.unraid.net/topic/120567-unassigned-devices-preclear-a-utility-to-preclear-disks-before-adding-them-to-the-array/ Dynamix File Manager - https://forums.unraid.net/topic/120982-dynamix-file-manager/ Dockers: DiskSpeed - https://forums.unraid.net/topic/70636-diskspeed-hard-drive-benchmarking-unraid-6-version-292/ ESPHome (until I get VM running) - https://forums.unraid.net/topic/72033-support-digiblurs-docker-template-repository/ Home-Assistant-Core - https://registry.hub.docker.com/r/homeassistant/home-assistant/ netdata - https://forums.unraid.net/topic/47828-support-data-monkey-netdata/ --- Home-Assistant-Core doesn't have the add-ons module (dockers) because it is a docker. There's also the linuxsystems developed core (docker) version? Idk, too many limitations; decided VM was best route. Unfortunately HA is now setup as CORE and I need to move files to the VM... hOw? idk. But I know that Dynamix is amazingly helping me with all that. Unfortunately installing HA as a VM was rather troublesome... qemu-img convert -p -f qcow2 -O raw -o preallocation=off "/mnt/user/domains/Home Assistant/haos_ova-8.2.qcow2" "/mnt/user/domains/Home Assistant/vdisk1.img" Solved here though: --- I am slowly condensing my data on old HDD's [1x 16T, 2x 10T, 2x 2T] while running pre-checks on the new drive(s). Ordering 10G components (sw and svr already have 10G, but I need a card for my desktop (5950X/3070/64G,NVMe) 1T 980 NVMe Cache drives arrived [need to lookup the temp issue and apply the fix again] Todo: -5900X -Setup cache -Move HA-Core config -Condense -10G -Move Data -Preclear -DNS docker -test failure modes Edited July 18, 2022 by NathanR Added plugins & dockers Quote Link to comment
SimonF Posted July 18, 2022 Share Posted July 18, 2022 3 minutes ago, NathanR said: Solved here though: Why did you not just use the qcow2 file? Quote Link to comment
NathanR Posted July 18, 2022 Author Share Posted July 18, 2022 7 minutes ago, SimonF said: Why did you not just use the qcow2 file? No matter what I did I could not get the qcow2 file to launch after EFI-Shell came up. I googled for hours finally finding that as the fix (after seeing stuff about CSM and Secure Boot all for older versions 4.xx/5.xx, ~2019) Quote Link to comment
NathanR Posted July 18, 2022 Author Share Posted July 18, 2022 (edited) In other news... I debated on preclear and decided on it eventually. WD's new 20T Gold drives are amazing. 295-peak read Edited July 18, 2022 by NathanR Added Disk Speed Image Quote Link to comment
NathanR Posted July 21, 2022 Author Share Posted July 21, 2022 (edited) Preclear signature was lost on the 20T Now it will have to do it again I wonder if this has to do with Unassigned Devices or Dynamix File Manager being installed? --- Finished my VM vs HW testing. Interesting results... Overall, seems not a huge difference. Fairly impressive IMO Drive speed differences are rather strange. (Strange because image 1 is the passthru and image 2 is hw, lol) Who cares tho. Edited July 21, 2022 by NathanR Quote Link to comment
NathanR Posted July 23, 2022 Author Share Posted July 23, 2022 Ran into the dreaded Ryzen Linux power states issue. I was curious if I was going to have an issue with it or not. I had BIOS set for typical idle current, but when I replaced IPMI/BMS chip I think it got reset somehow. I haven't had a need for disabling global c-states yet, but I am curious once I have more going on if it will be necessary. My Threadripper TrueNAS build definitely needed the fix and would reboot constantly before fixing those issues. --- Solved via FAQ for v6, this is a very good post that quickly explains the various issues running Ryzen and acceptable parameters. Quote Link to comment
NathanR Posted August 1, 2022 Author Share Posted August 1, 2022 (edited) Had a crash even after fixing Idle current. Probably will disable global C-states now. Upgraded to 6.11-RC2 to see if system is more stable for X570 builds. Tried to get temp/sensors working. Foundout the nerdtools plugin for Pearl is broken on 6.11 - hopefully we seen an update soon https://www.reddit.com/r/unRAID/comments/w9647x/unraid_6110rc1_now_available_notes_in_comments/ Version 6.11.0-rc2 2022-07-27 Improvements With this release there have been many base package updates including several CVE mitigations. The Linux kernel update includes mitigation for Processor MMIO stale-data vulnerabilities. The plugin system has been refactored so that 'plugin install' can proceed in the background. This alleviates issue where a user may think installation has crashed and closes the window, when actually it has not crashed. Many other webGUI immprovements. Bug fixes Fixed issue in VM manager where VM log can not open when VM name has an embedded '#' character. Fixed issue where Parity check pause/resume on schedule was broken. Change Log vs. Unraid OS 6.10 Base distro: aaa_base: version 15.1 aaa_glibc-solibs: version 2.35 aaa_libraries: version 15.1 adwaita-icon-theme: version 42.0 appres: version 1.0.6 at-spi2-core: version 2.44.1 atk: version 2.38.0 bind: version 9.18.5 btrfs-progs: version 5.18.1 ca-certificates: version 20220622 cifs-utils: version 6.15 coreutils: version 9.1 curl: version 7.84.0 dbus: version 1.14.0 dmidecode: version 3.4 docker: version 20.10.17 (CVE-2022-29526 CVE-2022-30634 CVE-2022-30629 CVE-2022-30580 CVE-2022-29804 CVE-2022-29162 CVE-2022-31030) editres: version 1.0.8 etc: version 15.1 ethtool: version 5.18 file: version 5.42 findutils: version 4.9.0 freeglut: version 3.2.2 freetype: version 2.12.1 fribidi: version 1.0.12 fuse3: version 3.11.0 gdbm: version 1.23 gdk-pixbuf2: version 2.42.8 git: version 2.37.1 glib2: version 2.72.3 glibc: version 2.35 gnutls: version 3.7.6 gptfdisk: version 1.0.9 harfbuzz: version 5.0.1 hdparm: version 9.64 htop: version 3.2.1 icu4c: version 71.1 inotify-tools: version 3.22.6.0 iproute2: version 5.18.0 iptables: version 1.8.8 json-c: version 0.16_20220414 kernel-firmware: version: 20220725_150864a kmod: version 30 libX11: version 1.8.1 libXcursor: version 1.2.1 libaio: version 0.3.113 libcap-ng: version 0.8.3 libdrm: version 2.4.110 libepoxy: version 1.5.10 libevdev: version 1.12.1 libgcrypt: version 1.10.1 libgpg-error: version 1.45 libidn: version 1.41 libjpeg-turbo: version 2.1.3 libmnl: version 1.0.5 libnetfilter_conntrack: version 1.0.9 libnfnetlink: version 1.0.2 libnftnl: 1.2.2 libnl3: version 3.6.0 libtiff: version 4.4.0 libtiff: version 4.4.0 liburcu: version 0.13.1 libusb: version 1.0.26 libxcb: version 1.15 libxkbcommon: version 1.4.1 libzip: version 1.9.2 libX11: version 1.8.1 listres: version 1.0.5 logrotate: version 3.20.1 lsof: version 4.95.0 lzip: version 1.23 mc: version 4.8.28 mcelog: version 184 mkfontscale: version 1.2.2 nano: version 6.3 nettle: version 3.8 nfs-utils: version 2.6.1 nghttp2: version 1.48.0 ntfs-3g: version 2022.5.17 oniguruma: version 6.9.8 openssh: version 9.0p1 openssl: version 1.1.1q (CVE-2022-1292 CVE-2022-2097 CVE-2022-2274) openssl-solibs: version 1.1.1q (CVE-2022-1292) pango: version 1.50.8 pciutils: version 3.8.0 pcre2: version 10.40 php: version 7.4.30 (CVE-2022-31625 CVE-2022-31626) rsync: version 3.2.4 samba: version 4.16.4 (CVE-2022-2031 CVE-2022-32744 CVE-2022-32745 CVE-2022-32746 CVE-2022-32742) setxkbmap: version 1.3.3 shared-mime-info: version 2.2 sqlite: version 3.39.2 sudo: version 1.9.11p3 sysfsutils: version 2.1.1 tdb: version 1.4.7 tevent: version 0.12.1 tree: version 2.0.2 util-linux: version 2.38 wget: version 1.21.3 xauth: version 1.1.2 xclock: version 1.1.1 xdpyinfo: version 1.3.3 xfsprogs: version 5.18.0 xkeyboard-config: version 2.36 xload: version 1.1.4 xmodmap: version 1.0.11 xsm: version 1.0.5 xterm: version 372 xwud: version 1.0.6 Linux kernel: version 5.18.14 (CVE-2022-21123 (CVE-2022-21123 CVE-2022-21125 CVE-2022-21166) oot: md/unraid: version 2.9.23 CONFIG_IOMMU_DEFAULT_PASSTHROUGH: Passthrough CONFIG_VIRTIO_IOMMU: Virtio IOMMU driver CONFIG_X86_AMD_PSTATE: AMD Processor P-State driver CONFIG_FIREWIRE: FireWire driver stack CONFIG_FIREWIRE_OHCI: OHCI-1394 controllers CONFIG_FIREWIRE_SBP2: Storage devices (SBP-2 protocol) CONFIG_FIREWIRE_NET: IP networking over 1394 CONFIG_INPUT_UINPUT: User level driver support CONFIG_INPUT_JOYDEV: Joystick interface CONFIG_INPUT_JOYSTICK: Joysticks/Gamepads CONFIG_JOYSTICK_XPAD: X-Box gamepad support CONFIG_JOYSTICK_XPAD_FF: X-Box gamepad rumble support CONFIG_JOYSTICK_XPAD_LEDS: LED Support for Xbox360 controller 'BigX' LED Management: rc.nginx: enable OCSP stapling on certs which include an OCSP responder URL rc.wireguard: add better troubleshooting for WireGuard autostart rc.S: support early load of plugin driver modules upc: version v1.3.0 webgui: Plugin system update Detach frontend and backend operation Use nchan as communication channel Allow window to be closed while backend continues Use SWAL as window manager Added multi remove ability on Plugins page Added update all plugins with details webgui: docker: use docker label as primary source for WebUI This makes the 'net.unraid.docker.webui' docker label the primary source when parsing the web UI address. If the docker label is missing, the template value will be used instead. webgui: Update Credits.page webgui: VM manager: Fix VM log can not open when VM name has an embedded '#' webgui: Management Access page: add details for self-signed certs webgui: Parity check: fix regression error webgui: Remove session creation in scripts webgui: Update ssh key regex Add support for ed25519/sk-ed25519 Remove support for ecdsa (insecure) Use proper regex to check for valid key types webgui: misc. style updates webgui: Management access: HTTP port setting should always be enabled webgui: Fix: preserve vnc port settings webgui: Fix regression error in plugin system webgui: Fix issue installing registration keys webgui: Highlight case selection when custom image is selected webgui: fix(upc): v1.4.2 apiVersion check regression Edited August 1, 2022 by NathanR added 6.11-RC2 info Quote Link to comment
ConnerVT Posted August 3, 2022 Share Posted August 3, 2022 On 7/23/2022 at 2:11 PM, NathanR said: Ran into the dreaded Ryzen Linux power states issue. I was curious if I was going to have an issue with it or not. I had BIOS set for typical idle current, but when I replaced IPMI/BMS chip I think it got reset somehow. I haven't had a need for disabling global c-states yet, but I am curious once I have more going on if it will be necessary. My Threadripper TrueNAS build definitely needed the fix and would reboot constantly before fixing those issues. I only needed to set Typical Idle Current (and appropriate DRAM settings) for my 1st Gen Ryzen 1500X. The 1st gen were the most notorious for Linux issues, but hasn't had one hang/freeze/crash for 1.5+ years since setting it correctly. As for Nerdtools, it will get caught up eventually. This is the price one pays to be an early adopter of a new revision of software (especially a release candidate). If a test bed system, no worries. For one you rely on, well... 😞 Quote Link to comment
NathanR Posted August 4, 2022 Author Share Posted August 4, 2022 (edited) 20 hours ago, ConnerVT said: I only needed to set Typical Idle Current (and appropriate DRAM settings) for my 1st Gen Ryzen 1500X. The 1st gen were the most notorious for Linux issues, but hasn't had one hang/freeze/crash for 1.5+ years since setting it correctly. As for Nerdtools, it will get caught up eventually. This is the price one pays to be an early adopter of a new revision of software (especially a release candidate). If a test bed system, no worries. For one you rely on, well... 😞 Thanks! I haven't had time to look into why it crashed the second time. Any pointers on where to look in the logs? I read that the logs don't show pre-boot logs IIRC. Idk, I'll investigate in the future. Re: RC - I completely agree, I usually don't run RC stuff, but I saw all the significant improvements (in-general & x570/kernel) and wanted to try it out. My SVR right now is just a test-bench and is running things that aren't critical (yet) just so I can get used to Unraid in a relaxed environment. That said, I am in the process of moving computer cases, installing 10G, and swapping drives so I'll be ready soon to move the server towards production Edit: Just read your story I think know I am in good company Edited August 4, 2022 by NathanR Quote Link to comment
ConnerVT Posted August 4, 2022 Share Posted August 4, 2022 (edited) From what I think I know, the normal logging for Unraid doesn't help much after a "crash" where you must boot the system to access anything. Knock on wood - I haven't found myself there yet. As Unraid loads and runs in memory (RAM), it also keeps its syslog there as well. So when you boot, POOF it is gone. You can set up a syslog server, which will write the log to disk. Believe the intent of normal logging to memory is to reduce disk I/O, important for those who wish to keep disks spun down (or not hammering/filling the flash drive). Though with Ryzen lock ups, there really isn't much of anything logged. The CPU bug is in the chip, so when it happens, things just freeze up. So not much help from the syslog, regardless of how you collect it. Glad you like the story. I did really intend just to put an appliance down in the basement and forget about it. But it has grown a life of its own. Since writing that, I procured another gently used 8TB drives - in cold store backup along with a 6TB waiting in the same drawer of the server desk. Plus another portable drive for off-site backup storage. Unraid is a pretty decent platform, and it amazes me how much (and how easily) my little 4c/8t 1st gen Ryzen server can handle. Edited August 4, 2022 by ConnerVT speeling Quote Link to comment
NathanR Posted October 2, 2022 Author Share Posted October 2, 2022 (edited) Update. Now running Version 6.11.0 2022-09-23 Pearl & iperf3 included with the kernel so...yay. My 10G NIC's are confirmed working; woot! I was having issues confirming speeds from within the W10 VM. Also, I have my 1T 980's in what I think is a Raid 1 pool. But I've read conflicting posts about whether this is operating properly or not. Pressing the rebalance as Raid 1 does nothing. I also modified the temp warning/errors to 85c so I didn't get the annoying Samsung 980 bug. I really wish they would do a firmware release. BTW...how do I do a NVMe firmware update? Oh! and I'm super happy that the temp sensors work out of the box now (well, there is a tiny config todo). Pearl or Linux Kernel with AMD tunings, whatever, don't care, I'm happy figured out exactly which by looking at IPMI & running P95 in a VM. CPU @ 70c and x570 @ ~50c lol Edited October 2, 2022 by NathanR Quote Link to comment
NathanR Posted October 3, 2022 Author Share Posted October 3, 2022 I have a NH-D9L on the CPU now NF-A4x10 PWM on the x570 chipset, which gets super hot. (ASRock really should have had a fan & larger cooler on x570 chipset) pwm cpu-rpm x570-rpm 100% 1900 5000 75% 1500 4000 50% 1100 2700 25% 600 1600 Quote Link to comment
NathanR Posted October 3, 2022 Author Share Posted October 3, 2022 Brainstorming/thoughts post... Looking into converting my .xva (xcp-ng) vms into Unraid format (qcow?) Maybe this? - https://forums.unraid.net/topic/69244-convert-vmware-vm-to-kvm/ Looking into converting my physical disk into unraid drive. https://kmwoley.com/blog/convert-a-windows-installation-into-a-unraid-kvm-virtual-machine/ https://kmwoley.com/blog/reduce-shrink-raw-image-img-size-of-a-windows-virtual-machine/ A little worried I can't shrink my 500GB NVMe drive down to a manageable 50GB lol Looking into creating new W10 VM I was looking here > https://wiki.unraid.net/Manual/VM_Management#Basic_VM_Creation But that is wrong, this is correct steps to install HDD driver > https://wiki.unraid.net/index.php/UnRAID_Manual_6#Installing_a_Windows_VM Latest virtio drivers: https://github.com/virtio-win/virtio-win-pkg-scripts/blob/master/README.md Quote Link to comment
NathanR Posted October 17, 2022 Author Share Posted October 17, 2022 Expanded my VM's HDD (vdisk1.img) size from 30GB to 60GB. Deleted the recovery partition so I could extend the OS drive space from 30GB to 40GB. https://www.partitionwizard.com/partitionmagic/delete-recovery-partition.html Quote Link to comment
NathanR Posted October 17, 2022 Author Share Posted October 17, 2022 VM is now running d-tools server on Win10x64 Pro. Server has been up for 8 days since upgrade to 6.11 Hoping for lots of stability now that I have something production-esque running on the server. Next will be Plex server, video-card passthru, and adding HDD's. Quote Link to comment
NathanR Posted October 19, 2022 Author Share Posted October 19, 2022 Plex server crashed again Tis why I'm hesitant to move Plex/Unifi/Syncthing/etc. over I Setup syslog/kiwi, hopefully I can catch the error now. https://documentation.solarwinds.com/archive/pdf/kss/kss_getting_started_guide.pdf Quote Link to comment
NathanR Posted January 22, 2023 Author Share Posted January 22, 2023 (edited) 95 days, 1 hour, 58 minutes uptime on 6.11.0 Running: 1x W10 VM Handful of dockers Some apps Just updated to 6.11.5 Needed a few reboots to get stats working properly. Hopefully stable for a long time again Found this useful command for USB Flash Drive (OS) backups from this reddit post: https://www.reddit.com/r/unRAID/comments/wp44pp/reminder_backup_your_unraid_usb_drive/ zip -r /mnt/user/backups/unraidbackup/flash`date +%d-%m-%Y`.zip /boot Fan 1 - CPU 80mm Noctua Fan 2 - x570 Chipset 40mm Noctua Fan 3 - Chassis Fan(s) 120mm arctic p12 PWM PST CO [25% = 500rpm, 50% = 900rpm, 75% = 1200rpm, 100% = 1800rpm] Fan 4 - HDD chassis fan 80mm arctic p8 PWM PST CO [25% = 900rpm, 50% = 1500rpm, 75% = 2300rpm, 80% = 2500rpm, 100% = 2900rpm] Fan 5 - future 120mm Fan 6 - future 120mm Edited January 22, 2023 by NathanR Added Info Quote Link to comment
NathanR Posted February 4 Author Share Posted February 4 (edited) Been running 6.11.5 for a while now. Current uptime is 2 months 22 days. Can't remember the last thing I did that required me to reboot it. I remember now, added a USB 3.0 card so I could pass-through for my W10 VM so I could run LOR for my Christmas lights. No crashes. Been very happy. Future Wishes: VM cloning, backup, snapshot, etc. Rebuild server with 13900K, X13SAE-F, 128GB ECC quicksync (GPU) encoding for plex more USB Plugins: Community CA Backup / Restore Appdata Dynamix Auto Fan Control Dynamix File Manager Dynamix System Statics Dynamix System Temperature Network UPS Tools (NUT) Unassigned Devices Unassigned Devices Plus Unassigned Devices Preclear Unraid Connect User Scripts Docker: blueiris (not on) DiskSpeed (not on) docker-diag-tools (not on) ESPHome glances Grafana-Unraid-Stack (not on) luckyBackup (not on) MongoDB (not on) OpenProject Phoronix-Test-Suite (not on) plex qbitorrent (not on) syslog-ng (not on) unifi-controller UniFi-Protect-Backup UptimeKuma VMs Dev-Home Assistant Home Assistant W10-SVR Win10 (not on) Decided to make a development home assistant VM to play around with things and not break my house. Process still works in Feb of 2024 Download Home Assistant .qcow2 file Copy to Array somehow Shared folder Upload via Dynamix File Manager app Create VM (linux) 32GB HDD Delete the vdisk vdisk1.img /mnt/user/domains/Dev-Home Assistant/vdisk1.img Run the convert command qemu-img convert -p -f qcow2 -O raw -o preallocation=off "/mnt/user/domains/Dev-Home Assistant/haos_ova-11.4.qcow2" "/mnt/user/domains/Dev-Home Assistant/vdisk1.img Boot the VM Launch VNC Get IP Login Woot/FIN Edited February 4 by NathanR Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.