Everything posted by Rhynri
-
[6.6.0-rc2] VM Memory allocation across numa boundaries
So, after getting into RC2, I was trying to optimize my pinning using the new interface. I looked up my NUMA boundaries in the process: numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23 node 0 size: 32040 MB node 0 free: 256 MB # <<< Make note of this value node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31 node 1 size: 32243 MB node 1 free: 19974 MB node distances: node 0 1 0: 10 16 1: 16 10 I'm currently running two VMs as of this command. VM 1: 16gb RAM, CPUs 4-7, 20-23 (So, numa node 0, in CPU pairs) VM 2: 16gb RAM, CPUs 8-11, 24-27 (Numa node 1) But as you'll note, all the RAM is being allocated to node 0. Uh oh. Let's check: numastat qemu Per-node process memory usage (in MBs) PID Node 0 Node 1 Total ----------------------- --------------- --------------- --------------- 13479 (qemu-system-x86) 16473.43 0.25 16473.68 27148 (qemu-system-x86) 13259.18 3204.48 16463.66 ----------------------- --------------- --------------- --------------- Total 29732.60 3204.74 32937.34 Well, crap. That's no good. I then tried to force it using the <numatune> tags. This works fine for VM 1, which is completely in it's own node, but for VM 2, this makes it take forever to start up, because it tries to force the second qemu instance onto node 1 (where it should be) and you get a bunch of numa misses when the memory is allocated to node 0 anyway. This can also cause some NVRAM corruption in combination with other numa-optimizations and xml configuration settings, though I'm not able to remember exactly which one borked up the VM so bad I had to restore the .img file, nvram and xml to get nvidia drivers working again. I imagine this will be extra important for 2990 users as two of the cores have significantly better memory access than the others and you'd want to keep VMs nicely in line with these boundaries for optimum performance. Obviously we don't want this boundary crossing to happen with other processors (like my 1950) for performance reasons as well. Bonus bug: WebTerminal is really slow this release once you have some text in the window compared to last release. Bonus question: Any chance of getting 'numad' baked in so we can use "auto" in numatune?
-
[6.6.0-rc1] xHCI dead, incomplete boot Threadripper 1950
My apologies upfront - no logs available as the xHCI controller dies, and there is no PS2 support on the board for CLI login. Additionally, since the UNRAID drive is unreachable during a critical point in boot, no networking is available. As you can see from the attached screen, the xHCI controller is listed as dead by the kernel during a critical juncture in the boot-up sequence, preventing successful boot. Just before this, you can see my array drives being listed, and they are all attached via USB, so clearly the controller is up just prior to this message. The first time this happened, the machine hung so severely it required a full hard power-switch cycle to properly reboot and be able to enumerate the USB attached drives again. From what I can see the boot precedes according to previous boot logs until this point. The same thing happens on safe boot. Rolling back to previous version was successful and Unraid booted without incident. Please let me know how I may assist troubleshooting. Diagnostic file in separate post below. Motherboard is an Asus Zenith Extreme.
-
Shared storage for VMs
Any suggestions, no matter how small, would be appreciated. Thank you!
-
Shared storage for VMs
Hello! I'm looking at setting up an unRAID box for the specific purpose of running several gaming machines out of one tower. Watching LTT's 7 head gaming machine video*, they mention something about BTRFS "snapshots" that allow you to combine multiple VM's disks into one that will combine same data and not delete it when one of the VM's does. This would be ideal. Unfortunately my google-fu is not up to par, and I haven't been able to dig this arrangement up. I'm sure it's there, I just don't know what to search for. In a related note, it currently has three 500gb 960 Evo drives. Ideally, I'd like to combine these for speed, and make virtual drives out of them for the VM's. I'm not really concerned about any data security to speak of, as these machines will be "disposable" clones of each other, essentially. I'm also willing to add additional drives as needed for an optimum configuration. Any assistance on how to configure this storage for optimal performance would be welcome. If the answer is just "stripe them and plop your VM's on them", perfect! If anything else, please elaborate, I'm all ears. Thanks for your time! *Link should bring you to the correct time in the video for your viewing pleasure. It's 10:45 if it doesn't.