jbartlett

I found a domain feature which may enable hyperthreading in AMD CPU's that don't support it. <feature policy='require' name='topoext'/> I will experiment with it. Utilizing the following gave a 19% improvement in RAM scores with negligible differences between 1/32/1 & 1/62/2. VM had 12 GB of RAM assigned and this evenly spread it between the two numa nodes that had the memory attached to it - VM was likewise pinned to those nodes so the VM matched the physical. <numatune> <memory mode='interleave' nodeset='0,2'/> </numatune> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='32' threads='1'/> <numa> <cell id='0' cpus='0-15' memory='6291456' unit='KiB'/> <cell id='1' cpus='16-31' memory='6291456' unit='KiB'/> </numa> </cpu>

Already seeing a 19% improvement in the memory score with this set. I'll also test hugepages.

This is what it took to get it to divide up the memory between the nodes. <numatune> <memory mode='interleave' nodeset='0,2'/> </numatune> Couldn't use any of the "auto" methods because numad isn't part of the unraid package.

Ya know, I had a feeling someone would pop in and tell me all my tests were invalid because there was another optimization. Ha! Sa'right. I'm in processing of recreating the VM with the numa setting in place from the start and I'll retest the numa config with the memory pinned to nodes 0 & 2. It was grabbing all 12G of RAM from node 0.

CPU Threadripper 2990WX with a VM running Windows 10 fully patched, RAM is G.SKILL Ripjaws 4 Series 64GB (8 x 8GB) DDR4 2133 (PC4 17000) Motherboard is a ASUS ROG Zenith Extreme Alpha X399. MB & RAM are at stock settings, CPU governor set to Performance. VM is pinned to NUMA nodes 0 & 2 which has the PCIe & RAM attached, utilizing all CPUs and the emulator pin is on NUMA node 1, CPU 16. Total OS memory assigned is 12GB. No NUMA || NUMA Benchmark CB R20 PT CPU PT RAM | CB R20 PT CPU PT RAM || Benchmark CB R20 PT CPU PT RAM | CB R20 PT CPU PT RAM CPU Topo 1/32/1 1/32/1 1/32/1 | 1/16/2 1/16/2 1/16/2 || CPU Topo 1/32/1 1/32/1 1/32/1 | 1/16/2 1/16/2 1/16/2 Average 6572 20944 1261 | 6515 20831 1257 || Average 6408 20617 1389 | 6539 20958 1300 Highest 6620 21085 1263 | 6443 20873 1258 || Highest 6525 20728 1391 | 6589 21144 1306 Lowest 6537 20810 1255 | 6484 20805 1254 || Lowest 6438 20455 1385 | 6511 20746 1297 Variance 83 275 8 | 59 68 4 || Variance 87 273 6 | 78 398 9 The left set has no NUMA node configuration. The 1/16/2 paring shows a roughly 0.7% drop in CPU performance and a negligible difference in RAM performance. However, the variance was much lower - or the difference between the high & low scores which indicates increased stability in processing speeds at a slight performance loss. With a NUMA configuration, things were different. CineBench showed roughly the same performance but PerformanceTest 9.0 showed a larger variance in scores in which there was several much higher scores that were dropped in order to bring the variance down below a thousand. What the NUMA configuration clearly benefited is in RAM test scores if you create the numa node in the guest OS to match the host. <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='32' threads='1'/> <numa> <cell id='0' cpus='0-15' memory='6291456' unit='KiB'/> <cell id='1' cpus='16-31' memory='6291456' unit='KiB'/> </numa> </cpu> In short, since AMD CPU's do not support hyperthreaded CPU's to the Guest OS, setting cores=16/threads=2 shows mixed results based on if you specify a numa node or not. It's probably recommended to always have threads=1 so it matches what the guest OS sees.

I'll give that a shot. For informational sake, having an emulated CPU gave a 9% boost in CPU performance with PerformanceTest 9.0 on 1/16/2 but only on every OTHER test. On the odd test, it scored the same as 1/32/1. Twenty test over two runs showed the same pattern. Cinebench R20 showed comparable scores between 1/32/1 & 1/16/2.

CPU is half if threads=2. Did you mean a quarter? I've tested to see if such a thing would even boot with CPU & threads reversed, it did, but I didn't do any benchmarks with it. That was back when I was trying to figure out how to get the AMD guest OS to see hyperthreaded CPUs before I discovered that AMD doesn't support it.

My testing shows that setting up a numa configuration in your guest benefits memory speed but not really CPU performance on an AMD system. Best to leave threads=1 for a slightly improved CPU performance over threads=2. Edit: ARGH! Some how, the CPU Mode ended up set to Emulated instead of passthrough. I didn't make that change but let me re-do these tests yet again.

Just learned something new. Using Microsoft's "Coreinfo" utility, Intel CPU's that support hyper threading will show up as hyper threaded CPU's to the guest regardless if you have threads=1 or threads=2. AMD CPU's, notably the Threadripper series (the only ones I have), do not. I was clued in when I was viewing the VM logs and saw a warning that AMD doesn't support the feature (which doesn't show up all the time either). For the unRAID GUI, it seemed to auto flag my Intel CPU for threads=2 'cause I don't recall making that change myself (my mileage will vary).

SeaBIOS VM scores roughly 2% higher than OVMF BIOS on CPU benchmarks with Windows 10.

I see many people commenting that they always change their VM XML from something like the following to improve system performance. I've done the same but I came across Microsoft's "coreinfo" utility which revealed that the VM wasn't actually seeing any hyper-threaded CPUs. So I decided to benchmark it using the following topologies on an existing Win10 VM. Iteration 1: <topology sockets='1' cores='32' threads='1'/> Iteration 2: <topology sockets='1' cores='16' threads='2'/> Based on my testing, I do not see any improvements. I used Cinebench R20 & PerformanceTest 9.0, running each ten times and taking the max & average score, discarding any test that scored too far off of the high & low end of the variance (difference between the high & low scores) until the variance fell into an acceptable value based on my observations of running these benchmarks dozens of times. I picked these because I needed a couple and they were quick & easy to set up (I lost count on how many Win10 VM's I've set up in the past month). For the initial test, I ran with cores=32 threads=1 and then tried to get close to or better variance on the cores=16 threads=2 test. In my scenario, I'm running a Threadripper 2 2990wx with the CPU's 0-15 & 32-47 pinned (numa 0 & 2 which has the PCIe & RAM attached) and the emulator pin on CPU 16 (numa 1). This particular config is my intended use case with video broadcasting using Livestream Studio outputting a 1080p@60fps stream to YouTube plus at least two NDI streams of a 1080p@60fps video feed to be consumed by other VM/PC's. OS is Windows 10 fully updated, GPU is a Quadro P2000. Benchmark CB R20 PT CPU PT RAM | CB R20 PT CPU PT RAM CPU Topo 1/32/1 1/32/1 1/32/1 | 1/16/2 1/16/2 1/16/2 Average 6572 20944 1261 | 6515 20831 1257 Highest 6620 21085 1263 | 6443 20873 1258 Lowest 6537 20810 1255 | 6484 20805 1254 Variance 83 275 8 | 59 68 4 The average scores are lower in the 1/16/2 config but they're also tighter. I'm currently running this test again passing in the numa configuration to match the host. Past test runs have shown marked improvements.

FYI - These post came from an 6.8.0 release thread but replying here on this thread as it's more on-topic. I've been having issues getting VM's to run with a video passthrough if I OC the MB by any amount. I haven't seen any difference between halving the cores and setting the threads to 2. Based off of Microsoft's "coreinfo" utility, the VM still sees single CPU's, none being hyper-threaded. I have not been successful in finding an XML config that passed the cpu pairing to the OS where the VM saw a hyper-threaded CPU. I'm doing another round of benchmarks with 10 iterations, I'll comment again when done. I still wasn't able to OC with the P2000 in the 3rd PCIe slot after moving it from the 2nd PCIe slot. In addition, I wasn't able to run any SeaBIOS VM's at all passing the card in slot #3. The most I'd see is the monitor activate but never even a POST. Graphics scores on the Quadro P2000 were in the same ballpark running in the x8 slot and the x16 slot. The 2nd & 3rd slots attach to different numa nodes but taking that into consideration with CPU pinning made no difference. My current guess is that it's due to having a different GPU (GeForce GT 1030) in the 1st PCIe slot for unraid to bind to. Unfortunately, I don't have two identical GPU's that I could put in it to test that theory.

Based on my own benchmarks with a 2990wx, the "Infinityfabric" for letting the different dies communicate has a barely noticeable effect when I forced GPUs and memory to talk off die. No difference in GPU performance, slight dip in memory performance.

How much total memory do you have and how much free when you tried to launch?

Version 2.4: Added changes submitted by @doron which should help those who got the "-s option" error by ignoring "platform" devices Display the correct throughput value when both the controller LnkSta & LnkCap are the same (current & max link speed) Reference table for PCIe controller link speed max throughput updated to add x32 Omit "platform" devices such as floppy drives from the scan Ignore "Logical/Physical Sector size" returned from hdparm as it does not properly handle SAS drives. Will implement a better fix later when I get a SAS drive

After installing a new BIOS, my Windows 10 VM was extrealy slow to start, even just to get to the startup animation. Loading the BIOS defaults corrected.

This issue may be resolved in a patch another user submitted to me. I'll post an update shortly.

I pushed an update after pulling up BeyondCompare to do a code sync but didn't actually sync the code. That'll teach me to try to work on bugs after being up for nearly 24 hours after a whirlwind vacation of New York City & Long Island! Try again, I show 2.3 on my main rig. Instead of just increasing the timeout to accommodate your rig, I just bumped it up to 9999 seconds for the controller benchmark (2.78 hours).

I need more information from you. I put in the same version of the card as yours but did not get the same drive breakout paths on the OS. Please describe everything between your motherboard and the hard drive.

I didn't know there was a 24 port controller. I have the timeout set to five minutes to try to nip any rogue processes in the bud but at 24 drives testing for 15 seconds each, it would take at least six minutes to do a complete pass. I've updated the default timeout to ten minutes. I just pushed version 2.3 - please update the Docker and try again.