Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

[6.6.0-rc2] VM Memory allocation across numa boundaries

  • Closed
  • Minor

So, after getting into RC2, I was trying to optimize my pinning using the new interface.  I looked up my NUMA boundaries in the process: 

 

numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 16 17 18 19 20 21 22 23
node 0 size: 32040 MB
node 0 free: 256 MB # <<< Make note of this value
node 1 cpus: 8 9 10 11 12 13 14 15 24 25 26 27 28 29 30 31
node 1 size: 32243 MB
node 1 free: 19974 MB
node distances:
node   0   1
  0:  10  16
  1:  16  10

I'm currently running two VMs as of this command.

 

VM 1: 16gb RAM, CPUs 4-7, 20-23 (So, numa node 0, in CPU pairs)

VM 2: 16gb RAM, CPUs 8-11, 24-27 (Numa node 1)

 

But as you'll note, all the RAM is being allocated to node 0.  Uh oh. Let's check:

numastat qemu

Per-node process memory usage (in MBs)
PID                               Node 0          Node 1           Total
-----------------------  --------------- --------------- ---------------
13479 (qemu-system-x86)         16473.43            0.25        16473.68
27148 (qemu-system-x86)         13259.18         3204.48        16463.66
-----------------------  --------------- --------------- ---------------
Total                           29732.60         3204.74        32937.34

Well, crap.  That's no good.  I then tried to force it using the <numatune> tags.  This works fine for VM 1, which is completely in it's own node, but for VM 2, this makes it take forever to start up, because it tries to force the second qemu instance onto node 1 (where it should be) and you get a bunch of numa misses when the memory is allocated to node 0 anyway.  This can also cause some NVRAM corruption in combination with other numa-optimizations and xml configuration settings, though I'm not able to remember exactly which one borked up the VM so bad I had to restore the .img file, nvram and xml to get nvidia drivers working again.

 

I imagine this will be extra important for 2990 users as two of the cores have significantly better memory access than the others and you'd want to keep VMs nicely in line with these boundaries for optimum performance.

 

Obviously we don't want this boundary crossing to happen with other processors (like my 1950) for performance reasons as well.

Bonus bug: WebTerminal is really slow this release once you have some text in the window compared to last release.

 

Bonus question: Any chance of getting 'numad' baked in so we can use "auto" in numatune?

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.