My VM was slower then I was accustomed to under windows.  The GUI was generally slow and laggy.  Copying files over samba to unraid, even to cache drive, was also very slow.

 

Looking at top and viewing all cores I could see my vmware process thrashing around between processor cores.  I could set affinity for my virtual machine but not for other programs.  All of this bouncing around between cores cause expensive context switches.   I sought a better way to give vmware a dedicated core in my quad core box.

 

The typical way to do such a thing is to modify your /etc/rc.d/sysinit files to set the affinity on the "init" process.  This will set affinity on all of its children and would help to clear out a core for vmware.  I thought that since unraid loads from the bzimage I can't get to sysinit files without modifying the bzimage.  A step I very much did not want to take.

 

So I sought to write a program that would allow me simple control over process affinity.  (note: it is possible to set affinity when a process starts up from your go file (e.g. emhttp).  I did initially do this but took out that code so all affinity is managed in one place.

 

Here is my file setaffin.sh:

#!/bin/bash

function setAffinityByPID {
#$1=cpu list or range
#$2=PID
taskset -pc $1 $2 > /dev/null
return $?
}

function setAffinityByName {
#$1=cpu list or range
#$2=name of process
local COUNT
for i in `pgrep $2`;
do
	setAffinityByPID $1 $i
	let COUNT=COUNT+1
done
return $COUNT
}

function setAffinityWithChildrenByPID {
#$1=cpu list or range
#$2=PID
local COUNT
let COUNT=COUNT+1
setAffinityByPID $1 $2
for i in `pgrep -P $2`;
do
	setAffinityWithChildrenByPID $1 $i
	let COUNT=COUNT+$?
done 
return $COUNT
}

function setAffinityWithChildrenByName {
#$1=cpu list or range
#$2=name of process

#note:  this does work as intenteded but there is a flaw whereby, if a parent and child 
	#are of the same name affinity is set multiple times on children (e.g. httpd, smbd)

local COUNT
for i in `pgrep $2`;
do
	setAffinityWithChildrenByPID $1 $i
	let COUNT=COUNT+$?
done
return $COUNT
}

#most processes are in here (vmware*, uu, emhttp, etc).  This should go first.
setAffinityWithChildrenByName "0,1" "init"

#heavy time user (only because of parity 12/1 parity calc?)
setAffinityByName "0" "unraidd"          

#heavy time user (only because of parity 12/1 parity calc?)
setAffinityByName "1" "mdrecoveryd"		
#heavy time user
setAffinityByName "1" "smbd"

#management interfaces.  they'll sleep the rest of the time.
setAffinityWithChildrenByName "0-2" "uu"
setAffinityByName "0-2" "emhttp"
setAffinityWithChildrenByName "0-2" "vmware-watchdog"

#vmware components
setAffinityByName "2" "vmnet-bridge"
setAffinityByName "2" "vmnet-dhcpd"
setAffinityByName "2" "vmnet-natd"
setAffinityByName "2" "vmware-authdlau"

#heavy time user.  is the vmware gui in here?
setAffinityByName "2" "vmware-hostd"
setAffinityByName "2" "vmnet-netifup"

#vmware virtual machine.  heavy time user.
setAffinityByName "3" "vmware-vmx"             

 

Some comments within the code but my general thoughts were:

Processes that are used in server management can go to any available core other then my vmware core.  You want this fast.  This is stuff like unmenu, emhttp, vmware management web interface.

VMWare virtual machine gets core 3 - this is the vmware-vmx process.

VMWare helper processes get core 2 - the big user here is vmwares hostd process.

My other heavy proc time users are unraidd, mdrecoveryd and smbd.  I put smbd on core 1 and then of the remaining two processes mdrecoveryd uses less time then unraidd so i put mdrecoveryd on core1 with smbd and unraidd goes on core 1.

 

Many other children process of the init process are also on cores 0 & 1.

unraidd and mdrecoveryd are children process of process 0.  these are the only two descendants of processes 0 I played with.  other descendant processes of 0 remained untouched.

 

I started and finished this last night in maybe 20 minutes (my first time writing bash).  Next might some additional information passed to sdout but the code generally does what it says.

 

My hopes with this setup is that when copying data from my virtual machine to the unraid the speed would increase because data would be serialized across the cores.  so the virtual machine on core 3 uses the vmware processes on core 2 to copy over the network to smbd on core1 where it goes to unraidd on core0.  i have no idea how unraidd and mdrecoveryd processes are used by unraid.  there may be answers on this forum but my first goal was to make a tool.  now comes how best to use it.

 

First question:  When I copy to a user share with a cache drive do both unraidd and mdrecoveryd get used during the copy? 

 

Regards,

I wish I had some answers to your questions, but I have the same questions as you evidently.  I see the same performance issues as you do with the VMware guests and for some reason setting the affinity in the VMware configuration file does not always show a performance improvement.  I didn't think about setting these options in Linux directly.  Now for a couple question to you:

 

1.  Have you tested this and does it give you a noticeable performance improvement?

2.  Where do you initiate this at?  I assume the very first line of the go script?

 

It sounds like we have very similar systems and probably the same problems.  I do know if your CPU does not have hardware virtualization instructions, performance is absolutely miserable.  I hope someone is able to help you with your questions as I'm sure it would help a lot of us VMware users that see slow-downs here and there.

 

I wish I had some answers to your questions, but I have the same questions as you evidently.  I see the same performance issues as you do with the VMware guests and for some reason setting the affinity in the VMware configuration file does not always show a performance improvement.  I didn't think about setting these options in Linux directly.  Now for a couple question to you:

 

1.  Have you tested this and does it give you a noticeable performance improvement?

 

I should mention that in the past I had my single processor VM set to use processor3 through edits in my .vmx file.  Now I set processorN.use = "true" for all processors and set affinity with this script.  The two methods seem equivalent for the vmware-vmx process but the rest of the system will hop on your vmware-vmx core and I don't think the other vmware processes were affected by the affinity settings of the .vmx.

 

I can only say it "feels" faster using the vmware GUI but I have no real test evidence.  I haven't timed file copies from vmware windows guests to unraid smb shares with this configuarion but again it "feels" faster.  Before I got to that point I wanted the community to input on what makes the most sense for the unraidd and mdrecoveryd processes or do the research myself.  Until I wrote this script I had no real configuration change to benchmark.

 

2.  Where do you initiate this at?  I assume the very first line of the go script?

 

Actually no.  You can set affinity on the "init" process in the beginning of go.  But if you tried to set affinity for process "vmware-vmx" at the start of the go script it would fail because that process does not exist.  there is not PID to act on.  You can start other processes with certain affinity using the taskset command in the go script but it just seemed easier to manage it all in one place.

 

This script would go last.  You could even do a big sleep and let things settle down first.  Running it with an active system doesn't seem to make anything unstable.  If vmware-vmx is running an active VM with utorrent, you're copying stuff to a protected array over samba, serving media from unraid, etc. you can still go ahead and run this script from what I've seen everything remains super stable.

 

It sounds like we have very similar systems and probably the same problems.  I do know if your CPU does not have hardware virtualization instructions, performance is absolutely miserable.  I hope someone is able to help you with your questions as I'm sure it would help a lot of us VMware users that see slow-downs here and there.

 

I have an Intel Quad Core Q6600 and yes it has hardware VT.  I do not have a hardware virtualized MMU though which is available in the Core i7. 

 

I would really like to give vmware direct access to its disks instead of using a .vmdk file.  Do you know if that would help?

I also do not have a linux swap partition (5gb ram).  Do you know if that would help?

I put in a gigE pci network card that is currently eth1 (It is not in use).  I know unraid is forced to use eth0 but I could bridge the vmware virtual nic to eth1 pretty easily.  Do you know if that would help?

 

What do you get when you put all unraid, samba and vmware helper processes on realcpu0 and realcpu1 and use realcpu2 and realcpu3 with a 2 processor VM?  In general people say dual proc vm's are bad because when the virtualcpu0 is waiting for virtualcpu1 to finish work but virtualcpu1 is waiting for realcpuN to finish work you have two virtualcpu's idle even though its possible that the realCPU that virtualcpu0 is on is available for work.  But if most of the processes are off of realcpu2 and realcpu3 a boost seems more plausible.  More config changes to make.

 

I do have 5gb of RAM and run unRaid 4.4.2.

 

I would really like to give vmware direct access to its disks instead of using a .vmdk file.  Do you know if that would help?

I am not sure, never tried or even heard of being able to do this.  I only know of the vmdk file option.

I also do not have a linux swap partition (5gb ram).  Do you know if that would help?

I have not used a linux swap partition either.  My assumption is it should help, but I'm not sure how much.  I would be interested in trying this, but my linux knowledge is little to none.  I am great at following guides though

I put in a gigE pci network card that is currently eth1 (It is not in use).  I know unraid is forced to use eth0 but I could bridge the vmware virtual nic to eth1 pretty easily.  Do you know if that would help?

I disabled my on-board nic and use an add-in intel Pro 1000 card.  I have noticed no differences since doing this.  The max transfer rate I get through a VM is 10-14 MB/s max.

What do you get when you put all unraid, samba and vmware helper processes on realcpu0 and realcpu1 and use realcpu2 and realcpu3 with a 2 processor VM?  In general people say dual proc vm's are bad because when the virtualcpu0 is waiting for virtualcpu1 to finish work but virtualcpu1 is waiting for realcpuN to finish work you have two virtualcpu's idle even though its possible that the realCPU that virtualcpu0 is on is available for work.  But if most of the processes are off of realcpu2 and realcpu3 a boost seems more plausible.  More config changes to make.

 

I do have 5gb of RAM and run unRaid 4.4.2.

 

I have not tried your script yet, although I am curious and may do so soon.  I have noticed whenever I run a dual CPU VM, performance absolutely sucks - there is no better way to put it.  I should do some more research on this as I would love to have a dual CPU VM working as well as it does at work in ESX 4.0.

 

I am running a Q9400 CPU with 6 GB of memory (2 GB allocated to my only VM running at the moment).  I noticed little difference allocating 2GB or 4GB to the VM, so I set it to 2 and let unRAID have the rest for caching.  I am on VMware 2.0.2 running on unRAID 4.5b11 for reference.  I did notice a great improvement moving to beta 11 when some tweaks to SAMBA were made to better cache directory listings and handle larger directories.  File transfers were no faster, but things were snappier.  I will move to 4.5 final soon as there is another tweak for the drive spin-up issue I think i hit on a regular basis.

 

vmware direct access to disk used to be called "raw disk" support.  is still available in workstation, esx and was in server v1.  server v2 doesn't have it in the gui so you have to get it in through unofficial means.  i'm guessing that it'll be worth it. 

 

re: two nics.  i use host only networking to communicate b/t win2k3 guest and unraid host.  it shouldn't be faster but to bridge vm nic to eth1 and communicate with unraid through physical switch and to eth0 may be better?

 

The two cpu vm's were slow due to the scenario i described most likely.  it's not uncommon. 

 

I haven't played with any beta releases.  i use the updated mover script only.  other then that it's just v4.4.2.  my stuff is very stable.  i just want more power.

 

I have turned off cache_dirs because i thought that perhaps it running so often was affecting vmware performance.  i didn't see any appreciable difference.

 

We both have some nice hardware and i hope that we can eek out better performance. 

 

Do you have your .vmdk file on your cache drive?  It makes some sense for that to be slow as you're copying to and from the same spindle.  i have an unprotected non cache 750gb disk in my unraid box.  my windows guests uses a vmdk on that disk for pagefile and other data.  it is this disk that i copy to unraid from.  so i'm copying from 1 physical disk to another, the cache drive. 

 

--sld

 

I would really like to give vmware direct access to its disks instead of using a .vmdk file.  Do you know if that would help?

I am not sure, never tried or even heard of being able to do this.  I only know of the vmdk file option.

I also do not have a linux swap partition (5gb ram).  Do you know if that would help?

I have not used a linux swap partition either.  My assumption is it should help, but I'm not sure how much.  I would be interested in trying this, but my linux knowledge is little to none.  I am great at following guides though

I put in a gigE pci network card that is currently eth1 (It is not in use).  I know unraid is forced to use eth0 but I could bridge the vmware virtual nic to eth1 pretty easily.  Do you know if that would help?

I disabled my on-board nic and use an add-in intel Pro 1000 card.  I have noticed no differences since doing this.  The max transfer rate I get through a VM is 10-14 MB/s max.

What do you get when you put all unraid, samba and vmware helper processes on realcpu0 and realcpu1 and use realcpu2 and realcpu3 with a 2 processor VM?  In general people say dual proc vm's are bad because when the virtualcpu0 is waiting for virtualcpu1 to finish work but virtualcpu1 is waiting for realcpuN to finish work you have two virtualcpu's idle even though its possible that the realCPU that virtualcpu0 is on is available for work.  But if most of the processes are off of realcpu2 and realcpu3 a boost seems more plausible.  More config changes to make.

 

I do have 5gb of RAM and run unRaid 4.4.2.

 

I have not tried your script yet, although I am curious and may do so soon.  I have noticed whenever I run a dual CPU VM, performance absolutely sucks - there is no better way to put it.  I should do some more research on this as I would love to have a dual CPU VM working as well as it does at work in ESX 4.0.

 

I am running a Q9400 CPU with 6 GB of memory (2 GB allocated to my only VM running at the moment).  I noticed little difference allocating 2GB or 4GB to the VM, so I set it to 2 and let unRAID have the rest for caching.  I am on VMware 2.0.2 running on unRAID 4.5b11 for reference.  I did notice a great improvement moving to beta 11 when some tweaks to SAMBA were made to better cache directory listings and handle larger directories.  File transfers were no faster, but things were snappier.  I will move to 4.5 final soon as there is another tweak for the drive spin-up issue I think i hit on a regular basis.