XML Performance Tuning for SSDs [deprecated]

[jonp]

NOTE:  THIS GUIDE IS NO LONGER RELEVANT OR RECOMMENDED.

 

Hey guys, just thought I'd share something interesting I found the other day that seems to be having a good impact on performance for my virtual machines.  Of course it is use at your own risk, but generally speaking, a lot of the issues I had noticed with IO hangups from time to time seem to have resolved themselves.  Anyone experiencing bad storage performance in VMs on KVM should consider trying this to get a nice speed bump.

 

NOTE:  this only applies to virtual machines running under KVM and where the virtual disk you're dealing with is in RAW format (no QCOW/QCOW2) and stored on an SSD / pool of SSDs (not HDDs).

 

#1:  Your XML file must have the following at the top:

 

<domain type='kvm' id='26' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

 

The bolded part is critical as it is what will allow you to add some of the custom statements I have in here.

 

#2:  Add IOthreads tag to your XML above the <devices> section:

 

<iothreads>6</iothreads>

 

Set the # of iothreads to match the # of vCPUs you've assigned your VM (and do not exceed the amount of physical cores you have available on your host).

 

#3:  Change your <disk> settings and specify an alias:

 

    <disk type='file' device='disk'>

      <driver name='qemu' type='raw' cache='directsync' io='native'/>

      <source file='/mnt/cache/domains/OVMF-GTX780/games2.raw'/>

      <backingStore/>

      <target dev='hde' bus='virtio'/>

      <alias name='virtio-disk4'/>

    </disk>

 

Set cache to directsync and make sure you follow the naming convention for the alias (the only thing you should change from the alias code above is the disk #.

 

#4:  Add the following XML after the </devices> tag but before the </domain> tag:

 

  <qemu:commandline>

    <qemu:arg value='-set'/>

    <qemu:arg value='device.virtio-disk4.x-data-plane=on'/>

  </qemu:commandline>

 

That's all there is to it!

 

Please report back if you test this and let me know if you see an improvement in your virtual machines!!

 

[jude]

What is the benefit of having the disk image in raw format and what is required to convert it from QCOW2 to RAW?

[jonp]

What is the benefit of having the disk image in raw format and what is required to convert it from QCOW2 to RAW?

 

QCOW adds additional operations in an effort to support thin provisioning and snapshots, which can have an impact on performance in general (more IO related I think than bulk transfer write speed).  RAW images do not support these functions, but I'm not necessarily sold on either of those benefits for every use-case that's out there.  Example:  if you're just creating a gaming VM, performance trumps all other aspects.

 

To convert from QCOW to RAW is very easy, but the command may take some time to perform depending on the size of the virtual disk you created.  The syntax for the command is as follows:

 

qemu-img convert image_name.qcow2 image_name.raw

 

Replace image_name obviously with the name of yours.

[archedraft]

(the only thing you should change from the alias code above is the disk #.

 

How do I determine the disk number of my SSD drive? I tired 0-6 but the alias tag keeps disappearing...

[jonp]

(the only thing you should change from the alias code above is the disk #.

 

How do I determine the disk number of my SSD drive? I tired 0-6 but the alias tag keeps disappearing...

 

With your VM running, drop to command line and type the following command:

 

virsh qemu-monitor-command --hmp YOURVMNAME 'info qtree' | less

 

Start scrolling and you'll find a section like this:

 

dev: virtio-blk-pci, id "virtio-disk4"

 

That tells you right there that you have a block device with the name virtio-disk4.

[archedraft]

Sweet OK so mine was disk0 but when you save the XML it disappears. Once you start the VM the alias tag shows up correctly.

 

My Windows VM is up and running! I'll report back if I notice any improvments.

[jonp]

Sweet OK so mine was disk0 but when you save the XML it disappears. Once you start the VM the alias tag shows up correctly.

 

My Windows VM is up and running! I'll report back if I notice any improvments.

You can also write that alias tag directly into the xml before starting the VM. No problem.

[dlandon]

Reading this post gave me some inspiration regarding a Windows 7 Media Center VM (qcow2 image) that I have been running for some time that has been having issues.  I have an InfiniTV ETH TV tuner with an Xbox 360 and a Ceton Echo extender for watching TV and movies.  There are intermittent pauses and pixelation when watching TV.  I converted the VM qcow to a raw image and it has solved the problems.

 

I guess I didn't realize that the qcow2 image had that much overhead.  After Jonp explained the issues, it makes sense.

[jude]

Converted Win8.1 VM from qcow2 to raw format and then made the changes that are referenced in the first post.

 

Definitely seems quicker and appears to have reduced some of the issues I am having with occasional crackling sound.

 

Boot times are also much faster.

 

Nice tweak.

 

[jonp]

Converted Win8.1 VM from qcow2 to raw format and then made the changes that are referenced in the first post.

 

Definitely seems quicker and appears to have reduced some of the issues I am having with occasional crackling sound.

 

Boot times are also much faster.

 

Nice tweak.

Awesome feedback. Thank you.

[jude]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

[dlandon]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

This is how I've done it with a Debian VM raw image.  Probably works the same for Windows.

 

### How to expand the size (10 GB) of a VM disk image.
dd if=/dev/zero bs=1G count=10 >> /mnt/user/domains/DebianVM/debian.img

 

Set count=30 for 30GB.

[GHunter]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

Look at the drive under disk management and see if there is unallocated space. You might need to resize the partition in Windows.

 

Gary

[jude]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

Look at the drive under disk management and see if there is unallocated space. You might need to resize the partition in Windows.

 

Gary

 

Yeah, that is what I am used to with qcow2 images but it didn't work that way this time with the raw image.

[jude]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

This is how I've done it with a Debian VM raw image.  Probably works the same for Windows.

 

### How to expand the size (10 GB) of a VM disk image.
dd if=/dev/zero bs=1G count=10 >> /mnt/user/domains/DebianVM/debian.img

 

Set count=30 for 30GB.

 

just to be clear. I would log into the server via terminal/putty. Go to the folder that the raw image resides in and then enter the following code?

 

dd if=/dev/zero bs=1G count=30

 

Is that correct? Does it need to be done through Virsh?

[jude]

I just checked the VM Management main tab and it doesn't show that the disk size was increased even though the qemu-img command appeared to be succesful.

[dlandon]

A little off topic but I just tried to resize the raw win8.1 image that I am playing around with right now.

 

code used

 

qemu-img resize Win8.1ProN.raw +30G

 

response from system was .. image resized

 

After starting the VM I went to Disk/Partition Management where in the past with qcow2 images the disk would immediately show the additional unallocated disk space.

 

For some reason It does not see an increase in the size of the disk.

 

Is this a known issue with raw disk images and is there a work around?

 

Thanks

 

This is how I've done it with a Debian VM raw image.  Probably works the same for Windows.

 

### How to expand the size (10 GB) of a VM disk image.
dd if=/dev/zero bs=1G count=10 >> /mnt/user/domains/DebianVM/debian.img

 

Set count=30 for 30GB.

 

just to be clear. I would log into the server via terminal/putty. Go to the folder that the raw image resides in and then enter the following code?

 

dd if=/dev/zero bs=1G count=30

 

Is that correct? Does it need to be done through Virsh?

 

Do it on the command line (Putty).

 

### How to expand the size (30 GB) of a VM disk image.
dd if=/dev/zero bs=1G count=30 >> Win8.1ProN.raw

[jude]

That worked great!

 

output at command line (adding 10G)

 

root@Tower:/mnt/disk/vmdisk/Image Media# dd if=/dev/zero bs=1G count=10 >> Win8.1ProN.raw
10+0 records in
10+0 records out
10737418240 bytes (11 GB) copied, 32.2871 s, 333 MB/s

 

successfully extended volume in windows.

 

Thanks

 

[archedraft]

Converted Win8.1 VM from qcow2 to raw format and then made the changes that are referenced in the first post.

 

Definitely seems quicker and appears to have reduced some of the issues I am having with occasional crackling sound.

 

Boot times are also much faster.

 

Nice tweak.

Awesome feedback. Thank you.

 

Boot time is noticeably faster. I cannot really say that once it boots it is any faster because it's already really snappy but I would venture to guess that it is faster.

[reluctantflux]

Worked great on my Gaming VM on Virtio bus.  I could not get it to work on my HTPC with Sata bus.  I figured that'd be the case, but figured I'd post so others knew.

 

ETA: I successfully converted my Sata bus vm over to Virtio!

[jonp]

This is great feedback guys.  Have any of you had any issues shutting down your VMs after doing this?

[archedraft]

This is great feedback guys.  Have any of you had any issues shutting down your VMs after doing this?

 

As in issues with the web gui KVM tab shutdown or just regular Windows start shutdown?

[jonp]

This is great feedback guys.  Have any of you had any issues shutting down your VMs after doing this?

 

As in issues with the web gui KVM tab shutdown or just regular Windows start shutdown?

 

Either.  My reason for asking is that I'm currently testing pass through of an SSD as a block device for my desktop OS, but use vdisks in a cache pool for my games / applications.  I am using this setting on the block device and when I go to shutdown the VM, the shutdown seems to hang after the OS is stopped, forcing me to "force shutdown."  I think this is because this tweak isn't meant for block devices, but rather, vdisks only.  Not sure, but curious if anyone has had this issue with vdisks yet.

[jude]

This is great feedback guys.  Have any of you had any issues shutting down your VMs after doing this?

 

As in issues with the web gui KVM tab shutdown or just regular Windows start shutdown?

 

Either.  My reason for asking is that I'm currently testing pass through of an SSD as a block device for my desktop OS, but use vdisks in a cache pool for my games / applications.  I am using this setting on the block device and when I go to shutdown the VM, the shutdown seems to hang after the OS is stopped, forcing me to "force shutdown."  I think this is because this tweak isn't meant for block devices, but rather, vdisks only.  Not sure, but curious if anyone has had this issue with vdisks yet.

 

I am having this issue with a vdisk on a separate ssd that is SNAP mounted using xfs file system. Shutdown from the VM Manager does not complete. If I shutdown from the VM Win8.1 the machine appears to shutdown but the VM Manager says it still running as does virsh: list. Once again I have to use the destroy option to close it right down.

This only happened once I converted the VM to raw and made the qemu adjustments.

 

 

[archedraft]

This is great feedback guys.  Have any of you had any issues shutting down your VMs after doing this?

 

As in issues with the web gui KVM tab shutdown or just regular Windows start shutdown?

 

Either.  My reason for asking is that I'm currently testing pass through of an SSD as a block device for my desktop OS, but use vdisks in a cache pool for my games / applications.  I am using this setting on the block device and when I go to shutdown the VM, the shutdown seems to hang after the OS is stopped, forcing me to "force shutdown."  I think this is because this tweak isn't meant for block devices, but rather, vdisks only.  Not sure, but curious if anyone has had this issue with vdisks yet.

 

I am having this issue with a vdisk on a separate ssd that is SNAP mounted using xfs file system. Shutdown from the VM Manager does not complete. If I shutdown from the VM Win8.1 the machine appears to shutdown but the VM Manager says it still running as does virsh: list. Once again I have to use the destroy option to close it right down.

This only happened once I converted the VM to raw and made the qemu adjustments.

 

Yes, I am also having the same results. My VM Disk is mounted through my GO file using EXT4 file system.  If I shutdown from the VM Win8.1 the machine appears to shutdown but the VM Manager says it still running as does virsh: list. Once again I have to use the destroy option to close it right down. This only happened once I made the qemu adjustments.

 

I may revert back and get rid of the SSD tuning settings, unless you think there is a way that this can be fixed? I am not in any immediate rush but I do like to shutdown the Win 8 VM and start a Win 7 VM using the same graphics card from time to time.