After array rebuild. VM's no longer working


Recommended Posts

After having to do a swap-enable on my array, my disk's have all been rebuilt and my data looks fine.  However when I go to turn on any of my VM's I get the following error

Execution Error
cannot write data to file '/var/run/libvirt/qemu/openelec.xml.new': No space left on device

This is happening with all of my vm's, not just Openelec.  Everything in docker is working fine  The Cache drive where I store the VM's still has approximately 100GB of free space.

 

How do I fix this?

 

Thanks for any help

Link to comment

This issue has reappeared.  Although now it is only effecting one of my VM's.  My other 2 are working properly.  In fact this somehow turned off the VM while it was in operation.  I'm totally at a loss as to where to start with trying to resolve this issue.

 

Thanks

 

Can't help you, but you might want to edit your thread again so it isn't marked as solved..

Link to comment

I suspect your flash device is nearing the end of its life. The configuration for VMs is stored in a loopback image on the flash device.  Typically a file can get renamed to .new when a system crash or other major event occurs while trying to add a new or update an existing VM.  How old is the USB flash device you are using?

Link to comment

The steps would be to format a new USB flash device (preferably a quality brand as we recommend on our website) and then copy the files over from the old flash to the new, then running make bootable on the new flash stick.  You will then need to boot up and initiate a registration key transfer which you can do through the webgui.  This will transfer your key to the new flash device.

Link to comment

Changed out my USB key for another one that I had, it is even newer than the one that was replaced.  Am still having the same issue.  It seems to take somewhere between 12 and 20 hours before this issue arises.  Rebooting temporarily does fix it.

 

Ok, on your USB flash device, go to config -> plugins -> dynamix.kvm.manager and locate the domain.img file.  Please upload that file to somewhere you can paste a link here (either dropbox, google drive, or some other equivalent).  The file should be about 3MB in size.

Link to comment

 

Ok, took a look at the file here and it seems fine.  Shouldn't be having this "no free space" issue that it's reporting.  I'm honestly a little perplexed.  Will need to ask Tom and Eric to assist with this sometime next week, but in the meantime, here's what you could try.

 

1 - With the array started (and the VMs tab visible), click on each vm and select Edit XML.  From there, copy the entire contents of the XML and store it in a notepad somewhere for safe keeping temporarily.

2 - Goto Settings -> VM Manager and set Enable VMs to No, then click Apply.

3 - On your flash device, delete the domain.img file I directed you to upload previously.

4 - Go back to Settings -> VM Manager and set Enable VMs to Yes, then click Apply.

5 - Go to the VMs tab.  It will be empty (your VMs will not show up there; this is expected).

6 - Click Add VM and select XML Expert from the template drop down.

7 - Paste the XML for one of your VMs there (that you copied to a notepad earlier).

8 - Click Add VM.

9 - Repeat the above process for each VM you wish to re-add.

 

Let me know if the issue occurs again after you've followed this process.

Link to comment

Attached are my diagnostics.

 

Motherboard Tyan S5512

Proc Intel Xeon 1230

Ram 8GB Kingston ECC

1 x 4TB Seagate HDD (Parity)

2 x 3GB HDD (One Seagate, 1 Western Digital) (Data)

1 x Kingston 16GB USB3 Flash drive

1 x Asus GT730 graphics card (for Openelec VM)

 

The mobo, proc, Ram, and hard drives have been in use for over a year and a half now without any observed hardware issues.

galactica-diagnostics-20150919-1700.zip

Link to comment

Attached are my diagnostics.

 

Motherboard Tyan S5512

Proc Intel Xeon 1230

Ram 8GB Kingston ECC

1 x 4TB Seagate HDD (Parity)

2 x 3GB HDD (One Seagate, 1 Western Digital) (Data)

1 x Kingston 16GB USB3 Flash drive

1 x Asus GT730 graphics card (for Openelec VM)

 

The mobo, proc, Ram, and hard drives have been in use for over a year and a half now without any observed hardware issues.

 

Ok, I'd like you to try recreating the issue when booted into safe mode.  This will disable all plugins on the system.  Something is going wrong with the domain.img file on the flash device and I'm wondering if the unassigned devices plugin may have something to do with it.  After booting in safe mode, if you can recreate the issue, please grab diagnostics again (while in safe mode) and upload so I can research this further...

Link to comment

I uninstalled the unassigned devices plugin as it wasn't being used at the moment.  Since uninstalling it, my system has run for 24 hours without issue, not in safe mode, but in regular.  I have even been able to stop and restart VM's, which I wasn't able to before.  I believe this issue may be resolved now, however I'm curious why this plugin would cause these issues.  The version installed was the most current one available, and I had not had any issues with it, until I had to remove the one drive that I was using it for.  I'm not sure why physically removing that drive  though would cause this issue.  I'm just curious about the correlation at this point.

 

Thank you for all the help jonp!

Link to comment

I uninstalled the unassigned devices plugin as it wasn't being used at the moment.  Since uninstalling it, my system has run for 24 hours without issue, not in safe mode, but in regular.  I have even been able to stop and restart VM's, which I wasn't able to before.  I believe this issue may be resolved now, however I'm curious why this plugin would cause these issues.  The version installed was the most current one available, and I had not had any issues with it, until I had to remove the one drive that I was using it for.  I'm not sure why physically removing that drive  though would cause this issue.  I'm just curious about the correlation at this point.

 

Thank you for all the help jonp!

 

Please report this finding to the thread on unassigned devices.  I think folks need to be aware that this plugin could be causing issues.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.