Adventures in restoring VMs


Recommended Posts

(I posted earlier in the Docker Engine forum, but I don't think that's the right place, as this is more about VMs.)

 

I'm on 6.5.3 and have 4 VMs, which were previously all running OK. I was fiddling around trying to install virt-manager on my LinuxMint VM and when I rebooted the unRAID server I found that I had a blank VM tab (no Add VM buttons or anything else). Based on looking at many forum threads, I checked to see that all the paths in VM Settings were correct (including Libvirt storage location specifying the file, not just the folder). I don't think I had changed anything, so it didn't seem like an incorrect path could be the cause of the blank tab.

 

But then I noticed that the Libvirt storage location was /mnt/cache/libvrt.img, which it has been all along (I checked previous diagnostics files). On changing this to /mnt/user/libvirt.img (note "user" instead of "cache" and spelling of libvirt), I managed to get the VM tab to show the Add VM and other buttons, although no VMs were listed.

 

As an experiment, I tried to create a new Windows 10 VM, just using the default template. What surprised me was that my existing Win10 VM came up, even though I had made some minor changes to the XML file for that existing VM. Fortunately I had a backup of that customised XML, so I shut down the "new" VM and then overwrote the XML entries with the backed up ones. On restarting I got an error message: "operation failed: unable to find any master var store for loader: /usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd". Unfortunately I can't recall exactly what I did after that, but I definitely didn't edit the XML file since I have no idea what most of the entries do. In the end I just restarted the VM (perhaps after rebooting the server) and now my Win 10 VM is working just as before.

 

Now I'm going to try restoring the other VMs. But if there's a better way of restoring the VMs, I'd appreciate some pointers. (Also any suggestions as to why the VMs went AWOL in the first place - so that I can avoid this happening again.)

 

 

 

tower-diagnostics-20190110-2122.zip

Link to comment

I had these kinds of issues when I accidentally filled up the cache drive...  KVM/QEMU doesn't handle the cache drive filling up very well...  I would try freeing up some space on it and see what happens...  Make sure to change the libvirt back if it fixes it...

 

A command that worked when I had this situation for my system:

fallocate -d <filenamegoeshere>

Will "Dig" all the zeros out of large image files that are taking up too much space (like VM image files in the domain folders)...  Just make backups, and be careful, you can accidentally overwrite files typing that command wrong...

 

There are other more permanent solutions to this if that does help, look into changing the VM's image files to use the SCSI driver, and setting the 

discard='unmap'

command for them in the VM configuration... But that can be a bit of a project on an already setup machine...

Link to comment

Thanks for the suggestion, but I think it's unlikely to be a full cache drive problem. I have around 200GB free on the cache drives and very rarely go below 100GB free.

 

Fortunately I have backups of the VM image files and the XMLs and when I have a bit of time I'll work on restoring the VMs. I didn't have a libvirt.img backup, but I'll make sure I back that up in future. Still a bit confused about this file, as it seems to be recreated when booting up.

Link to comment
9 hours ago, Warrentheo said:

Will "Dig" all the zeros out of large image files that are taking up too much space (like VM image files in the domain folders)... 

I'm not sure, but I think it can only deal with trailing zeros, the upshot being that a fully defragmented vdisk image file will get much smaller than a "busy" image that has the same amount of used space but is heavily fragmented.

Link to comment

I managed to get my VMs back relatively easily. It seems that the error message "operation failed: unable to find any master var store for loader: /usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi.fd" refers to a missing nvram file in /etc/libvirt/qemu/nvram. The relevant file (which I had fortunately backed up) has a string of charcters in the front and then '_VARS-pure-efi.fd'. This string of numbers corresponds to the UUID in the VM's XML file.

 

(A brief aside: one thing that I got stuck on for a while happened while I was using Midnight Commander (mc) from a terminal on my Windows PC. I wanted to use mc to copy the .fd file to /etc/libvirt/qemu/nvram. In mc the nvram files had an asterisk in front of the name. I've learned to be very scared of unusual things in Linux (so basically most of Linux for me as a Windows user).  A lot of Googling only described filenames with asterisks after their names and mc discussions only mentioned colours of filenames. Worryingly, the .fd files in /etc/libvirt/qemu/nvram didn't have this asterisk. I took the bold step of just copying the file, and everything seemed OK. After a bit more guesswork it turns out that the asterisk represents an executable file, which is a bit weird for what I thought an nvram file would be. My guess is that this is a result of the backing up of the nvram files which somehow added the executable flags.)

  • Thanks 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.