VMs not surviving reboot - V6.10.3

[skylineboy]

Last night around 9pm GMT I had some erratic behaviour on my server, that started with Unraid rebooting itself, i'm unclear at this point if it was triggered via a series of mini powercuts that the UPS couldnt sustain. The after effect was that the VM tab was no longer visible, and two of my cache pools (4 drives) had become "Unmountable - No File System". 

 

I was unable to restore the libvert.img and my backup's we're misconfigured, so I manually recreated the VM XML templates and successfully rebooted the VM's (Linux and Windows). 

 

Following a reboot, the VM's had disappeared again, at this point I noticed the unmountable drives, one pool "cache_img" containing the docker and VM Images and Disks, the other "cache_files". I still had full read access to these and backed them up, followed by repairing them via btrfs repair. This found no errors but on restarting the array the drives mounted succesfully. 

 

Around 4/5am this morning the server appears to have rebooted again for some reason and again the VM's were gone. I've checked the location of the libvert.img and no file exists. 

 

I've tried disabling and re-enabling the VM's to recreate the image but still to no success. As a temp measure i've rebuilt the VM template to get the VM's up and running for now, but expect they wont survive another reboot and i'm unclear why the system is rebooting. 

 

Any help appreciated. 

 

Diagnostics enclosed.  

 

Thanks 

 

 

tower-diagnostics-20221025-1110.zip

[JorgeB]

First check that you don't have multiple libvirt images, post the output of:

 

find /mnt -name libvirt.img

 

[skylineboy]

Two returns.

 

/mnt/cache/system/libvirt/new/libvirt.img   - This location is when I tried to create a new .img in a new location this morning

/mnt/cache/system/libvirt/libvirt.img - Original location 

 

 

I can see no ".img" file in either location though

Edited October 25, 2022 by skylineboy

[JorgeB]

If you currently have a VM working reboot and post new diags after array start.

[skylineboy]

New diagnostics, different behaviour on this reboot. 

 

VM Tab is showing this time but getting "LibVert Service failed to start", assuming due to two images been listed as above

 

Thanks

tower-diagnostics-20221025-1334.zip

[JorgeB]

Not seeing any attempt to start the VM service, disable then re-enable the service and post new diags.

[skylineboy]

I noticed the ISO path in the configuration was incorrect, I must have accidently changed that earlier, corrected that. VM tab now present but no VM's listed. 

 

Stopped and Restarted the services as advised and diags attached. 

tower-diagnostics-20221025-1733.zip

[JorgeB]

Create a new VM using that libvir.img, can be a test VM with a new vdisk, reboot and post new diags.

[skylineboy]

Created VM - Disabled VM Manager and Then re-enabled (VM still present in Tab) - File ending - 1800

 

Rebooted Unraid - VM tab present - No VM's listed - File ending - 1807

tower-diagnostics-20221025-1800.zip tower-diagnostics-20221025-1807.zip

[JorgeB]

A new libvirt.img file was created on disk7, system share is set to use pool "cache" but there's no pool named "cache", change that to the pool you want to use and try again.

[skylineboy]

Success! Not sure how it'd defaulted to "Cache" - i've not had a singular cache drive for a long time. I've done a few reboots and all appears stable once more. Huge thanks!