NB. this particular bug is specific to: Version: 6.11.5 

Problem: After restarting Unraid or the Array, some(or all) docker and VM, images are dropped from their respective management panels.

That said, and after a little investigating, I discovered that this behaviour is specific to assigning the system shares(appdata, domains and system), to an nvme drive, prior-to, and following initial Unraid server setup.

After which, and upon creating docker and vm images, a restart of the array or Unraid server, will result in the disappearance of said images from their respective management panels.
 

Edited May 25, 20233 yr by JohnBee

I managed to recreate this 

After re-creating a vm, I then stopped and started the Array and found the VM gone from the manager listing
That said, the .img and relevant data appears to be intact, though I cannot for the likes of me, figure out why the VM keeps disappearing from the list 
 

Please post the diagnostics 

I have since reinstalled, but find the same issue.
Here's the diag file 

 

bertnas-diagnostics-20230524-1211.zip

Would add that this behavior also extends to dockers, as well as reinstallation of my USB /Unraid OS server.
Only thing left, is the drives, which appear to be the culprits in this behavior?

Diags show no devices assigned at array start, is this a new config? It also shows a new docker and libvirt images being created at array start, so it's normal for both Docker container and VMS to be empty, question is what happened to the old ones.

The weird part is, this is a fresh install, and so after initial configuration, I proceeded to create a docker, then stopped and started the array, and the docker disapeared from the manager window.

 

Thing of it is, this was happening in my previous install as well, which prompted me to scrap the server configuration and start over.

 

Really not sure what could cause this, and it's got me stumped.

 

Guess my only recourse, is to nuke the disks and see if that makes a difference, though all of the metrics say the disks are 100%, which is weird.

Going down the syslog, initial docker image was create here, at first array start:

 

May 24 11:05:09 BERTNAS  emhttpd: shcmd (245): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20
May 24 11:05:09 BERTNAS root: Creating new image file: '/mnt/user/system/docker/docker.img' size: 20G

 

You then stopped and re-start the array a couple of times and it used the existing image, e.g.:

 

May 24 11:29:19 BERTNAS  emhttpd: shcmd (515): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20
...
May 24 11:30:31 BERTNAS  emhttpd: shcmd (665): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20

 

Then the last time you started the array a new image was created, this means it didn't exist, suggesting it was deletes or the storage where it existed removed:

 

May 24 12:09:03 BERTNAS  emhttpd: shcmd (958): /usr/local/sbin/mount_image '/mnt/user/system/docker/docker.img' /var/lib/docker 20
May 24 12:09:03 BERTNAS root: Creating new image file: '/mnt/user/system/docker/docker.img' size: 20G

 

It's the same for the libvirt.img.

This seems like very odd behavior.

The question now is; why would Unraid conclude the images or storage deleted following the start and stop of the array?

That said, do you have any tips on how I might close-in on this particular bug?
I would add that this is a test server, and that all of the disks can be deleted, formatted, etc

NB. I was able to reproduce this by rebooting the NAS as well, all VM's and Dockers removed from the tabs
 

 

 

 

Here's another apparent discovery toward this issue.
If I restore the VM, by mounting the image within a newly created shell, the VM will now persist through array restarts, shutdown, reboots etc.

That said, I am no closer to identifying the culprit on this most intriguing behavior 

 

 

9 hours ago, JohnBee said:

why would Unraid conclude the images or storage deleted

For some reason it's not being able to access the old image, so a new one us created, it didn't happen always, first couple of times you stopped the array it used the existing one, so something changed, reboot and post new diags right after array start.

 

 

I made progress on this after reinstalling  Unraid from scratch, and starting with a clean slate.

 

That said, and to the best of my abilities, it would seem that the issue is linked to assigning the docker and vm image files to a separate nvme drive, this in-turn, leading to the system dropping these from their respective management panels.

 

In addition to this, and equally perplexing, is that I can only reproduce this one time, after which, both the dockers and vm's will be retained regardless of whether the array is cycled, or the Unraid is restarted. Which seems rsther odd to say the least.

 

Therefore and with that being said, I would like to cover the recommended method to provision an nvme drive for docker and vm images, so-as to determine whether this can be avoided, as well as whetherthis is a genuine bug.

 

- thanks for your continued participation on this btw. I  do appreciate the help

This is quite strange, don't remember a similar issue, using separate poos is quite common, if it happens again post new diags, ideally clean ones, e.g., like if you can repeat the behavior after a reboot.

13 hours ago, JorgeB said:

This is quite strange, don't remember a similar issue, using separate poos is quite common, if it happens again post new diags, ideally clean ones, e.g., like if you can repeat the behavior after a reboot.

Here is what I have found;

[Scenario 1]
Upon installing Unraid and setting; appdata, domains and system(cache pool settings), to 'nvme', prio-to all other things, the docker and vm images will drop from management panels following a restart of array, or the Unraid server.

[Scenario 2]
Upon installing Unraid, and creating docker and vm images prior-to changing; appdata, domains and system cache, to 'nvme', the dockers and vm's will not drop from their respective management panels following array restarts, or restarting the Unraid server

That said, it would appear that there are discrepancies with setting-up and/or changing system shares(appdata, domains and system) prior-to specific server activities, though I have yet to interrogate this further...

9 hours ago, JohnBee said:

[Scenario 1]
Upon installing Unraid and setting; appdata, domains and system(cache pool settings), to 'nvme', prio-to all other things, the docker and vm images will drop from management panels following a restart of array, or the Unraid server.

Seems odd that if it's a bug it was never reported before, can you post a more detailed step by step of what you are doing? Still using v6.11.5 right?

 

P.S. going to move this to the general support forum since it's not really a KVM issue.

 

 

I would like update this thread and post that I have identified the cause of this behaviour as well as found a suitable solution.

The first thing to note, is that this particular behaviour seems specific to assigning relevant paths and directories to alternative devices - such as nvme's or SSD drives
The second to note is that the Docker and VM services, will continue to point to their original paths until these are either stopped or cycled.

That said, and should this not be done, the respective Docker and VM services can potentially report false information, and as a result of polling the now original paths and folders.
And so, the solution to this, and at this time, appears to that of cycling both the Docker and VM services after the changes to their respective devices are made.

- hope this helps and thanks to JorgeB, for following-up on this

PS. it would seem that the ideal scenario on this, is that of cycling said services following devices changes

5 hours ago, JohnBee said:

he second to note is that the Docker and VM services, will continue to point to their original paths until these are either stopped or cycled.

Not following, the paths can only be changed with the services stopped, how are you changing them?