rebooted now vm's not showing

[fagostini]

I recently rebooted unraid 6.8.2 and when it came back up the vm's and dockers are no longer showing up. I have checked the domains share and i still have vdisk.img for both vm's but under the VMS tab nothing... any help would be great

[trurl]

Go to Tools-diagnostics and attach the complete Diagnostics zip file to your NEXT post 

[fagostini]

gargantua-diagnostics-20200224-1621.zipok here it is

[fagostini]

ok so i just made a new vm and pointed it to the old vm vdisk and that seems to work fine but, i dont know how to do that for the docker containers

 

[trurl]

Why are your appdata, domains, and system shares set to cache-yes? They have files on the array instead of all on cache where they belong.

[trurl]

Go to Settings - Docker and disable Docker service. Go to Settings - VM Manager and disable VM service.

 

Set appdata, domains, and system shares to cache-prefer (this is their default so you did this to yourself).

 

Run Mover. When it completes post new diagnostics.

[fagostini]

ok so im a litter curious why i would want my appdata to all be in cache, my cache is only 500g and my appdata folder has several TB of data in it, i only wanted to use the cache to dump files onto the machine and then move them into the array periodically. if i switch to prefer it will fill my cache all the time

[trurl]

Why would your appdata have several TB? Do you have your container applications configured to write downloads into appdata or something?

 

appdata should only contain the "working storage" of your containers. Downloads and other things the applications write should go to other shares.

 

appdata, domains, and system shares should be on cache because cache will give your applications and VMs better performance if their working storage is not impacted by the slower parity writes. And they should be on cache so they won't keep array disks spinning, since the files in these shares are in constant use.

 

Those shares are cache-prefer by default for those reasons.

 

I don't know what you have done but it is not the way people normally do it.

 

 

[fagostini]

i think i did make a big oof, im going to make a new share to transfer all the vm data into and change the paths and keep the move the other shares to cache-prefer, and use the mass storage to cache-yes for faster write times and this you are saying should help the vm's / dockers reappear ?

[trurl]

2 minutes ago, fagostini said:

this you are saying should help the vm's / dockers reappear ?

No it won't fix your problem, but the fact that you set things up this way may have something to do with the cause of your problems.

 

Which dockers do you normally run?

 

Apps - Previous Apps will let you reinstall your dockers just as they were, but just as they were doesn't seem like the correct way.

[fagostini]

the dockers are less of a concern for me right now, so if i move all the excess data from appdata to a new share and change the share settings for the main ones to prefer the vm should stay after a reboot at least right?

 

originally i had plex docker and it saved the data to appdata so i changed the cache to yes so it wasn't full all the time, everything worked fine for months that way even after reboots. i changed a cpu pinning for the vms and when it came back up they disappeared.

 

thanks for all the info, when i got unraid it was difficult to get best practices from online and now it is biting me

 

so you think migrating the bulk storage of plex to a new share and changing the cache settings wont really solve the issues?

[trurl]

Just now, fagostini said:

the dockers are less of a concern for me right now, so if i move all the excess data from appdata to a new share and change the share settings for the main ones to prefer the vm should stay after a reboot at least right?

appdata doesn't have anything to do with VMs normally so I don't know what you have done there.

 

2 minutes ago, fagostini said:

originally i had plex docker and it saved the data to appdata so i changed the cache to yes so it wasn't full all the time, everything worked fine for months that way even after reboots. i changed a cpu pinning for the vms and when it came back up they disappeared.

2 minutes ago, fagostini said:

so you think migrating the bulk storage of plex to a new share and changing the cache settings wont really solve the issues?

The only things plex stores are its library, which can get somewhat large but nothing like what you have. Transcodes are also stored as a subdirectory of that library by default but that can be changed.

 

Other than that plex doesn't really write anything, except possibly DVR, which should be on another share. And any media you want plex to play should be on other shares and not appdata.

 

 

Looking again at your diagnostics it seems you don't actually have any shares except appdata, domains, isos and system. These shares are not intended for any of your media, downloads, or any other files you may want to write to your server.

 

appdata is strictly for the working storage of your containers. By working storage, I mean anything it might need to keep track of what it is doing. Plex, for example, has a database that it keeps track of where all of your media is stored and other information about that media. The media itself does not belong in appdata.

 

domains is strictly for your VMs. This is where the OS for each of your VMs goes. The VMs will typically access storage from other shares on your Unraid server.

 

isos is where the installation files for your VMs go. This will be where you save downloaded distributions of whatever OS you want to install as a VM.

 

system is where your docker image and libvirt images go. The docker image actually contains the downloaded executable code for each of your docker containers.

 

You should not be storing anything in these shares. They are for the system to use to manage your dockers and VMs.

 

Sounds like you need to rework almost everything. Do you have backups that would allow you to just start over?

 

[fagostini]

i don't think i need to start over, i got the vm running in the beginning and i am already transferring everything out of the appdata folder, outside of the plex media nothing else should be an issue. originally the pex was a docker and i thought i had to store all its files together in the appdata section. major noob here

 

i do want to say thanks for explaining all the to me, i feel like an idiot for not figuring that out when i started.

Edited February 25, 2020 by fagostini

[trurl]

You should delete your appdata and at least start over with that. I know your plex library will all be wrong after you move the media. I don't know what other dockers you have but they will likely not work correctly either after you change their mappings.

[fagostini]

will mover not just move from array to cache ? 

[trurl]

Mover moves cache-prefer shares from array to cache. Mover moves cache-yes shares from cache to array. Mover ignores cache-no and cache-only shares.

 

Mover cannot move open files, so it won't be able to move the system share and possibly the others unless you disable the Docker and VM Services in Settings.

[itimpi]

Just now, fagostini said:

will mover not just move from array to cache ? 

No.   Mover transfers files from cache to array for shares that have Use Cache=Yes.     It will transfer files from array to cache for shares that have Use Cache=Prefer.

[fagostini]

okay so it took all night but mover has finally filled the cache with 200g of the appdata domain and system files, it all looks mirrored onto the array as well, im guessing this is the setup that i want?

[trurl]

1 hour ago, fagostini said:

it all looks mirrored onto the array as well

This is likely just a misunderstanding, but could you show us what your mean here?

[fagostini]

it shows all the files in /mnt/users/appdata and /mnt/cache/appdata.

[JonathanM]

50 minutes ago, fagostini said:

it shows all the files in /mnt/users/appdata and /mnt/cache/appdata.

That's the same location. /mnt/users includes all of /mnt/diskX and /mnt/cache

 

2 views of exactly the same files.

[fagostini]

so does it move the data back to array when the system powers off?

[trurl]

2 hours ago, fagostini said:

so does it move the data back to array when the system powers off?

I'll leave out the part of your question about "when the system powers off" because I can't make any sense of it.

 

There is a process called Mover that runs on a schedule. Daily in the middle of the night is the default but you can change that. Mover is intended to run during idle time.

 

Each user share has a Use cache setting which controls how that user share uses cache. Mover uses these settings to decide whether to move the share from cache to array, from array to cache, or to just skip that share.

 

There is Help in the webUI. You can toggle Help for the whole webUI using Help (?) in the main menu. You can also toggle help for a specific setting by clicking on its label.

 

Rather than explaining all the details of how Unraid uses cache, I will give you some links and you can ask more questions after reading them.

 

https://wiki.unraid.net/UnRAID_6/Overview#Cache

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/page/2/#comment-537383