I created a share to mount persistent data for a docker container on my nvme pool set as prefer. What happens if I change this to “No”?

 

if I haven’t moved it yet what is the expected behavior at this stage when that docker container writes new data to its new share? How does it read data if this was toggles?

 

when parity is complete

when parity isn’t complete

1 hour ago, moogoos said:

I created a share to mount persistent data for a docker container on my nvme pool set as prefer. What happens if I change this to “No”?

 

if I haven’t moved it yet what is the expected behavior at this stage when that docker container writes new data to its new share? How does it read data if this was toggles?

 

when parity is complete

when parity isn’t complete

Parity doesn't have anything to do with file data, and it doesn't effect share behavior.

 

If you change a share from prefer to no, all existing data files will stay on the existing drive, new data files will be written to the parity array to whichever drive meets parameters. Reads happen from whichever drive currently contains the file. If you change the share back to prefer, files that aren't open will be moved to the designated pool when mover runs. Cache no disables the mover for the share, and mover can never move open files.

 

What are you trying to accomplish? If you would have stated an end goal I probably could have given direction on how to accomplish it.

 

 

Hello.

 

I don't have an end goal perse. I'm just trying to understand the behavior and trying to avoid corruption until I understand these toggles before the "mover" runs. Ive never used this type of setup before and am understanding why the name now. lol. I come from a Proxmox and OMV world and standard RAID usages of various types.

 

If data was on the NVMe pool as "prefer" and I change this to "no" without the mover job running and I write data still, I would assume then data is split across the NVMe share and the array? This is the part I'm struggling with because if the mover hasn't run and I change the share behavior then I would assume I would have data in two places now. Then at some scheduled time the mover job will run and will move the old data to the array.

Edited June 11, 20215 yr by moogoos

I don't know what "toggles" are, unless it's "the thing" turning things off and on (like a light switch). Assuming I understand what we're talking about... I would be really careful moving data from "preferred" or "cache only" or "several other things" shares. I'm no expert, but I've learned to; run "Mover" before making changes that involve my cache drive(s), and everything seems to work out just fine.

 

6.

5 minutes ago, 6of6 said:

I don't know what "toggles" are, unless it's "the thing" turning things off and on (like a light switch). Assuming I understand what we're talking about... I would be really careful moving data from "preferred" or "cache only" or "several other things" shares. I'm no expert, but I've learned to; run "Mover" before making changes that involve my cache drive(s), and everything seems to work out just fine.

 

6.

 

Toggles meaning the different cache options. I still dont really understand the behavior, because If I toggle from "prefer" to "no", so I can then run the mover so its moved off the pool to the array, data is still being split among the two shares until the mover is complete. How does an application, like a docker container that has data persisted on a preferred share in a pool like this deal with that?

3 hours ago, moogoos said:

Then at some scheduled time the mover job will run and will move the old data to the array.

it won't be moved if share is set to cache=no, existing data on cache for that share will remain there, any new data written to that share will go to the array.

1 hour ago, JorgeB said:

it won't be moved if share is set to cache=no, existing data on cache for that share will remain there, any new data written to that share will go to the array.

 

So how does that work if the share is for a docker container volume that now needs data from the old pool and the new array data being written? I'm beginning to think I dont want an array at all and would rather just move all my HDDs to a btrfs pool? 

 

This concept of "cache" shares which are really raid volumes that are persistent, and not temporary really. The term cache kinda throws me off.

28 minutes ago, moogoos said:

So how does that work if the share is for a docker container volume that now needs data from the old pool and the new array data being written?

It's not a problem if the docker is using a user shares path (/mnt/user/share) since all share data will be accessible.

8 hours ago, JorgeB said:

It's not a problem if the docker is using a user shares path (/mnt/user/share) since all share data will be accessible.

 

That's interesting. So /mnt/user is a path to the files combined over the two physical disks? Disk 1 and Disk 2 both have a media folder obviously with different assets. How does the system merge these things under a different path into a single resource? How does this magic work?

3 minutes ago, moogoos said:

 

That's interesting. So /mnt/user is a path to the files combined over the two physical disks? Disk 1 and Disk 2 both have a media folder obviously with different assets. How does the system merge these things under a different path into a single resource? How does this magic work?

Yes - they are merged into a single view.    The view will include any files in the same folder located on any array disk and any pool (cache).

 

The details of how this is managed are not really relevant unless you are a programmer - in which case look into Linux Fuse support.

2 minutes ago, itimpi said:

Yes - they are merged into a single view.    The view will include any files in the same folder located on any array disk and any pool (cache).

 

The details of how this is managed are not really relevant unless you are a programmer - in which case look into Linux Fuse support.

 

It's relevent to understand it. You don't need to be a linux programmer to want to know the simple idea of how this works and what technology they're implementing to do it. My personal experience using Fuse, although this was 10 years ago, was not favorable. Im just trying to learn how it manages the data so I don't screw something up myself.

23 minutes ago, moogoos said:

Im just trying to learn how it manages the data so I don't screw something up myself.

Just don't mix /mnt/user paths with /mnt/diskX or /mnt/<poolname> paths when manipulating files manually. Typically I only use /mnt/user paths to consume media or files, and disk paths to manage stuff in the background, but less technical users are encouraged NOT to directly access the disks, just use the user shares and the accompanying settings to allow Unraid to manage the initial and final file locations.

 

Much work has gone into removing the need to directly use disk paths for almost any scenario.

 

One of the ways I manually manipulate the system is to use the /mnt/user/domains stock location with cache : only settings, which means that when I use the wizard to create a VM, it's initially created on the assigned cache pool. However, I can manually move the VM subfolder for VM's I don't use very often from /mnt/cache/domains/NewVM to /mnt/disk1/domains/NewVM and everything still works without changing any VM path definitions. All my VM's show up in /mnt/user/domains, and I control which ones take up fast expensive SSD space, and which are relegated to slow cheap array space.

 

cache : yes and cache : prefer both tell the mover to do something, cache : no and cache : only tell the mover to ignore the share. 

27 minutes ago, jonathanm said:

cache : yes and cache : prefer both tell the mover to do something, cache : no and cache : only tell the mover to ignore the share. 

 

so cache:no means straight to the array and cache:only means use the pool as a persistent store. Spaceinvader video, I think it was, questioned the need for cache:only but to me I see value in it as I dont want the mover to attempt anything on some larger pools is my thinking. 

Edited June 11, 20215 yr by moogoos