Some questions about first array build

[Excypher]

Hi everyone, I have a couple questions about the setup before going ahead with my unraid build, just to clear up a couple potential concerns.

 

Currently I have 4x4TB drives in raid 1 as my storage, I have about 5TB used of the 7.25TB usable.

 

I am planning on putting those 4x4tb, plus an additional 4TB into the unraid array, In addition I have 2x 1TB SSD's I plan to Mirror as a cache volume. 

 

Obviously since I'm going to have to store the data in the mean time, I have 2x new 18TB drives, one of which I'm going to use to copy the array to before building it, and the other is going to be the parity drive, once the array is built I'm going to add the other 18TB that I've used for temporary storage to the array. 

 

So, My question is: since the cache is going to be 1TB, and I'll have 5TB to copy to the array once completed, what steps need I take, if any, to ensure this transfer onto the array goes cleanly?

 

As my understanding of the cache setup is that you schedule a worker task to move the data from the cache onto the actual redundant storage overnight during normal use, does this mean I'm going to have to copy 1TB ish worth, run the worker, rinse and repeat or is there a way to just transfer the whole shebang without interruptions and just leave it to it overnight?

 

My other question is a general usability question, My understanding of how unraid works is it only spins up the drives required if you access them, basically, does this apply to metadata too? As in, if I have the share mapped as a network drive, if I go to click on it to browse the file structure, but not necessarily access files, am I going to expect some spin up time for the drives when this happens if I have them configured to spin down? Or is it smart enough to save that file structure elsewhere? 

 

Thanks all!

[trurl]

Usual recommendation is to not cache initial data load. To make those writes faster, don't install parity until initial data load is complete, then build parity at the end.

 

Keep the source data disks intact with all their contents until parity build complete. And you might keep them as backup if they aren't already backed up somewhere. You will need a backup plan, of course. Parity is not a substitute for backups.

[Excypher]

So in effect, the best bet would be to copy from the existing drives onto one of the 18TB drives that is effectively alone in the share, then build the parity, then add the original source disks, then add the cache?

[trurl]

You can add cache without using it for the initial data load. Each User Share has Settings which control whether and how it uses cache.

 

In any case, you should definitely have cache going before you enable Docker or VM Manager so those User Shares get created on cache.

 

https://wiki.unraid.net/Manual/Shares#Default_Shares

 

Ideally, appdata, domains, system shares should be on fast pool (cache) so Docker/VM performance isn't impacted by slower array, and so array disks can spin down since these files are always open.

 

If these aren't created on cache to begin with, it's a little more trouble to get them moved since nothing can move open files.

 

 

[Excypher]

Gotcha so:

1. set up array with 1x 18TB disk and the SSD cache
2. load data from old array onto this share
3. enable cache
4. add parity
5. add remaining disks?

The machine is going to exist solely as a network share... so unless I'm missing something, I shouldn't need Docker or VM's 

 

[trurl]

15 hours ago, Excypher said:

load data from old array onto this share

  

Not clear you understand the difference between a disk and a user share (click the link). I recommend not sharing disks on the network.

 

[Excypher]

On 12/27/2022 at 3:53 PM, trurl said:

  

Not clear you understand the difference between a disk and a user share (click the link). I recommend not sharing disks on the network.

 

Would you clarify your statement here please?

 

My understanding, as loose as it is, is that a share is effectively unraids method to map what is recognised by other systems as disk to a physical space on the disks themselves?

 

IE this share equals this space on these disks, and you then map that drive onto the user systems (in this case windows, so mapping a network drive)

 

My plan is to build the unraid server on a new machine, and keep my existing array in an old machine temporarily, to allow the data to be transferred onto the unraid pool, then add the old array disks to the new unraid pool once this is populated.

 

Or is this a definition thing, where a share or network share is different to a user share?

 

Thanks!

 

 

[JonathanM]

11 minutes ago, Excypher said:

Would you clarify your statement here please?

Did you read the link?

 

What exactly needs more explanation? It would be helpful to know what is unclear in the manual so it can be better explained.

[Excypher]

8 minutes ago, JonathanM said:

Did you read the link?

 

What exactly needs more explanation? It would be helpful to know what is unclear in the manual so it can be better explained.

 

OK I see the confusion now.

 

I thought he was referring to the act of sharing the space at all over the network was the issue, rather then the definition of disk shares VS user shares.

 

I gave the link a read (albeit skimming a lot of what I considered to be irrelivent to this discussion, IE the allocation method sections), and honestly, I couldn't really make much sense of the distinction until I just googled 'unraid difference between user and disk shares' where the following statement finally gave clarity:

 

Quote

An Unraid “User” share is a volume backed by Unraid's FUSE filesystem, while a “Disk” share is a volume directly backed by the disk's native filesystem.

 

The article itself even says:

 

Quote

It is sometimes important to realize that these are two different views of the same underlying file system. Every file/folder that appears under a User Share will also appear under the Disk Share for the physical drive that is storing the file/folder.

 

Which doesn't suggest there's much of a concern or reason to use either.

 

So I guess if you are looking for feedback on that article, it doesn't really give a clear definition between disk and user shares that at least, made sense to me. 

 

But to clarify the original concern I quoted, no I do not intend on using disk shares.

[trurl]

The additional material you quote doesn't really seem to clarify anything. Still not sure you get it since you think the wiki link was irrelevant.

 

The whole point of user shares is they allow folders to span disks. Did you understand that most important point when you looked at the wiki?

 

A disk is just a disk, a user share can be on multiple disks.

 

If you don't use user shares, then folders can't span disks.

 

If you also share disks over the network, then it is easier to get into trouble mixing disks and user shares. See here for an explanation of what can happen (the surprising results):

 

 

 

[Excypher]

1 hour ago, trurl said:

The whole point of user shares is they allow folders to span disks. Did you understand that most important point when you looked at the wiki?

 

No, that particular page of the wiki did not lead me to that conclusion, it was the exterior summary that pointed out user shares allow unraids management that make it make sense to me.

You say: not sure you know the difference between these two shares, here read this, article says, there's two types of shares, here's what this one does, and here's what the other one is called, but doesn't explain why it's different directly to a disk share.

 

To be fair, that's maybe not the purpose of that particular page you linked but that's the source of my confusion.

 

Anyways, all is clear now so, thanks for the help

[trurl]

Early in the text at that link

Quote

What they do is provide an aggregated view of all top level folders of the same name across the cache and the array drives.

 

But it's probably clearer in the Overview.

https://wiki.unraid.net/Manual/Overview#User_Shares

[Excypher]

50 minutes ago, trurl said:

Early in the text at that link

 

But it's probably clearer in the Overview.

https://wiki.unraid.net/Manual/Overview#User_Shares

 

 

Again, what these articles are really good at showing is what user shares are and how they operate, they don't show the difference between them and disk shares. 

 

When you say 'disk share' within the context of unraid it sounds much more like it's a method or definition within the unraid ecosystem, it leads you to think it's something special or another method of writing to an array rather than you using what might be the common sense definition, that being simply the disk itself.

 

In this case, honestly, it was simply a definition problem, I said disk as In my application when I do the initial data load the array would consist of a single disk. I'm not at that point talking about an unraid definition (at that point I didn't even know disk shares were a thing!)

 

When it came to setting up the array I would have created a user share because that's the default behaviour and what all the 'getting stated' guides indicate you should do, when you said disk share Initially and expressed concern my immediate thought was you were advising against any kind of sharing on the network at all, hence the confusion.

[trurl]

I usually use the word "disk" when working with a disk directly on the server. It is only when a disk is shared on the network that it becomes a disk share.

 

But the "surprise" behavior when mixing them with user shares as mentioned in that other post I linked applies whether you are working with disks on the server, or disk shares on the network.

[Excypher]

3 hours ago, trurl said:

I usually use the word "disk" when working with a disk directly on the server. It is only when a disk is shared on the network that it becomes a disk share.

 

But the "surprise" behavior when mixing them with user shares as mentioned in that other post I linked applies whether you are working with disks on the server, or disk shares on the network.

 

Yes I see why you felt the need to clarify, seems there's potential for some problems if you mix the two by the looks of it.

 

May I ask whilst I have your attention, my plan involves putting my entire storage onto a single disk temporarily, is it easy enough to have it distributed once all the disks are in place and the full array is built? Is there any appropriate reading for that operation?

 

Thanks!

 

 

[trurl]

2 hours ago, Excypher said:

have it distributed

unBALANCE plugin