Number of Drives Limit


Recommended Posts

Hi, would someone be able to clarify what is the limit on the number of drives in an array?  I see the Pro license says Unlimited, but reading some of the forum posts, I also see a 30 drive limits.  Which is correct?  I see this in the manual, "NOTE: The attached device limits do NOT refer to how many devices you can assign to the array or cache. Those limits are imposed by the software, not the license policy."  But I'm not sure what that means.  Eg., is there a lower limit due to "the software" compared to the "unlimited" Pro license?  If there is a lower limit, what is that limit?  In other words, what is the max number of drives I can have in an array if I have a Pro license?

 

Also, is it possible to have two arrays in a single machine?  I have a bunch of older 3 and 4 TB drives I would like to use in one array and a bunch of 8-10 TB drives in a second array.  I'm happy to pay for two keys...my issue is to not have another motherboard/system running a completely separate array.  Besides not having to have another system up, by having two separate arrays, I can also store my more "important" data on the new drives to minimize the risk of drive failure.

 

Thanks!

Edited by Hammer8
Link to comment
5 minutes ago, JorgeB said:

Array can only have 30 devices max (28 data + 2 parity), but you can have additonal multiple pools and unassigned devices, hence unlimited devices, just not unlimited array devices.
 

Thanks for the clarification.  What does multiple pools mean? For example, I currently have a cache pool with 1 SSD that is dedicated to my array.  I've read having multiple pools is a workaround to the 30 drive array limit, but I didn't really understand what that means.  Suppose I create a new pool with 3 8TB drives, will I be able to add shares to those drives similar to what I can do with an array?  And is the difference that they will not be parity protected?  Can the shares span multiple drives so that I can have a share that is say 12 TB in size?  Thanks again for your help.

Link to comment
16 minutes ago, Hammer8 said:

What does multiple pools mean? For example, I currently have a cache pool with 1 SSD that is dedicated to my array.

When speaking of the array, we mean parity and just those disks protected by parity, so your cache pool is not part of the array (though it is part of user shares).

 

V6.9+ supports multiple pools. These are handled like cache, and like cache, they are part of the user shares. And like cache, they can be used however you want, including caching user share writes for moving to the array (cache-yes shares), or cache-only which don't get moved, or cache-prefer which get moved TO a pool. Keeping some files on pool makes sense for many things to improve performance, like dockers and VMs.

 

You just name these pools however you want, and each pool can have multiple disks if they are configured as btrfs. "Cache" is just another pool that happens to be named "cache" and you don't even have to have a pool named "cache".

Link to comment
23 minutes ago, trurl said:

When speaking of the array, we mean parity and just those disks protected by parity, so your cache pool is not part of the array (though it is part of user shares).

 

V6.9+ supports multiple pools. These are handled like cache, and like cache, they are part of the user shares. And like cache, they can be used however you want, including caching user share writes for moving to the array (cache-yes shares), or cache-only which don't get moved, or cache-prefer which get moved TO a pool. Keeping some files on pool makes sense for many things to improve performance, like dockers and VMs.

 

You just name these pools however you want, and each pool can have multiple disks if they are configured as btrfs. "Cache" is just another pool that happens to be named "cache" and you don't even have to have a pool named "cache".

Ah okay...I think I understand.  Suppose my array of 3 and 4 TB drives fill up the 30 drive array limits.  I can create a new pool with my 8 TB drives.  These 8 TB drives will not be protected, but I can create shares and assign the shares to these drives using the "cache-only" option.  Two things I really like about UnRAID is the ability to add new drives and if a drive fails (assuming no parity), I still have the data on the non-failing drives.  Do these features also apply to pools if I choose to format the drives using btrfs, but not use RAID?  I say btrfs only because I've read this is the fs I need in order to expand the pool with new disks in the future.  Is not using RAID an option if I would like to maximize my storage space?  Also, when I add a new disks to the pool, will I lose my existing data?  Thank you for helping me understand UnRAID!!

Edited by Hammer8
Link to comment
39 minutes ago, trurl said:

A pool with multiple drives must use btrfs raid. Depending on the btrfs raid mode, they could have redundancy.

 

https://wiki.unraid.net/Manual/Storage_Management#Change_Pool_RAID_Levels

Thanks!  And so RAID0 is striped which is no redundancy.  If I select RAID0 to maximize storage space, does that mean if one of the disks fail, I lose all the data in the pool and not just the data on the failed drive?  Just want to confirm in this use case, UnRAID behaves like traditional RAID.  Thanks again!

Link to comment
1 hour ago, Hammer8 said:

Can the shares span multiple drives so that I can have a share that is say 12 TB in size?

Shares span drives, the parity array drives are still separate volumes so any one file can only exist on one drive, no spanning files across drives. Cache pools use BTRFS RAID, so files can span across all drives in the pool. Free space on the parity array is limited by the free space available on any single drive. So, if you have 2 1TB parity data disks, you would not be able to store a single file larger than 1TB, but you could have a folder with 2 TB worth of smaller files. Clear as mud?

Link to comment
12 minutes ago, jonathanm said:

Shares span drives, the parity array drives are still separate volumes so any one file can only exist on one drive, no spanning files across drives. Cache pools use BTRFS RAID, so files can span across all drives in the pool. Free space on the parity array is limited by the free space available on any single drive. So, if you have 2 1TB parity data disks, you would not be able to store a single file larger than 1TB, but you could have a folder with 2 TB worth of smaller files. Clear as mud?

Hmmm...I think I'm almost there.  Suppose my pool of 5 drives is formatted with RAID0 and 1 of the drives fail.  Will I lose the data on all 5 drives (as in traditional RAID 0) or just the data on the drive that failed?  Since any one file will reside entirely on a single drive, does that mean the files in the non-failing drives will be safe?

Link to comment
Just now, Hammer8 said:

Hmmm...I think I'm almost there.  Suppose my pool of 5 drives is formatted with RAID0 and 1 of the drives fail.  Will I lose the data on all 5 drives (as in traditional RAID 0) or just the data on the drive that failed?  Since any one file will reside entirely on a single drive, does that mean the files in the non-failing drives will be safe?

Nope, got it backwards. On the parity array, all non-failing drives are fine, and you can rebuild as many failing drives as you have parity drives, max of 2.

 

The cache pools are purely BTRFS RAID for multiple disks, so a RAID 0 is completely broken with no recovery if a drive fails.

Link to comment
6 minutes ago, jonathanm said:

Nope, got it backwards. On the parity array, all non-failing drives are fine, and you can rebuild as many failing drives as you have parity drives, max of 2.

 

The cache pools are purely BTRFS RAID for multiple disks, so a RAID 0 is completely broken with no recovery if a drive fails.

Thanks for clarifying!  Much appreciated!

 

Since each installation of UnRAID can only have 1 array, is there a way besides using pools to create a separate "unprotected array" such that in this unprotected array, if 1 drive fails, only the data in that failing drive is lost?  Basically, all I want UnRAID to do is expose the shares on this unprotected array as single folders that can span multiple drives so that the total size of the contents in the folders can be larger than the size of a single drive (but no single file can be larger than the remaining capacity of a drive).

 

Thanks!

Link to comment
8 minutes ago, Hammer8 said:

Thanks for clarifying!  Much appreciated!

 

Since each installation of UnRAID can only have 1 array, is there a way besides using pools to create a separate "unprotected array" such that in this unprotected array, if 1 drive fails, only the data in that failing drive is lost?  Basically, all I want UnRAID to do is expose the shares on this unprotected array as single folders that can span multiple drives so that the total size of the contents in the folders can be larger than the size of a single drive (but no single file can be larger than the remaining capacity of a drive).

 

Thanks!

Pools can participate in user shares even if there is only one device in each pool. Since the introduction of multi pools is new, I don't remember the limit of number of pools that can be defined, but you could possibly just keep adding XFS single device pools, but you would have to manage space allocation manually, there is currently no mechanism to send writes to multiple pools based on free space like there is in the parity array data disks. All pool disks with the share name root folder would be read, but new files written would go to the share's named pool only or overflow to the parity array if the named pool went below specified free space and configured for cache prefer.

 

A user share is simply a root folder on any array or pool disk, so you can span whichever volumes you choose.

Link to comment
17 minutes ago, Hammer8 said:

is there a way besides using pools to create a separate "unprotected array" such that in this unprotected array, if 1 drive fails, only the data in that failing drive is lost?

You can create multiple single device "pools", some limitations though, all existing data in a specific share using those pools would still be visible when browsing that share but you'd need to work with disk shares to copy data to all the pools except the one set to use that share, currently any share only can be configured to use one pool.

Link to comment

Thanks everyone for your help!

 

Does anyone know the reason for the 30 drive limit?  Is it if the limit was higher then the chance of 3 drives failing is too great?  While not perfect, the ability to recover data on the non failing drives seems to be a huge offset to that risk.

Edited by Hammer8
Link to comment
  • 1 month later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.