Idea here is to have multiple "redundancy groups", e.g., two separate arrays each with their own parity,disk1,disk2...

 

Probably will never do this for unRAID, what's the point?

 

I am interested in this feature. I would suggest the point is that there is some economy of scale in building larger boxes. The days of PCI are over and with PCI-e and port multipliers supporting large numbers of disks is limited only by the current lack of large enclosures.

 

However on that point there are a number of "blackblaze" type projects on the go.

 

Even if you didnt decide to use vast numbers of disks smaller arrays are faster. With 4 or even 5TB disks on the horizon smaller arrays could become more practical.

 

Either way I think there is a case for the feature the question is really the effort of implementation which I cannot answer.

Hi NAS,

 

For economy of scale, larger array sizes are needed (30+ drive beasts).

 

 

 

For faster arrays, maybe this is a better solution (presents itself as one big array):

 

Support daisy-chained servers

 

Requires two ethernet ports per server. Primay server connects to network. Secondary server connects to second ethernet port of primary server. All user shares on secondary server appear integrated with primay server. All system management for both servers is done through primary server. Not sure how many such servers could be realistically daisy-chained (probably a lot).

 

 

 

 

I'm personaly not in favour of 2 arrays in one machine (MB). Better to take the above approach.

 

I also think that this is a good alternative to P+Q parity. If you think your array gets to big, and has a chance of dual drive failure, just build a lot of 6 disk arrays and daisy chain them together. P+Q parity is more cost effective, though.

 

 

 

 

The thing you could also do is ask support for ESXi VM. Then you can build as many small arrays on one system as you want.

 

Even though I have an existing virtual system that could handle this I would not use vmware for unRAID. I would not add a layer of abstraction onto a NAS. unRAID is elegant and that is a prime reason i moved to it.

 

Having two servers act as one sounds like alot more dev work to me and I dont like the extra complication either. To my eye simply having the option to have more than one array comes with a number of benefits. However having no clue about how much recoding that would entail I cant comment on effort vs. gain. What I can say is the Liemtech have added the feature to the todo list as "never" but asked a question if it would be of benefit.

 

Hence this thread, for me this now it would be a big benefit.

 

It used to be in PCI days if you have 20 drives you would have a slow array. This is not the case now. Extreme case...With a modern low end Motherboard (6 SATA $80) +(PCI-e 8 port card ($100)  + 14 five port multiplers ($700) you would have 70 SATA ports for < $1000. You could use 42 of these of these and still not be blocking. These numbers are dropping daily.

The benefit of multiple arrays is capacity/power. There is a limit to the number of drives an array can support. It is now 21. That might grow. But I don't think that is the answer.

 

Allowing multiple arrays is one method for supporting many more drives. Above, a system with ~70 drives was mentioned. But when seeking capacity, not performance gains, the number can be much larger. When the desired performance envelope is supported by 5-6 active drives, the performance impact of blocking goes unnoticed. Currently the way to satisfy growing capacity demand by adding an entire server.

 

Adding multiple arrays is a better way to achieve the capacity.

 

I have seen dozens of posts with people stating they have reach X limit and are now replacing drives of 1TB size with 2TB size, hoping to find a use for the 1TB drive(s). Multiple arrays would give those drives extended life/value.

 

Please reconsider moving multiple arrays up the roadmap. Are there problems in making, managing, supporting multiple arrays?

With the limited time that limetech can spend on innovating or finishing beta cycles, I do not see multiple arrays as practical.

 

If this were targeted at enterprise use, P+Q parity and large arrays would seem more practical. But for the average home user, I question if we need multiple arrays in, or on, one machine vs a reliable pluggable environment that lets us consolidate file services.

 

I do see a practicality in support for vmware/esx as that helps users consolidate servers/services and could accomplish the multiple arrays in one chassis.

 

Granted I would like to see multiple arrays per server just to help me compartmentalize similar data.

I.E. A smaller array for pictures, downloads, music,  a larger array for backups,  a larger array for videos.

 

However, when I think about it. I'm not sure if I want to build a monster (larger then 32 drive) system to house these.

 

With a 20 drive server now, I've been considering a smaller 5 drive server for the most important files and the 20 drive server for video archives alone.

 

Once you have to carry or move a massive server due to impending issues (cough irene cough), you do question how your array should grow or be laid out.

With the limited time that limetech can spend on unRAID, it is important to properly prioritize enhancements.

 

P+Q parity is more of requirement for large drives, than large arrays. The larger drive increases the probability of hitting a non recoverable read error during rebuild. The large array just increases unit count and rapid repair can address that. So, P+Q is going to be required sooner for average home user who grabs the latest largest available. 3TB drives were consumer products for nearly a year before enterprise.

 

The virtualization process adds so much complexity it is just not manageable. Running unRAID on ESXi, swapping disks is a much longer process since the RDM has to manually change. Scale that to multiple unRAID arrays(machines), and move disks between them. RDMs are a lot of trouble. I don't want to even try and cover the steps of incorporating say 4 new 3TB drives into 2 arrays made of 42 drives with sizes of 0.7TB, 1TB, and 2TB.

 

The problem is trying to make use of a growing volume of smaller capacity disks. While "free", these small capacity disks can be "not worth the power". I thought unRAID might be a solution since the disks get spun down. Adding another physical machine to re-use 20 small capacity drives just raises the power bill.

 

Larger than 32 drive systems are currently available. Backblaze is 45 drives, and could really do 60 (they do not use all the ports). Norco has a 64 drive case. And looking at Xyratex will show you systems scaling to 100s of drives.

 

The Irene situation is a driving factor for multiple arrays. You do not need to move an array which is already remotely replicated. By supporting multiple arrays, the capacity burden is lowered.

The virtualization process adds so much complexity it is just not manageable. Running unRAID on ESXi, swapping disks is a much longer process since the RDM has to manually change. Scale that to multiple unRAID arrays(machines), and move disks between them. RDMs are a lot of trouble. I don't want to even try and cover the steps of incorporating say 4 new 3TB drives into 2 arrays made of 42 drives with sizes of 0.7TB, 1TB, and 2TB.

 

Under ESXi, if you pass through the entire controller, the disks belong solely to unRAID and are free to be swapped just as if the server was a simple unRAID machine.

 

The Irene situation is a driving factor for multiple arrays. You do not need to move an array which is already remotely replicated. By supporting multiple arrays, the capacity burden is lowered.

 

Frankly, I don't want to purchase double drives for everything at the current time. I'm sure there are many people in that boat too.

 

The virtualization process adds so much complexity it is just not manageable. Running unRAID on ESXi, swapping disks is a much longer process since the RDM has to manually change. Scale that to multiple unRAID arrays(machines), and move disks between them. RDMs are a lot of trouble. I don't want to even try and cover the steps of incorporating say 4 new 3TB drives into 2 arrays made of 42 drives with sizes of 0.7TB, 1TB, and 2TB.

 

I'm planning to go down this road in the future. As many have stated, passing the whole controller to the virtual machine works and helps consolidate servers.

 

There's another discussion raised about hot spares and I can see that being a benefit, the p+q parity is another feature members have been asking for.

 

 

It would be an interesting debate/poll on what the biggest bang for the buck would be.

 

There's another discussion raised about hot spares and I can see that being a benefit, the p+q parity is another feature members have been asking for.

 

A second parity disk is at the top of my list and has been for a couple of years. The lengthy 5.0 beta process has made me think that it may never arrive, but perhaps that isn't so. Implementation would not be trivial, but beta testing should be straight forward. If the code for the 1st parity disk remains the same, then most likely a bug would only affect the second parity disk. Maybe I'll start getting my hopes up again...

The virtualization process adds so much complexity it is just not manageable. Running unRAID on ESXi, swapping disks is a much longer process since the RDM has to manually change. Scale that to multiple unRAID arrays(machines), and move disks between them. RDMs are a lot of trouble. I don't want to even try and cover the steps of incorporating say 4 new 3TB drives into 2 arrays made of 42 drives with sizes of 0.7TB, 1TB, and 2TB.

 

Under ESXi, if you pass through the entire controller, the disks belong solely to unRAID and are free to be swapped just as if the server was a simple unRAID machine.

 

 

Dedicating slots gets away from RDM, but slots are limited. How many slots you got for those controllers? At 2 or 3 slots per unRAID machine, this is not really going to address multiple arrays.

 

PS: we should keep this thread to multiple arrays. P+Q and hot spare/auto rebuild are two other roadmap topics already.

PS: we should keep this thread to multiple arrays. P+Q and hot spare/auto rebuild are two other roadmap topics already.

 

Sorry, couldn't help myself.

I can't help but feel that multiple arrays in a single box would be a good feature. I have 2 very distinct usages for my array - backup, and media storage.

 

I have one large user share for media (6 disks), and a few shares for backups (one for each Mac for Time Machine, and one for backing up the media share).

 

It seems to me that having independent arrays would distribute some of the risk of drive failure reducing the chance of a double drive failure. Since my TimeMachine is backing up to my unRaid box every hour, that data and parity disk are getting a lot of use. That load is surely putting a larger failure risk on my parity drive and since it's common with my media share, the chances of my media share losing it's parity must be higher than if it had it's own parity drive that could spend most of it's life spun down as that share is mostly read.

 

Currently I am backing the media up to another share within the same parity group - that just seems odd to me. I know the backup is on different drives because my shares are configured that way, but I can't help the feeling that an independent array would provide better redundancy.

 

The hard facts of the matter are that the more co-dependant drives you add, the greater the risk of 2 failing at the same time. 2 arrays of 11 drives are safer than 1 array of 22 - at least with 2 there is a chance that a dual failure will happen in separate arrays.

 

Obviously my ideal solution would be a second box for backups, but that defeats one of my primary objectives with setting this thing up - to only have one machine running all the time.

 

Anyway.. +1 for multiple arrays.

 

The hard facts of the matter are that the more co-dependant drives you add, the greater the risk of 2 failing at the same time. 2 arrays of 11 drives are safer than 1 array of 22 - at least with 2 there is a chance that a dual failure will happen in separate arrays.

 

Obviously my ideal solution would be a second box for backups, but that defeats one of my primary objectives with setting this thing up - to only have one machine running all the time.

 

Anyway.. +1 for multiple arrays.

 

I would really hope that this could get moved from the "Never" pile as well (like others have said, assuming the time to code is not so great).  I mean, I would even pay for a second array (like I would have to if building a second box).

 

But as whiteatom said; kinda defeats the purpose of setting up the single backup/storage device.

Also, a second box increase power usage (as not all components can be powered down).  You don't need another CPU not really being used, etc.

 

As array sizes increase (and drive sizes), some way to add another layer of protection would be greatly appreciated.

 

Using VMWare really isn't easy (putting the virtual layer debate aside).

 

Or would it simply be easier to move to a Raid 60 array when moving beyond 6 drives?