June 18, 201214 yr be able to configure 'sub-arrays' lets say you have a 24 drive case, instead of a single 22 drive array (we'll assume we could do double pairity) you have 2 11 drive arrays in the same case. each would have it's own parity drive, but the drives would be drive1 thru drive22... drive1 to 11 in sub-array 1, 12 thru 22 in sub-array 2. you could take either sub-array off-line, or the whole -super-array. IN THEORY you could lose 2 drives, one from each sub-array and your data is still safe. you lose two drives in a single parity 20-24 drive array and you're screwed. IN THEORY you could maybe assign up to 'x' sub-arrays to the super-array.. let's do 24 drives again.. set the sub array to 6, 5 data and 1 parity, you have 4 sub-arrays total, 4 active parity drives. 20 drives worth of data. You still have to lose 2 drives in a sub-array to lose data, but the smaller you make the sub-array, the less likely you have a catastrophic, multidrive failure in that sub-array. just a thought, I could see this raising the memory requirements.
June 18, 201214 yr I really think this a good idea, but again don't believe it belongs in the RC support thread. This sub forum should be about getting 5 to final. Is the road map sub forum still current?
June 18, 201214 yr I consider a cache drive an absolute must (many do), so with your method i'd need a cache drive for each array. That's 4 drive slots for 2 arrays and then i'm back to where I began, 20 data drives. The only pro I see with this method is having 2 parity drives and less likely chance of losing data. However, it's still not as reliable as 2 parity drives in a single array. For example, if you have 2 drives fail you are playing russian roulette and hope each failure is on separate arrays, with a single array and 2 parity it doesn't matter which data drives fail. 1 array with 1 parity: 2 data drives die = 100% chance of data loss 2 arrays with 1 parity each: 2 data drives die = 50% chance of data loss 1 array with 2 parity: 2 data drives die = 0% chance of data loss I ended up building a second server because I needed more space and all 20 data drives were either 2TB or 3TB drives. It would of been nice to be able to easily add 6TB (2 more data drives), but I don't see it happening until 5.1 or later. I wish we had a time frame, but now I'll have 40 data drives between my 2 unRAID servers. Definitely not cheap, but i'll be able to expand to 120TB using 3TB drives without replacing drives. By the time I actually need to fill all the bays, I assume we will have 22 data drives and 5+ TB drives.
June 18, 201214 yr If you need to get more security then just set up a totally different system next to it.. That extra costs for that are basically limited considering the amount of money allready would need to be spent on drives... Also, this thread needs to be moved to feature suggestions..
June 18, 201214 yr I consider a cache drive an absolute must (many do), so with your method i'd need a cache drive for each array. There is no reason at all that has to be true. Nothing stops a single cache drive servicing both sub-arrays. As for *your* desire for more data drives, then you simply don't have to choose the dual sub-array route. But for others who are more concerned about the risks involved in 20+ drives being protected by a single parity drive this is one way to segment a huge array without the added overhead cost of enclosures, motherboards, cpu, PSU and memory. Double bonus that increasing drive count means this is now possible without reducing data-drive slot count.
June 18, 201214 yr I consider a cache drive an absolute must (many do), so with your method i'd need a cache drive for each array. There is no reason at all that has to be true. Nothing stops a single cache drive servicing both sub-arrays. As for *your* desire for more data drives, then you simply don't have to choose the dual sub-array route. But for others who are more concerned about the risks involved in 20+ drives being protected by a single parity drive this is one way to segment a huge array without the added overhead cost of enclosures, motherboards, cpu, PSU and memory. Double bonus that increasing drive count means this is now possible without reducing data-drive slot count. I guess it's possible that the cache drive could support more than 1 array. However, this setup is still far inferior to a true 2 parity drive + 22 data drive setup. Tom has stated that these things are possible, so why would he spend time on allowing us to make sub arrays? I'm not saying the suggestion is bad, it's just that tom probably isn't going to spend time on coding all of this when he can just work on the "real thing". As stated above: 1 array with 1 parity: 2 data drives die = 100% chance of data loss 2 arrays with 1 parity each: 2 data drives die = 50% chance of data loss (assuming your arrays are the same size) 1 array with 2 parity: 2 data drives die = 0% chance of data loss I would not waste a hard drive based on a 50% chance that I won't be able to recover data if 2 drives fail on the same sub array. Most people are not going to base their data recovery on luck, especially ones that are wanting dual parity drives. When talking about critical data, 50% chance of data recovery should be considered 0%. I work in IT, and i've seen many companies lose thousands of dollars because of setups like the one OP is describing. It's "better" than what we have now, but it's not a solution, never base data recovery on a chance of success. If this was implemented, the demand for 2 parity drives on a single array would still be there.
June 18, 201214 yr I actually do agree with you on dual parity being the desired endstate. However options are good and dual parity may take time to implement as well as have unacceptable write-speed impacts for some people. Depending on Tom's program plan it might be a reasonable interm step to for folks getting itchy about having so many drives protected by a single parity drive. I don't know, but that is why we bring up Brilliant Ideas For Other People I do know Tom has mentioned something along these lines already be it sub-arrays in the same box, or even fusing n boxes via a master box and user shares.
June 18, 201214 yr This is what I was alluding to on the thread of the OP who has maxed out his drives, by using a HW RAID card underneath unRAID. You have to have 4 simultaneous drive losses (2 drive on 2 different RAID sets) in order to lose data as seen from the view of the outer system. The underlying (I use HW) RAID can be any type of parity-protected subsystem. But you need dedicated parity calc/rebuild for EACH subsystem, plus the outer system. The negative to an outer unRAID made up of sub-RAIDs, is the ratio of data drives to parity drives decreases.... but then the high data:parity ratio is what scares me from a 24-drive monster with a single parity drive!
June 19, 201214 yr Not tested, but... For people who already run ESXi, just run 2 unraid VM's side by side. One with 16 disks, (2x M1015) and one with 8 disks (1x M1015). Like I said, not tested, but hopefully should work...
June 19, 201214 yr Theoretically, couldn't you have three parity drives? 1st drive: Array A 2nd drive: Array B 3rd drive: Parity of A & B parity drives. That way you can have quite a few disk failures without data loss? (Someone check my maths!)
June 19, 201214 yr Not tested, but... For people who already run ESXi, just run 2 unraid VM's side by side. One with 16 disks, (2x M1015) and one with 8 disks (1x M1015). Like I said, not tested, but hopefully should work... I am essentially doing that now. I have 2 M1015s in my X9SCM ESXi box. One M1015 goes to a SAS expander to get me 16+ drive unRAID server. The other is passed through to Windows VM to provide 8 drives for my SageTV server. That second one could also go to a 2nd unRAID server just as easily and with another SAS expander I could have 16+ on it as well.
June 19, 201214 yr Not sure that works. At least, that is, it won't let you lose two data drives from Array A (A1 and A2 of An drives) because even with Pab (Parity A&B) you can't reconstruct the data lost in A1 and A2. Sure you'll know what Pa is but unless you have done diagonal or PQ parity you're SOL becuse you still have two unknowns and only one equation; A1 + A2 = Pa and no other combination of parity/data includes A1 or A2 All you get from having Pab is the ability to lose a data drive and and parity drive. Then you can reconstruct parity from Pb and Pab (assuming you lost Pa) and then follow that with a rebuilt of Array A (assuming we lost a drive in A) now someone check my math ... that is right math, singular you silly Brit >;-)
June 19, 201214 yr Not sure that works. At least, that is, it won't let you lose two data drives from Array A (A1 and A2 of An drives) because even with Pab (Parity A&B) you can't reconstruct the data lost in A1 and A2. Sure you'll know what Pa is but unless you have done diagonal or PQ parity you're SOL becuse you still have two unknowns and only one equation; A1 + A2 = Pa and no other combination of parity/data includes A1 or A2 All you get from having Pab is the ability to lose a data drive and and parity drive. Then you can reconstruct parity from Pb and Pab (assuming you lost Pa) and then follow that with a rebuilt of Array A (assuming we lost a drive in A) now someone check my math ... that is right math, singular you silly Brit >;-) you are correct. His idea does not work mathematically. Joe L.
July 5, 201214 yr As stated above: 1 array with 1 parity: 2 data drives die = 100% chance of data loss 2 arrays with 1 parity each: 2 data drives die = 50% chance of data loss (assuming your arrays are the same size) 1 array with 2 parity: 2 data drives die = 0% chance of data loss While this is true it is a very pessimistic way to look at it. For starters, while dual parity is a much better solution overall, it is not without its own downfalls. For starters, dual parity would likely further limit the speed at which data is written to the array (already one of unRAIDs biggest drawbacks) whereas having two "sub-arrays" with independent parity drives would not suffer this performance hit even when both arrays are being written to at the same time. There would still be the same limits on your PCI-E bus as a single large array and your CPU would be working a little harder (but we're all cool with that becuase let's be honest, our CPUs have been pretty lazy running unRAID so far). Some more optimistic points to note: Sub-arrays should be significantly easier to impliment as an option than dual parity would be, as Tom has already stated how "just add dual parity" is easier said than done. There is no reason you couldn't use a single cache drive for multiple sub-arrays as each array would have its own shares and own disks. You obviously lose a disk slot for each sub-array, but you also lost a disk slot with dual parity. Parity checks could be run at different times on each sub-array, reducing controller/bus overhead, PSU, CPU & memory load, heat.. etc. [Dual parity] checks require all drives spinning. While it is not a failsafe for losing two drives, it is 50% more likely to save your ass then your current setup is. I'll take a 50% chance on losing a second disk over a 0% chance any day.
July 5, 201214 yr If you haven't seen this yet, it appears Tom is implimenting "multiple array" in 5.1 among other features. See: http://lime-technology.com/forum/index.php?topic=18773.msg188648#msg188648
Archived
This topic is now archived and is closed to further replies.