shEiD Posted July 14, 2017 Author Share Posted July 14, 2017 That is the most reasonable way of doing it, imho. I could buy another license and put up a 2nd unraid, but it's more convenient to have all the stuff in one place. Although, the 2nd one could be a backup, that's always online also. Hmm, gonna think about it some more... Quote Link to comment
Frank1940 Posted July 14, 2017 Share Posted July 14, 2017 (edited) 14 minutes ago, shEiD said: gonna think about it some more... You can probably find the beginnings for a backup server at a garage sale. The MB and CPU requirements are rather light for a strictly NAS server. (If you want dual parity and fast parity check speeds, you need a CPU no later than ~2015. But slow parity checks on a backup server are not near the issue that they can be an production server...) Edited July 14, 2017 by Frank1940 Quote Link to comment
shEiD Posted July 14, 2017 Author Share Posted July 14, 2017 I have an old i7 rig (about year 2010, IIRC) I could use. How would that cope with dual parity? So unraid's attraction was, it was always running smoothly on old an not power-full hardware. Did that change with dual parity? I mean, forget docker and VMs - just using dual parity...? Quote Link to comment
SSD Posted July 14, 2017 Share Posted July 14, 2017 Might want to look here: https://www.backblaze.com/blog/hard-drive-benchmark-stats-2016/ https://www.backblaze.com/blog/hard-drive-failure-rates-q1-2017/ The 6T REDS don't have the best stats, at nearly 3% failure rate. The Seagate 8Ts are looking pretty good, and look at how many they bought! (That's usually a sign of a good value drive). They are not the archive version (at first I thought they were but they are not). Seagate was sued for quality issues and since their quality has improved. The HGST 8Ts are stellar, but they only bought a few because they were the Helium drives and high dollar. There is nothing out on the WD 8T or 10T REDS. I can tell you as a brand name the HGST have been the highest for some time now, and I tend to buy them. But Seagates were 3 for the price 2 HGSTs (24T for the price of 16T). So it was hard to resist. May pay for it in reliability, though, but it was a calculated risk. The archives are good performers. 5900 RPM and high density. There is a possibility of it slowing down due to excessive random writes, but with media data it is normally sequential and the slowdown should never happen. Many users here use them successfully, even for parity, and report they work great. These are not recommended for RAID, but unRAID is not RAID. Not trying to sell you on anything, just providing info. Quote Link to comment
JorgeB Posted July 14, 2017 Share Posted July 14, 2017 30 minutes ago, shEiD said: So unraid's attraction was, it was always running smoothly on old an not power-full hardware. Did that change with dual parity? I mean, forget docker and VMs - just using dual parity...? Any dual core Sandy Bridge or newer is enough. Quote Link to comment
shEiD Posted July 14, 2017 Author Share Posted July 14, 2017 @bjp999 @johnnie.black Thanks for info, guys. And I will look into those Seagates, if their reliability has become better. Because in my own experience - Seagates were the worst, actually more than half of mine are dead already. I really can't remember exactly, but I think I have 10+ 1.5TB and 10+ 3TB dead Seagate drives. At the same time - I have like ~30 old as hell 500GB and 750GB Samsung drives - most of them still work perfectly. My go to testing drives Maybe 3-5 dead from ~30, some I have stopped using just because SMART gave warning about reallocated sectors. Quote Link to comment
tdallen Posted July 14, 2017 Share Posted July 14, 2017 Haswell and newer Intel CPUs contain support for the AVX2 instruction set. The unRAID dual parity calculations leverage this for faster performance, but your 2010 i7 should be able to handle dual parity without issue. The AVX2 instruction set probably benefits low end processors the most since they have less raw horsepower to work with. Quote Link to comment
Opawesome Posted September 6, 2019 Share Posted September 6, 2019 (edited) On 7/14/2017 at 11:30 AM, shEiD said: I am still kinda confused, how does dual parity work. Does it really mean, that having unRAID use dual parity I could recover from ANY 2 drives failing? Like they say, a picture is worth a thousand words... Greetings, Sorry for bumping this old thread but after quite intensive research and carefull reading of this thread, I still could not find documentation explaning how does *dual* parity work and how it can manage to recover data from *any* two drives (i.e. data and/or parity drive). Many thanks for your replies. G Edited September 6, 2019 by Opawesome typo Quote Link to comment
trurl Posted September 6, 2019 Share Posted September 6, 2019 20 minutes ago, Opawesome said: Greetings, Sorry for bumping this old thread but after quite intensive research and carefull reading of this thread, I still could not find documentation explaning how does *dual* parity work and how it can manage to recover data from *any* two drives (i.e. data and/or parity drive). Many thanks for your replies. G The parity2 calculation is different and independent of the parity calculation. The exact algorithm is a bit more complicated and I haven't really tried to understand it in detail, but the fact that these calculations are different and independent is what allows them to recover 2 drives. The contents of parity and parity2 are different, so these disks are not interchangeable and cannot be swapped without rebuilding parity. Here is a thread I found on the forum that discusses it a bit more. If you google "Q parity" you can probably find more technical discussions. https://forums.unraid.net/topic/51362-pq-or-dual-parity-in-detail/ Quote Link to comment
Opawesome Posted September 6, 2019 Share Posted September 6, 2019 (edited) 43 minutes ago, trurl said: The parity2 calculation is different and independent of the parity calculation. The exact algorithm is a bit more complicated and I haven't really tried to understand it in detail, but the fact that these calculations are different and independent is what allows them to recover 2 drives. The contents of parity and parity2 are different, so these disks are not interchangeable and cannot be swapped without rebuilding parity. Here is a thread I found on the forum that discusses it a bit more. If you google "Q parity" you can probably find more technical discussions. https://forums.unraid.net/topic/51362-pq-or-dual-parity-in-detail/ Hi trurl, Many thanks. I will investigate towards that direction. Have nice day, G Edited September 6, 2019 by Opawesome typo Quote Link to comment
Frank1940 Posted September 6, 2019 Share Posted September 6, 2019 (edited) 1 hour ago, Opawesome said: Many thanks. I will investigate towards that direction. Here is a good one to start with. It breaks it down to a simple explanation avoiding most of the rigorous supporting theory as to why it will always work. https://igoro.com/archive/how-raid-6-dual-parity-calculation-works/ Edited September 6, 2019 by Frank1940 Quote Link to comment
dingsda Posted May 2, 2020 Share Posted May 2, 2020 (edited) On 7/14/2017 at 11:30 AM, shEiD said: I am still kinda confused, how does dual parity work. Does it really mean, that having unRAID use dual parity I could recover from ANY 2 drives failing? Like they say, a picture is worth a thousand words... Sorry for bumping this thread again, but I think, it's not answered corretly. Tis question was answered with a "Yes" to "having unRAID use dual parity I could recover from ANY 2 drives failing?" and the following link was provided: Quote There's no magic: https://wiki.lime-technology.com/Parity#How_parity_works ....but.... behind this link at the bottom of the page I found the following information: Quote For large arrays, ‘dual parity’ – or, the facility to have a second parity disc that is not simply a mirror of the first – would be useful. This would permit two simultaneous drive failures without losing data. unRAID does not have dual parity at present, but ‘P + Q redundancy’ is part of the future roadmap.(Dead Link) In a P + Q redundancy system (as in a RAID-6 system), there would be two redundancy disks: ‘P’, which is the ordinary XOR parity, and ‘Q’, which is a Reed-Solomon code. This would allow unRAID to recover from any 2 disk errors, with minimal impact on performance. Further discussion: [1], [2] This means: - the second parity disc is a mirror of the first one - there is no possibility to recover from ANY 2 drives failing - from the image above it's possible to recover from the 1st and 2nd scenario, but the third one sums up in data-loss Or do I miss something? cheers, d. Edited May 2, 2020 by dingsda Quote Link to comment
JorgeB Posted May 2, 2020 Share Posted May 2, 2020 12 minutes ago, dingsda said: the second parity disc is a mirror of the first one That's not correct 2nd parity uses a different calculation and isn't interchangeable with parity1, some info here. 13 minutes ago, dingsda said: there is no possibility to recover from ANY 2 drives failing There is, you can recover any two failed drives. Quote Link to comment
itimpi Posted May 2, 2020 Share Posted May 2, 2020 13 minutes ago, dingsda said: This means: - the second parity disc is a mirror of the first one - there is no possibility to recover from ANY 2 drives failing - from the image above it's possible to recover from the 1st and 2nd scenario, but the third one sums up in data-loss Or do I miss something? Parity2 uses a different algorithm to parity1 to calculate its contents so they are NOT mirrors. Unlike parity1, the algorithm for parity2 includes disk slot number as one of its inputs. I am not sure if Limetech have ever given details of the exact calculations that are used other than to say they are more demanding on the CPU than the ones used for parity1. Quote Link to comment
dingsda Posted May 2, 2020 Share Posted May 2, 2020 (edited) So, you say, the offical WiKi is providing wrong informations? Edited May 2, 2020 by dingsda Quote Link to comment
Frank1940 Posted May 2, 2020 Share Posted May 2, 2020 If you google P+Q parity you will find numerous articles on it. The following link will take to to one of them https://igoro.com/archive/how-raid-6-dual-parity-calculation-works/ This post also discusses the Parity2 (or Q) calculation as it applies to Unraid. https://forums.unraid.net/topic/49598-unraid-server-release-620-rc4-available/#comment-487759 This has come up before and I usually tell folks that to fully understand what it going on, you will have to had a graduate-level course in Matrix Algebra. The problem with any proposed Dual Parity solution is that you must have a rigorous mathematical proof that the method being implemented actually works 100.000000% of the time. Without that proof, no one would commit their data to being protected by it. Imagine that you are the IT manager of Chase Manhattan bank and you had to explain to your Board of Directors that all of their customer account information had been lost because the disk failures 'happened' to be one of .00001% of the cases where data recovery fails because of a flaw in the parity scheme. The proof for single parity is almost trivial in comparison! Quote Link to comment
dingsda Posted May 2, 2020 Share Posted May 2, 2020 (edited) ergo... the official wiki is wrong, because of the following statement: Quote unRAID does not have dual parity at present, but ‘P + Q redundancy’ is part of the future roadmap.(Dead Link) Edited May 2, 2020 by dingsda Quote Link to comment
JorgeB Posted May 2, 2020 Share Posted May 2, 2020 1 hour ago, dingsda said: ergo... the official wiki is wrong, It's just out of date, and not sure if that's even part of the official wiki or user contributed content. Quote Link to comment
itimpi Posted May 3, 2020 Share Posted May 3, 2020 It is part of the User Contributed content section of the wiki. The Official Documentation part of the wiki has UnRAID_6 as part of the URL. Quote Link to comment
bombz Posted June 3, 2020 Share Posted June 3, 2020 (edited) Hello, I have read through some of the posts here and got lost in some of the conversations. Thought to reach out on this topic as I am in the middle of an upgrade. I will try to keep this short and simple Current array size in disk - 11 disk total Current chassis physical drive bays - 12 disk total Parity was upgraded successfully from 6TB to 10TB Disk 3 was upgraded (currently rebuilding) 4TB to 10TB I currently have 1X physical drive bay OPEN for one of the following purposes that I was considering: 01. Dual parity (add another 10TB disk in the future) 02. Data disk (add more storage space) **If i go with option 01, does the second parity disk need to match @ 10TB or can it be 8,6,4,etc?** As there are many experienced users in the community, and what you would recommend? Option 01 or Option 02? Being my array is not large in physical disk (currently 11 disk), would dual party would be worth wild? As disk size in the world today has improved, I am remaining with this chassis at the 12X drive bay limit. Suggestions (if you were me)? Thanks, Edited June 3, 2020 by bombz Quote Link to comment
trurl Posted June 3, 2020 Share Posted June 3, 2020 18 minutes ago, bombz said: Disk 3 was upgraded (currently rebuilding) 4TB to 10TB 18 minutes ago, bombz said: does the second parity disk need to match @ 10TB or can it be 8,6,4,etc? Each parity disk must be at least as large as the largest single data disk. Since disk3 is 10TB parity2 must be at least that large. Quote Link to comment
bombz Posted June 3, 2020 Share Posted June 3, 2020 6 minutes ago, trurl said: Each parity disk must be at least as large as the largest single data disk. Since disk3 is 10TB parity2 must be at least that large. OK. Good to know when running dual parity, both parity disks need to be as large or larger then the largest, so same rules apply. I thought I could get away with a smaller disk as parity2, now thinking about it, that wouldn't make sense. Now the decision time. Do I run single parity, or dual parity, hmm hmm Quote Link to comment
Frank1940 Posted June 3, 2020 Share Posted June 3, 2020 21 minutes ago, bombz said: As there are many experienced users in the community, and what you would recommend? Option 01 or Option 02? Depends on your adversion (or tolerance) to Risk! Single parity will permit you to recover from a single disk failure. That part is obvious. Dual parity allows you to recover from a two disk failure. While it is possible for a prudent person to monitor his server so that a two disk failure 'would' not occur because at the first sign of a problem with a disk, he would replace it with a new disk. But what would happen if a second disk were to start acting up while the first disk is being rebuilt. One could lose the contents of both disks in a single parity situation. With dual parity, the rebuild would simply continue and the second disk would be flagged as having a problem. (Which of course a risk- adverse person would address immediately!) For more information, see this thread: https://forums.unraid.net/topic/50504-dual-or-single-parity-its-your-choice/ By the way, it appears that failure rates of hard disks are about the same for low capacity disks as for high capacity disks. So you should always opt to replace a small capacity drive with a larger rather than adding another small capacity disk to the array when adding capacity to the array. It may cost more initially but that older small capacity drive is nearer to its end-of-life and you will be doing it anyway at some time in the future. Always be looking at the sweet spot of cost/TB when buying disks... Quote Link to comment
bombz Posted June 3, 2020 Share Posted June 3, 2020 (edited) 7 minutes ago, Frank1940 said: Depends on your adversion (or tolerance) to Risk! Single parity will permit you to recover from a single disk failure. That part is obvious. Dual parity allows you to recover from a two disk failure. While it is possible for a prudent person to monitor his server so that a two disk failure 'would' not occur because at the first sign of a problem with a disk, he would replace it with a new disk. But what would happen if a second disk were to start acting up while the first disk is being rebuilt. One could lose the contents of both disks in a single parity situation. With dual parity, the rebuild would simply continue and the second disk would be flagged as having a problem. (Which of course a risk- adverse person would address immediately!) For more information, see this thread: https://forums.unraid.net/topic/50504-dual-or-single-parity-its-your-choice/ By the way, it appears that failure rates of hard disks are about the same for low capacity disks as for high capacity disks. So you should always opt to replace a small capacity drive with a larger rather than adding another small capacity disk to the array when adding capacity to the array. It may cost more initially but that older small capacity drive is nearer to its end-of-life and you will be doing it anyway at some time in the future. Always be looking at the sweet spot of cost/TB when buying disks... Thanks for your input. I am juggling the thought if I do or don't. Dual parity may be a year out still, to match the 10TB disk. I am often monitoring my server daily if not, weekly, and review SMART, and if there are any error counts within the 'main' tab menu. Generally if I see SMART or errors start to populate (as it has happened in the past) I immediately swap the disk as soon as I can. Trying to think if I burn another 10TB on dual parity setup for the satisfaction and protection. I have some thinking to do on that empty drive bay, as I do have some 4TBs sitting around after this upgrade that are not being used. Never a bad idea to have hot swap available for the current 4s in the array currently. I will think on it. Thanks again! Edited June 3, 2020 by bombz Quote Link to comment
JonathanM Posted June 4, 2020 Share Posted June 4, 2020 1 hour ago, bombz said: I have some thinking to do on that empty drive bay, Leave it empty. Having a spare slot makes troubleshooting and recovery SO much easier. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.