April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I do think he meant two separate parity protected arrays in the same server. There have been so many requests for dual parity, and I would like to see the feature as well (probably for different reasons than others), but do think it will come with considerable "baggage" (e.g., spinning up the entire array to write, slow down writes, etc.) that will make it undesirable to many users now saying they want it. But we'll have to see. But if we look closely at the ways real users have lost real data, none of them (that I can remember anyway) involve 2 drives failing at the same time. In fact SINGLE drive failures are so rare I can hardly remember one of those either. More commonly a user sees a drive is failing based on its SMART attributes and replaces / rebuilds it, a procedure that could just as easily have been done by adding a new drive and copying the data from the failing drive to the good one. So we, as a community, are asking for a feature to combat a problem that is only remotely possible at the far edge of reasonable possibility, while ignoring specific problems with easily achievable solutions that cause real data losses for real users.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I do think he meant two separate parity protected arrays in the same server. There have been so many requests for dual parity, and I would like to see the feature as well (probably for different reasons than others), but do think it will come with considerable "baggage" (e.g., spinning up the entire array to write, slow down writes, etc.) that will make it undesirable to many users now saying they want it. But we'll have to see. But if we look closely at the ways real users have lost real data, none of them (that I can remember anyway) involve 2 drives failing at the same time. In fact SINGLE drive failures are so rare I can hardly remember one of those either. More commonly a user sees a drive is failing based on its SMART attributes and replaces / rebuilds it, a procedure that could just as easily have been done by adding a new drive and copying the data from the failing drive to the good one. So we, as a community, are asking for a feature to combat a problem that is only remotely possible at the far edge of reasonable possibility, while ignoring specific problems with easily achievable solutions that cause real data losses for real users. While I'm not pushing for the dual parity feature, I can say that I did have a dual drive failure. Not in the sense of both drives failing (going red) at the same time. A second drive failed and as I was rebuilding the failed drive. One of the other data drives had a pending sector and caused the rebuild to fail. This is one of the reasons I've been advocating the presentation of smart attributes from the main screen and having the icon change color if any smart attributes are out of spec. Knowing you have pending sectors or a growing number of unrecoverable sectors is important. Since emhttp already reads the SMART attributes, we only need to request Tom to inspect the other attributes and what the warning or out of spec values are. Combining the monthly or weekly parity checks with a visible state change on SMART attributes is one way to combat pending drive failures. The dual parity feature is to help prevent the situation where the rebuild would have failed for a drive that was in a marginal state. As drives get larger, I think this becomes more important. How many people have a large spare 4tb drive laying around?
April 15, 201412 yr There have been so many requests for dual parity, and I would like to see the feature as well (probably for different reasons than others) ... Weebo - some of your points are in line with what I was eluding to here. A second parity drive may also be able to do much more than handle 2 simultaneous failures, including help pinpoint data issues introduced by a less than perfect parity when a rebuild is done. While I'm not pushing for the dual parity feature, I can say that I did have a dual drive failure. Not in the sense of both drives failing (going red) ... What do you mean "going red"? The huge majority of the time a disk going red has to do with a cable coming loose more than a drive failing. A true "failed" drive (as in, dead as a doornail) is pretty unusual. If a disk red-balled due to a cabling issue, the problem could be fixed and the rebuild could be restarted and the disk recovered. ... at the same time. A second drive failed and as I was rebuilding the failed drive. One of the other data drives had a pending sector and caused the rebuild to fail. can you elaborate on this failure. To be nitpicky, a rebuild doesn't fail from a pending sector. Many pending sectors just silently appear in the SMART report. But a READ ERROR caused by a bad sector would generate a failed rebuild (and probably generate a pending sector as well). Is this what happened? How did this fail the rebuild? Did unRAID keep retrying and retrying and retrying? Did it hang? Or did it continue the rebuild leaving a corrupted block on the rebuilt disk? Curious as this is a very unusual situation, and knowing how it presents is very useful. When unRAID hits a read error in a protected array, what it is supposed to do is reconstruct that sector and rewrite it to the drive. That should cause a remapping event and that issue is fixed. But in the middle of a rebuild it can't reconstruct. One more point - people are quick to say, with emphasis, that a red ball only occurs if a write fails. But lets say that a cable is loose and a read fails as a result. unRAID will reconstruct the sector and try to write it back. Well, the cable is still loose so the write fails. Poof - red ball. So it was in fact the read (from the end user's perspective) that triggered the red ball. He wasn't writing. So an unresponsive drive will always get red balled, whether it was detected on the red or the write. This is one of the reasons I've been advocating the presentation of smart attributes from the main screen and having the icon change color if any smart attributes are out of spec. Knowing you have pending sectors or a growing number of unrecoverable sectors is important. Since emhttp already reads the SMART attributes, we only need to request Tom to inspect the other attributes and what the warning or out of spec values are. Small plug for myMain. It provides a single screen view of smart issues on all drives in the machine. I think it is the most used feature. Combining the monthly or weekly parity checks with a visible state change on SMART attributes is one way to combat pending drive failures. Again, pending sectors may or may not generate read errors back to the OS. I have seen too many of them crop up with no OS impact. But I do agree that users should know about them and try to force them to clear (I have seen that happen), or reallocate them. The dual parity feature is to help prevent the situation where the rebuild would have failed for a drive that was in a marginal state. As drives get larger, I think this becomes more important. How many people have a large spare 4tb drive laying around? I agree. But still when you look at the many and numerous ways people shoot themselves in the foot and loose data with perfectly healthy drives, these far less common scenarios seem lower priority to me. Why the comment about a spare 4T drive? I prefer to leave mine on the store shelf. They gladly store it for me for FREE! But there is a fee to retrieve it. So I could get one very quickly if I needed to.
April 15, 201412 yr There have been so many requests for dual parity, and I would like to see the feature as well (probably for different reasons than others) ... Weebo - some of your points are in line with what I was eluding to here. A second parity drive may also be able to do much more than handle 2 simultaneous failures, including help pinpoint data issues introduced by a less than perfect parity when a rebuild is done. While I'm not pushing for the dual parity feature, I can say that I did have a dual drive failure. Not in the sense of both drives failing (going red) ... What do you mean "going red"? The huge majority of the time a disk going red has to do with a cable coming loose more than a drive failing. A true "failed" drive (as in, dead as a doornail) is pretty unusual. If a disk red-balled due to a cabling issue, the problem could be fixed and the rebuild could be restarted and the disk recovered. A permanent failure where the drive is no longer reliable. ... at the same time. A second drive failed and as I was rebuilding the failed drive. One of the other data drives had a pending sector and caused the rebuild to fail. can you elaborate on this failure. To be nitpicky, a rebuild doesn't fail from a pending sector. Many pending sectors just silently appear in the SMART report. But a READ ERROR caused by a bad sector would generate a failed rebuild (and probably generate a pending sector as well). Is this what happened? How did this fail the rebuild? Did unRAID keep retrying and retrying and retrying? Did it hang? Or did it continue the rebuild leaving a corrupted block on the rebuilt disk? Curious as this is a very unusual situation, and knowing how it presents is very useful. When unRAID hits a read error in a protected array, what it is supposed to do is reconstruct that sector and rewrite it to the drive. That should cause a remapping event and that issue is fixed. But in the middle of a rebuild it can't reconstruct. The rebuild failed due to an unreadable sector, unRAID threw the drive out of the array and two drives were now offline. My parity drive that I was trying to rebuild and the data drive that had the failed unreadable/pending sector. FWIW, I did do a parity check the night before. As far as 'a rebuild doesn't fail from a pending sector' that's only half true. If the pending sector needs to be read by some allocated data and it is unreadable, and the drive returns that back to the kernel, the rebuild will fail. If the pending sector is in an unused portion of the drive, and can somehow be reconstructed from CRC, it will be remapped when written. This is one of the reasons I've been advocating the presentation of smart attributes from the main screen and having the icon change color if any smart attributes are out of spec. Knowing you have pending sectors or a growing number of unrecoverable sectors is important. Since emhttp already reads the SMART attributes, we only need to request Tom to inspect the other attributes and what the warning or out of spec values are. Small plug for myMain. It provides a single screen view of smart issues on all drives in the machine. I think it is the most used feature. This should be part of unRAID, perhaps by enabling an advanced option or some other kind of screen. Every other NAS I''ve used present the SMART attributes for review and alerted you via email if the SMART attributes had questionable values. Combining the monthly or weekly parity checks with a visible state change on SMART attributes is one way to combat pending drive failures. Again, pending sectors may or may not generate read errors back to the OS. I have seen too many of them crop up with no OS impact. But I do agree that users should know about them and try to force them to clear (I have seen that happen), or reallocate them. Pending sectors should be seen on the front screen with perhaps a yellow light as this is a warning condition to the drive's health. It can cause the drive to go offline. I've seen it.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I mean multiple arrays each with its own parity disk, so keep what ever disk limits but allow me to have 6+P, 10+P etc all in the same system.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I mean multiple arrays each with its own parity disk, so keep what ever disk limits but allow me to have 6+P, 10+P etc all in the same system. This can 'almost' be accomplished with virtualization. I.E. Two unique arrays in the same host. What would not be shared is the management and/or the user share. I.E. they would look like two separate servers and require two separate license keys.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I mean multiple arrays each with its own parity disk, so keep what ever disk limits but allow me to have 6+P, 10+P etc all in the same system. This can 'almost' be accomplished with virtualization. I.E. Two unique arrays in the same host. What would not be shared is the management and/or the user share. I.E. they would look like two separate servers and require two separate license keys. Indeed but I would like the user shares to span the arrays I don't want much do I. I just don't like the rebuild times for a 20+P disk array.... and they seem to be getting longer.
April 15, 201412 yr Multiple Arrays on the same system. (Sure this was mentioned, eg one parity + 10 disks x2) Use of a different filesystem Not sure why you would want two arrays on the same system if you only had one parity drive? I could see a case for two independent arrays each with its own parity. Having said that I would much prefer dual parity on a single array as that improves resilience against multiple failures. I mean multiple arrays each with its own parity disk, so keep what ever disk limits but allow me to have 6+P, 10+P etc all in the same system. This can 'almost' be accomplished with virtualization. I.E. Two unique arrays in the same host. What would not be shared is the management and/or the user share. I.E. they would look like two separate servers and require two separate license keys. Indeed but I would like the user shares to span the arrays I don't want much do I. I just don't like the rebuild times for a 20+P disk array.... and they seem to be getting longer. I've often wanted to do this too. I wanted to have each set of 5 drives as it's own array or each set of drives on a particular controller in it's own array.
June 15, 201412 yr But if we look closely at the ways real users have lost real data, none of them (that I can remember anyway) involve 2 drives failing at the same time. In fact SINGLE drive failures are so rare I can hardly remember one of those either. More commonly a user sees a drive is failing based on its SMART attributes and replaces / rebuilds it, a procedure that could just as easily have been done by adding a new drive and copying the data from the failing drive to the good one. So we, as a community, are asking for a feature to combat a problem that is only remotely possible at the far edge of reasonable possibility, while ignoring specific problems with easily achievable solutions that cause real data losses for real users. Google "RAID 5 dead" for a vigorous discussion of single drive failure protection versus dual drive failure protection. The short of it is that you should never use RAID5 unless you have a really small array because the chance of a second drive failure during rebuild increases with the size of the array (to be fair, a lot of the discussion is based on THEORETICAL failure rates of consumer grade drives). There's even an argument that RAID6 is becoming obsolete for the same reasons. This is the same reason why dual or even triple parity is desirable for unraid, even if it comes with downsides (you can put a big warning for people who enable it). The other item on my wishlist is SMART monitoring/notifications.
July 17, 201411 yr I would like "self preservation" features. - If temperatures on disks goes beyond set level, the disk or the whole system is shut down. Obviously with an appropriate notification or indication. - If a disk starts having errors, the data is moved to a hot-spare, and the sick disk is put on quarantine.
July 17, 201411 yr 1 - dual parity (however implemented) 2 - replacement filesystem as we're soon to exceed 16TB disks
July 17, 201411 yr 2 - replacement filesystem as we're soon to exceed 16TB disks Really?!?!? What is your definition of "soon" ?
July 17, 201411 yr 2 - replacement filesystem as we're soon to exceed 16TB disks As these are due to start arriving with v6 which is due in the next few months I think it is going to happen well before you can get 16TB disks.
Archived
This topic is now archived and is closed to further replies.