January 10, 201511 yr Hi guys, I thought I had done my research prior to what I'm about to explain, but I might have done something out of order. Here goes: 1) My disk2 had been getting errors in the webGUI and was marked FAILED as SMART overall health. My parity and disk2 were both 2TB drives. Parity has had zero errors since the beginning. 2) I bought new WD 3TB drives with the intention of increasing space since I was replacing drives. 3) I precleared the first 3TB drive, stopped the array, and assigned the 3TB to parity. 4) Parity was re-built 5) This morning, I started the parity check. The check is currently at about 25% and shows that 1700+ errors have been corrected. This is concerning to me as I'm not sure if what is being corrected is "right" since disk2 already had errors. My question is: -- Did I screw up by replacing the parity drive BEFORE replacing disk2? If that's the case, am I required to buy a 2TB drive to replace it since my parity is 2TB? I currently have the original 2TB parity disk unassigned and the array has not been written to during this entire process. Thanks!!
January 10, 201511 yr -- Did I screw up by replacing the parity drive BEFORE replacing disk2? Yes -- once parity was rebuilt you lost the ability to rebuild disk #2 If that's the case, am I required to buy a 2TB drive to replace it since my parity is 2TB? No, you COULD have done a "swap-disable" ... which would have rebuilt parity onto the 3TB drive; then done a rebuild of the failed disk #2 onto the old parity drive. But that's not possible now UNLESS a couple of things are true (read below) ... I currently have the original 2TB parity disk unassigned and the array has not been written to during this entire process. This MAY be your saving grace. Did you by any chance backup the flash drive contents BEFORE you started this process? If so, you can restore the original flash drive contents; replace the original parity drive; boot the system (to confirm all looks well); and then shut it down and do a "swap-disable" with the new 3TB drive. Here's the instructions for a swap-disable from the Wiki: ------------------------------------------------------------------------------------------------------------ You must replace a failed disk with a disk which is as big or bigger than the original and not bigger than the parity disk. If the replacement disk is larger than your parity disk, then the system permits a special configuration change called swap-disable. For swap-disable, you use your existing parity disk to replace the failed disk, and you install your new big disk as the parity disk: 1. Stop the array. 2. Power down the unit. 3. Replace the parity hard disk with a new bigger one. 4. Replace the failed hard disk with you old parity disk. 5. Power up the unit. 6. Start the array. When you start the array, the system will first copy the parity information to the new parity disk, and then reconstruct the contents of the failed disk. ------------------------------------------------------------------------------------------------------------ If you didn't back up the flash drive, there MAY still be a way. If you're running v5, you can do a New Config with the "Trust Parity" option using the OLD parity drive; then I THINK you can immediately do a swap-disable. I'm not certain of that, however, as the system may want to do a parity check when you first start it ... but if that's the case you can probably immediately abort that and then do the swap-disable. Probably the best alternative at this point is, if you have good backups, to just do a New Config with the new parity drive and without the failed drive (perhaps adding the old parity drive as a data drive) ... and then just copying the data that was on the failed drive to the array from your backups.
January 12, 201511 yr Author Ok, I'm hoping that there's a way to recover this mess as I screwed up and did not do a backup... I am running v5.0. I don't fully understand why a backup of the config is needed. Shouldn't replacing the old parity drive put the system exactly as it was? Or is it not that simple? Based on the lack of a backup and that I'm running v5.0, is my only approach the method you mentioned here: If you didn't back up the flash drive, there MAY still be a way. If you're running v5, you can do a New Config with the "Trust Parity" option using the OLD parity drive; then I THINK you can immediately do a swap-disable. I'm not certain of that, however, as the system may want to do a parity check when you first start it ... but if that's the case you can probably immediately abort that and then do the swap-disable. I really appreciate all the help. I will post the syslog tonight when I'm home after work. Thanks, Dave
January 12, 201511 yr Ok, I'm hoping that there's a way to recover this mess as I screwed up and did not do a backup... I am running v5.0. I don't fully understand why a backup of the config is needed. Shouldn't replacing the old parity drive put the system exactly as it was? Or is it not that simple? Based on the lack of a backup and that I'm running v5.0, is my only approach the method you mentioned here: If you didn't back up the flash drive, there MAY still be a way. If you're running v5, you can do a New Config with the "Trust Parity" option using the OLD parity drive; then I THINK you can immediately do a swap-disable. I'm not certain of that, however, as the system may want to do a parity check when you first start it ... but if that's the case you can probably immediately abort that and then do the swap-disable. I really appreciate all the help. I will post the syslog tonight when I'm home after work. Thanks, Dave When you put in the new parity drive and started the array, unRAID updated your array configuration. That configuration is stored in /boot/config/super.dat. Now, if you try to put the old smaller parity drive back in unRAID will not let you because it would consider that a replacement and the replacement has to be at least as large as the drive it is replacing. The only way to get it to accept that old drive is new config, or using a backup of super.dat that was taken when the old drive was still in place. I always make a backup of my flash drive whenever I make any changes to the array or unRAID settings. It's best to do that backup with the array stopped, since super.dat also keeps track of whether the array is started or not. If you try to boot and super.dat says the array was already started, unRAID will assume an unsafe shutdown and start a correcting parity check. If you export your flash share, you can access the flash contents over the network even with the array stopped, and just copy it somewhere on your pc.
January 12, 201511 yr Trurl explained it nicely -- as soon as you ran the array with changes, your old config was gone, so you can't simply restore the old parity drive. What you MAY be able to do is what I noted earlier ... a New Config with the old parity drive using the "Trust Parity" option. I'm simply not sure if you'll still be able to do a swap-disable immediately after that; or if it will just start a parity check when you first start the array. Most likely you simply need to recover what you can from the failed drives -- and hopefully learn a lesson about maintaining good backups of your data.
January 12, 201511 yr Author I've done some searching on the "trust parity" option and there are posts saying that it does not work on v5.0 and later. Is that true? Most of the threads I found are 2+ years old...
January 12, 201511 yr Author Also, as a precaution, should I copy as much data as possible off of disk2 to an external drive?
January 12, 201511 yr I've done some searching on the "trust parity" option and there are posts saying that it does not work on v5.0 and later. Is that true? Most of the threads I found are 2+ years old... Actually it was only implemented as of v5 => there was no "Trust Parity" option in v4.7 or earlier.
January 12, 201511 yr Also, as a precaution, should I copy as much data as possible off of disk2 to an external drive? Certainly not a bad idea -- it never hurts to have backups of your data [i have complete backups of all 3 of my UnRAID arrays].
January 12, 201511 yr Author There should be away to recover using mdcmd. Ask lime tech. Thanks for the suggestion, I just sent an email to support.
January 13, 201511 yr Author Haven't heard from support yet, hoping they get back to me soon. In the meantime, I will be copying all the data off of disk2 onto an external drive. After that completes, I want to perform the New Config procedure. Can you guys please look over the steps I have outlined below and make sure they are correct? 1. Stop array (already complete) 2. Unassign new 3TB drive from Parity slot. 3. Re-assign old Parity drive 4. Navigate to Utils/New Config 5. Apply the New Config 6. Navigate back to the main page 7. Check "Parity is correct" box 8. Start the array 9. For step 9 and thereafter, what should I do? Or is step 8 the last? Thanks so much again!!! Dave
January 13, 201511 yr You don't "Apply the New Config" ... when you select New Config, it wipes out all configuration info; then you re-assign all of your disks. So the steps you listed are not correct. Basically you simply (1) Select New Config; (2) Assign all of your disks as desired (i.e. using the old parity drive as parity and all of the other disks in their current slots -- be sure to note the current assignments before you start this) AND check the "Trust Parity" box; and (3) Start the array. Note: IF a parity check is initiated when you start the array, IMMEDIATELY abort it. Next, you Stop the array, unassign the failed disk, and Start the array so it notices it's missing. Then Stop the array and then follow the steps for a swap-disable to rebuild the failed disk and upgrade your parity to the new, larger parity disk.
January 14, 201511 yr Author Ok, thanks. Now I understand. Here's my plan for tonight: 1. Stop array (already complete) 2. Write down current drive assignments. 3. Navigate to Utils 4. Click on New Config 5. Navigate back to the main page 6. Re-assign all disks in correct spots (including old parity drive) 7. Check "Trust Parity" box 8. Start the array 9. If necessary, immediately stop Parity check Then, perform the swap-disable: 10. Stop the array 11. Unassign disk2 12. Start the array 13. Confirm disk2 is recognized as missing 14. Stop the array **Unclear here - do I need to shut the machine down? Or can I: 15. Assign new 3TB to parity and assign old parity to disk2 spot 16. Start the array and let unRAID copy parity over to the new drive then rebuild disk2 One final question I have: last time my parity check ran (January 1st) with the old parity drive, there were zero errors. Would this lend me to want to run a parity check after step 8? My understanding is that the array will be in the exact same state it was back on January 1st. Or is not worth the risk? Thanks, Dave EDIT: Posting this link here for reference: http://lime-technology.com/forum/index.php?topic=29529.0
January 14, 201511 yr One final question I have: last time my parity check ran (January 1st) with the old parity drive, there were zero errors. Would this lend me to want to run a parity check after step 8? My understanding is that the array will be in the exact same state it was back on January 1st. Or is not worth the risk? You do NOT want to run a parity check if you are intending a rebuild. If you run a correcting parity check then a disk playing up could cause bad parity to be generated which mean a rebuild would not produce the desired results. A non-correcting check is not worth running with a problem disk as it would almost certainly produce errors due to the problem disk.
January 14, 201511 yr As I noted (and itimpi also said), you do NOT want to run a parity check. This could "correct" bad data ... and would make a good rebuild impossible. What I'm not certain of is whether or not the array will automatically start one when you first start it after a New Config with "Trust Parity". I don't think it will ... but if it happens to, just IMMEDIATELY click on the button to stop it. Hopefully it won't have "fixed" anything in the couple seconds it ran -- if it did, there's nothing you can do about it, so just go ahead with the Swap-Disable.
January 14, 201511 yr Author One final question I have: last time my parity check ran (January 1st) with the old parity drive, there were zero errors. Would this lend me to want to run a parity check after step 8? My understanding is that the array will be in the exact same state it was back on January 1st. Or is not worth the risk? You do NOT want to run a parity check if you are intending a rebuild. If you run a correcting parity check then a disk playing up could cause bad parity to be generated which mean a rebuild would not produce the desired results. A non-correcting check is not worth running with a problem disk as it would almost certainly produce errors due to the problem disk. As I noted (and itimpi also said), you do NOT want to run a parity check. This could "correct" bad data ... and would make a good rebuild impossible. What I'm not certain of is whether or not the array will automatically start one when you first start it after a New Config with "Trust Parity". I don't think it will ... but if it happens to, just IMMEDIATELY click on the button to stop it. Hopefully it won't have "fixed" anything in the couple seconds it ran -- if it did, there's nothing you can do about it, so just go ahead with the Swap-Disable. Thanks you guys. I'll report back tonight or tomorrow morning with my results.
January 14, 201511 yr ... 14. Stop the array **Unclear here - do I need to shut the machine down?... No, you can simply proceed as you outlined ... ... 15. Assign new 3TB to parity and assign old parity to disk2 spot 16. Start the array and let unRAID copy parity over to the new drive then rebuild disk2 ...
January 14, 201511 yr Author Thanks for clarifying! Here's my final plan for tonight: 1. Stop array (already complete) 2. Write down current drive assignments. 3. Navigate to Utils 4. Click on New Config 5. Navigate back to the main page 6. Re-assign all disks in correct spots (including old parity drive) 7. Check "Trust Parity" box 8. Start the array 9. If necessary, immediately stop Parity check Then, perform the swap-disable: 10. Stop the array 11. Unassign disk2 12. Start the array 13. Confirm disk2 is recognized as missing 14. Stop the array 15. Assign new 3TB to parity and assign old parity to disk2 spot 16. Start the array and let unRAID copy parity over to the new drive then rebuild disk2
January 14, 201511 yr That should do the trick. The only real question is whether anything got inadvertently changed during the earlier attempts you've made to resolve this. Hopefully not -- in which case all should be well when it completes (sometime tomorrow).
January 15, 201511 yr Author Ok, everything went fine up until the last step! I've attached my current configuration. As you can see, I have the option to check the box then Start the array. However, it does not do anything. I think it is because it says "Wrong" right next to my parity drive. Any ideas? Thanks!!!
January 15, 201511 yr I have no idea why it won't proceed -- it's clearly giving you the option to do so, which is how the Swap-Disable is supposed to work. If you check the "Yes I want to do this" box, do you then get the option to Start the array? And if you then click on Start does ANYTHING happen?
January 15, 201511 yr Author I have no idea why it won't proceed -- it's clearly giving you the option to do so, which is how the Swap-Disable is supposed to work. If you check the "Yes I want to do this" box, do you then get the option to Start the array? And if you then click on Start does ANYTHING happen? Yep, when I check the "Yes..." box the Start button becomes active. When I click it, the page refreshes and nothing changes. I've attached a snippet of the log that contains me clicking the Start button. You can see in there that there is "Jan 14 21:51:19 Tower emhttp: swap_dsbl copy not complete" log.txt
January 15, 201511 yr I simply don't know why it won't proceed. There have been some previous notes about Swap-Disable not working correctly in the new versions, but others have said it did, so I assumed it was simply a matter of getting the process right. But you've clearly done that -- as evidenced by the box that SAYS it's going to do it once you check the box. I'd send Tom a PM to see if he can help get this working for you.
Archived
This topic is now archived and is closed to further replies.