Hi,

 

I just tried to replace a drive (not faulty, but has a HPA issue preventing me from upgrading off of 4.5). The manual simply states "start the array" after replcing the drive, and the wiki and the FAQ both seem inaccurate:

 

"click the little check box under the Start button that says "I'm sure I want to do this", then click the Start button to Start the array and start the rebuild of the replaced drive"

 

The 'Im sure' was under a button called 'refresh', and checking it only enabled the refresh button not the start button. so I did the refresh, then was able to start, and was then prompted to format the drive. The format is now complete but data rebuild is not occurring.

 

Does anyone have any ideas?

Hi, thanks for responding!

 

Attached is a screenshot, I hope it helps? I do still have the old drive which I expect could be re-added to the array (and once parity ran might give me another chance at this?).

 

Disk1 is the drive that I replaced, and it is slightly larger than the one I removed (only by a few k).

It appears you formatted the new disk1 disk you installed.  In addition, you do not have valid parity.  The parity disk looks to have been taken out of service. (it shows over 300 read errors)

 

Step 0.  Post a syslog.

 

Step 1.

Get a SMART report for the parity disk.      If it is going bad, no sense re-calculating it.

 

Step 2.

Put the original disk1 back in.    Since it is smaller you will need to set a new disk configuration in order to have the array start.

This will immediately invalidate parity, but since you do not have valid parity at this time, you have nothing to lose there.  When you then start the array it will begin a new initial parity calculation.

 

Once done, you are back to where you were.  (but probably with a failing/near-failing parity disk)

 

Step 3. 

Since the parity disk has issues, replace it with the disk you were initially attempting to use to replace disk1. 

When you start the array, it will completely re-calculate parity.

 

Now, you are back to healthy parity, and disk1 with an HPA. 

 

Step 4.  Run a parity check to make sure you can read the parity you just wrote to the parity disk.

 

Wait a week..  to make sure all is stable. In that time you can consider RMA'ing the old parity disk, or purchasing an additional disk.

 

Do NOT press a Format button... ever ... unless you are adding an ADDITIONAL drive to your array.  It is NEVER used when replacing an existing disk.    Do not press the Format button if there are multiple disks showing as "unformatted" when you only added a single additional disk.  Seek assistance.

 

Hi Joe, thanks for the reply.

 

I have attached the syslog but cannot generate a SMART report for the drive. I have tried the following without success:

 

smartctl  -a  -d  ata  /dev/sdd >/boot/smart.txt

smartctl  -a  -A  /dev/sda >/boot/smart.txt

smartctl  --all /dev/hda >/boot/smart.txt

 

syslog.zip

Hi Joe, thanks for the reply.

 

I have attached the syslog but cannot generate a SMART report for the drive. I have tried the following without success:

 

smartctl  -a  -d  ata  /dev/sdd >/boot/smart.txt

smartctl  -a  -A  /dev/sda >/boot/smart.txt

smartctl  --all /dev/hda >/boot/smart.txt

The parity disk device is /dev/sdd

 

You might try

smartctl -a /dev/sdd >/boot/smart.txt

and see if it responds.  (it might not if really dead, or locked up, as there are LOTS of errors in the syslog.)  You may have to power cycle the server before it will respond again, or it might not ever respond again.  In any case, writes to it failed, so it was taken out of service. (disabled)

 

As of the time you captured the syslog, unRAID was trying again and again to spin down the parity disk... (apparently, with no success) 

 

I'd try a power cycle (power down, make sure the power and data cables to the parity disk are re-seated and secure, then power up)

 

Joe L.

Joe L.

Thanks Joe.

 

That worked, and attached is the SMART report.

 

However... Since the point of all this was to resolve HPA issues so I can upgrade, I tried hdparm on disk 2 before rebooting. It seemed to work, but upon reboot the drive is offline. I ran "hdparm -N /dev/sdf" and get "max sectors = 1953525168/1953525168 HPA disabled" which is what I would expect, so not sure what's wrong... screenshot and syslog attached.

 

(sorry, had to zip files to meet upload limits)

unraid.zip

It thinks the disk changed (since the size is different)

 

Now that you removed the HPA you need to set the size of the partition to include the extra blocks.

 

I wrote a script to help another user that does exactly that.  unraid_partition_disk.sh

It is described here:

http://lime-technology.com/forum/index.php?topic=5072.msg47122#msg47122

 

Since you are running an older version of unRAID you want your partition to start on sector 63. 

Therefore, DO NOT USE the "-A" option.

 

It will set the partition so unRAID will accept it as an unRAID partitioned disk.

 

After you use it you then need to set a new disk configuration.  On your version of unRAID this is done by using the "Restore" button (it is really a "Set New Disk Configuration" button)    it was so poorly labeled it was eventually removed entirely as too many people thought it restored data.  It immediately invalidated parity, making it impossible to recover a failed disk.

 

Since you have no parity drive that is valid, once you set the partition on disk1 so unRAID will recognize it, you can set a new configuration, then all the indicators will turn BLUE.  When you start the array, a new initial parity calc will occur.

 

Oh yes, the reiserfs on that disk where you removed the HPA will be re-sized to fill the partition.

 

Once parity is calculated, then you need to perform a parity check to make sure you can read what was written.

 

DO NOT PRESS ANY FORMAT BUTTON... 

 

Joe L. 

 

Reminder - disk1 is the new drive with no data. I still have the old disk1 and it is healthy (only pulled because of HPA)

              - disk2 is the drive I tried to remove HPA from

              - parity drive is failing and needs replacing

 

Since I seem to be in a cycle of doing more harm than good, I'm trying to break it down into idiot proof steps for myself... so, my apologies but can you please confirm that the steps are:

 

1) run "unraid_partition_disk.sh -p /dev/sdf" (to fix disk2)

2) replace current disk1 with old disk1 (adds all data back into the array, and free's up a drive for step 3)

3) replace the parity drive with new drive (which was reclaimed from the disk1 slot)

 

Will this lose the data on disk2?

 

Since Joe has a handy script to properly recover from HPA removal without losing data, I'd investigate the possibility of removing the HPA on the old disk 1 before you build parity onto your new disk.

 

Joe, it's your script, would you recommend HPA removal on all drives before building parity again?

I downloaded the zip from the link provided, and extracted unraid_partition_disk.sh to the root of my flash drive.

 

I then went to the server console and entered "unraid_partition_disk.sh -p /dev/sdf" but recieved an error "-bash unraid_partition_disk.sh: command not found"

 

Please help - I need to get my array back up!

Can I disconnect disk2 for now to safely bring the array back online (sans any data on that disk)?

Scripts need to be called with a preceding ./

 

i.e. "./unraid_partition_disk.sh -p /dev/sdf"

Thanks BetaQuasi, now I get "no such file or directory" when I run the script.

 

disk2 is definitely sdf

make sure the script is executable - "chmod +x unraid_partition_disk.sh"

Since Joe has a handy script to properly recover from HPA removal without losing data, I'd investigate the possibility of removing the HPA on the old disk 1 before you build parity onto your new disk.

 

Joe, it's your script, would you recommend HPA removal on all drives before building parity again?

 

One disk at a time is required. Remove HPA from a single disk. Rebuild disk. Parity check. Repeat.

make sure the script is executable - "chmod +x unraid_partition_disk.sh"

 

unfortunately that generates the following error:

chmod: cannot access 'unraid_partition_disk.sh': no such file or directory

What does "pwd" and "ls" show?

pwd returns "/root"

ls returns "mdcmd* powerdown@ samba@"

Enter "cd /boot" and then try again.

wow... i guess you've already figured out you're dealing with a total Linux noob... 

 

That worked (thankyou!) and the script ran and reports that the disk "IS partitioned for unraid properly". Unfortunately, the drive still appears offline, and my guess is that it's still seen as a different drive because it now reports as being 1953525105 in size (it was 1953523055 with HPA).

 

what do i do now...??

Does the reported size match the other drives? Since parity and disk2 have failed the contents of disk 2 is not recoverable and must be restored from an independent backup.

There seem to be 2x values for the size (see attached).

 

Should I be selecting the "restore" option now?

Yes.

thanks so much - array is back up.

 

not sure what's been lost yet, but my photo's seem to be ok which is the main thing! It's currently running parity, and even though I haven't yet replaced that drive, it's not yet reporting any errors. I'll let it run for now.

Since Joe has a handy script to properly recover from HPA removal without losing data, I'd investigate the possibility of removing the HPA on the old disk 1 before you build parity onto your new disk.

 

Joe, it's your script, would you recommend HPA removal on all drives before building parity again?

 

One disk at a time is required. Remove HPA from a single disk. Rebuild disk. Parity check. Repeat.

Since you have no valid parity at this time, it really does not matter.  If parity existed, then one at a time is the correct advice.

What DOES matter is that the partition size MUST be properly set with either the utility I wrote, or if on a 5.0 release, the "mkmbr" command lime-tech supplies with unRAID 5.X betas.  That must occur AFTER you remove the HPA, but before starting the array.

 

If the partition is not set properly, the disk will not be recognized as valid. (and re-partitioned AND made available for re-formatting when you start the array.)

Edit: I see you have started calculating parity.  As such, I'd wait before doing ANYTHING ELSE.  You need reliable parity to fix the next HPA using the re-construction method