unRAID Server Release 4.7 "final" Available

[Joe L.]

Rajahal,

 

Thanks for the response.  The reason I am tempted to remove the jumper on my 1 TB drive is that it is the only other advanced format drive in my box.  My thought is that as I continue to replace drives they will in all likely hood be advanced format drives, and who knows what the future will hold for performance or some other benefit.

 

Dan

It does not matter. You can have a mixed set of advanced-format and non-advanced.

 

You put your data at risk by removing the jumper and re-building the drive.  If you do it wrong, and accidentally leave an existing MBR on the disk, or by removing the jumper the original one is now visible, you'll end up with a non-4k aligned partition and WORSE performance.

 

My advice, leave a jumpered drive alone.  There is NO benefit in removing it, and substantial risk if you do.

 

Joe L.

[Rajahal]

There is NO benefit in removing it, and substantial risk if you do.

 

Kind of sounds like Pascal's Wager...nothing to gain and everything to lose

[Carpet3]

Kind of sounds like Pascal's Wager...nothing to gain and everything to lose

 

Careful

[bigup]

hi guys, im currently running unraid 4.6 and would like to upgrade to 4.7, while am at it i would like to add a WD20EARS 2Tb drive to my array to make it the parity drive and release my current 1.5Tb party drive to use for storage

 

current drives:

 

 

im a bit scared of doing this incase i lose any data

 

what are the steps for me to take?

 

thanks in advance

[BRiT]

Here is what I suggest doing...

 

1) run parity check under your existing 4.6 setup.

2) upgrade to 4.7, ensure array is started and everything works fine

3) run parity check under 4.7 setup

4) power down system

5) connect WD EARS to system

6) power up system

7) invoke preclear_disk.sh -A on your new drive, the WD EARS

If the preclear_disk.sh reports fine, then proceed to stop the array and assign WD EARS as parity drive

9) Keep the old Parity drive unassigned.

10) Start Array

11) Let Parity Calculation complete

12) Perform a Parity Check to ensure the newly calculated parity is correct.

13) If all is well, then invoke preclear_disk.sh -A on your old parity drive, the 1.5TB unit.

14) If the preclear_disk.sh reports fine on the 1.5TB drive, then proceed to stop the array and assign the drive as a data drive, perhaps disk 3.

15) Start Array

16) Perform a Parity Check to ensure system is stable with all drives assigned.

 

 

I might have missed a step or two, but I think that's the general steps to keep your system with parity protection for the longest time duration. It might seem to be a lot of steps, but this way if something goes wrong you wont be second guessing what the issue is or how to proceed with some complex steps.

 

[bigup]

Here is what I suggest doing...

 

1) run parity check under your existing 4.6 setup.

2) upgrade to 4.7, ensure array is started and everything works fine

3) run parity check under 4.7 setup

4) power down system

5) connect WD EARS to system

6) power up system

7) invoke preclear_disk.sh -A on your new drive, the WD EARS

If the preclear_disk.sh reports fine, then proceed to stop the array and assign WD EARS as parity drive

9) Keep the old Parity drive unassigned.

10) Start Array

11) Let Parity Calculation complete

12) Perform a Parity Check to ensure the newly calculated parity is correct.

13) If all is well, then invoke preclear_disk.sh -A on your old parity drive, the 1.5TB unit.

14) If the preclear_disk.sh reports fine on the 1.5TB drive, then proceed to stop the array and assign the drive as a data drive, perhaps disk 3.

15) Start Array

16) Perform a Parity Check to ensure system is stable with all drives assigned.

 

 

I might have missed a step or two, but I think that's the general steps to keep your system with parity protection for the longest time duration. It might seem to be a lot of steps, but this way if something goes wrong you wont be second guessing what the issue is or how to proceed with some complex steps.

 

 

fantastic mate, many thanks for the detailed steps, much appreciated

[goinsnoopin]

Rajahal and Joe,

 

Thanks for the additional information, I will leave the jumper on!

 

Dan

[Orbi]

Here is what I suggest doing...

 

...

13) If all is well, then invoke preclear_disk.sh -A on your old parity drive, the 1.5TB unit.

...

 

Assuming his old parity drive is non-advanced format, is it correct to preclear it with the -A option? Does it make a difference on 'old' format style hard drives?

[dgaschk]

It does not matter.

[BRiT]

For non-advanced format drives, all they require is to be aligned on a 512 Byte boundary, which sector 64 satisfies. I suggest using advanced formatting on all drives added to an unRAID 4.7 array to ease their ability to be replaced with advanced formatted drives, of course without the jumper applied to WD EARS.

[olympia]

^^^^ I am confused about that a little bit.

 

Is that means if I want to replace a non-advanced formated drive with an advanced formatted drive and let unRAID rebuild and extend it, then it won't work?

 

Or how about a non-advanced formatted EARS WD with the jumper on? If I want to replace it with an advanced formatted EARS WD without the jumper, then let unRAID rebuild it won't work?

[Joe L.]

^^^^ I am confused about that a little bit.

 

Is that means if I want to replace a non-advanced formated drive with an advanced formatted drive and let unRAID rebuild and extend it, then it won't work?

Yes, it will work.  you can replace ANY drive with ANY drive. (as long as it is big enough and not bigger than the parity drive)

Or how about a non-advanced formatted EARS WD with the jumper on? If I want to replace it with an advanced formatted EARS WD without the jumper, then let unRAID rebuild it won't work?

It too will work.  You will want to either blank out the MBR of the new drive before using it as a replacement and set the MBR-4k-align option in the unRAID settings, or pre-clear it with the "-A" option, as those will result i the best performace, but even if you do not, the drive will still function and will still be rebuilt. 

 

unRAID will use the partition it finds as it already exists, and if no valid partition exists, it will create one based on the MBR-4k-align setting.  Therefore, if you want a 4k-aligned partition, make sure there is not a current partition starting on sector 63.

 

Check with

fdisk -lu /dev/sdX

 

Joe L.

[hklt]

 

<snipped>

 

If you are currently running unRAID Server 4.6 or higher, please copy the following files from the new release to the root of your Flash device:

   bzimage

   bzroot

 

<snipped>

 

 

I'm a unix/linux n00b: If I wanted to backup the original unRAID install (to go back to in case everything goes belly up), do I just copy the older versions of those two files somewhere else? Then copy them back to "revert?"

 

[Joe L.]

yes, exactly.

[hklt]

Great, thanks as always Joe!

[lionelhutz]

^^^^ I am confused about that a little bit.

 

Is that means if I want to replace a non-advanced formated drive with an advanced formatted drive and let unRAID rebuild and extend it, then it won't work?

 

Or how about a non-advanced formatted EARS WD with the jumper on? If I want to replace it with an advanced formatted EARS WD without the jumper, then let unRAID rebuild it won't work?

 

The unRAID "advanced formatting" simply consists of moving the partition one sector so it starts at sector 64 instead of sector 63. You can switch back and forth however you want when you rebuild a drive. On a stock unRAID the "Default Partition Format" drop-down on the Settings page tells unRAID which sector to start the partition on. 4k-Aligned is sector 64. Set this variable and the new or replacement disk uses the corresponding starting sector. When you replace a disk the starting sector on the old disk does not matter and you can pick either starting sector on the new disk. Any combo of old disk and new disk starting sector is possible as follows.

old -------> new

sector 63 -> sector 63

sector 63 -> sector 64

sector 64 -> sector 63

sector 64 -> sector 64

 

 

A non-advanced EARS would not have a jumper.

 

Peter

 

[limetech]

^^^^ I am confused about that a little bit.

 

Is that means if I want to replace a non-advanced formated drive with an advanced formatted drive and let unRAID rebuild and extend it, then it won't work?

 

Or how about a non-advanced formatted EARS WD with the jumper on? If I want to replace it with an advanced formatted EARS WD without the jumper, then let unRAID rebuild it won't work?

 

The unRAID "advanced formatting" simply consists of moving the partition one sector so it starts at sector 64 instead of sector 63. You can switch back and forth however you want when you rebuild a drive. On a stock unRAID the "Default Partition Format" drop-down on the Settings page tells unRAID which sector to start the partition on. 4k-Aligned is sector 64. Set this variable and the new or replacement disk uses the corresponding starting sector. When you replace a disk the starting sector on the old disk does not matter and you can pick either starting sector on the new disk. Any combo of old disk and new disk starting sector is possible as follows.

old -------> new

sector 63 -> sector 63

sector 63 -> sector 64

sector 64 -> sector 63

sector 64 -> sector 64

 

 

A non-advanced EARS would not have a jumper.

 

Peter

 

 

This is true but be aware of one other detail.  Any drive you plug into an unRAID server that already has a valid 'unRaid-style' MBR (Master Boot Record), will never have it's MBR re-written by unRaid OS.

 

So what is a valid unRaid-style MBR?  It is an MBR that has been written by unRaid OS sometime in the past, or is created using Joe L.'s excellent pre-clear script.

 

Of course you can log into the server and wipe out the MBR using the appropriate 'dd' command as has been documented earlier in this thread.  In this case when unRaid next "sees" the disk, it will no longer have a valid unRaid-style MBR & hence will be created according to setting of "Default partition format".

 

For completeness:

- if you plug in a disk that has been in another system, maybe Windows, maybe another variety of linux, chances are, it will not have a valid unRaid-style MBR, but I suppose it's possible (esp. if it's coming from another linux OS where you ran 'fdisk' and created exactly one partition that spanned the entire drive - but I have not looked at this).

- if you plug in a brand new disk straight from the factory it will typically come to you completely cleared (though no spec says that's how it should be), and hence also no valid unRaid-style MBR.

[ClawSS]

Ok I think I got it...finally.  I have a new 2TB EARS on the way.  I have 2 x @TB EARS that are not MBR 4k aligned and they are without jumpers running in the array holding huge files.

 

---first thing is verify parity---

1)  Stop Array

2)  Shutdown server

3)  Note new serial number and Insert new drive into system

4)  Power up server

5)  Run preclear_disk.sh -A on the new drive, a few hours later if all is good then

6)  stop array

7)  shut down server and remove 1 of the existing 2TB drives

  replace with newly precleared drive

9)  reboot server

10)  stop array  (if it restarted)

11)  assign new drive to the disk position removed using new serial number as guide (should say it was missing)

12)  start array and format to MBR 4k aligned default

13)  recover from parity (unRAID should ask for confirmation)

14)  run preclear_disk.sh -A on "old" drive to clear off old MBR (I think this is all I have to do is run this script to fix)

15)  repeat steps 6-14

16)  add last drive to pool following preclear_disk.sh -A

 

 

Maybe...please tell me what I missed?

 

Thanks

[goinsnoopin]

I just upgraded to 4.7 and have a problem.  Disk 6 shows up with a red dot, says replacement disk is too small and therefore the array does not start.  The odd think is the size is off by 4.  Please note this is a regular sata drive and not an advanced format drive.  I have attached the syslog and screenshot.  I reverted to 4.6 and disk 6 has a green dot, the array starts and everything works fine.

 

Does anyone have any ideas on how to proceed?  I want to get to 4.7 to install a new 2TB parity drive.

 

Thanks,

 

Dan

syslog-2011-01-28.txt

[Joe L.]

I just upgraded to 4.7 and have a problem.  Disk 6 shows up with a red dot, says replacement disk is too small and therefore the array does not start.  The odd think is the size is off by 4.  Please note this is a regular sata drive and not an advanced format drive.  I have attached the syslog and screenshot.   I reverted to 4.6 and disk 6 has a green dot, the array starts and everything works fine.

 

Does anyone have any ideas on how to proceed?  I want to get to 4.7 to install a new 2TB parity drive.

 

Thanks,

 

Dan

Your disk has an HPA (Host protected area)  You probably have a Gigabyte motherboard, or the disk was used with one in the past.

 

Jan 28 22:02:17 Tower kernel: ata2.00: HPA detected: current 625140335, native 625142448

Jan 28 22:02:17 Tower kernel: ata2.00: ATA-7: ST3320620AS, 3.AAC, max UDMA/133

Jan 28 22:02:17 Tower kernel: ata2.00: 625140335 sectors, multi 16: LBA48 NCQ (depth 31/32)

Jan 28 22:02:17 Tower kernel: ata2.00: configured for UDMA/100

Jan 28 22:02:17 Tower kernel: scsi 2:0:0:0: Direct-Access    ATA      ST3320620AS      3.AA PQ: 0 ANSI: 5

Jan 28 22:02:17 Tower kernel: sd 2:0:0:0: [sdb] 625140335 512-byte logical blocks: (320 GB/298 GiB)

Jan 28 22:02:17 Tower kernel: sd 2:0:0:0: [sdb] Write Protect is off

Jan 28 22:02:17 Tower kernel: sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00

Jan 28 22:02:17 Tower kernel: sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA

 

You need to disable the BIOS feature that creates the HPA and then remove it.

 

User with similar issue here: http://lime-technology.com/forum/index.php?topic=10483.msg99685#msg99685

Basic instructions to correct it here: http://lime-technology.com/forum/index.php?topic=10483.msg99686#msg99686

and in the few posts following.

 

In your case it is /dev/sdb

 

Perform a parity check. If there are any errors resolve them before continuing.

Then... Disable the "feature" in the BIOS that saves a copy of itself to the disk.

Stop the array, run the hdparm command to set the HPA to the full natural size of the disk.

Then the disk will re-construct onto itself when you next start the array. Because the size changes it will think you've replaced the disk with a larger one.

 

Joe L.

[goinsnoopin]

Joe,

 

I am using the unofficial supermicro  C2SEE motherboard...I didn't think that had the HPA issues like the gigabyte boards have.  The only gigabyte board I have ever owned is in my current desktop... I can not remember it this drive was ever installed on it.  I just did a parity check before the attempted 4.7 upgrade and I had no errors.  Please confirm that the HPA issue can not come from the supermicro board...if this is the case can I just run the hdparm command?

 

Thanks,

 

Dan

[Joe L.]

You must have had the drive on the other motherboard.  It cannot come from the supermicro.

 

Go ahead, run the hdparm command ( on /dev/sdb )

 

Stop the array first

Then run

hdparm -N p625142448 /dev/sdb

You can see if it took with

hdparm -N /dev/sdb

Then, start the array and let it rebuild the disk onto itself.

 

Joe L.

[RobJ]

Tom may want to have a look at this, because I don't believe he intended this to be a problem.  The drive size calculation has been changed between v4.6 and v4.7, certainly because of the added support for 4k drives (Advanced Format), and as you can see from goinsnoopin's drive, is off by 4K, one AF sector.  This drive was detected as a Seagate 320GB drive:

kernel: ata2.00: HPA detected: current 625140335, native 625142448

 

However, the v4.7 unRAID web page shows the drive size as 312570132, but claims it should be 312570136, which is probably what v4.6 calculated it as.  I strongly suspect that Tom did not intend this to be a problem for so many.  And I do believe there are quite a few users who have this problem, many like goinsnoopin who don't even know it yet, until they try to upgrade.  Is it possible that the calculation can be tweaked in a v4.7.1, to avoid unnecessarily requiring this repair, for those with the Gigabyte HPA?  It is easy to detect this exact situation, by detecting the native vs current sector count difference of 2113.  I think I just heard Tom groaning!

[BRiT]

I only ask one thing, can we please get unRAID 5.0 beta 3 out the door first before any possible 4.7.1?

[olympia]

Thank you for the responses guys. I thought the same, but got confused by BRiT words. Probably I've just misread it