unRAID Server Release 4.7 "final" Available

[prostuff1]

Images and stuff

 

Look back through this thread for instructions on how to fix the issue you are seeing and why it is happening.

 

 

The short explanation is that you have HPA on three of your drives.

[MartinQ]

Look back through this thread for instructions on how to fix the issue you are seeing and why it is happening.

 

The short explanation is that you have HPA on three of your drives.

 

Shouldn't those instructions be included in the first post under the heading,  'Upgrading an existing Flash Device' ?

 

I might try 4.7 again after the drive-upgrade is finished.  There's no significant reason why I need native advanced-format support right now.

[prostuff1]

Shouldn't those instructions be included in the first post under the heading,  'Upgrading an existing Flash Device' ?

 

Maybe... but not really

 

The crux of your problem is your motherboard.  The problem only happens on some boards and LimeTech can't know everyone's configuration.

[davephan]

I am rebuilding my unRAID server, replacing a Gigabyte GA-EX580UD5 system board with a Supermicro MBD-X8SIL-F-O. HPA cannot be disabled in the Gigabyte system board. I moved all my data off my unRAID server to Windows formatted drives. Eventually all my drives are being removed from the old unRAID server and formatted with Windows, using a computer that does not have a Gigabyte system board. If HPA existed on any of the drives, does the Windows system formatting remove the HPA?

 

I was planning on running the preclear_disk.sh with the -A option when rebuilding the new Supermicro unRAID server for MBR-4K aligned drives. The old runRAID server was using version 4.5.6. I plan to build the new unRAID server with 4.7. My 2 TB WD Advanced Format drives all had the jumper installed. My plan is to not use the jumper and use the -A switch during the preclearing process. The MBR-4K aligned drives will not work in unRAID versions below version 4.7.

 

It has been mentioned before that old AF drives should have the jumpers left in and be used as MBR-4K non-aligned drives. One advantage to leaving the drives MBR-4K non-aligned is that you could go back to the older unRAID versions. Is there an upside to using the -A option in the preclearing process to have MBR-4K aligned drives? If there isn't an upside to using the MBR-4K aligned drives, then I'm not sure why anyone would switch to using MBR-4K aligned drives, even with new drives. Maybe there is a performance reason, or some other reason. Since I'm going through the whole rebuild process, I plan to build all the drives as MBR-4K aligned drives, one drive at a time, as I move all the data back from the Windows formatted drives over the next several weeks.

 

Dave

[dgaschk]

Windows formatting does not remove HPA. See here: http://lime-technology.com/forum/index.php?topic=10866.0

 

EARS with jumpers are the only drives that should be MBR-Unaligned. All other drives including EARS without jumpers should be MBR 4K Aligned. There is no benefit to removing the jumpers from your EARS drives. AF drives perform better with MBR 4K Aligned. Non-AF drives could be formatted either way but it is easiest to format all new drives with MBR 4K Aligned.

[Joe L.]

I am rebuilding my unRAID server, replacing a Gigabyte GA-EX580UD5 system board with a Supermicro MBD-X8SIL-F-O. HPA cannot be disabled in the Gigabyte system board. I moved all my data off my unRAID server to Windows formatted drives. Eventually all my drives are being removed from the old unRAID server and formatted with Windows, using a computer that does not have a Gigabyte system board. If HPA existed on any of the drives, does the Windows system formatting remove the HPA?

No, it will not.  You must use one of the utility programs such as Seatools to remove the HPA.

I was planning on running the preclear_disk.sh with the -A option when rebuilding the new Supermicro unRAID server for MBR-4K aligned drives. The old runRAID server was using version 4.5.6. I plan to build the new unRAID server with 4.7. My 2 TB WD Advanced Format drives all had the jumper installed. My plan is to not use the jumper and use the -A switch during the preclearing process. The MBR-4K aligned drives will not work in unRAID versions below version 4.7.

They would not be recognized as pre-cleared and would be cleared by unRAID itself again if you were if put them in a pre 4.7 version of unRAID.

It has been mentioned before that old AF drives should have the jumpers left in and be used as MBR-4K non-aligned drives. One advantage to leaving the drives MBR-4K non-aligned is that you could go back to the older unRAID versions. Is there an upside to using the -A option in the preclearing process to have MBR-4K aligned drives? If there isn't an upside to using the MBR-4K aligned drives, then I'm not sure why anyone would switch to using MBR-4K aligned drives, even with new drives. Maybe there is a performance reason, or some other reason. Since I'm going through the whole rebuild process, I plan to build all the drives as MBR-4K aligned drives, one drive at a time, as I move all the data back from the Windows formatted drives over the next several weeks.

 

Dave

There is only an advantage for those drive that do not have a "jumper" feature that makes them comparable.

[davephan]

Thanks for the responses. Will the Seagate tool work with Western Digital WD20EADS and WD20EARS drives? Could you post a link to the utility?

 

Dave

[SSD]

There is only an advantage for those drive that do not have a "jumper" feature that makes them comparable.

 

I think I speak for all the moderators in suggesting that users follow these simple rules:

 

1.  All users using unRAID 4.7 and 5.3+, make sure your default alignment is set to 4k aligned (preclear AND unRAID use this as their default)

 

2.  If you have an already running jumpered EARS, don't do anything.  The alignment is already taken care of.

 

3.  If you get a new EARS, don't install a jumper on it.

[Joe L.]

i think 4.7 uses un-aligned as its default.  5.0Beta4 uses 4k aligned as its default.

 

The preclear script uses the setting you elected in the unRAID settings as its default.

 

 

[Hex]

 

With all your experience, you'll understand nobody on these forums work for lime-tech other than "limetech"   Good luck with your server.

 

Joe L.

 

Running 4.7 at last. When 2TB drives are $80, you just replace them. Took about 16 hrs to rebuild. I will attach the hpa drive to my pc and fix it from there and then add it back again. Thanks for your help.

 

Lime, y'know when the bootloader loads it asks for your selection for unraid or memtest, how about putting a program there to check and remove any HPA issues?  Sounds like a winner to me!

 

Hex

 

[BRiT]

Limetech already includes a set of programs capable of detecting and removing HPA in unRAID. It's called the Linux kernel. It logs to your system log /var/log/syslog. The program suitable to remove HPA is hdparm.

 

Also it's not that simple. All programs capable of removing HPA require user interaction.

[prostuff1]

Also it's not that simple. All programs capable of removing HPA require user interaction.

 

Not to mention that if HPA exists on more than one disk and HPA is removed from every disk all at once it will screw up unRAID.

[RobJ]

Lime, y'know when the bootloader loads it asks for your selection for unraid or memtest, how about putting a program there to check and remove any HPA issues?  Sounds like a winner to me!

 

That would be really nice, but I'm afraid there are currently no suitable tools that could be added to the boot menu.  Even the hdparm (v9.27) that loads with v4.7 has not proven to be reliable, and apparently can actually mess up certain drives, if you try to use it to remove an HPA.  Plus it requires a running Linux system, which you don't have at the boot menu.  The other tools require either moving the drive to another machine, or installing a floppy or CD drive in your server.

 

I could be wrong, but it is very possible that Tom at Limetech has never seen a Gigabyte HPA.  I have never seen any mention from him of ever using Gigabyte boards, especially in the systems he builds and supports.  I'm quite sure he did not know there was an HPA problem when he released v4.7.  It was discovered by other users installing v4.7, users with past or present Gigabyte boards.

 

I'm hoping that someone can confirm that a more recent hdparm (v9.37?) can reliably remove the HPA, needs to be tested on multiple drive models.

 

Edit:  I've never seen an HPA myself, just in the syslogs of other users.  Doesn't make it any less of a problem, but not one that Tom could have expected/predicted/dealt with ahead of time.

[avilada]

 

I'm hoping that someone can confirm that a more recent hdparm (v9.37?) can reliably remove the HPA, needs to be tested on multiple drive models.

 

Edit:  I've never seen an HPA myself, just in the syslogs of other users.  Doesn't make it any less of a problem, but not one that Tom could have expected/predicted/dealt with ahead of time.

 

me too.... I tryed to remove the HPA issue in two of my HD when i discovered (thanks to the post) in my syslog 2 lines warning me about HPA.

 

from UNraid 4.6 I removed HPA errors from Logs using the hdparm -NXXXXXXX /dev/sdX . It seemed to work OK (4.6) after REbuilding

 

Now I dont HAve any line in my syslog "talking" about HPA but when I tray to Upgrade to 4.7  both HD still says  about different size.

 

when i typed hdparm -N  it said HPA setting seems invalid. Should i Try to remove (Again) HPA from 4.7 version???

 

any Idea?

 

 

[Joe L.]

 

I'm hoping that someone can confirm that a more recent hdparm (v9.37?) can reliably remove the HPA, needs to be tested on multiple drive models.

 

Edit:  I've never seen an HPA myself, just in the syslogs of other users.  Doesn't make it any less of a problem, but not one that Tom could have expected/predicted/dealt with ahead of time.

 

me too.... I tryed to remove the HPA issue in two of my HD when i discovered (thanks to the post) in my syslog 2 lines warning me about HPA.

 

from UNraid 4.6 I removed HPA errors from Logs using the hdparm -NXXXXXXX /dev/sdX . It seemed to work OK (4.6) after REbuilding

 

Now I dont HAve any line in my syslog "talking" about HPA but when I tray to Upgrade to 4.7  both HD still says  about different size.

 

when i typed hdparm -N  it said HPA setting seems invalid. Should i Try to remove (Again) HPA from 4.7 version???

 

any Idea?

 

 

The "seems invalid" is usually a  buggy kernel driver.  Most can be ignored.

What do you see returned by hdparm -N ?

 

 

[avilada]

root@Tower:~# hdparm -N /dev/sda

 

/dev/sda:

max sectors  = 2930277168/11041584(18446744072344861488?), HPA setting seems invalid (buggy kernel device driver?)

root@Tower:~# hdparm -N /dev/sde

 

/dev/sde:

max sectors  = 2930277168/11041584(18446744072344861488?), HPA setting seems invalid (buggy kernel device driver?)

 

 

As you can see I get the same values in 2 disks. one of them never have had any HPA  the other was changed for me with the hdparm -nXXXXXXX  and both are working flawlessly in 4.6 but when i tryed to upgrade to 4.7 (sda) says about different size , just as if i never had changed the hdparm value. I thought it could be the version of hdparm . i used the 4.6 one.

 

any tool to force size . i was thinking about the seatools but i was trying not to get my HD out of my unraid tower.

 

Thank you.

 

 

[Joe L.]

Your disks do not have an actual HPA.

 

Do not attempt to change it.

 

If there was one, the apparent size would not end with 168.

 

Joe L.

[avilada]

When I try to upgrade to 4.7 the different size still happens.....

 

 

mmmm  then maybe  the problem is on the flash  ( I have 2  pro ) is it possible then any info about sizes (out of date) was  messing it up?

I always upgrade using another flash to avoid  dont be able to go back to a "secure status". Is any info about sizes on any config file on the flash?? maybe when i copyed the config folder the info about sizes was wrong. is it possible?

 

when i finish to check parity i'll try to copy a new unraid flash and try to boot from it.

 

 

I'll report results

 

thank you

 

 

[Joe L.]

When I try to upgrade to 4.7 the different size still happens.....

 

 

mmmm  then maybe  the problem is on the flash  ( I have 2  pro ) is it possible then any info about sizes (out of date) was  messing it up?

I always upgrade using another flash to avoid  dont be able to go back to a "secure status". Is any info about sizes on any config file on the flash?? maybe when i copyed the config folder the info about sizes was wrong. is it possible?

 

when i finish to check parity i'll try to copy a new unraid flash and try to boot from it.

 

 

I'll report results

 

thank you

 

 

You'll not be able to upgrade to 4.7 until a patched version is available.  For you, it is a math issue in 4.7.

 

Send an e-mail to lime-tech.  Let them know you are waiting to upgrade and cannot.    I think the fix for the math is now in 5.0beta5a, but it has other issues, so don't go running to upgrade to it.

[avilada]

finally i got it.

 

After seeing you so sure about my HPA was OK. you made me think that my flash usb could be the problem.

 

I made a copy of my flash ( with the config folder) before fixing HPA issue in a new 4.7 flash usb .

 

I checked every .cfg file looking for a reference of my unfixed HD sizes but i guessed it could be in a binary file.

 

When i booted from this pre-fixed ( config folder with not up to date information about sizes) usb flash(4.7) with my HDs already fixed . unraid showed me the same "different size HD" error . but my HD's were OK....( thanks Joe you made me confident about my HD's)

 

So I re-copy the updated( fixed) /config in my new 4.7 usb flash  and the different size error fixed it

 

 

I'll wait over a month working Ok before activate 4k - aligment mode

 

Thanks Joe

[Joe L.]

finally i got it.

 

After seeing you so sure about my HPA was OK. you made me think that my flash usb could be the problem.

 

I made a copy of my flash ( with the config folder) before fixing HPA issue in a new 4.7 flash usb .

 

I checked every .cfg file looking for a reference of my unfixed HD sizes but i guessed it could be in a binary file.

 

When i booted from this pre-fixed ( config folder with not up to date information about sizes) usb flash(4.7) with my HDs already fixed . unraid showed me the same "different size HD" error . but my HD's were OK....( thanks Joe you made me confident about my HD's)

 

So I re-copy the updated( fixed) /config in my new 4.7 usb flash  and the different size error fixed it

 

 

I'll wait over a month working Ok before activate 4k - aligment mode

 

Thanks Joe

You can change the 4k-alignment setting at any time since it is only used when partitioning newly added disks and only then if they are not already partitioned or pre-cleared.  It has absolutely no effect on any disks in your array.

 

Best to wait a month (at least) before upgrading to the 5.? series.  Way too un-stable for production use.  There were 4 or 5 releases today, each with a different critical bug requiring another immediate release.

[avilada]

I know I can change aligment at any time, but when i'll change it i wanna change one "odd" disk preclearing with -A .

 

(In the begining this disk used jumper but later i remove it and  I would like to see if any corruption will show up).

 

 

So I'll wait for a month to enjoy my new version  

 

 

And yes  I'm looking forward to use the 5 series. but till it was stable i preffer to wait.... too much data to risk

 

Regards.

[Flomaster]

to run Joe.L 's preclear script or not?

 

Big changes for my system kinda sorta.

bought the plus license

I upgraded from 4.5.x to 4.7 all was fine and dandy...

added a 250GB cache drive

added another WD20EARS  drive.

 

I arranged the Parity, Disk1, and Disk2 in the proper order. all green lights. the cache drive is green but I'd assume it needs formatting. it has a NTFS partition on it.

 

but the new WD20EARS was blue dot. I changed the setting to enable the 4K sector, I clicked the little box and then started. that drive is now in the "clearing" process and is at 5%.

 

is this the same clearing processes as I used with my other 3 drives way back when I had to manually invoke the pre-clear script?

 

or should I cancel the clearing and do it like did with my other drives?

 

 

-=jason=-

[Joe L.]

to run Joe.L 's preclear script or not?

 

Big changes for my system kinda sorta.

bought the plus license

I upgraded from 4.5.x to 4.7 all was fine and dandy...

added a 250GB cache drive

added another WD20EARS  drive.

 

I arranged the Parity, Disk1, and Disk2 in the proper order. all green lights. the cache drive is green but I'd assume it needs formatting. it has a NTFS partition on it.

 

but the new WD20EARS was blue dot. I changed the setting to enable the 4K sector, I clicked the little box and then started. that drive is now in the "clearing" process and is at 5%.

 

is this the same clearing processes as I used with my other 3 drives way back when I had to manually invoke the pre-clear script?

 

or should I cancel the clearing and do it like did with my other drives?

 

 

-=jason=-

It is not identical to the pre-clear script in that there are no read phases and no verification that the the writing of zeros is subsequently readable.

 

It is a "write" of zeros to the drives.  It takes about 1/4 of the time because of that.  (it takes about the same time as the "write" phase of the preclear script.)

[Flomaster]

Doh... well I guess I'll just un-assign that drive and run the preclear script on it.

 

-=Jason=-