Description:

unRAID does not warn users of drives that have HPA partitions when present, and allows the creation of arrays with these drives.
This could lead to data lose as it interferes with the parity drive if it has one, nor does the os warn the user so that they can take corrective measures
to disable this and prevent trouble.

 

How to reproduce:
Set up unRAID on a machine that creates HPA partitions on HDDS. Seems to be common with Gigabyte motherboards with virtual dual bios. (This is what happened to me)

 

Expected results:

For unRAID to warn the user and prevent array creation, starting or configuration without it being corrected or explicitly overridden by the user when it detects any hdd/s with a HPA partition.

 

Actual results:

No warnings were given, and a user is able to create or start arrays that have a high risk of preventable data lose.

Other information:
I only found out about this after installing a plugin called "Fix Common Problems". Looking into the information, and after finding a link here: https://lime-technology.com/wiki/index.php/UnRAID_Topical_Index#HPA, I found out that I had just shifted all of my data off of a failing hdd into a brand new unRAID server, my first unRAID server to boot, that could have issues. This os is meant to be about protecting data, yet it just logs a detected problem, says nothing and just lets a user do something that could lead to lose of data.

I am aware that an HPA partition is just a tiny segment at the end of a hdd drive, but considering it's an external source from unRAID managing this, this is still a massive flag for potential data lose.
There was a thing when I could select only one of the 3 TB hdds as parity when setting the array up, but I had discounted this as as just one being a tiny bit smaller than another, not knowing anything about HPA partitions.

I have found a beta bios that lets me disable the virtual BIOS feature on the motherboard, and have so far removed it from my parity drive. I'm currently undergoing a data rebuild after removing it from another disk. The array is in maintenance mode while this is being done (There's no explanation on what should be done when rebuilding drives other than starting the array. But if data changes on another hdd, parity is lost, yes? The manual just doesn't explain what's to do/what will happen fully). Also not sure if a parity check after that is needed as well, so I'm thinking of doing one just to be sure. So all up I'm looking at 27 hours of downtime, and I'm crossing my fingers that I won't have any data issues. And I only have 4 drives and under 6 TB of data.

All up this has made for a horrible learning experience. I now know about HPA's and Virtual BIOS's now though.

Edited January 7, 20188 yr by Ryonez
Clarification

In the past (and I would assume, it remains this way), if you had a parity disk with a HPA partition on it, you would encounter a problem when you went to install a data disk of the same capacity size (3TB data drive with a 3TB parity drive) without the HPA partition.  unRAID would not allow you to add it to the array since it was larger than the parity disk.  There is no problem using a HPA drive as a data drive.  

 

The problem first cropped when Gigabyte added an 'feature' to its BIOS to automatically create this partition on a new disk connected to the MB. They intended it to be used to store the BIOS settings.  (I am not privy to what thought process lead them to believe that this was really necessary.)   Originally, it defaulted to 'on' but later they added an option to turn it 'off' but the default was 'on'.  (I am not sure that the state of the setting is in current production MB's.)

 

There have some reports that shucked drives from USB cases had drives that contain a few less sectors than OEM and consumer-packaged drives.   If these reports are true, these drives would have the same issues. 

 

The check that unRAID makes is that the no drive can be larger than the current parity.  The only way that unRAID could implement your suggestion would be to incorporate a table with the exact number of sectors (or bytes) for every disk ever made.  IF a new HD size were to be introduced, unRAID would have to be update to reflect that new size.  Hopefully, you can see the problems this could cause.  

Edited January 8, 20188 yr by Frank1940

Hi there Frank.

I disagree with this. As mentioned in the original post unRAID already detects it itself, it just doesn't warn a user:

Jan  7 04:21:40 Atlantis kernel: ata3: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
Jan  7 04:21:40 Atlantis kernel: ata1.00: HPA detected: current 5860531055, native 5860533168
Jan  7 04:21:40 Atlantis kernel: ata1.00: ATA-9: WDC WD30EFRX-68EUZN0,      WD-WMC4N2545090, 80.00A80, max UDMA/133
Jan  7 04:21:40 Atlantis kernel: ata1.00: 5860531055 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
Jan  7 04:21:40 Atlantis kernel: ata1.00: SB600 AHCI: limiting to 255 sectors per cmd
Jan  7 04:21:40 Atlantis kernel: ata2.00: HPA detected: current 5860531055, native 5860533168
Jan  7 04:21:40 Atlantis kernel: ata4.00: ATA-7: ST3360320AS, 9QF9MW93, 3.CHN, max UDMA/100
Jan  7 04:21:40 Atlantis kernel: ata4.00: 703282608 sectors, multi 16: LBA48 NCQ (depth 31/32)
Jan  7 04:21:40 Atlantis kernel: ata4.00: SB600 AHCI: limiting to 255 sectors per cmd
Jan  7 04:21:40 Atlantis kernel: ata2.00: ATA-9: WDC WD30EFRX-68EUZN0,      WD-WMC4N2556899, 80.00A80, max UDMA/133

 

HPA detected: current 5860531055, native 5860533168

^ This line here is where HPA is detected.This is happening on load, is logged, and then is ignored. Leaving the user to go on completely unaware unless they knew to look out for it.

 

Quote

In the past (and I would assume, it remains this way), if you had a parity disk with a HPA partition on it, you would encounter a problem when you went to install a data disk of the same capacity size (3TB data drive with a 3TB parity drive) without the HPA partition.  unRAID would not allow you to add it to the array since it was larger than the parity disk.  There is no problem using a HPA drive as a data drive. 

 

This did happen to me, but not being aware of HPA I swapped the drives around and continued on, unaware of what had happened. And being OCD, I actually changed the drive connection so parity was in the SATA0 port. The bios saw this, gave it a HPA partition, but as the other drive already had it their sizes matched, letting me continue on to create the array and start shifting data.

 

So far from what I can tell, the drive don't lose sectors, they're just marked as reserved? 

This is why I've reported this as a defect. It just shouldn't be allowed from a system that's meant to be about protecting data. HPA being detected is a sign that a BIOS is playing with your hdds, something that shouldn't really exist in the first place in my opinion. 

 

Quote

Originally, it defaulted to 'on' but later they added an option to turn it 'off' but the default was 'on'.  (I am not sure that the state of the setting is in current production MB's.)

 

After I preformed the update to a beta bios for my current unRAID bios, this option appeared and seemed to be defaulted to off. That was a 2009 bios. Though I'd let you know what I saw with that.

Edited January 8, 20188 yr by Ryonez
Spotted a spelling mistake.

i use HPA with Hardware RAID0 as parity. cos my 2x2TB RAID0 array is a bit smaller than 4TB hdd. i know, it depends on Raid card, but i can live with that - when i decide expand my array, just remember to create small HBA on every data disk.

Sounds like a messy set up >.<

But, that HPA invalidates the parity, and you're also allowing your motherboard/non unRAID sorces to modify your hdd.

But, you know about this and have chosen to take that risk, which is your choice. I don't want any outside source playing with my hdds, and am extremely unpleased it was detected and I was uninformed. I don't want that risk, and feel I should be warned from the OS I'm paying money for to look after my data.

I also would personally find it easier to just through a hdd in, and not worry about shuffling hdds around to add HPA partitions. I don't even think that would be needed. The parity drive is the biggest issue.

14 minutes ago, Ryonez said:

But, that HPA invalidates the parity, and you're also allowing your motherboard/non unRAID sorces to modify your hdd.

not in my case - my MB don't create any HPA - i do it myself one time before new disk is assigned to the array.

if your MB is creating HPA, you can try disable this in BIOS, if there is such option.. if no, then i would suggest to change it..

You do? I'm curious as to your reason for doing so.

Though, yes, in my case my original post detailed how I found out my motherboard was creating them, and that I did disable this fiction.
Please note that my expected result suggested:
 

Quote

For unRAID to warn the user and prevent array creation, starting or configuration without it being corrected or explicitly overridden by the user when it detects any hdd/s with a HPA partition.

 

This would allow users like yourself who are aware of HPA's to override and continue, whilst helping/saving users who don't have a value about this themselves from trouble.

2 minutes ago, Ryonez said:

You do? I'm curious as to your reason for doing so.

for speed mainly, but it affects only multiple concurrent writes. and i had these 2x2TB disks already, so just reuse them.  

How does having a HPA on the... hardware raid is it? make them faster?
From my understanding is they are reserved storage areas of the hdds.

I think I just clicked. It's your raid card creating a HPA partition while it's managing them, and because of that anything that's 4TB has to have an HPA added so the size is equal to or smaller than the parity raid.

Your setup makes sense, and that's what you've chosen to do. So you'd defiantly  do the manual override. 
This is something you know of though. For most I'd say they wouldn't have a clue of HPA's.

Just now, Ryonez said:

How does having a HPA on the... hardware raid is it? make them faster?

i have no HPA on Raid0 - 2x2TB RAID0 is a bit smaller than 4TB disk, so HPA resides on data disks.

and according speed - i mean why i use RAID0 for parity, not HPA.. HPA is used to get size of data disks smaller than parity.

I see, thank you for taking the time to explain this for me!