I think many more people have 4k sector drives than Macs.  Maybe I'm wrong, but I see it as a bigger issue especially with the January swap to only 4k.

I think many more people have 4k sector drives than Macs.  Maybe I'm wrong, but I see it as a bigger issue especially with the January swap to only 4k.

 

I agree with you, but there are many questions about implementation that aren't solved yet. 4K drives support can't be dissociated from 3TB drives support. AFP is more feasible right now, so IMHO is that it should be implement first.

A little perspective.... unRAID works fine with all 4K drives right now... right out of the box.  No errors, no lost data, they all work.  The only issue is a moderate speed penalty if they are misaligned.  unRAID "supports" 4K drives right now.  The whaling and gnashing of teeth is over supporting them "better" by aligning the partition, and the result is a moderate speed benefit. 

 

4K drives support can't be dissociated from 3TB drives support.

 

Yes, it can.  4K drive "support" is a partition alignment issue... nothing more.  And that issue is a minor one, as the drives work even is misaligned, just with a slight performance hit.

 

The only connection between the two is that drives >2TB will need GPT partitions (but that is not all they need) and mucking around in the unRAID source to change the partitioning scheme to deal with the alignment issue for 4K drives, is the same place that some eventually mucking around will have to be done to implement GPT partitions.  They are related in the same way that filling the radiator reservoir is related to changing the battery in your car.... they are both located in the engine compartment under the hood.

The more I think about it, the more I'd say that getting a stable release first would be a good idea. Add the driver support to recognize both sector 63 and sector 64 formatted drives but do not change the formatting. Then, add the support for formatting to a later release. This way, there is a previous unRAID 5.x stable release to fall-back to if something goes wrong with the 64-sector formatting version.

 

 

4K drive "support" is a partition alignment issue... nothing more.

 

Yup, current 4k drives appear to be 512b sector drives just like any other drive you have purchased in the past. The only "issue" is placing the partition at the correct 512b sector so that the partition internally starts at a 4k sector.

 

The released 3T drives are not true 4k sector drives. They are 4k sectors on the platters and 512b sectors at the interface. Presently, unRAID will support drives with up to 2^32 sectors which equals about 2.19 terabytes in size. Adding support for a larger number of sectors (48-bits or 64-bits for the sector counter instead of 32-bits) is not the same thing as putting the partition in a different location.

 

It's possible that true 4k sector support could one day go hand-in-hand with 3T drive support. You could continue to use the present 32-bit sector addressing and be capable of accessing a drive size somewhere around 17.6 terabyes assuming you had a true 4k sector drive (4k sectors at the interface). However, there are no such drives available today so 4k sector support and 3T drives do not go hand-in-hand today.

 

Peter

 

unRAID will support drives with up to 2^32 sectors

 

Agree but want to point out that that LBA addresses have been 48 bits (roughly 128,000TB addressability with 512k sectors) since 2003. Unles unRAID was coded to use only 32 bits when doing LBA calculations (assuming it even gets to that level in the drive interface) the current 32 bit limit is a MBR partitioning limit. So we don't ever need true 4K sector support for any purpose other then partition allignment and maybe some performance improvement. I suspect the key to unlocking 3TB drives is support for GPT partitioning. Don't know what file system limits might exits. Things like BIOS support and bootability don't matter to unRAID. I really don't know if connecting a 3TB drive to a MV8, creating a GPT partition and addressing the full 3TB using LBA address will work or not. Such things need to be carefully tested but 3TB drives are still a bit pricy to buy one just to experiment. Who knows maybe it works so long as you don't try to boot from it. Don't we all disable INT13 anyway? Would be nice if Supermicro would do the testing and isssue a statement regarding compatibility, now that 3TB are available in the retail market. I would think that we will eventually see a list of HBAs that have been tested to be compatible with > 2TB drives.

the current 32 bit limit is a MBR partitioning limit.

 

Yes, it's the MBR limit. Well, I believe the MBR would allow a 3T drive to have 2 or more partitions as long as all were under 2.1TB in size. unRAID uses a single partition though so that's pretty much irrelevant to this issue.

 

I believe GPT will allow 64 bits for a partition, allowing a 9.4 x 10^21 byte partition (=1.2 x 10^12TB??). That should solve the partition issue of our storage servers for a little while anyways. 

 

Peter

unRAID will support drives with up to 2^32 sectors

 

Agree but want to point out that that LBA addresses have been 48 bits (roughly 128,000TB addressability with 512k sectors) since 2003. Unles unRAID was coded to use only 32 bits when doing LBA calculations (assuming it even gets to that level in the drive interface) the current 32 bit limit is a MBR partitioning limit. So we don't ever need true 4K sector support for any purpose other then partition allignment and maybe some performance improvement. I suspect the key to unlocking 3TB drives is support for GPT partitioning. Don't know what file system limits might exits. Things like BIOS support and bootability don't matter to unRAID. I really don't know if connecting a 3TB drive to a MV8, creating a GPT partition and addressing the full 3TB using LBA address will work or not. Such things need to be carefully tested but 3TB drives are still a bit pricy to buy one just to experiment. Who knows maybe it works so long as you don't try to boot from it. Don't we all disable INT13 anyway? Would be nice if Supermicro would do the testing and isssue a statement regarding compatibility, now that 3TB are available in the retail market. I would think that we will eventually see a list of HBAs that have been tested to be compatible with > 2TB drives.

 

Yes this is exactly correct (and yes, everything is built using large file support) and I agree that most modern h/w should work as long as you don't try to boot (and that should work too with EFI bios).

 

What I'm considering is, when putting in the 4K alignment fix, is using GPT partitions.  Everything should work fine with this except GPT defines a backup partition table at the end of the disk.  I am trying to determine if it's possible to not have this (or else existing disk size will have to shrink somewhat - not good).

or else existing disk size will have to shrink somewhat - not good

 

Not sure I understand why loosing space at the end would be a problem unless it was a 2TB or less parity drive. Can't imagine that the lost space on a data drive is significant. Maybe GPT could only be used on drives > 2TB. Obviously to use a 3TB drive you need to replace the parity drive first so all existing data drive partitions would still be smaller than the parity drive partition. Adding a 3TB data drvie would get the same backup partition table so the data partition would be the same size as the parity partition. Of course it would take at least two 3TB drives to do proper testing.

 

Am I missing something?

Am I missing something?

 

Yes... rebuilding a failed drive with data at the end of the drive.

Am I missing something?

 

Yes... rebuilding a failed drive with data at the end of the drive.

 

If GPT was used only drives that are > 2TB in size, wouldn't a replacement drive always have a partition that is => the failed drive? Space would only be lost if the replacement drive is > 2TBs. If replacing a 2TB or less drive with a 3TB the partition would be much bigger. If replacing a 3TB drive it would be the same size. If the replacement drive were the same size as the original and 2TBs or less, no space would be lost so the data partition would be the same size using either 63 or 64 allignment.

The issue is that Tom is considering combining the GPT and the 4k alignment into one combined upgrade. It won't work though because replacement drives become an issue. For example, I have a 2T WD with the jumper and it fails so I chose a Seagate 2T as the replacement. I could not use the new GPT partitioning since that would mean the partition becomes slightly smaller. So, I'd have to use the current MBR partitioning and not get the proper alignment for the drive.

 

Tom could impliment the sector 64 fix into the MBR partitions as well to get around this issue but it adds more work, which takes time away from other improvements.

 

Peter

 

The issue is that Tom is considering combining the GPT and the 4k alignment into one combined upgrade. It won't work though because replacement drives become an issue. For example, I have a 2T WD with the jumper and it fails so I chose a Seagate 2T as the replacement. I could not use the new GPT partitioning since that would mean the partition becomes slightly smaller. So, I'd have to use the current MBR partitioning and not get the proper alignment for the drive.

 

I understand that which is why I am wondering if using GPT only on drives > 2TB might solve the problem. Guess I don't know what the work effort is to:

1. Align on 64

2. Use MBR if drive is 2TB or less

3. Use GPT if drive is > 2TB

 

Is there any reason this would not work? Is it just a matter of work effort?

What about replacing a < 2TB failed drive with a > 2TB replacement drive? Doesn't that mean you now have additional complexity that doesn't need to be there with the multiple types? Maybe it won't add any complexity because only the partition is parity protected and not anything beyond it...

so, there will be no issues in the future if i have an advanced format drive now as my parity drive with a jumper?

so, there will be no issues in the future if i have an advanced format drive now as my parity drive with a jumper?

 

That is a goal.

I believe that you could still use MBR on a 3TB drive if it were truly using 4K sectors. You'd have 8 times the bits addressed per sector, so it would top out at 16TB or so.

 

GPT is only necessary in order to BOOT to a larger than 2^32 bit partition, because BIOS itself doesn't support such.

 

If this is correct, and assuming the industry does move to true 4K sectors next year, then GPT/UEFI support becomes something that can be added to a later version.

 

As for when to add 4K support:

It's not currently broken, so I support the idea of adding this to version 5.1. That's partially in self-interest, since I seem to be suffering from the mvsas driver issue, but I think it makes sense.

...I seem to be suffering from the mvsas driver issue...

 

I think the Supermicro card might be heat-sensitive.  If you are using this card, and can make errors happen, as an experiment, try running fan over the card and see if that helps.

I also have the mvsas driver issue. Cannot make it happen on demand, but so far, the three times it happened, have all been while playing in the GUI. Never while doing a parity check or other disk activity that you think of as stressing the card.

 

My card is in a very cool basement with drives idling in the teens and maxing in mid 20s. I think it extremely unlikely it is heat related based on my symptoms.

so, there will be no issues in the future if i have an advanced format drive now as my parity drive with a jumper?

 

That is a goal.

 

If I understand what the two of you meant, it seems like this might be the tough case to deal with.  From what I understand, there would be no need to ask people whether to start at sector 63 versus 64, except that if you are formatting a jumpered WD advanced format drive you'd want to start at sector 63.  For all other drives its safe to start them at 64.  As I said before, I think the best thing to do is just to align all newly formatted drives to sector 64.  People formatting/reformatting Western Digital advanced format drives would just have to know to remove the jumper.

 

Of course, a jumpered WD drive that when formatted was previously aligned at sector 63 would still work fine.  Maybe that's all you meant.  I'm just a little nervous because a previous post talked about asking people if they had an advanced format drive or not, which seems like it would do more harm than good.

Would we have to start the partition at 64. How about sector 16 or 8. My understanding is that After the MBR, there are a bunch of zeroed sectors that could be used if the drive were partitioned differently. Is there enough there to make up for that lost at the end with the GPT?

My understanding is that After the MBR, there are a bunch of zeroed sectors that could be used...

Not entirely true.  Different bootloaders use some sectors between the MBR and 63, and

you can't be sure how many.  Remember, we can insert a disk into unRAID that's already been

formatted and contains data.  It may be a boot disk from somewhere, and we want to be able to

later put it back where it came from   So we can't mess with its first 62 sectors.

 

It may be a boot disk from somewhere, and we want to be able to

later put it back where it came from   So we can't mess with its first 62 sectors.

 

I don't follow.  If you add the drive to your array, unRAID is going to zero the drive.  If, per chance, it does not zero the first 62 sectors (I think it does but if it doesn't), your boot partition is gone.  The bootloader code in there is useless.  And if you want to pull it from your array later and add it to a workstation, you could repartition / reformat it and, if appropriate, new code could be put in the early sectors.

 

To me this doesn't sound like any reason to not use those sectors.

 

Of course these sectors represent minimal space - so from that perspective it doesn't matter.  But if they offset the lost sectors are the end of the disk to GPT partition, and allow partition to remain exactly the same size as in current unRIAD, it would be worth doing.

It may be a boot disk from somewhere, and we want to be able to

later put it back where it came from   So we can't mess with its first 62 sectors.

 

I don't follow.  If you add the drive to your array, unRAID is going to zero the drive. 

 

 

Why would it zero the drive?  If it's already reiserfs formatted and mountable, you just

add it to the array and rebuild the parity.  All existing files on the disk stay intact.

 

 

It may be a boot disk from somewhere, and we want to be able to

later put it back where it came from   So we can't mess with its first 62 sectors.

 

I don't follow.  If you add the drive to your array, unRAID is going to zero the drive. 

 

 

Why would it zero the drive?  If it's already reiserfs formatted and mountable, you just

add it to the array and rebuild the parity.  All existing files on the disk stay intact.

 

 

If you are just adding the drive to an existing array it will probably clear the drive even if it has a reiserfs.

 

Only if you do an "initiconfig" to force a new initial configuration and force a complete parity calculation will the disk not be cleared.

(and only if it is partitioned exactly as unRAID would have partitioned it)

Is there anything wrong with putting a line in the sand that says as of unraid 5.x.x all new partitions start at sector 64 ?