Need Community input on 4K-sector drives

[Joe L.]

Is it not possible to have two versions of unRaid, one for <=2TB and one for >2TB. Would this not be easier  to manage.

 

No, it can be (i.e. more than likely will) be more of a pain in the arse to release 2 different versions.

Besides, there would be no upgrade path. 

[lionelhutz]

Tom;

 

There are 2 different issues when talking 4k sector drives.

 

Presently, the 4k sector drives actually emulate 512b sector drives. Despite many posts or opinions to the contrary, unRAID presently supports these 4k sector drives but the performance can suffer if reading and writing very small files. Since most media is much larger than 512b I personally think the issue has been blown way out of proportion. However, to take full advantage of these drives the filesystem partition needs to start on a sector that is divisible by 4, ie chose sector 64 instead of 63. You don't need an "advanced format" drive to do any tests of this since the fact the drive is using 512b or 4k sectors internally really has no bearing on the partition alignment the filesystem uses. You can test a new partition alignment with any old drives you have lying around.

 

The next issue will be when drives with true 4k sectors (no 512b emulation) will begin to appear on the marketplace. The BIOS or EFI, filesystem, SATA controller, OS and all the file system tools will have to support 4k sectors to use these drives. Testing this will require a 4k drive where the 4k sectors are exposed to the SATA port. I have no knowledge of any such drives being presently available. Even the new Seagate and WD 3T drives use 512b emulation.

 

Peter

 

[lionelhutz]

3 - Hybrid array (existing disks aligned at sector 63 and new ones at sector 64).  Would require new method to replace a failed or upsize a disk:

- New (replacement) disk added to array and cleared (or pre-cleared)

- New disk partitioned / formatted with sector 64 alignment

- Data from failed or to-be replaced disk copied at file level to new disk

- Disk to be removed zeroed (or if simulated, the simulated disk is zeroed to adjust parity) and removed from array

- Done!

 

Why can't you read from the other disks, re-create the required data, and then write it with the correct offset on the new disk? If upgrading 63 to 64 then write with a sector offset.

 

Peter

 

[prostuff1]

]It was only two years ago that my company upgraded my PC from Windows2000 to WinXP.  The last remnants of 2000 (all over the company) were removed this year.  Don't think from a business perspective Microsoft CAN get rid of XP as soon as everybody (individual users) or Microsoft would want.  They extended 2000 support about 5 years longer than they originally said they were going to just because of that.  My company will NOT install a new OS until the first service pack has been produced and then it is a slow migration and only updated if service calls to our help desk become too frequent.  And I doubt Vista will ever hit a machine here.

 

I understand why MS does it, but I don't agree with it.  Supporting old systems, with legacy OS's, leads to a lot of extra work.  EVERYONE knew Vista was coming (Vista sucked until SP2, and still kind of does), but it seemed like no one wanted to update there software to work with it.

 

The university I worked for updated the XP fairly quickly, and is still running XP for most of there stuff.  I do know that they are going to be moving to Windows 7 here pretty soon.  The majority of the apps the employees need run just fine on Windows 7, it is the IT guys that have to support stuff that are left behind.  For that case they are planning on keeping around VM's for the apps that will not work quite correctly.

[Joe L.]

3 - Hybrid array (existing disks aligned at sector 63 and new ones at sector 64).  Would require new method to replace a failed or upsize a disk:

- New (replacement) disk added to array and cleared (or pre-cleared)

- New disk partitioned / formatted with sector 64 alignment

- Data from failed or to-be replaced disk copied at file level to new disk

- Disk to be removed zeroed (or if simulated, the simulated disk is zeroed to adjust parity) and removed from array

- Done!

 

Why can't you read from the other disks, re-create the required data, and then write it with the correct offset on the new disk? If upgrading 63 to 64 then write with a sector offset.

 

Peter

 

That works for all but the last sector.  Ouch... no place to put it.

[prostuff1]

We had a productive conversation HERE concerning how the migration to 4k sectors might work.

 

In summary, these options were debated / discussed:

 

1 - Offering 2 separate versions (sector 63 and sector 64 alignment) - Pros: Easy to implement; Cons: More difficult to support and maintain, confusion about what advice applies to what version

 

2 - Offering 4byte aligned (sector 64 alignment) version only.  Users would have a time-consuming one-time conversion effort.  Pros: Easy to implement; Cons: Risk of data loss while migrating, very time consuming for users

 

3 - Hybrid array (existing disks aligned at sector 63 and new ones at sector 64).  Would require new method to replace a failed or upsize a disk:

- New (replacement) disk added to array and cleared (or pre-cleared)

- New disk partitioned / formatted with sector 64 alignment

- Data from failed or to-be replaced disk copied at file level to new disk

- Disk to be removed zeroed (or if simulated, the simulated disk is zeroed to adjust parity) and removed from array

- Done!

 

Methods to recover/restart if crash or power failure during rebuild would need to be considered during each step in process.

 

BubbaQ explains this concept in more detail in the thread above.

 

Pros: Easy for users to migrate, data stays protected throughout; Cons: more difficult to implement, time consuming disk replacement

 

4.  Another option was discussed briefly - moving the partition forward one sector on a disk.  A tool might be able to do this with some finese based on a knowledge of RFS and avoid the brute force method.  No one was aware of a tool to do this with an RFS disk.  The brute force method would be lengthy and non-restartable - a bad combination.

 

 

I am not up to speed completely on moving partitions and the like but gparted might work and I did a quick search and came up with this and this

[greybeard]

3 - Hybrid array (existing disks aligned at sector 63 and new ones at sector 64).  Would require new method to replace a failed or upsize a disk:

- New (replacement) disk added to array and cleared (or pre-cleared)

- New disk partitioned / formatted with sector 64 alignment

- Data from failed or to-be replaced disk copied at file level to new disk

- Disk to be removed zeroed (or if simulated, the simulated disk is zeroed to adjust parity) and removed from array

- Done!

 

I like the idea of all newly formatted drives using sector 64 alignment.

I don't like the idea of anything moving during rebuild, when an arrary is most vulnerable. I would rather see a misalligned partition than risk moving things during a rebuild.

Simulating zeroing (not actually clearing) for removal would be a nice to have but is not strictly require for 4K support. It would make manual migration from 63 to 64 easier.

 

I can make good use of sector 64 alignment right now even if there is no upgrade path from sector 63 alignment.

I have no immediate need for support of >2TB drives. No desire to buy them until they are much cheaper.

All my recent drive purchases have been WD and Samsung. (I missed those $60 F4s or I would have several more of them.)

 

Just my opinions.

[lionelhutz]

That works for all but the last sector.   Ouch... no place to put it.

 

Does the file system use every sector right to the end?

 

You know, the bigger question is should Tom even bother at all. The 4k drives which emulate 512b drives do work and they are an intern fix until full 4k drives are released.

 

The rebuild from a 512b drive to a 4k drive might be interesting...

 

Peter

 

[limetech]

Thanks to everyone who participated in this thread - I think I know the correct way to proceed with this.

 

Here is the disk device size calculation done by the unRaid driver.  This is the size reported on the 'Main' page for a disk, and actually represents the size of partition 1:

 

 temp = (real_size - offset) / 2;  /* temp counts 1K-blocks */

 size = temp - (temp % 4)  /* size is multiple of 4K-bytes, expresses as a number of 1K blocks */

 

where

 real_size is the actual number of 512-byte sectors reported by the drive itself

 offset is the number of sectors from start of drive for first partition (ie, 63)

 (note: all 64-bit arithmetic)

 

Partition 1 is created to start at sector 'offset', for 'size' number of 1K blocks.

 

Since I've never seen a drive with an odd number of sectors, the current calculation for 'temp' guarantees there's an extra sector unused at the end of the disk.  So a disk with partition 1 at offset 63 should report the exact same size if partition 1 starts at offset 64.

 

So it will not be difficult to add code (in version 5.x) to put the partition 1 offset at 64 for new drives installed in the server.

 

Also, note that the way unRaid currently works is all I/O to hard drives via unRaid driver reference partition 1, not the whole disk.  It was designed this way because early on I envisioned a feature where you could define more than one partition on an array drive, say partition 1 and partition 2, and only partition 1 would be part of the parity group.  This would mean that writes to partition 2 would be faster, but data unprotected.  This feature was ultimately supplanted by the cache drive feature.

 

This design makes it "easy" to put a fix in for a 64-sector offset, but this has also made it difficult to support being able to plug in already-formatted drives into a server and using existing 'partition 1' in the array.  This is because, since the partition table is not parity-protected, if that disk needs to be reconstructed, I can not reconstruct the partition table (or partitions outside partition 1).  Anyway, this latter point should maybe be a separate "ask the community" topic.

 

Open questions:

 

o  Whether to create a utility to "covert in place" data on an existing drive?

 

o  If you format a WD20EARS with jumper in place, then remove the jumper, where does the extra sector show up?  If at end disk, that's a bummer, if at beginning of disk, then writing a new 64-sector MBR would make the disk available immediately.

 

 

EDIT: I should add.. if a drive gets formatted with partition 1 starting in sector 64, then you will NOT be able to run any previous version of unRAID OS, because since the partition table would be different, all current versions of unRAID OS will think the drive is 'unformatted'.

[bubbaQ]

If you format a WD20EARS with jumper in place, then remove the jumper, where does the extra sector show up? 

 

It doesn't.  The sector count remains the same... there is room for extra physical sectors at the end of the drive, so using the +1 offset jumper will push usage to 1 more sector past the logical end that it would have w/o the jumper.

 

Of course, the FUBAR is installing the jumper, and then aligning it in unRAID to 64 instead of 63, adn recreating misalignment!

[ohlwiler]

o  Whether to create a utility to "covert in place" data on an existing drive?

 

I don't think this is necessary, and I for one would not use it. I would rather copy my data off the drive and then replace it myself. Imagine the howling if your utility lost data!

 

My question is, what would happen when replacing a 63 drive (because of failure, upgrading or just because), would you create another 63 drive or make it a 64? I think keeping it a 63 is safer and that is what I would prefer.

[SSD]

I like the idea of all newly formatted drives using sector 64 alignment.

 

Me too.

 

I don't like the idea of anything moving during rebuild, when an arrary is most vulnerable. I would rather see a misalligned partition than risk moving things during a rebuild.

 

Not sure what "anything moving" means.  The proposed approach to allow a hybrid array and being able to replace a 63 sector alignment disk with a 64 sector alignment disk is by adding a new disk to the array and then doing a file level copy from the failed/simulated disk or from the disk to be replaced.  One side benefit of this approach, depending on how it is implemented, is that you might be able to replace a larger disk with a smaller disk! 

 

But if this were Tom's approach (allowing a hybrid array), I would likely do this procedure on every disk in my array, a few a week, to get to sector 64 alignment on every disk in my array, hopefully before any failures.

 

Simulating zeroing (not actually clearing) for removal would be a nice to have but is not strictly require for 4K support. It would make manual migration from 63 to 64 easier.

 

The problem with not actually clearing the disk is that it becomes non-restartable.  If you get 1/2 way through and then have a crash, parity is 1/2 updated with no way to know where to pick up to complete.  If the data is being zeroed at the same time, you could start again from the beginning (data is already zeros for the first half so it causes no harm, and then it would pick up where is left off).  If the disk were failed, zeroing the disk would just affect parity.

 

I can make good use of sector 64 alignment right now even if there is no upgrade path from sector 63 alignment.

I have no immediate need for support of >2TB drives. No desire to buy them until they are much cheaper.

All my recent drive purchases have been WD and Samsung. (I missed those $60 F4s or I would have several more of them.)

 

Just my opinions.

[SSD]

Also, note that the way unRaid currently works is all I/O to hard drives via unRaid driver reference partition 1, not the whole disk.  It was designed this way because early on I envisioned a feature where you could define more than one partition on an array drive, say partition 1 and partition 2, and only partition 1 would be part of the parity group.  This would mean that writes to partition 2 would be faster, but data unprotected.  This feature was ultimately supplanted by the cache drive feature.

 

This design makes it "easy" to put a fix in for a 64-sector offset, but this has also made it difficult to support being able to plug in already-formatted drives into a server and using existing 'partition 1' in the array.  This is because, since the partition table is not parity-protected, if that disk needs to be reconstructed, I can not reconstruct the partition table (or partitions outside partition 1).  Anyway, this latter point should maybe be a separate "ask the community" topic.

 

A throught came to mind reading your post.  What if parity were maintained based on sectors from the start of the partition vs sectors from the start of the physical disk.  If so, it would make the starting sector offset completely irrelevant to parity, and allow you to replace a sector 63 offset partition with a sector 64 offset partition the same way you do today.  Because on the first disk, sector 1 would be physical sector 63, and on the second sector 1 would be on pysical sector 64.  But parity would be maintained on sector 1 (which could be stored starting at physical sector 64 on the parity disk).  When reconstructing a partition, it wouldn't matter where the partition begam (sector 63, 64, or 1,000,000).  Parity would reconstruct the parition based on the relative sector within the partition.  Would this work?

 

Also, might want to re-read THIS POST which outlines some approaches to handling the problem of supporting hybrid arrays with both 63 and 64 sector offset disks.

 

 

[bubbaQ]

Parity would reconstruct the parition based on the relative sector within the partition.  Would this work?

 

Yes, but it can't be done with the MD driver though.  It would require a complete rewrite of unRAID from scratch.

[bubbaQ]

FWIW, here is an interesting discussion of 4K drives from a Linux perspective.

 

https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues

[limetech]

Parity would reconstruct the parition based on the relative sector within the partition.  Would this work?

 

Yes, but it can't be done with the MD driver though.  It would require a complete rewrite of unRAID from scratch.

 

A "hybrid" array is no problem.  The Parity disk, like all the other disks, has a single partition: partition 1.  All I/O in the unRaid driver is executed relative to the start of each disk's partition 1.  It doesn't matter the relative positioning of partition 1 within a particular disk... make sense?

[hellbringer]

Wait.. WHAT?!, the Samsung HD204UI aka 'F4' is NOT supported?

What are the issue's with this disk? My unraid array is working fine (or seems to be) with 2 of these (2TB) drives?

 

Should I be worried?

[BRiT]

Wait.. WHAT?!, the Samsung HD204UI aka 'F4' is NOT supported?

What are the issue's with this disk?

 

All the info you could ask for is here http://lime-technology.com/forum/index.php?topic=8012.0 -- search is your friend.

[mezz64]

Wait.. WHAT?!, the Samsung HD204UI aka 'F4' is NOT supported?

What are the issue's with this disk? My unraid array is working fine (or seems to be) with 2 of these (2TB) drives?

 

I use two HD204UI drives in my array as well with no apparent issues.  I've read the thread on them, but don't personally have any performance complaints myself.  I average 35-40 MB/s writes with a 7200rpm Hitachi parity drive over a gigabit network.  No issues streaming multiple files to multiple computers at once.

 

I've never run any benchmarks, but during my everyday use I have zero problems.

[SSD]

Parity would reconstruct the parition based on the relative sector within the partition.  Would this work?

 

Yes, but it can't be done with the MD driver though.  It would require a complete rewrite of unRAID from scratch.

 

A "hybrid" array is no problem.  The Parity disk, like all the other disks, has a single partition: partition 1.  All I/O in the unRaid driver is executed relative to the start of each disk's partition 1.  It doesn't matter the relative positioning of partition 1 within a particular disk... make sense?

 

A picture is worth a thousand words.

 

Does this picture resemble the model you are describing.

 

[greybeard]

A "hybrid" array is no problem.  The Parity disk, like all the other disks, has a single partition: partition 1.  All I/O in the unRaid driver is executed relative to the start of each disk's partition 1.  It doesn't matter the relative positioning of partition 1 within a particular disk... make sense?

 

Yes, I think so. The XOR for parity works on block 1 of all partition 1s, then block 2, etc. So it makes no differenc where the partitions actualy start. You can rebuild to a relocated partition without "moving" anything. No need for a file level copy. Right? Cool!

[greybeard]

The problem with not actually clearing the disk is that it becomes non-restartable.  If you get 1/2 way through and then have a crash, parity is 1/2 updated with no way to know where to pick up to complete.  If the data is being zeroed at the same time, you could start again from the beginning (data is already zeros for the first half so it causes no harm, and then it would pick up where is left off).  If the disk were failed, zeroing the disk would just affect parity

 

Interresting point. I was looking at it more from the perspective that logically removing the drive would allow me to take it out and set it on a shelf until I am sure I no longer need anything that is on it. If you look at it from the perspective that the drive is part of the arrary up until the zeroing is finishd then I agree you need to erase as you go. However doing so completely destroys the integrity of the file system withing the partition. Not sure what the affect of that would be, if anything.

[greybeard]

If you format a WD20EARS with jumper in place, then remove the jumper, where does the extra sector show up?  If at end disk, that's a bummer, if at beginning of disk, then writing a new 64-sector MBR would make the disk available immediately.

 

Adding or removing the jumper after a WD20EARS has been used is risky business. In a recent experiment I took a new drive and ran a multipass disk sanitization (with read verify) against it. Then installed the jumper and put it in a test unRAID system. The result was a large number of hardware IO errors. Something in unRAID (or unMenu) was trying to read the very last sector of the drive which for some reason was unreadable. SMART even indicated a pending sector reallocation. Started a preclear and continued to get 1,000s of errors throught the preread. Wasn't until preclear finished it's write pass that all was good again. The post read was normal. It took 33 hours for one pass. That is a long time even for a 2TB drive, again the post read ran at normal speed. In the end there was no pending reallocate and no relocated sectors. This could all be a coincidence caused by one marginal sector in exactly the right spot on the drive but I doubt it. I have one more I think I will install and format before adding the jumper just to see what happens.

[lionelhutz]

Since I've never seen a drive with an odd number of sectors, the current calculation for 'temp' guarantees there's an extra sector unused at the end of the disk.  So a disk with partition 1 at offset 63 should report the exact same size if partition 1 starts at offset 64..

 

This is good news, a spare sector at the end of the drive makes the fix much easier.

 

 

Also, note that the way unRaid currently works is all I/O to hard drives via unRaid driver reference partition 1, not the whole disk.  It was designed this way because early on I envisioned a feature where you could define more than one partition on an array drive, say partition 1 and partition 2, and only partition 1 would be part of the parity group.  This would mean that writes to partition 2 would be faster, but data unprotected.  This feature was ultimately supplanted by the cache drive feature.

 

This is also good news. You should be able to create a partition at sector 64 on a new drive and it will work. Also, if a sector 63 drive fails and a new drive is installed then unRAID can format that drive as a sector 64 and then rebuild the data correctly onto the new drive.

 

 

o  Whether to create a utility to "covert in place" data on an existing drive?

 

I would not bother. If the drive is working then it's working (say EARS for example). However, if a user wanted to do so, it should be possible by stopping and unassigning the disk, clearing the partition table or the first part of the drive before re-assigning it and rebuilding it as if it was new drive. Of course, when it gets rebuilt unRAID would do so as a sector 64 drive.

 

o  If you format a WD20EARS with jumper in place, then remove the jumper, where does the extra sector show up?  If at end disk, that's a bummer, if at beginning of disk, then writing a new 64-sector MBR would make the disk available immediately.

 

There really is no point in trying to "fix" a EARS drive that already has the jumper. The only thing you accomplish is the removal of the jumper. The drive will perform just as well before as after.

 

Peter

[Rajahal]

There really is no point in trying to "fix" a EARS drive that already has the jumper. The only thing you accomplish is the removal of the jumper. The drive will perform just as well before as after.

 

Glad to hear that, I was not looking forward to removing the jumpers from all my EARS drives one by one.