Is there a quick test to check whether it will accept the change after the fact? purely by speed after adding the jumper?

 

Cheers, thanks

In my experience, if you have the bugged firmware then Unraid will not boot after you add the jumper.

 

Is there a quick test to check whether it will accept the change after the fact? purely by speed after adding the jumper?

 

Cheers, thanks

In my experience, if you have the bugged firmware then Unraid will not boot after you add the jumper.

 

 

Mhh, ok so its booted with all 4 disks now with jumpers :-/

 

Not really sure if it has worked....parity is at 21,357 KB/sec

It doesn't seem there is one clear "best option" for the 2TB drives right now.  I guess that doesn't come as a surprise as they are relatively new.

 

I think I should just go with my gut and avoid them right now.  For $80 CDN and free shipping, I can get:

Seagate Barracuda 7200.11 1500GB 1.5TB SATA2 7200RPM 32MB Cache NCQ 3.5IN Hard Drive OEM

 

The cost per GB is still cheaper this route and I do have the space to fit 6 more drives without any further investment.

Hello,

 

I got one of the buggy EARS firmware drives and installed it without jumper at first and then as list discussion changed on it's merits I tried changing the jumper and got the endless error-ific boot cycle that others have encountered.

 

When I found this out I emailed Western Digital about getting an upgraded firmware for the drive and removed the jumper ... and waited and waited and waited ...

 

My drive finally started to fill up ...

 

And I said no firmware ... then guess I'm RMA'ing it ...

 

Now I have my RMA drive and am starting a preclear on it with the jumper installed ...

 

When the preclear is done I was curious what the best course of action was?

 

On main unRaid page ... Stop Array ...

On Devices Page ... Unassign buggy firmware drive ...

On Devices Page ... Assign precleared RMA drive ...

On Main unRaid Page ... Start Array ...

 

I know switching that drive slot from an un-jumpered drive to a jumpered drive shouldn't be a problem for unraid ... but I have to ask ;-)

 

Thanks for your time and help,

Bobby

After researching this issue here is the fix I want to put in the code:

 

a) Any newly Formatted drive will have partition 1 start in sector 64, instead of 63 as is historical.  This will cause partition 1 to be aligned on a 4K boundary.

 

A concern with this change however, is what effect, if any, will this have on the size of the disk from unRAID's point of view, e.g., will it cause the loss of a block?  The algorithm unRAID uses to determine the size of the disk is as follows:

 

size = number of 512-byte sectors reported by disks

size = size - 63  # account for "first track" used for MBR & boot code

size = size / 2    # convert to number of 1024-byte blocks

size = size - (size modulo 4)  # trim size to make it an exact multiple of 4K

 

In spot checking a number of IDE and SATA hard drives, it appears that there is NO change to the total number of blocks available, due apparently to the fact that I haven't found a drive that has an odd number of sectors.

 

b) Existing drives with partition 1 starting in sector 63 will not be affected by this change, with possible exception to WDxxEARS drives.  The exception has to do with the Advanced Format Jumper (pins 7-.

 

With this jumper NOT installed, 512-byte LBA addressing corresponds 1:1 with 512-byte LBA addressing on the media.  This causes a write penalty if data being written to the drive is not 4K aligned (which is the case when partition 1 starts in sector 63).

 

With the jumper installed, it appears that what WD does is simply add 1 to each 512-byte sector LBA to get the media LBA.  Thus it aligns the start of partition 1 in media sector 64.  This is fine for XP (and also unRAID, btw), but is not a good long-term solution because eventually other manufacturers will start producing drives with 4K sectors and they might not include this jumper hack.

 

So after this change is in the code:

- if you are adding a new WDxxEARS drive, do NOT install the jumper.

- if you already have a WDxxEARS drive in your array, and the jumper is installed, no change is necessary, leave the jumper installed.  (If you later remove the drive and intend to reformat, you should remove the jumper before reformatting.)

- if you already have a WDxxEARS drive in your array, and the jumper is NOT installed, then you can also leave it alone, but the drive will suffer a write performance hit.  Your choices in this case are:

 1. do nothing, get an adult beverage and go watch a movie, or

 2. copy all the data off the disk, reformat it, and let parity-rebuild re-create your data, or

 3. reply to this post that you really really want a partition moving utility.

 

I'm mainly interested in hearing from anyone who is running jumper-less at the present time.  Also, I want to provide a vesion 4.5.5 "pre-release" to anyone willing to test with WDxxEARS drives.

 

Finally, something else that should work:  if you have the jumper installed, then you should be able to remove the jumper, do an "initconfig", and upon array Start,  unRAID will write a new mbr, but since you had the jumper on previously, partition 1 should already be in sector 64, and all your data should be fine.  I don't have an EARS drive at the moment so I'm looking for a volunteer to try this.

 

One more thing:  Joe L. will need to modify his 'preclear' script coincident with the new release in order to define partition 1 starting in sector 64 instead of 63.

 

Edit: the over-striked paragraph above won't work.

One more thing:  Joe L. will need to modify his 'preclear' script coincident with the new release in order to define partition 1 starting in sector 64 instead of 63.

No problem... the issue I see is I have no easy way to know which version of unRAID a user is running to know when to start in sector 64 vs. 63.

(Actually, I really don't care about the version itself, but the capability to use a starting sector of 63 or 64) 

I need to make the command usable on older versions of unRAID and newer ones too, without the user having to mess with various options.

 

Is there a possibility of you including an "unraid_sector64" clue? or something I can use to determine when sector 64 is appropriate? 

It can be as simple as a file  /usr/local/sbin/unraid_sector64 that has the contents:

echo "reiserfs partition 1 may start at sector 63 or 64"

It could even be a file under /usr/src, and not executable, since that too will be in place regardless of any unRAID version.

 

Lastly, will a disk pre-cleared using sector 63 as the start of the first partition still be recognized as pre-cleared after this is in place?  (once you put your change in place, either starting sector would be valid, and still considered pre-cleared?)

 

You will obviously need to change how you determine if a disk has been pre-cleared, or even a valid unRAID reiserfs, since a single simple memory compare of the MBR with your expected values will not match both partitions starting in sector 64 and the older ones starting in sector 63.

 

Joe L.

After researching this issue here is the fix I want to put in the code:

 

[...]

So after this change is in the code:

- if you are adding a new WDxxEARS drive, do NOT install the jumper.

- if you already have a WDxxEARS drive in your array, and the jumper is installed, no change is necessary, leave the jumper installed.  (If you later remove the drive and intend to reformat, you should remove the jumper before reformatting.)

- if you already have a WDxxEARS drive in your array, and the jumper is NOT installed, then you can also leave it alone, but the drive will suffer a write performance hit.  Your choices in this case are:

  1. do nothing, get an adult beverage and go watch a movie, or

  2. copy all the data off the disk, reformat it, and let parity-rebuild re-create your data, or

  3. reply to this post that you really really want a partition moving utility.

 

I'm mainly interested in hearing from anyone who is running jumper-less at the present time.  Also, I want to provide a vesion 4.5.5 "pre-release" to anyone willing to test with WDxxEARS drives.

 

Finally, something else that should work:  if you have the jumper installed, then you should be able to remove the jumper, do an "initconfig", and upon array Start,  unRAID will write a new mbr, but since you had the jumper on previously, partition 1 should already be in sector 64, and all your data should be fine.  I don't have an EARS drive at the moment so I'm looking for a volunteer to try this.

 

One more thing:  Joe L. will need to modify his 'preclear' script coincident with the new release in order to define partition 1 starting in sector 64 instead of 63.

 

Hi Tom,

I just received 2 2TB EARS Drives and did "best practice" after reading here - this is: setting jumper 7-8 on the drives and did a preclear with JoeL.'s script.

To be more exact: The drives are the newer drives with only 3-platter.

The drives are planned to replace older 300GB drives - but after reading your last post I am unsure what to do - my plan was to use those "jumpered, precleared" drives to replace older drives and let them rebuild.

So what would be the official recommendation now?

Thanks, Guzzi

 

PS: Any plans, when 24 drive support will be included? In this case I would add instead of replace those drives...

Lastly, will a disk pre-cleared using sector 63 as the start of the first partition still be recognized as pre-cleared after this is in place?  (once you put your change in place, either starting sector would be valid, and still considered pre-cleared?)

 

Here's what I'm going to do.  I will continue to use the current signature for 'pre-cleared' disk.  When a pre-cleared disk is detected,  unraid will simply write a new mbr to it that sets the start of partition 1 to sector 64.  No need to change your script.

I just received 2 2TB EARS Drives and did "best practice" after reading here - this is: setting jumper 7-8 on the drives and did a preclear with JoeL.'s script.

To be more exact: The drives are the newer drives with only 3-platter.

The drives are planned to replace older 300GB drives - but after reading your last post I am unsure what to do - my plan was to use those "jumpered, precleared" drives to replace older drives and let them rebuild.

So what would be the official recommendation now?

If you need to get those disks in service now, then you should install the jumper.  If you want to try a 4.5.5 prelease I should have one up later today.

 

PS: Any plans, when 24 drive support will be included? In this case I would add instead of replace those drives...

 

4.5.5 added support for 21 drives.  The main issue with 24 drives is that I can't test it.

I just did a preclear on one of these drives with the jumper in place and it showed 0 errors before and after the preclear. I couldn't tell you anything about speed since I have loaded it up in an active environment since some of my parts are not hear as of yet. I can tell you it took around 35 or 36 hours to preclear 2TB and man was it every quite spinning. I do recall seeing it reading at 90Mbs and at one time around 57Mbs in a P4 3.ghz with 1gigram.

 

I'm not sure if its the newer model or not, since well I had no idea there was a newer model. I did pick it up a few days ago before NewEgg decided to have a sale. Of course.

Joe's preclear script does a lot more than just write zeros to the drive.  To just write zeros to a 2TB drive I would expect to take 7-10 hours.

 

PS: Any plans, when 24 drive support will be included? In this case I would add instead of replace those drives...

 

4.5.5 added support for 21 drives.  The main issue with 24 drives is that I can't test it.

 

How do you count the drives? We have today support for "21 drives" (19 + parity + cachedisk) - so when you're talking about 21 drives you mean 21 datadrives + parity (thus adding 2 more drives)? This would just allow me to add my 2 new drives...

 

Back to topic: In case I would want to test your new version, I should remove the jumper - and should I also redo the preclear? I assume "yes", because currently there should be 1 sector uncleared, because the preclear with jumper set added 1 sector, right?

Joe's preclear script does a lot more than just write zeros to the drive.  To just write zeros to a 2TB drive I would expect to take 7-10 hours.

 

I hope you don't think I said that. I said it had "0 errors" and it did it in 35 hours. Just from looking through some of the code it looks like it beats the drive up something good, which I'm a big fan of so I know my drive can be trusted more since it has been stressed some.

How do you count the drives?

Right, should have been more clear: 21 means 21 array drives including parity (so 1 parity + 20 data).  Cache is outside this, so including cache it would be 22 total drives.

Back to topic: In case I would want to test your new version, I should remove the jumper - and should I also redo the preclear? I assume "yes", because currently there should be 1 sector uncleared, because the preclear with jumper set added 1 sector, right?

Yes, but if look at the original post with pre-clear script description, there are switches you can set to skip some steps and speed things up.  http://lime-technology.com/forum/index.php?topic=2817.0

Also you could let unRAID clear the disk, but then your array will be offline until clear completes.

Firstly after a very hot day getting to work on the tube in London, you have made my day thanks for your efforts..

 

I can finally get my Unraid server going at last!

 

So I have 4 ears drives with no data on them; so I can test whatever you need.

 

Can i just remove the jumpers from the drives and use your latest test version and let it do the work? or do I need to do preclears?

 

Thanks

Firstly after a very hot day getting to work on the tube in London, you have made my day thanks for your efforts..

 

I can finally get my Unraid server going at last!

 

So I have 4 ears drives with no data on them; so I can test whatever you need.

 

Can i just remove the jumpers from the drives and use your latest test version and let it do the work? or do I need to do preclears?

 

Thanks

 

If you are building a "new" array with these drives (ie, not plugging into existing array to expand storage), then you don't have to run 'preclear' script, though preclear will exercise the disks and perform some basic diagnostics.

 

You will want to install without jumpers once I release the code for you to test.

 

So, here's what I'm debating about this.  Once a drive gets formatted with this new release, there's "no going back", that is, earlier releases of unRAID will think the drives will need to be formatted.  As a result I'm still thinking about the best way to update the software.

So, here's what I'm debating about this.  Once a drive gets formatted with this new release, there's "no going back", that is, earlier releases of unRAID will think the drives will need to be formatted.  As a result I'm still thinking about the best way to update the software.

 

Maybe finally a reason to put out a 5.x release? If you are gonna break backwards compatibility, then doing it on a "major" release makes the most sense (or at least that is what Apple/MS  have conditioned me to believe).

 

I have a 2TB WD on the way personally, but I am happy to jumper it.

Maybe two releases? One that supports without the jumper and one that is jumpered.

 

I know from experience how much of a pain in the butt it is to maintain two releases thou.

 

Releasing as a beta with the normal caveats would then allow it to be tested first, giving you more time to decide on the release pattern.

 

Once the functionality is stable, then I am not sure of the reasons why you would need to go back (he says naively).

 

Obviously users with mixed setups should perhaps wait until fully stable before upgrading or even jumpering the one drive, preclearing and adding that one drive to the array ... so I presume your new version would identify this drive as normal rather than advanced and would do nothing different...

 

 

This is exactly why i think we should go back to the old model unRAID used where you would have many betas then one stable. Currently we have either one beta and a stable (which in itself had changes never seen in any beta) or more recently no betas at all, just stables released without community testing.

 

I can see only praise coming from low frequency stables, that is what uNRAID users came to expect. Stability and data integrity is all. Nothing else matters

It will still work with all other (older) drives, and ones with jumpers as unchanged? So that this will only change new Drives or will we need to start fresh boxes and format all old and new drives?

 

Thank you

For the time being, I will hold off on this change.  I believe the only drives out there with 4K sectors are the WDxxEARS's.  The recommended way to run with these drives in unRAID is to install the pin 7/8 jumper.

 

What I'm not 100% sure about is this.  If you later remove the jumper, is it true that all the sector addresses are now shifted up one?  I can not verify this until I get a EARS drive in here (some are on order).  Someone with command line experience can determine this by using 'dd' command to copy specific sectors to a file and then look at them in a dump utility.  If the jumper is installed, and you format the drive, you should see a MBR in sector 0.  If you then remove the jumper, you should see the MBR now in sector 1.

 

I think this issue has to be addressed eventually for these reasons:

a) At some point other manufacturers are going to release 4K-sector drives.  They may or may not include the sector-shifting-jumper-hack.

b) Mucking around with jumpers is just silly & leads to confusion.

 

Disappointing.

 

Hopefully your drives will arrive and further testing can be done.

For the time being, I will hold off on this change.  I believe the only drives out there with 4K sectors are the WDxxEARS's. 

And, from what I've read, most all of the SSD drives

The recommended way to run with these drives in unRAID is to install the pin 7/8 jumper.

 

What I'm not 100% sure about is this.  If you later remove the jumper, is it true that all the sector addresses are now shifted up one?  I can not verify this until I get a EARS drive in here (some are on order).  Someone with command line experience can determine this by using 'dd' command to copy specific sectors to a file and then look at them in a dump utility.  If the jumper is installed, and you format the drive, you should see a MBR in sector 0.  If you then remove the jumper, you should see the MBR now in sector 1.

You'll need to verify what happens to the very last sector on the disk in addition to sector 0.  When shifted upward, it might be out of the addressable range with the jumper removed.

I think this issue has to be addressed eventually for these reasons:

a) At some point other manufacturers are going to release 4K-sector drives.  They may or may not include the sector-shifting-jumper-hack.

b) Mucking around with jumpers is just silly

True, but we all did it for years with master/slave jumpers on IDE drives

& leads to confusion.

You can see that just from the few comments in response to your announcements of a possible change to accommodate this in this thread.

 

I think it is smart to not rush into this.  Even if the WDxxEARS drives do shift everything up one by adding the jumper, and is is accessible again after removing it, who's to say the next manufacturer will do the same with whatever method they choose to implement a switch to accommodate both older and newer OS.  I actually think you need to wait and see what the next manufacturer(s) does before deciding on a final solution.

 

Joe L.

 

Seems likely that an update is 6 months away then if we are waiting to see what the other manufacturers do.

 

Can you perhaps create a time expiring beta so I can at least try it? if I get some good results that may help to decide if its worthwhile or not...