How to upgrade from 2tb to 3tb hdds?

[pras1011]

Whats the best way to do this?

[dgaschk]

One at a time.

[pras1011]

Of course but once a rebuild has finished on the upgraded drive do I really need to do a parity check?

[dgaschk]

Of course but once a rebuild has finished on the upgraded drive do I really need to do a parity check?

 

That is the only way to confirm success.

[pras1011]

That would mean that I would need to carry out a parity check 12 times (once per upgraded drive). A rebuild would take 12 hours? and I know a parity check will take 12 hours. So potentially 12+ days to upgrade all discs to 3tb!!!

[jowi]

Yep... or just replace all disks at once with new, empty ones and restore your backup, and do one parity build over the new disks or if you have enough free slots, add the new disks to the array, move the data from the old to the new disks under parity protection, and remove the old disks and do a new parity build? Or, maybe replace the old disks with new empty ones, creating an empty array and use a reiserfs tool to mount each old disk under windows or something and copy the data on them back to the protected array? If you like some adventure Or build a second, temporary unraid server with some old hardware you have lying around, and copy the old data to the new array and switch hardware?

[Frank1940]

If you are  truly replacing 12 drives, I would replace them over a period of a few months.  My reasoning is that all new electronic devices (including disk drives) are much more likely to fail early (call 'infant morality').  And the failure of two drives close together would result in the loss of both drives. 

 

My strategy would be to preclear a drive, Replace one 2Tb drive, rebuild the data and use the server for  a minimum of a week.  Start the next drive on the preclear cycle.  Do a non-correcting parity check after the time was up.  If no errors are found, replace the next drive.  IF you have an error, stop the replacement process until you find the problem and solve it.

[pras1011]

I have decided to change the 2tb to 4tb hdds with a few 3tb hdds mixed in. Once the parity hdd is upgraded to 4tb I will pull the first 2tb hdd (disk 1) and replace with a 3tb or 4tb. Then rebuild, then carry out parity check. I will then transfer the data in the last 2tb hdd (disk 14) to the new 3tb/4tb disk 1. And so on. Now the problem is I will have quite a few 2tb hdds that will be empty. Is there a way of permanently taking these out of the array without unraid go mad?

[dikkiedirk]

Whats the best way to do this?

 

Parity disk first, then one by one all the others.

[pras1011]

Whats the best way to do this?

 

Parity disk first, then one by one all the others.

 

I know this already.

 

But I need opinions to this:

 

I have decided to change the 2tb to 4tb hdds with a few 3tb hdds mixed in. Once the parity hdd is upgraded to 4tb I will pull the first 2tb hdd (disk 1) and replace with a 3tb or 4tb. Then rebuild, then carry out parity check. I will then transfer the data in the last 2tb hdd (disk 14) to the new 3tb/4tb disk 1. And so on. Now the problem is I will have quite a few 2tb hdds that will be empty. Is there a way of permanently taking these out of the array without unraid go mad?

[Joe L.]

Whats the best way to do this?

 

Parity disk first, then one by one all the others.

 

I know this already.

 

But I need opinions to this:

 

I have decided to change the 2tb to 4tb hdds with a few 3tb hdds mixed in. Once the parity hdd is upgraded to 4tb I will pull the first 2tb hdd (disk 1) and replace with a 3tb or 4tb. Then rebuild, then carry out parity check. I will then transfer the data in the last 2tb hdd (disk 14) to the new 3tb/4tb disk 1. And so on. Now the problem is I will have quite a few 2tb hdds that will be empty. Is there a way of permanently taking these out of the array without unraid go mad?

last step,

perform a parity check to ensure all is healthy.

 

Stop the array, un-assign all the drives you no longer wish in the array

Go to the Utils tab, press the "New Config" button

This will set a new disk configuration based on the currently assigned disks.  It ALSO immediately invalidates parity.  The next time you start the array a full complete parity calculation will occur.  Once it is done you will be protected by parity once more.

 

Joe L.

[ptmurphy]

I have decided to change the 2tb to 4tb hdds with a few 3tb hdds mixed in. Once the parity hdd is upgraded to 4tb I will pull the first 2tb hdd (disk 1) and replace with a 3tb or 4tb. Then rebuild, then carry out parity check. I will then transfer the data in the last 2tb hdd (disk 14) to the new 3tb/4tb disk 1. And so on. Now the problem is I will have quite a few 2tb hdds that will be empty. Is there a way of permanently taking these out of the array without unraid go mad?

 

Wouldn't you just go into the main unRAID management page, stop the array, and then in the drop down disk of the disk you want to remove, change it from the disk/serial number to Unassigned?

[pras1011]

I removed the 2tb parity drive and replaced with a 4tb parity drive and then did a parity rebuild. However I did not do a parity check after the parity rebuild. I then replaced a 2tb data drive with 3tb data drive. I did a full data rebuild and now I am doing a parity check.

 

Was it ok not to check the parity after the parity rebuild?

[Frank1940]

I removed the 2tb parity drive and replaced with a 4tb parity drive and then did a parity rebuild. However I did not do a parity check after the parity rebuild. I then replaced a 2tb data drive with 3tb data drive. I did a full data rebuild and now I am doing a parity check.

 

Was it ok not to check the parity after the parity rebuild?

 

Yes, but doing the parity check (I would do non-correcting as a personal choice), you would have verified that your parity rebuild was without error.  The check merely adds another level of confidence that everything is OK and helps control your anxiety levels.  IF you would have had an error at that point, the presumption would be that it was on the new parity disk. 

 

Since you elected not to check parity after the parity rebuild, I would hang onto that old data drive until your parity check on the data rebuild is completed without error.  That way if something is wrong, you still have all of your original data intact.  Carrying on with the issue of a problem being found after this parity check, you now don't know whether the error is on the parity drive or on the data drive. (Hopefully, you will not have that issue to deal with!)

[JonathanM]

Was it ok not to check the parity after the parity rebuild?

That depends. Did you preclear the new drives at least twice with verified good result? Did you keep the old drive intact so you can verify the contents of the newly rebuilt drive against the old data? If you can verify the entire contents of the new drive matches the drive you pulled, then it's ok to not check parity. It would have taken much less time to do a parity check than it is to verify the data is ok, but that's the only way you will know if the parity wrote correctly.

 

tl;dr No, the only way to know if parity was built correctly is to immediately do a check. You rebuilt a drive with unchecked parity, so any disk errors will result in corrupt data. Statistically you probably got lucky, but you don't know until you verify the contents of the rebuilt drive.

[pras1011]

I am carrying out a parity check at this moment. The rebuilt data drive has bluray iso films on it. I checked each film by mounting it and watching it via powerdvd. So far all the films have played without problems. I thought the parity drive simply builds data in relation to the data drives present and therefore as the data drives have not changed (yet) there would be no point in rechecking the parity build (I hope you understand what I mean)?

 

[pras1011]

And I also did a parity check twice before I started this procedure with no errors. I do a parity check once a month and I have never had any parity errors.

 

What I have noticed is that on the test1 and test2 software the parity check speed went from 75MB/s (RC5) to 50MB/s (this was the speed before the drive upgrade). I am using test2 and my parity check speed right this moment is about 45MB/s.

 

Could I go back to RC5 and continue with the drive upgrades or is this unwise?

[Joe L.]

I am carrying out a parity check at this moment. The rebuilt data drive has bluray iso films on it. I checked each film by mounting it and watching it via powerdvd. So far all the films have played without problems. I thought the parity drive simply builds data in relation to the data drives present and therefore as the data drives have not changed (yet) there would be no point in rechecking the parity build (I hope you understand what I mean)?

Itis true, that parity would be the same as previously, but you only wrote to the new parity drive, you made no attempt to read what was written until you then attempted to use it to reconstruct a different drive.

 

All it would take to have an issue is a sector on the parity disk that could not be read back successfully after being written.    That is why a parity check after an initial parity calc is a good idea.

[pras1011]

What are my options to check whether the data drive has been built properly. I have checked the SMART data on the new parity drive and all seems ok. I purposefully bought Hitachi drives as replacements as they are so solid! How do I check the old data drive with the new data drive?

 

I assume that if there is a problem with the parity rebuild 1) I would not know about and 2) the problem will filter down to any other drive I rebuilt and 3) everytime I do a parity check it would report zero errors?

[Frank1940]

What are my options to check whether the data drive has been built properly. I have checked the SMART data on the new parity drive and all seems ok. I purposefully bought Hitachi drives as replacements as they are so solid! How do I check the old data drive with the new data drive?

 

I assume that if there is a problem with the parity rebuild 1) I would not know about and 2) the problem will filter down to any other drive I rebuilt and 3) everytime I do a parity check it would report zero errors?

 

Has your Parity check after rebuilding your data drive finished without finding any errors?  You said that all of the data on that drive was bluray iso's.  You further stated that you played all of the iso's successfully.  All of this is true, it means that you are more than likely OK.  (Plus, since all of your data on this disk were bluray iso's, you can easily recover any loss of files if there was an (most unlikely) error in the parity drive rebuild.) 

 

Remember, modern hard drives are designed not to deliver erroneous data.  They are designed to either provide the correct data or no data at all (a read failure).  I believe that they all verify that the data has been written correctly to the drive at the time of the initial write. Otherwise, the drive would not 'know' that it had a write failure and to lock that sector out as being defective. 

 

Having said all of this, you should realize that all hard drives will have lots and lots read errors.  The drive designers all know this and provide very complex error correcting schemes so that we, as end users, never experience a data error.  All of these schemes attempt to prevent to the worst possible outcome of error correction--- delivery of 'bad' data.  They prefer to deliver no data.  But there is always the very, very small possibility that their scheme may encounter a situation with more errors than the scheme can correct and not realize that condition and provide bad data.  Before anyone who reads this panics, when I said small, I meant small to the point of almost non-existent.  That is why schemes such as single parity will work when protecting arrays as large as 480 Terrabits in size!  The drives themselves are far, far more likely to fail that the error correction scheme delivering bad data. 

 

The big problem with unRAID error correcting scheme comes from that portion of any hard drive that has not been written to.  (That is why the preclear procedure is so important.)  When you are rebuilding a drive which is larger than some of the other drives in an array, you have to be able to read all of bits on those small drives.  If you have just one bit on those drives that can't be read, you can't rebuild the new drive.  That is why those periodic parity checks are so important, they will test the readability of every bit in the current array.

[pras1011]

Thanks Frank!

 

It hasn't finished the parity check yet. I have about 16 hours to go! In the past few months I began checking the parity every three weeks. I have been using Unraid for two years and so far I have never had any errors or problems with my server. It runs so much better than that rubbish RAID server that I had.

 

Is there a way for me to check once and for all if the data on my replaced drive is ok?

[JonathanM]

Is there a way for me to check once and for all if the data on my replaced drive is ok?

What I have done in the past was mount the old drive and do a binary compare between the files. You can accomplish that in several ways, which way is easiest depends on what tools you are familiar with. A full binary compare takes ages, you can get a 99.999% sure result by comparing md5 checksums instead.

 

Are you a unix command line guy, a windows gui guy, or somewhere in between? Tools are available for any environment you are comfortable with.

[pras1011]

I only really know Windows. Do you have a step by step guide? I need a very fast method to compare the drives!

[JonathanM]

I only really know Windows. Do you have a step by step guide? I need a very fast method to compare the drives!

Do you have a spare PC you can temporarily turn into a single drive unraid machine? The simplest method I can think of is to prepare a usb key with the free version of unraid, mount the old drive as a single data drive in that spare pc (unplug ALL current drives in your temporary box so you don't accidently erase them), map the old drive and new drive to a windows drive letter on another windows box, and use something like par2 or clonespy to verify that all files are duplicates.

 

Somebody else may have a better faster method.

[Joe L.]

Remember, modern hard drives are designed not to deliver erroneous data.  They are designed to either provide the correct data or no data at all (a read failure).  I believe that they all verify that the data has been written correctly to the drive at the time of the initial write. Otherwise, the drive would not 'know' that it had a write failure and to lock that sector out as being defective.

No disk that I am aware of tests the success of a "write"  by re-reading it from the physical platter.    The only time sectors are marked for re-allocation is when they fail to be read.  (and, as you said, they have internal CRC checks, and try several times before giving up)

 

The OS knows when a write fails, as the disk simply fails to respond that a write was completed.  I suppose if a track seek fails, then it might be affiliated with a "write" operation, but the actual failure is not that the "write" was tested for correctness, it was the disk could not seek to the desired track.  Checking the results of a "write" would take a full revolution of the disk platter, to bring the same sector under the read head and to read it.  I've not read of any disk that slows write operations in that way.  (although, I'm sure some may exist, in really critical hardware.  Certainly nothing we use today works that way.)

 

The issue is, the disk could not inform the OS the operation succeeds, and then test afterwards, it would have to slow everything down and only inform a write's success after a full disk revolution.

 

Unraid reads a sector to get the current contents, calculates parity, and then writes it, but does not re-read it.  (It would be pointless with most disks regardless, as the subsequent "read" request from the OS would be from RAM, the disk cache memory having still having the most recent data.  The disk would not even need to go to the physical disk.    You would need to force the disk to read from the physical disk,(and make a full revolution of the disk platter) to perform any verification, and that would slow things down considerably.

 

Joe L.