Greetings,

 

I am planning both a S/W and H/W upgrade - moving from 4.3.3 to 4.5.??

 

I have read as much as I could about the releases.  Seems 4.5.3 has an "unformatted" bug and that 4.5.2 is a better release at this point, but I can't find it for download.  Any suggestions?

 

My set-up is in my sig.  Are there any other issues I am likely to face? I don't like making changes when things are working great, but increased writes sounds like a good reason to upgrade the S/W.

 

Also, I'm about to upgrade to my first 2Tb drive.  It'll be a WD-EARS (Most of my other drives are WD-EADS HDD's).  I have read that a jumper should be connected between pins 7-8. Is there anything else I should know about these drives?

 

I'm going to install two EARS - one for parity and one for data.  The data drive will be replacing 4x500Gb WD's.  This will finally leave me some spare slots.  My question is in regard to the best method for transferring the data from the other 3 drives onto the new 2Tb drive.  I know I can do it across the network via my XP machine.  What about MC.  I haven't used it. Any advise?

 

How does one check the files are transferred correctly (or compare it to the original drives?)?

 

Anything else I should know?

 

Things like this always make me fret, so thx for any help.

 

 

 

 

First of all, you should do all your hardware upgrades first, then do your software upgrade second.

 

I have read as much as I could about the releases.  Seems 4.5.3 has an "unformatted" bug and that 4.5.2 is a better release at this point, but I can't find it for download.  Any suggestions?

The safest bet at the moment is to upgrade to 4.5.1.  4.5.2 doesn't seem to exist, I don't know why.  You may also want to disable spin up groups as soon as you upgrade (on the Settings page), since some people have had problems with them.  Spin up groups were created to address a problem of stuttering video and music while another drive on the same controller is being accessed.  If you don't have this problem with your current setup, then you shouldn't need to use spin up groups.

 

You should see the speed increases immediately after upgrading, though, and yes, they are significant and worth the trouble.

 

My set-up is in my sig.  Are there any other issues I am likely to face? I don't like making changes when things are working great, but increased writes sounds like a good reason to upgrade the S/W.

Nope, looks fine to me.  My primary server uses a Promise TX4 as well, and I didn't have any trouble upgrading from 4.4.2 to 4.5.1.

 

Also, I'm about to upgrade to my first 2Tb drive.  It'll be a WD-EARS (Most of my other drives are WD-EADS HDD's).  I have read that a jumper should be connected between pins 7-8. Is there anything else I should know about these drives?

Nope, just jumper the WD EARS drives before installing them into your server and all should be well.  Make double-extra sure that you jumper pins 7-8 and not pins 1-2.

 

I'm going to install two EARS - one for parity and one for data.  The data drive will be replacing 4x500Gb WD's.  This will finally leave me some spare slots.  My question is in regard to the best method for transferring the data from the other 3 drives onto the new 2Tb drive.  I know I can do it across the network via my XP machine.  What about MC.  I haven't used it. Any advise?

I also haven't used MC, so I can't help you here.  Here's how I would do it (let's call it the slow and lazy way):

1) Run a parity check on your current setup.  If there are any errors, seek help here, do not proceed.

2) Power down, physically install new drives.

3) Power up, Preclear all new drives (after installing the jumpers).  You can preclear multiple drives simultaneously.  Do not assign the drives to unRAID yet.

4) Assuming all the new drives pass preclear, stop your array, go to the Devices page, and unassign your parity drive.  In it's place, assign the new parity drive (WD EARS).  Start the array.  You should get a message about upgrading parity, and a parity sync should begin.  Do not press either the restore button or the format button.

5) Once the parity sync finishes (which will take a long time), run a parity check (this will also take a long time).  Again, if the parity check returns any errors, seek help here.

6) Stop the array, power down.

7) Physically remove one of the 500 GB data drives (doesn't matter which).

Power up, unRAID will tell you something about a missing disk.  Go to the devices page and assign the second WD EARS to the missing disk's slot.  Go back to the main page.  unRAID will tell you that you are upgrading a disk.  Start the array, and a data rebuild should start.  Do not press either the restore button or the format button.

9) Once that finishes, run another parity check.  Seeing the pattern here?  Basically, run a parity check before and after doing anything.  Again, seek help if there are errors.

10) Manually copy the contents of the other three data disks to the new WD EARS data disk.  I would recommend doing them one at a time instead of simultaneously to avoid lots of disk fragmentation.  Just start one transfer each night as you go to bed and each morning as you leave for work/school.  Make sure you copy the data, not move (cut-copy) it.  That way you'll still have a backup copy in case something goes wrong during one of the transfers.

11) Once all your data is transferred off the three 500 GB disks, stop the array and power down the server.  Physically remove the three 500 GB disks.

12) Run a parity check.  Again.

13) Power up the server.  It will complain about missing disks and not let you start the array.  Press the restore button.  This is one of the two possible cases in which you actually do want to press the restore button.  What this does is forget about your old configuration (meaning it forgets about the three missing disks) and starts a new parity sync.  unRAID will stop complaining about missing disks and will start a new parity sync with only the disks that are currently installed in your server.  Note that you will be without parity protection for this step, until the parity sync completes (which is another over-night operation).  This is a risk you are going to have to take, and another reason why the initial preclears of your new disks is all that much more important.  If a disk does fail during this process, you will at least have all your data on the four 500 GB drives safely intact, since none of them have been overwritten.

14) Once the parity sync completes, run a parity check.  Yes, again.

 

This whole process will take you probably the better part of a week, many steps will take many, many hours to complete.  Figuring out MC or similar will save you time in step 10, but all the other steps are completely necessary.

 

 

How does one check the files are transferred correctly (or compare it to the original drives?)?

MD5 checksums?  I don't really know.  That's WeeboTech's department.  I just compare file sizes and play a random sampling of files.  I also perform a particular ritual dance involving a few domesticated animals, but I won't get into that here.

 

Anything else I should know?

A cow has four stomachs!

Thx Rajahal for the detailed reply.  Much appreciated.

 

I can see there is going to be some effort here.  My new drives arrived today, but I'm not sure when I'll start the operation - hopefully this week if time permits.

 

It appears pins 7-8 are the inner pins looking at the HDD from the top (spec sticker facing up).  I asked the supplier to provide some jumpers, but they obviously don't read the "comment" segment.  Thankfully, I found some in my box of bits.

 

All the parity checks are really going to hammer the box.  I guess the power bill will be higher this month  .

 

I'll get the H/W stuff sorted first as you said - give it some time to bed in, then tackle the S/W component.

 

Thx again for the help.

No problem.  The sticker on the WD EARS HDDs should have a diagram which identifies the pin numbers.  I believe that pins 7-8 are on the bottom-right if you holding the drive upright so that you can read the sticker, but that's just from memory, so it may be wrong.

 

Yeah, you will use up a bit of power during the week that you tackle this, but it will pay off in the long run since you are consolidating 4 non-green drives for a single green drive.

Just got my new parity drive installed and it's currently doing a sync.  I was a little shocked to see that initially it says it'll take 40+ hours to sync. I realise this will change as the smaller drives complete, but even half of that is a long time.

 

Am I right in assuming the array is "Unprotected" while the the new parity drive sync is taking place? I guess if this is the case - should something happen, I can put the old parity drive back in.  Will this work? I did take a backup of the flash before the upgrade.  Thx.

 

Edit: As an afterthought, why does it take longer to sync a new (bigger) parity drive when all the data disks remain the same?

Yes, your array is unprotected while parity is syncing.  As soon as the parity sync completes your array is technically protected, but you won't really know if that protection will do you any good until you run a parity check and ensure that your parity data can be read back.  Once both the parity sync and the parity check complete with no errors, then you can trust that your data is safe.

 

If a disk were to die during the parity sync, you could put your old parity disk back in.  I think you would have to use the 'Trust My Array' procedure, but I'm not 100% sure.  If this happens, we'll need someone with more experience to jump in and guide you.

 

Even when the parity drive is larger than the largest data drive, the parity calculations still need to run on the left over bits.  Say you have a 2 TB parity drive and your largest data drives is 1.5 TB.  Once the parity check passes the 1.5 TB drive you'll see the parity check speed increase dramatically for that final 500 GB's worth of calculations, but it will still take some time.  To be completely honest, I don't understand why it can't be an instantaneous operation since all the bits in that remaining 500 GBs are (or at least should be) zeros.  All I know is that during that period you'll see all your data disks spun down and your parity drive still spinning and working.  I imagine a faster CPU may help with this last part of the parity check as well.  Maybe another hero member can explain this all better.

Thx for the input.

 

I got up this morning and found that after 16 hours it is only 46% complete.  With a 1tb parity, the whole process only took about 18 hours. Maybe going to 2tb drives wasn't such a good thing.

Thx for the input.

 

I got up this morning and found that after 16 hours it is only 46% complete.  With a 1tb parity, the whole process only took about 18 hours. Maybe going to 2tb drives wasn't such a good thing.

and if 1TB took 18 Hours, you expected 2TB to be ??

 

Regardless, once you get past the 1TB mark it will dramatically speed up since it only has to write the parity drive.  It might even be done by now.

 

Since reading the other disks is not involved once you get above their size, you are limited only by the speed you can write to the parity disk. 

 

If writing is at a rate of 100MB/s it will take 10 seconds per Gigabyte.  For 1000 Gigabytes (1TB), that would then be 10,000 seconds or 166.66 minutes or 2.77 hours.  At 50 MB/s it would be just over 5 hours.  Odds are your parity calc speed once you get above the 1TB mark is somewhere in between if you have fairly modern hardware.

 

 

and if 1TB took 18 Hours, you expected 2TB to be ??

 

I guess I never really thought too much about it. I certainly didn't expect the estimate to be 42 hours.  Came as a surprise.

 

In the end it took 22 hours to complete parity.  I guess it's going to be longer once I put the first 2Tb data drive in.  It won't be long until I'll find out 

 

One of the things I don't really understand is why adding a bigger parity drive increases the calc time when all the other disks remain the same. I would've thought that the count would be the same until you add a bigger data drive.  Hopefully, the increased write speed with the later version will help somewhat.

In the end it took 22 hours to complete parity.

My calculations were correct.

 

unRAID works it magic of allowing us to use various sized drives in a RAID array by pretending all of the data disks are the same size as the parity disk.  When it needs to calculate a parity bit, if a given data drive is smaller than parity it substitutes a "zero" for that byte it would have read from the smaller drive if it was of equal size to the parity drive.

 

Therefore, unRAID still calculates parity for the second TB of your new 2TB drive, and still writes it to the 2TB drive, pretending all the data drives are 2TB, but substituting zeros for the data if the drives are not. 

 

Since all the existing data drives are smaller than 2TB, it would then write a "zero" in each bit position above 1TB to maintain an even number of "1" bits for even parity.

 

If your 1TB parity calc took 18 hours, and the new 2TB parity calc took 22 then it took 4 hours to write the additional 1TB.

 

4 hour = 240 minutes = 14400 seconds.  That's 14.4 seconds per Gigabyte, or 69.4MB/s write speed to the parity disk.

I had estimated in my prior post somewhere between 50 and 100 MB/s.

One of the things I don't really understand is why adding a bigger parity drive increases the calc time when all the other disks remain the same. I would've thought that the count would be the same until you add a bigger data drive.

Because, unRAID pretends all the disks are the same size as the parity disk.  It still needs to write parity for the initial calc, and will still need to read it when verifying subsequently.

  Hopefully, the increased write speed with the later version will help somewhat.

The "write" speed to the array has never had anything to do with the length of the parity calculation.  When writing to the array only the parity disk and the disk being written to are involved.  No matter whose hardware is involved, as long as the network is not a bottleneck the speed to write to the array is limited by the rotational speed of the slowest disk involved.  Hopefully, your new 2TB drive is as fast or faster in its rotational speed as your old 1TB parity drive.  If it is, the speed "writing" to the array will be the same.

 

Joe L.

Thanks Joe!  I had always wondered about that as well.  Adding it to the wiki (best of the forums).

Wow...thx Joe...your comments have helped me heaps to understand how the magic works.

 

So...if the parity is calculated on a 2Tb drive, does this mean that when I add the first 2Tb data drive the parity will stay at 22 hours or will the time increase because it has to physically read the data disk(s) - so the calc takes longer??

 

Did I understand this correctly?

 

 

Wow...thx Joe...your comments have helped me heaps to understand how the magic works.

 

So...if the parity is calculated on a 2Tb drive, does this mean that when I add the first 2Tb data drive the parity will stay at 22 hours or will the time increase because it has to physically read the data disk(s) - so the calc takes longer??

 

Did I understand this correctly?

 

 

I will guess it will increase.

Thought so.  Thx.