Upgrading HDDs with dual parity


Recommended Posts

I am in the process of replacing all my 6TB HDDs with 12TB HDDs. After some reading on this site, I found out that I am able to swap out BOTH of my dual parity drives at the same time. I realize this is slightly riskier than doing them one at a time. But that got me thinking. Since I will have valid parity, and dual parity drives running after it is done, am I able to swap out 2 data drives at a time? Again, I realize this would be slightly more risky, but I thought that was the entire point of dual parity, to allow for 2 drive to fail at the same time? So, in theory, it could also be helpful in speeding up the time it takes to upgrade correct?

Link to comment

This is possible in theory (with the increased risk you mention) but you may find that 2 rebuilds running in parallel actually take longer than doing them sequentially due to disk contention between the two operations.    It would all depend on whether that contention caused additional head movements on the drives or whether the fact they started at the same time means you get away with it.   If not then doing them sequentially may end up being faster.

 

Have  you tried it on the parity drives yet?  If they could both be replaced at the same time without adversely affecting the speed of the parity build then that might be a good indication of whether rebuilding two data drives at the same time would not end up being slower than doing them sequentially.

Link to comment
3 minutes ago, itimpi said:

This is possible in theory (with the increased risk you mention) but you may find that 2 rebuilds running in parallel actually take longer than doing them sequentially due to disk contention between the two operations.    It would all depend on whether that contention caused additional head movements on the drives or whether the fact they started at the same time means you get away with it.   If not then doing them sequentially may end up being faster.

 

Have  you tried it on the parity drives yet?  If they could both be replaced at the same time without adversely affecting the speed of the parity build then that might be a good indication of whether rebuilding two data drives at the same time would not end up being slower than doing them sequentially.

I am running it right now, and it has been going for about 30 minutes. Write speed seems to be hovering around 200-208 MB/sec. Not sure if it is running as fast as a single would be, as I have never done a single before. It estimated about 16.5 hours for completion. The parity check I ran before this (when they were all 6TB drives) took about 12 hours I think to complete. But the 6TB drives were also 7200 rpm, and the 12TB drives are only 5400 rpm drives. So I am sure they will be slower anyway. 

Link to comment
3 minutes ago, jebusfreek666 said:

I am running it right now, and it has been going for about 30 minutes. Write speed seems to be hovering around 200-208 MB/sec. Not sure if it is running as fast as a single would be, as I have never done a single before. It estimated about 16.5 hours for completion. The parity check I ran before this (when they were all 6TB drives) took about 12 hours I think to complete. But the 6TB drives were also 7200 rpm, and the 12TB drives are only 5400 rpm drives. So I am sure they will be slower anyway. 

That sounds promising then and sounds like there is little (if any) contention.   I would be interested to hear how replacing the data disks goes.

 

Normally the key driver for having dual parity is to handle the case of a second drive failing while recovering a failure of the first one.    However in your particular case of replacing good drives you have the original (good) drive still intact so the risk should be minimal.

Link to comment
you may find that 2 rebuilds running in parallel actually take longer than doing them sequentially due to disk contention between the two operations.

On a server with dual parity rebuilding two disks at the same time takes the same time as rebuilding only one, I always do my disk upgrades two by two to save on rebuild time.

 

 

  • Thanks 1
Link to comment
4 minutes ago, johnnie.black said:

On a server with dual parity rebuilding two disks at the same time takes the same time as rebuilding only one, I always do my disk upgrades two by two to save on rebuild time.

 

 

Good to hear that it tends to work fine :)

 

I suspect that is because the rebuild process tends to access roughly the same sector on both disks at the same time so no head movement gets involved?   Have you ever tried rebuilding 2 disks at the same time that have significantly different performance characteristics to see if this remains the case?

Link to comment
4 hours ago, itimpi said:

That sounds promising then and sounds like there is little (if any) contention.   I would be interested to hear how replacing the data disks goes.

 

Normally the key driver for having dual parity is to handle the case of a second drive failing while recovering a failure of the first one.    However in your particular case of replacing good drives you have the original (good) drive still intact so the risk should be minimal.

Just a quick update on this. It is still running the rebuild on the 2 parity disks at the same time. However my speed has fallen from 200 MB/sec to around 170. I think this is normal, as this is kind of what I saw with all my preclears as well, but wanted to make sure.

Link to comment
1 minute ago, jebusfreek666 said:

Just a quick update on this. It is still running the rebuild on the 2 parity disks at the same time. However my speed has fallen from 200 MB/sec to around 170. I think this is normal, as this is kind of what I saw with all my preclears as well, but wanted to make sure.

It is quite normal for array operations to drop in speed as they reach the inner tracks of drives as there the raw transfer speed is lower.

Link to comment
6 hours ago, jebusfreek666 said:

Just a quick update on this. It is still running the rebuild on the 2 parity disks at the same time. However my speed has fallen from 200 MB/sec to around 170. I think this is normal, as this is kind of what I saw with all my preclears as well, but wanted to make sure.

yea, and that 16 hour time is optimistic at best.  you're going to see speeds dropping to about half your start rate.

Link to comment

One more question about upgrading disks. Since the rebuild is basically the same procedure as a parity check with the added steps of emulating and writing the data to the new drives, do I need to do a parity check between completions? I mean right now I have it building parity 1&2. It is in maintenance mode, so nothing is being written to the drives anyway. When the parity drives complete, do I need to run a parity check before proceeding? Or can I just shut it down, pull 2 more drives, assign them in the place of the 2 I pulled, and start the rebuild again? 

Link to comment

Also, since I am going from 6TB to 12TB drives, I assume that means my first "high water" mark would be 6TB? Does that mean if I pull all the drives and replace them with the 12TB, then add back an empty 6TB, it will basically fill the 6TB (minus the amount I have set in reserve) before continuing to add to the 12TB drives?

Link to comment
5 minutes ago, jebusfreek666 said:

Also, since I am going from 6TB to 12TB drives, I assume that means my first "high water" mark would be 6TB? Does that mean if I pull all the drives and replace them with the 12TB, then add back an empty 6TB, it will basically fill the 6TB (minus the amount I have set in reserve) before continuing to add to the 12TB drives?

No.    It means that an empty 6TB drive will not start being used.    When the other drives get down to 6TB the next mark of 3TB comes into play and the drives will start being used (in ascending order) until they get down to 3TB free, etc

Link to comment
5 hours ago, itimpi said:

In Principle the parity check is not required although some people seem to like to carry it out as a confidence check !   If you do run it there should be 0 errors reported.

Would you suggest running it between each swap? Is that what you would do? After this, I have 3 more swaps to run (if I do 2 at a time). I am already looking at a full day for every 2 drives, and don't really want to add another half day in between each step if I can avoid it. That is a lot of down time. 

Link to comment

Have never done a large-scale replacement of drives like you are doing so have never really had to decide.   I would probably not bother doing it every time, but I think I would do it at least once to give me confidence that the replacement process was working as expected and I got 0 errors.   I would also do one before repurposing any removed drives.    What is the state of backups if at any point you DO get any errors reported?  
 

 

Link to comment
6 hours ago, itimpi said:

Have never done a large-scale replacement of drives like you are doing so have never really had to decide.   I would probably not bother doing it every time, but I think I would do it at least once to give me confidence that the replacement process was working as expected and I got 0 errors.   I would also do one before repurposing any removed drives.    What is the state of backups if at any point you DO get any errors reported?  
 

 

Finished the first round, the 2 parity drives. All looks good to me. 

 

Event: Unraid Parity sync / Data rebuild
Subject: Notice [TOWER] - Parity sync / Data rebuild finished (0 errors)
Description: Duration: 22 hours, 29 minutes, 55 seconds. Average speed: 148.2 MB/s

 

All vital files are backed up in the cloud and also on a seperate drive outside my home. All non-vitals are backed up just in the cloud.

Edited by jebusfreek666
Link to comment

Running 2 data disks at the same time has taken a major dump. Currently speeds are sitting around 105 MB/sec at 50% completion. Estimated time for completion is around 27 hours.

 

Edit:

Once it crossed the 50% threshold, speeds improved to around 170 MB/sec. I am guessing that it was on the inner tracks when I looked. New estimated time is around 20 hours, so it looks like the data disks will end up being about the same as the parity disks. 

Edited by jebusfreek666
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.