Replacing drive by harddisk that's bigger than parity drive


BartDG

Recommended Posts

  • Replies 92
  • Created
  • Last Reply

Top Posters In This Topic

Ok, after 30 hours of preclearing, the new 4TB drives are all precleared and I can go ahead with the procedure.

 

I figured I'd risk the "Swap Disable" procedure: I've got all my data backup up anyway, so if anything goes terrible wrong I can completely re-install the server no problem.

 

I just removed the faulty 3TB drive and attached one of the new 4TB drives in its place.

I booted the system up, and set the new HGST 4TB drive as the parity (no problem since all other drives are WD drives).  I used the old 3TB parity drive and set it as a data drive (replacing the faulty 3TB drive)

 

Parity and Disk 1 are now marked as "wrong". 

 

According to the manual I should now start the array, but alas, I cannot.  The option is greyed out, saying "Too many wrong and/or missing disks!".  Did I forget something?  I did everything the "swap disable" procedure told me to. 

Can somebody help me please?  Thx!

 

Edit: I found a more detailed explanation of the "parity swap" procedure.  .  It seems there was one important step in between that isn't mentioned on the other page, and that's starting the array with the faulty disk already disconnected.  Then immediately stop the array again.  Next, power down and add the new disk to the array, then power up, unassign the parity disk and swap it with the new disk.  Next assign the now unassigned disk to the "Disk 1" (in my case) slot. 

 

Then press the "copy" button in the "array operation section. 

 

Now it's busy copying.  Hope this won't take another 30 hours... :)

Link to comment

It'll take a long time, but not likely 30 hours  :)

 

Too late now ... but since you're (a) completely backed up and could easily restore the data to a completely new setup; and (b) want to switch your file system to XFS, which will require that you reformat the rebuilt drive anyway (which will delete all of the data on it) and then copy the data back from another location;  I would have simply created an entirely new array with the 4TB drives;  done a parity sync; and then copied all of your data from the backups to the array.

 

Done  :)

 

[And you would have still had the original disks untouched just in case anything went wrong -- i.e. some of your backup data wasn't readable]

 

THEN I would have pre-cleared the old disks you want to keep [NOT the "bad" drive that started you down this path]; and added them to the array.

 

Link to comment

Ah, yes, that would have been a (very good) option as well.  But I'm not 100% sure I got all the files with the backup, since some of the backup was done a long time ago (mostly DVD-rips) and I had to manually check if everything was there because some of the file names had changed over time.  I didn't want to go through the trouble of copying everything, because that would have added a lot of hours to an already long-taking task (but in hindsight...)  I'm pretty sure I got 99% of the files and even if I missed some it wouldn't be that bad since I still got the original DVD's, so I could always rip a few missing ones again (but I'd rather not since that is also a very tedious task). 

That's why I decided to take the chance and do the parity swap option because if all ended well, I would be absolutely sure I'd still have all my data.  A small, calculated risk. :)

 

The copy is now at 70 %.  It's indeed a lot quicker than a preclear.

 

I also wanted to ask : is the position of the drives in the "array of devices" important?  I'm asking because my next step will be to add another new 4TB drive drive, set it up as XFS and then move everything from the freshly created 3TB (reiser FS) drive to the new 4TB drive.  I then want to remove the 3TB drive (which sits in the Disk 1 slot) and replace it (in software) with the new 4TB drive (which now sits in Disk slot 3).  Can I do this without causing issue?  No real reason for this, this just seems "tidier" to me. :)

Link to comment

The PHYSICAL position of the drives doesn't matter ... but you can't change their assignments in the array without doing a New Config.

 

However, you CAN re-assign the drives to the slots you want by doing the following:

 

==>  Do a parity check to be CERTAIN all is well (no disk errors; no sync errors).  Also, be CERTAIN you know which drive is assigned as parity.

 

==>  Stop the array;  go to the Tools menu and do a New Config ... and reassign the drives to the slots you want them in [Do NOT change the actual set of drives assigned to the array at this point];  then, on the main page, check the "Parity is Already Valid" box and Start the array.

 

 

Link to comment

According to the manual I should now start the array, but alas, I cannot.  The option is greyed out, saying "Too many wrong and/or missing disks!".  Did I forget something?  I did everything the "swap disable" procedure told me to. 

Can somebody help me please?  Thx!

 

Edit: I found a more detailed explanation of the "parity swap" procedure.  .  It seems there was one important step in between that isn't mentioned on the other page, and that's starting the array with the faulty disk already disconnected.  Then immediately stop the array again.  Next, power down and add the new disk to the array, then power up, unassign the parity disk and swap it with the new disk.  Next assign the now unassigned disk to the "Disk 1" (in my case) slot. 

 

The updated procedure is found here ->  The parity swap procedure

Link to comment

I found another way, which I believe is easier to do.

 

If I use I use the information on this page, I'm thinking I can do it with less manual intervention.

 

This is my current setup

 

Parity - Hitachi 4TB

Disk 1 - WDC 3TB - ReiserFS

Disk 2 - WDC 2TB - ReiserFS

Disk 3 - Hitachi 4TB - XFS

 

Now, I want to move all the files from Disk 1 to disk 3.  I will then remove disk 1 and replace it with another Hitachi 4TB, after which I will move the files from disk 2 to disk 1.  By then both the old Disk 1 (WDC 3TB) and Disk 2 should be empty.  I would then reposition this old Disk 1 as Disk 4, after which I would like to reformat both disks with XFS.

 

Looking at the aforementioned page, this I believe is what I would need to do:

 

Disk 1 contains 4 folders : /Movies, /Music, /Photos, /Series

 

Would this be the correct command then?

nohup mv /mnt/disk1/* /mnt/disk3/ &

(what is the ampersand for at the end of the command?)

 

Then Disk 1 should be empty I believe.  I will then shutdown the system, remove Disk 1 (wdc 3TB), and put a Hitachi 4TB in its place.  Then power on, and assign the Hitachi to Disk 1 and also format it as XFS (all Hitachi disks have been precleared by now)

 

Disk 2 contains the same folders + my Docker image (which is only there for testing atm, so it's unused at the moment.  If this should be a problem, I can just remove it)

 

Next I do:

nohup mv /mnt/disk2/* /mnt/disk1/ &

This I believe should move the contents of Disk 2 to the new Hitachi at Disk 1.

 

Next, power down again.  Add the removed 3TB drive again and power up.  Add the 3TB as Disk 4.  Then format Disk 2 and Disk 4 as XFS.

 

Is this correct?  I'm not very linux savvy, so I would like to know if my commands are correct and if by doing it this way I cannot bump into the "user share copy bug".

 

I will hold off doing anything until someone here can give me the green light.  Thanks!

 

Link to comment

Ok, so I setup the plugin, which went fine.  I've setup one folder (photos) to move it from Disk 1 to Disk 3.  By default un-balance does a dry run first.  This went fine.  I then unchecked the "dry run" box, hit "move" again, but now nothing happens?  It does say "move operation in progress..3, but that's it.  The disks don't even spin up.  It's been like this for the last 15 minutes.  Any idea if I did something wrong?  And can I stop this somewhere?  (I don't see a stop button anywhere??)

 

Edit: tried again, but this time I didn't do a "dry run", just a simple calculate and then "move".  Now it's working.  Files are moving now! :D

I don't know what went wrong the first time, but I figured I'd mention it in the unbalance thread.  Maybe it'll be of some use to its author.

Link to comment

I found another way, which I believe is easier to do.

 

If I use I use the information on this page, I'm thinking I can do it with less manual intervention.

 

This is my current setup

 

Parity - Hitachi 4TB

Disk 1 - WDC 3TB - ReiserFS

Disk 2 - WDC 2TB - ReiserFS

Disk 3 - Hitachi 4TB - XFS

 

Now, I want to move all the files from Disk 1 to disk 3.  I will then remove disk 1 and replace it with another Hitachi 4TB, after which I will move the files from disk 2 to disk 1.  By then both the old Disk 1 (WDC 3TB) and Disk 2 should be empty.  I would then reposition this old Disk 1 as Disk 4, after which I would like to reformat both disks with XFS.

 

Looking at the aforementioned page, this I believe is what I would need to do:

 

Disk 1 contains 4 folders : /Movies, /Music, /Photos, /Series

 

Would this be the correct command then?

nohup mv /mnt/disk1/* /mnt/disk3/ &

(what is the ampersand for at the end of the command?)

 

Then Disk 1 should be empty I believe.  I will then shutdown the system, remove Disk 1 (wdc 3TB), and put a Hitachi 4TB in its place.  Then power on, and assign the Hitachi to Disk 1 and also format it as XFS (all Hitachi disks have been precleared by now)

 

Disk 2 contains the same folders + my Docker image (which is only there for testing atm, so it's unused at the moment.  If this should be a problem, I can just remove it)

 

Next I do:

nohup mv /mnt/disk2/* /mnt/disk1/ &

This I believe should move the contents of Disk 2 to the new Hitachi at Disk 1.

 

Next, power down again.  Add the removed 3TB drive again and power up.  Add the 3TB as Disk 4.  Then format Disk 2 and Disk 4 as XFS.

 

Is this correct?  I'm not very linux savvy, so I would like to know if my commands are correct and if by doing it this way I cannot bump into the "user share copy bug".

 

I will hold off doing anything until someone here can give me the green light.  Thanks!

 

The problem is that when you remove a drive, you invalidate parity, and if you remove a drive then replace it with another, you cause a complete drive rebuild, before you can then reformat the drive.  The only way to do what you want, using your steps, is to remove the parity drive first, which does make everything simpler and you can do it as you planned.  However, you will have to add the parity drive back after it's all finished, and let parity rebuild.

 

Another much smaller problem is that the shares will be disrupted during this, and/or you'll need to be constantly updating their share settings as to where they are currently located.

 

I don't know if you are interested (especially since you have already started), but I've just created a modified XFS conversion procedure that preserves parity throughout, and preserves all shares throughout too, in their current positions.  It's rather tricky, but (I think) is quicker and simpler, and I'd like to say safer, except that the instructions are tricky.  If you make a mistake, it could be disastrous.  The procedure also includes the recommended rsync command, which verifies the transfers.  If you are typing commands at a command line, I recommend it over cp and mv.

Link to comment

Thanks Rob! 

But as you say: I've already started, using the unbalance plugin.  It's working, but it's rather slow, I'm guessing no more than 15-20Mb/s.  That's fine, I've got the time.  But if there is a faster way, I'm always interested. 

 

Have you already posted your modified XFS conversion procedure somewhere?  I'd like to read about it, even if it is probably too late for me.

Link to comment

Ok, update!

 

Basically, everything went fine!

The parity swap procedure worked no problem.

I then added the other two 4TB HGST drives to the array. 

I then copied everything off of the 3TB and 2TB drives to the new 4TB drives via unBalance.

I then did a parity check, which was ok.

 

My array now looks like this:

 

-parity 4TB

- Disk 1 : WDC 3TB

- Disk 2 : WDC 2TB

- Disk 3 : HGST 4TB

- Disk 4 : HGST 4TB

 

I would now like to remove the two WDC drives from the array.  And then use the two 4TB drives as Disk 1 and Disk 2 instead (look tidier)

 

From what I was told before, I should stop the array, and then select: "new config"

Then I should re-arrange the drives the way I like them, leaving the two WDC drives as unassigned

 

Then I can power down and remove the drives.  Is this correct?

 

I'm also toying with the idea of using the 2TBV drive as a cache drive.  Would this be ok, or shouldn't I exptect too much of an improvement that way because it's not an SSD?

 

I also have an 128 GB SSD laying around (Crucial M4).  Would this be big enought or would the 2TB drive be better then?

 

Thanks!

 

 

Link to comment

...

My array now looks like this:

 

-parity 4TB

- Disk 1 : WDC 3TB

- Disk 2 : WDC 2TB

- Disk 3 : HGST 4TB

- Disk 4 : HGST 4TB

 

I would now like to remove the two WDC drives from the array.  And then use the two 4TB drives as Disk 1 and Disk 2 instead (look tidier)

 

From what I was told before, I should stop the array, and then select: "new config"

Then I should re-arrange the drives the way I like them, leaving the two WDC drives as unassigned

 

Then I can power down and remove the drives.  Is this correct?

And then rebuild parity. As already mentioned in this thread, you always have to rebuild parity when you remove drive(s).

 

I'm also toying with the idea of using the 2TBV drive as a cache drive.  Would this be ok, or shouldn't I exptect too much of an improvement that way because it's not an SSD?

 

I also have an 128 GB SSD laying around (Crucial M4).  Would this be big enought or would the 2TB drive be better then?

Depends on your use. I only have 120GB cache and it is only about half full, but I only use it for dockers and VMs, since I don't cache user shares.
Link to comment

I would now like to remove the two WDC drives from the array.  And then use the two 4TB drives as Disk 1 and Disk 2 instead (look tidier)

 

From what I was told before, I should stop the array, and then select: "new config"

Then I should re-arrange the drives the way I like them, leaving the two WDC drives as unassigned

 

Then I can power down and remove the drives.  Is this correct?

 

Yes, that's all there is to it.    As trurl noted, the system will do a new parity sync when you start the array ... you should do a parity check after that completes to confirm everything went okay.

 

 

... I'm also toying with the idea of using the 2TBV drive as a cache drive.  Would this be ok, or shouldn't I exptect too much of an improvement that way because it's not an SSD?

 

It'll work fine as a cache drive -- the fact it's not an SSD isn't a big factor, since when you write data to the array you're limited by the network speed anyway.    An SSD works much better when you're using it for application data storage, where the reads/writes are all local (not over the network) ... but for a cache for speeding up cached shares there's not much difference.

 

Link to comment

Ok, making a new config went perfectly.  I immediately assigned the 2TB WDC drive as a cache drive now, and left the WDC 3TB  drive as unassigned (I'll remove it after the parity check is done).

 

Then the parity check automatically started.  It's now humming along at about 38 MB/s, so it'll take about 1 day and 3 hours before it'll be finished.

 

This made me think: creating/checking parity is no other that doing XOR calculations, right?  So I imagine some CPU's are better suited than other for this?  I'm using a Core2Duo 8500, but using a different cpu could probably speed thing up a lot. 

Maybe even use a hardware co-processor?  Does unRaid support those?  Maybe even a dedicated GPU?  I would put in one of those cards into my system solely for this purpose if that would help. :)

Link to comment

Did a parity CHECK automatically start ... or are you referring to the parity SYNC, which is what should start when you first Start the array after you've done a New Config?

 

Assuming the SYNC is what it's doing now, you should then start a CHECK after it's finished -- this will confirm that the sync went perfectly (no undetected write errors).

 

As for the speed -- the XOR operations needed to conduct parity operations are not CPU intensive at all -- your Core 2 Duo 8500 can easily keep up with these.    Speed bottlenecks are generally due to either the controller(s) being used; or low-density disks.    But in your case, if I understand your final config, you just have 3 new 4TB Drives assigned -- is that correct?    If so, 38MB/s is a very slow speed for the parity sync -- indicating you may have an issue with one of the new disks ... UNLESS -- are you by any chance using the array during the parity sync?    [i.e. copying something to it; streaming something from it; etc. => these activities will slow things down a LOT and would easily explain the slow speed]

 

Link to comment

Sorry, it's doing a parity sync.  Indeed, the parity check needs to be done afterwards.  It's now at 38%.  It's been at it the whole day.

 

My hardware is the following:

An Asus P5E VM DO motherboard (using onboard VGA, no extra add-on cards)

16GB ram

3 HGST ultrastar 4TB 7200 rpm harddisks (running at AHCI mode)

 

I think this should be a good configuration.

 

I'm not using the unRAID server for anything at the moment.  It's happily purring along doing a parity sync, and that's all it's doing at the moment. 

 

This is a copy/paste of the PCI devices in the system devices tab:

00:00.0 Host bridge: Intel Corporation 82Q35 Express DRAM Controller (rev 02)
00:02.0 VGA compatible controller: Intel Corporation 82Q35 Express Integrated Graphics Controller (rev 02)
00:03.0 Communication controller: Intel Corporation 82Q35 Express MEI Controller (rev 02)
00:03.2 IDE interface: Intel Corporation 82Q35 Express PT IDER Controller (rev 02)
00:03.3 Serial controller: Intel Corporation 82Q35 Express Serial KT Controller (rev 02)
00:19.0 Ethernet controller: Intel Corporation 82566DM-2 Gigabit Network Connection (rev 02)
00:1a.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #4 (rev 02)
00:1a.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #5 (rev 02)
00:1a.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #6 (rev 02)
00:1a.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #2 (rev 02)
00:1d.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 02)
00:1d.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 02)
00:1d.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 02)
00:1d.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 92)
00:1f.0 ISA bridge: Intel Corporation 82801IO (ICH9DO) LPC Interface Controller (rev 02)
00:1f.2 SATA controller: Intel Corporation 82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] (rev 02)
00:1f.3 SMBus: Intel Corporation 82801I (ICH9 Family) SMBus Controller (rev 02)
00:1f.6 Signal processing controller: Intel Corporation 82801I (ICH9 Family) Thermal Subsystem (rev 02)

 

Does that give  you enough info?

 

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.