Question: Array Best Practices (full array swap)


Recommended Posts

Replacing 5 x 4 TB disk with 4 x 10 TB + 1 x 12 TB (parity).

 

All disks passed 3 preclear runs with clean SMART

 

Currently Syncing Parity on 12TB

 

Q1:  Is it best practices to run a parity check after swapping each of the four data disks (or does the data-rebuild operation preserve|rely on parity)?

Q2: Is it best to keep dockers | vms offline during sync | build operations (no writes, just reads for dockers + VMs run off of either cache-pool or dedicated nvme)?

 

I have done reading on how parity works and preserves (spreads) data - but I think I am fuzzy on the concept when it comes to disk access during build operations

 

Thank you.

 

Link to comment
13 minutes ago, J.Nerdy said:

Replacing 5 x 4 TB disk with 4 x 10 TB + 1 x 12 TB (parity).

 

All disks passed 3 preclear runs with clean SMART

 

Currently Syncing Parity on 12TB

 

Q1:  Is it best practices to run a parity check after swapping each of the four data disks (or does the data-rebuild operation preserve|rely on parity)?

Q2: Is it best to keep dockers | vms offline during sync | build operations (no writes, just reads for dockers + VMs run off of either cache-pool or dedicated nvme)?

 

I have done reading on how parity works and preserves (spreads) data - but I think I am fuzzy on the concept when it comes to disk access during build operations

 

Thank you.

 

The fastest way is to install both the new and old disks the mount new disks in Unassigned Devices then cp disk-to-disk (e.g. 4TB A -> 10TB A, 4TB B -> 10TB B etc.). Since the number of disks match (presumably your 5x4TB = 4 data + 1 parity), you can even run the cp in parallel e.g. using CA User Script run in background functionality. Then you remove the old disks, create a new config and build the 12TB parity.

A derivation of the above method is to add all the new disks to the array but remove parity and then again do cp disk-to-disk. It's slightly less safe (no parity protection during migration) but there's zero format risk with UD (not that there's any substantial risks, UD should do xfs format in such a way that the disks can be added to the array directly without requiring the disks being reformatted).

 

Your rebuild method is not wrong but it's a lot slower since you can only do it 1 disk at a time.

The instruction was intended for single disk replacement while in your case it's a wholesale migration.

 

Q1: no need to rerun parity check.

Q2: no need to turn off docker and VM, especially if you are sure there's no write. It may slow things down a little. On 6.7.0+ though, you may have terrible performance for any array access during rebuild due to the bug with read/write priority that was raised recently.

  • Upvote 1
Link to comment

Tremendous answer.

 

I already have the parity rebuild underway... and will be away for most of the day, I will let that finish just to have array protection.

 

However after that (all sata ports are full), would the following be appropriate?

 

1) Mount new disk 1 and new disk 2 (using UD) and either use screens (cp not dd, correct?)  or Krusader to clone old disk 1 and old disk 2 to new drives ... (I have 2 bay vertical dock usb 3.1).

 

2) Do the same for new disk 3 and new disk 4

 

3) shut down server

 

4) swap all disks

 

5) create new config

 

6) assign disks properly (disk 1-1 disk  2-2 ...)

 

7) start array and begin parity sync

 

 

Link to comment
5 minutes ago, J.Nerdy said:

1) Mount new disk 1 and new disk 2 (using UD) and either use screens (cp not dd, correct?)  or Krusader to clone old disk 1 and old disk 2 to new drives ... (I have 2 bay vertical dock usb 3.1).

2) Do the same for new disk 3 and new disk 4

3) shut down server

4) swap all disks

5) create new config

6) assign disks properly (disk 1-1 disk  2-2 ...)

7) start array and begin parity sync

Yes, that should work. A few things to keep an eye for

  • Make sure your 2-bay dock is mounting each disk as individual devices and not some weird JBOD or RAID nonsense. It shouldn't happen but check the manual just to be sure.
  • I do cp just because I have always used cp and I know it works every time.
  • Don't use Krusader (and mc) for large migration. I have found these more advanced tools to cause issues (e.g. Krusader grinding to a halt after a few hundred GBs, mc causing fragmentation etc.).
Link to comment
7 hours ago, testdasi said:

Yes, that should work. A few things to keep an eye for

  • Make sure your 2-bay dock is mounting each disk as individual devices and not some weird JBOD or RAID nonsense. It shouldn't happen but check the manual just to be sure.
  • I do cp just because I have always used cp and I know it works every time.
  • Don't use Krusader (and mc) for large migration. I have found these more advanced tools to cause issues (e.g. Krusader grinding to a halt after a few hundred GBs, mc causing fragmentation etc.).
  • The same dock I used for preclearing:  mounts the disk individually as /dev/sd#; /dev/sd% etc... so all good there
  • In CP we will trust

The New config should recognize|respect all shares, disk settings, etc... correct?

 

Thank you!

 

EDIT:  So began reading about write bottlenecking that @johnnie.black documented, which of course lead me down the SQLite thread as well.  While I have noticed degraded performance since the upgrade to 6.7.2 (especially HDR streams) I assumed it was a configuration derp that I introduced.  Considering the downgrade now.

 

Just to be sure: 

 

cp /mnt/disk1  /mnt/disks/sdW

    /mnt/disk2 /mnt/disks/sdX

    /mnt/disk3 /mnt/disks/sdY

    /mnt/disk4 /mnt/disks/sdZ

 

New config

 

Assign w to 1, x to 2, y to 3, z to 4

 

Starty array and begin parity sync

 

If this is the case, I will stop parity sync - take parity off line and proceed with the copies. 

 

 

Edited by J.Nerdy
process question
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.