Evict a disk before removing it?

[jrrjrr]

I'm doing a little bit of planning before I begin the evaluation period. To minimize how many new disks I need to buy, I intend to start with some older spare 4T disks, and then if I decide to keep Unraid, I'd swap in some 8T disks from the system I'm replacing.

 

Reading through documentation, I don't have a great feeling about the disk upgrade process:

https://wiki.unraid.net/Replacing_a_Data_Drive

 

I was hoping for a process like this:

1. Add new drive
2. Evict old drive
3. Remove now-empty old drive
4. Scan/Rebuild parity? (seems like this could be a no-op)

 

For #2, I imagine a feature where you tell Unraid that you want to retire the disk, and it moves all the data onto other disks.

 

Basically it feels like the "voluntarily replace a good disk" process should be gentler and faster and less-risky than the "replace a failed disk" process. (with parity protection for the whole time)

 

The closest thing I've found in the docs is the "Alternative method" for "Removing data disk(s)":
 

Quote

 

It is also possible to remove a disk without invalidating parity if special action is taken to make sure that the disk only contains zeroes as a disk that is all zeroes does not affect parity. There is no support for this method built into the Unraid GUI so it requires manual steps to carry out the zeroing process. It also takes much longer than the simpler procedure above.

 

There is no official support from Limetech for using this method so you are doing it at your own risk.

 

 

https://wiki.unraid.net/Manual/Storage_Management#Removing_data_disk.28s.29

 

Have I missed anything? Is something like this on the roadmap for the future, or available via plugins?

[Squid]

You can do that process that you'd like now.  No real issues.  Add the new drive, copy the files over, remove the old drive and rebuild parity (or alternatively zero out the drive first and then remove it)

 

The fastest method is to simply replace the drive.  And, since you still have the original drive intact from how it left it is the least risky.

[trurl]

1 hour ago, Squid said:

The fastest method is to simply replace the drive.  And, since you still have the original drive intact from how it left it is the least risky.

And rebuilding the data drive is as fast as rebuilding parity, even faster if the data drive is smaller than parity.

 

So, rebuild to a new data disk saves you from having to copy anything. Everything is just as it was before except all the data is on a new (larger if you want) disk assigned to the same slot.

 

Really, this is the whole point of having parity in the first place. Can't imagine why you would consider doing it any other way.

[trurl]

1 hour ago, jrrjrr said:

(with parity protection for the whole time)

At step 4 of your proposed solution, you HAVE TO rebuild parity, and you aren't protected until it is rebuilt.

[jrrjrr]

2 hours ago, trurl said:

At step 4 of your proposed solution, you HAVE TO rebuild parity, and you aren't protected until it is rebuilt.

 

I think I'm probably working with an incomplete picture of how unraid arrays work, or didn't go deep enough on how I imagine it could work . I'll expand my imaginary steps into more detail, and maybe you can point to the places where I'm wrong?

1. Add new disk
   1. Unraid zeroes the drive and adds it to the array. Parity is unchanged because XOR.
2. Evict old disk
   1. I tell unraid that I want to migrate all the data off of this drive in order to remove it.
   2. Unraid stops writing new files to the disk.
   3. Gradually move all the files off of the disk, onto the other disks, updating parity along the way.
   4. (at this point we have an empty filesystem, but not a zeroed disk)
   5. Zero the disk, updating parity along the way.
3. Remove now-empty old drive - to be more clear, a now-zeroed old drive
4. Scan/Rebuild parity? (just scan, I suppose, because parity should already be valid)

I think this is basically the same as the Alternative method from that wiki page. So I guess I just want it to be a first-class supported feature, driven by a few clicks in the UI instead of a series of dangerous manual steps.

 

[jrrjrr]

4 hours ago, Squid said:

Add the new drive, copy the files over,

 

Now I'll ask an even newbier question: how do you manually move files between disks that belong to an unraid array? Can you individually mount the component disks? Do writes to those disks update parity? Which manual page describes this, so that I can ask fewer dumb questions?

[ChatNoir]

4 hours ago, jrrjrr said:

Can you individually mount the component disks?

Individual drives CAN be shared. However you should NEVER mix disk share and 'regular' Unraid shares when doing file operations as there is a risk of data loss.

 

Most people do those disk operation through :

• a file explorer docker
• Midnight Commander (built in Unraid, 'mc' on the console) - that's my preference
• the Unbalance plugin

4 hours ago, jrrjrr said:

Do writes to those disks update parity?

Absolutly.

[hawihoney]

10 hours ago, jrrjrr said:

Reading through documentation, I don't have a great feeling about the disk upgrade process

 

Why do you have bad feelings about that process? I guess 99% of disk replacements are done just that way. In our servers I've done that over 100 times without one single problem.

 

- Stop the parity protected array

- Shutdown the server *1)

- Physically replace the old disk with the new disk

- Start the server

   --> With a replacement disk installed Unraid will not start the array automatically

- Replace the old disk with the new disk in the GUI

   --> Notice the warning beside the buttons on the Main Page (you are going to start a disk replacement)

- Click the checkbox "Yes, I'm sure"

- Start the array

   --> At that time, during Unraid overwrites the new disk with the data of the old disk, you can begin to work with the array again *2)

 

*1) With server grade hardware (professional backplane) you don't need to shutdown the server, stopping the parity protected array will be enough to swap the disks in the GUI. Simply refresh the Main Page and make the changes in the GUI mentioned above.

 

*2) If the old disk ist still readable, you have a backup of the data thats written to the new disk.

 

Edited February 7, 2022 by hawihoney

[trurl]

Another way to keep parity protection going throughout the whole process would be to assign the new, larger disk as parity2, then after that is built, remove parity and use it to replace a data disk.

 

Parity and parity2 are not interchangeable (different algorithms), but gives the same protection and you can have parity2 without parity.

 

7 hours ago, hawihoney said:

Why do you have bad feelings about that process? I guess 99% of disk replacements are done just that way. In our servers I've done that over 100 times without one single problem.

 

If you are having problems already then the more complicated methods you are considering are likely to be more problematic than the simpler methods.

 

If you are careful (always double-check connections when mucking about inside), and diligent (setup Notifications to alert you immediately by email or other agent as soon as a problem is detected), any data loss is rare. Probably most of the data loss I see on the forum are user error.

 

And you must always have another copy of anything important and irreplaceable, of course. Parity is not a substitute for backups, whether Unraid or some other system.

[trurl]

18 hours ago, jrrjrr said:

"voluntarily replace a good disk" process should be gentler and faster

And if you're not having problems already, none of your suggestions would be any faster. In fact, they would be (much) slower if data has to be moved to other disks in the array.

[jrrjrr]

Well, the whole thing just became moot: the old disks I dug out have errors, so I'm buying enough new disks to start from scratch. But also you've convinced me to give up on graceful gradual upgrades and just let rebuild do its thing. Thanks for your help everybody!