Reducing the number of disks.

[richardb]

Hey all, I have a slightly unusual question: I want to remove several disks from my array. I have been doing some rationalization on my system, and I realized that I have more space than I need: I have 32TB, but am only using about 8TB most of the time. So, I'd like to remove some of the drives to save power, simplify the system and use them elsewhere or keep them as cold spares. However, I can't find any specific instructions on this.

 

My initial thought is that it is probably a process of remove drive, rebuild array, rinse and repeat. Is that the case, or is there a less aggressive way to mark a drive for removal and shift the data off?

[CyberSkulls]

I'm sure someone will chime in and link you to the exact procedure from the wiki but the question that comes to mind is if you wanting to use them in a different system I would understand. If your looking to save power like you mentioned, why not set them to spin down after "xx" minutes and not have to worry with it at all?

 

 

Sent from my iPhone using Tapatalk

[ashman70]

Don't quote me on this, but I believe you would have to make sure there was no data on the disks you want to remove, stop the array, remove the disks you want to remove, do a new config based on the remaining disks, and start the array at which point it would have to recalculate and build parity from scratch based on the new config. Don't follow these instructions until someone more knowledgable confirms the correct steps.

[richardb]

I'm sure someone will chime in and link you to the exact procedure from the wiki but the question that comes to mind is if you wanting to use them in a different system I would understand. If your looking to save power like you mentioned, why not set them to spin down after "xx" minutes and not have to worry with it at all?

 

It's mainly about using them elsewhere. I already have them set to a fairly aggressive power-down, so the system runs at fairly low power when it isn't crunching. But I appreciate the thought ;-)

 

 

[richardb]

Don't quote me on this, but I believe you would have to make sure there was no data on the disks you want to remove, stop the array, remove the disks you want to remove, do a new config based on the remaining disks, and start the array at which point it would have to recalculate and build parity from scratch based on the new config. Don't follow these instructions until someone more knowledgable confirms the correct steps.

 

That's what I thought, but I was hoping for a more, erm, elegant solution. Rest assured I won't be rushing into anything.

[garycase]

There's no simply way to remove a disk while maintaining fault tolerance, so I'd just do the following ...

 

(a)  Do a parity check to confirm everything is good before you start (no disk errors; no sync errors).  If you find any issues, fix them before proceeding.

 

(b)  Now just do a New Config, and only include the disks you want to keep.  Be SURE you assign the correct disk as parity -- it will be wiped out in the new parity sync, so if you assign the wrong disk (i.e. a disk full of data) to parity you'll lose everything on it.  None of the other disks will be "touched" in the process.

 

©  Start the array and let the new parity sync finish.  Then do a parity check to confirm all went well.

 

Done 

 

You can now remove all of the disks that aren't part of the new configuration.  [You could, of course, have removed them after step (a) ]

 

I presume it's obvious, but just in case:  if there's any data on the disks you're removing, be sure you copy it to the disks you're keeping before doing all of the above.  If you forget, you can always attach the disk as an  "unassigned device" and copy the data to the array after you've done the new configuration.

 

 

[garycase]

One other caveat:  If you're copying data from a disk you plan to remove to a disk you plan to keep, be CERTAIN that you don't do it in a way that triggers the "user share copy bug" => this will result in complete data loss !!  The easiest way to be certain you avoid this is to copy the data using disk references --  NOT user share references.

 

[richardb]

if there's any data on the disks you're removing, be sure you copy it to the disks you're keeping before doing all of the above.  If you forget, you can always attach the disk as an  "unassigned device" and copy the data to the array after you've done the new configuration.

 

Can you give me some pointers on this? I haven't assigned shares to specific disks, and I don't see an option to change where the data is placed after the share is created. How would I move the data onto another disk?

[tdallen]

If you go to Settings -> Global Share Settings you can set Enable Disk Shares to Yes.  You can then browse individual disk contents in Windows or in a terminal window as /mnt/disk1, /mnt/disk2, etc.  That allows you to use a tool like Midnight Commander to see the data stored on each disk and copy/move it from one disk to another.  Don't mix and match copy operations between disk and user shares, though.  Disk to disk is fine, user share to user share is fine, but don't mix.

[richardb]

Excellent, thanks. I will give that a go.

 

I also realized that I can change the disk settings for user shares after they are created to exclude drives. However, that doesn't seem to move the data off the drive. I guess that only covers new data?

[JonathanM]

I also realized that I can change the disk settings for user shares after they are created to exclude drives. However, that doesn't seem to move the data off the drive. I guess that only covers new data?

Correct. Only overt action on your part will move files from an array data drive to another. If you search for the unbalance plugin that offers a function to clear off a specific drive, so that's one way to do it semi automatically. You still would want to verify that a drive truly is empty before removing it, so temporarily exposing disk shares or using MC at the console or one of the file manager dockers is still warranted.

 

If you were masochistic and were using a cache drive, I suppose you could probably manipulate share cache settings, global, and normal share excludes in a combination of mover runs to accomplish the same thing using the tools built in to unraid without touching the command line or adding any plugins or dockers, but it would be quicker to use 3rd party or command line tools. It might be "fun" to establish a procedure to do it from the unraid gui, but I use the word fun in the same way it's "fun" to debug lines of code. Some people enjoy the pursuit, others not so much.

[trurl]

If you go to Settings -> Global Share Settings you can set Enable Disk Shares to Yes.  You can then browse individual disk contents in Windows or in a terminal window as /mnt/disk1, /mnt/disk2, etc.  That allows you to use a tool like Midnight Commander to see the data stored on each disk and copy/move it from one disk to another.  Don't mix and match copy operations between disk and user shares, though.  Disk to disk is fine, user share to user share is fine, but don't mix.

If your using mc or just a terminal window it doesn't matter what your Global Share Settings are, you can still directly access the disks as /mnt/disk1, /mnt/disk2, etc.

[garycase]

... user share to user share is fine

 

NO !! This is what can cause the "user share copy bug" and result in lost data.  Disk share to disk share is fine; or a user share to ANOTHER user share is okay.

 

If you want to read all the nitty gritty on this issue, see this thread:  http://lime-technology.com/forum/index.php?topic=34480.0

 

There are a variety of ways to be CERTAIN you never encounter the bug.  I take the SAFEST approach -- which is to never move data around on the array itself.  If I want to move files from diskX to diskY, I first copy them to an external location [e.g. a folder on my desktop PC];  then delete them from the original location on UnRAID;  and then copy back to UnRAID to the disk I want them on.

 

e.g. I'd effectively do the following (I actually Windows Explorer w/TeraCopy so the copies are verified) ...

 

Copy  \\Tower\diskX\<Folder to Copy>  D:\Files for UnRAID

 

Delete \\Tower\diskX\<Folder to Copy>

 

Copy D:\Files for UnRAID\<Folder to Copy>  \\Tower\diskY\

 

 

 

[richardb]

... user share to user share is fine

 

NO !! This is what can cause the "user share copy bug" and result in lost data.  Disk share to disk share is fine; or a user share to ANOTHER user share is okay.

 

Ah, this makes sense. This is probably the easiest approach, as copying 8TB to another system is going to take a long time. I'll create another share (which excludes the drives I want to remove), copy it there, then move it back to the original. With verification, of course :-).

[garycase]

... user share to user share is fine

 

NO !! This is what can cause the "user share copy bug" and result in lost data.  Disk share to disk share is fine; or a user share to ANOTHER user share is okay.

 

Ah, this makes sense. This is probably the easiest approach, as copying 8TB to another system is going to take a long time. I'll create another share (which excludes the drives I want to remove), copy it there, then move it back to the original. With verification, of course :-).

 

Good plan.  Note that assuming you're maintaining parity during these copies, it's not really any slower to copy the data to another system and then copy it back, since a Gb network is faster than writes to the parity-protected array => in fact, it's likely faster.  However, that DOES require having a spare 8TB of space on the other system      Doing it via another share is fine -- and once you've removed the drives you can then just copy the files back to the original shares, since those shares will no longer already contain the files (since the component disks that contained them will be gone) -- thus the user share copy bug won't be an issue.

 

 

[garycase]

... by the way, the local writes will be MUCH faster if you enable "turbo write" [settings - Disk Settings - md_write_method - reconstruct write

 

This requires all disks be spinning, but is significantly faster than the normal method of read/modify/write required for parity-protected writes.    Since you're going to be copying a lot of data, it's a good idea to turn this on while you're doing it.