Replacing larger data drive with a smaller one

[PIMF]

How is this done? The usual way of setting a parity drive and then unplugging a data drive does not apply here since the parity drive cannot be smaller than any data drive.

 

In my case I have an array of 1x4TB and 3x3TB with just over 1TB free space. I want to replace the 4TB one with a 3TB, so that I can then add 2 more 3TB drives for parity. Is going into "mc" through telnet and copying over stuff from the 4TB drive to the other ones enough? I'm afraid that when it finishes and I remove the drive, that the data won't be recognized in my user shares.

[trurl]

How is this done? The usual way of setting a parity drive and then unplugging a data drive does not apply here since the parity drive cannot be smaller than any data drive.

 

In my case I have an array of 1x4TB and 3x3TB with just over 1TB free space. I want to replace the 4TB one with a 3TB, so that I can then add 2 more 3TB drives for parity. Is going into "mc" through telnet and copying over stuff from the 4TB drive to the other ones enough? I'm afraid that when it finishes and I remove the drive, that the data won't be recognized in my user shares.

No problem with the data and user shares if you do the copy correctly, making sure to copy from disk to disk and not disk to user share or user share to disk (User Share Copy Bug).

 

What do you intend to do with the 4TB drive though? I can sort of understand not wanting to buy another 4TB for parity just to save a few nickels, though I wouldn't do it that way, but it sounds like you are instead going to buy 2 x 3TB for parity instead, which of course will be more expensive than just adding another 4TB parity.

 

Maybe give us some more details about what you want to do and why and we can come up with a plan.

[PIMF]

Is copying from /mnt/disk1/media/ to /mnt/disk5/media/ correct? What about these folders: appdata, domains, isos, system?

 

I will just use the 4TB drive for my PC. I thought about switching to 4TB drives, but it's an unnecessary expense (in my country these "few nickels" are quite a lot of money). All the other disks in the array are 3TB so if I buy 2 4TB disks for parity I get no real benefit over 3TB drives.

 

In detail:

I currently have data spread across a 4TB drive and three 3TB drives (13TB total, just under 12TB used). I want to replace the 4TB drive with a 3TB drive so the data is spread across four 3TB drives. This way I can install two 3TB drives for parity, instead of having to install two 4TB parity drives just to gain 1TB of usable space.

[trurl]

When I went to dual parity, I debated whether or not it was worth it. In my case, not because of the expense, but because I would be using up another port that might better be used for data. Since I only had 5 data drives, and unRAID V6 makes it much easier to keep up with disk problems, the chances of multiple failures aren't that high. And I have backups. The main reason I went ahead with dual parity is just so I could learn more about it. And my usual method of acquiring more capacity is just to upsize drives anyway.

 

It is probably worth noting that some have reported performance issues on lower end hardware due to parity2 calculation. See the 6.2 release threads.

 

As long as you are copying disk to disk you shouldn't have any issues with the User Share Copy Bug, but it is probably a good idea to stop all other activity while you are doing all this, such as your dockers, VMs, any plugins you have that access your disks or user shares, etc.

 

Of course after you are done moving things off the 4TB, to remove it you will have to set a New Config and rebuild parity. You can build both parities at this point.

[PIMF]

Update: I have managed to move all the content through MC without losing anything. Basic instructions for how I did this:

 

HOW TO MOVE CONTENT BETWEEN UNRAID DRIVES

1. Open a telnet client such as PuTTY, point it to "tower".

2. Log in to your unRAID (default username is "root", no password).

3. Type "mc" without the quotes.

4. Select "/.." at the top of both columns, then go to /mnt/diskX where X is the number of the disk you want to copy from/to.

5. Use the "copy" command on the folders you want to copy to the other drive. Once it finishes copying you will have the same files on both drives.

6. Use the "delete" command to delete the old folders.

 

This does NOT result in any data loss. UnRAID does not panic when there are dupe files (like I was afraid would happen), even when in the same directory on both drives. For example if you copy /disk1/media/file.html to /disk2/media/ and have file.html in both directories, you can delete either one safely and it will still be accessible in your shares.

[trurl]

Update: I have managed to move all the content through MC without losing anything. Basic instructions for how I did this:

 

HOW TO MOVE CONTENT BETWEEN UNRAID DRIVES

1. Open a telnet client such as PuTTY, point it to "tower".

2. Log in to your unRAID (default username is "root", no password).

3. Type "mc" without the quotes.

4. Select "/.." at the top of both columns, then go to /mnt/diskX where X is the number of the disk you want to copy from/to.

5. Use the "copy" command on the folders you want to copy to the other drive. Once it finishes copying you will have the same files on both drives.

6. Use the "delete" command to delete the old folders.

 

This does NOT result in any data loss. UnRAID does not panic when there are dupe files (like I was afraid would happen), even when in the same directory on both drives. For example if you copy /disk1/media/file.html to /disk2/media/ and have file.html in both directories, you can delete either one safely and it will still be accessible in your shares.

The main thing I would add to your instructions is that anytime you are doing any lengthy operations remotely (not at an attached keyboard/monitor) you should work in a screen session. That way if the connection is lost for some reason (your PC goes to sleep for example) it will continue to work and you can resume the session. You can install screen from the NerdPack. Google "linux screen" for how to use.

[JonathanM]

This does NOT result in any data loss. UnRAID does not panic when there are dupe files (like I was afraid would happen), even when in the same directory on both drives. For example if you copy /disk1/media/file.html to /disk2/media/ and have file.html in both directories, you can delete either one safely and it will still be accessible in your shares.

This is true, but has some unseen effects that can bite you if you aren't aware. While the 2 files exist, only the lowest numbered disk's copy shows up in the user shares. So... if you make the copy, and before you delete the duplicate the file in the user share is accessed and modified, then the two files are no longer the same. If you subsequently delete the lower number disk's file, then those changes are lost.

 

One way to eliminate that from being an issue is turn off user shares for the duration of the operation. Another way is to globally exclude the source disk, so only the destination shows up over the network.

[RobJ]

Update: I have managed to move all the content through MC without losing anything. Basic instructions for how I did this:

 

HOW TO MOVE CONTENT BETWEEN UNRAID DRIVES

1. Open a telnet client such as PuTTY, point it to "tower".

2. Log in to your unRAID (default username is "root", no password).

3. Type "mc" without the quotes.

4. Select "/.." at the top of both columns, then go to /mnt/diskX where X is the number of the disk you want to copy from/to.

5. Use the "copy" command on the folders you want to copy to the other drive. Once it finishes copying you will have the same files on both drives.

6. Use the "delete" command to delete the old folders.

 

This does NOT result in any data loss. UnRAID does not panic when there are dupe files (like I was afraid would happen), even when in the same directory on both drives. For example if you copy /disk1/media/file.html to /disk2/media/ and have file.html in both directories, you can delete either one safely and it will still be accessible in your shares.

 

A few comments -

* Your server is wide open to 'admin' access from other local stations, PLEASE put a decent password on root!  Unless you have total control on all other stations and devices on the network, you are vulnerable to attack if one of them gets infected.

* Your server would be safer if Telnet was turned off and SSH used instead (easy through PuTTY), see the Tips and Tweaks plugin.

* Once the transfers are complete, just stop and restart the array, and a brand new User Share file system will be built, with all the right pointers to files in their new locations, all duplication forgotten.  This *may* not be necessary, but ensures it's clean and correct.

 

(Has it ever been proven that FUSE will correctly find and add the higher located file of a dupe pair, if you delete the lower one?)

[JonathanM]

(Has it ever been proven that FUSE will correctly find and add the higher located file of a dupe pair, if you delete the lower one?)

Yes, at least for the test I just did on 6.1.9

 

I created a text file in a folder on disk1, with the content "this file is on disk1", I then created a text file in the same name folder on disk2, with the content "this file is on disk2". I connected to the user shares, navigated to the text file, and opened it. It was the disk1 file. I deleted the file on disk1 using mc, and opened the same file in the user share window without F5 refreshing, it now had the disk2 contents.

 

Quick easy test.

[RobJ]

Thank you!  That's how I hoped it would work, but suspected it didn't, since that would require specific programming to add.

 

That means the file system held both entries, but only displayed one, which is somewhat unusual.  It makes you wonder if that's a way to hide stuff, create a file you want hidden, and give it a file name already in use on a lower numbered drive.  A bit tricky to use, but seems like it would work.