Before I start --

If your reply is going to be in the format of "Suck it up and do Option One" I would rather you left the thread clear for replies more directed toward my actual questions than airing criticism of my decisions. I know how it's Supposed To Be Done(tm) but have my own excuses for not doing it that way. While I will explain my reasoning, I'm not here to defend them; I'm here to discuss the nuance and methodology of the other options with users interested in or experienced with fiddling under the hood manually, and to document my experiences with it, for the goal of helping the next fellow like myself.

 

 

Hi all. I'm a long time Unraid user, I spend literally most of my time under the hood of either this server or one of my other spares. I need advice from some deep knowledge folks. When I'm done I'll try to rework this post to share some of that deep knowledge for similar-seekers. It is, after all, our way. If you're looking for the path to my goal, it's edited at the bottom of this first post IF I have found it. (It isn't. Yet.)

 

My server has grown to too many disks, and I'm looking to prune it. Power demand and parallel disk IO loads are getting a bit out of control. Too many SAS cards, can't add a second 1650 Super for transcoding without losing my 4x NVME-PCIE bifurcation card for the Docker containers and game servers.

 

Homelab problems, am I right?

 

Thusfar, I've got a 16tb disk replacing a single 3tb disk in-system. Data rebuild in progress, properly emulated, all is happy because Unraid is awesome like that. I -need- to get the 16tb up before removing the other two 3tb disks because they're pretty full and I don't have 6tb spare elsewhere.

 

After this disk comes online, I intend to remove another two disks. My next few upgrades will be the same -- install a single 16tb drive with the goal of removing three 3tb drives. Obviously step one is remove the data from the disks.

 

From here I know of three options, but I only know how to accomplish two.

 

What I'd like is for a sanity check (I've got time for advice before I can act, lol) and possibly a solution for Option Three from an experienced poweruser, maybe even a dev.

 

Option One: The Unraid GUI End User way -- Remove the two drives. New Config, rebuild parity.

- I don't want to do this. This means a 16tb parity rebuild, removing two 3tb disks.

 

 

Option Two: "The Math Works Out(tm)" -- In parity calculations, two identical disks cancel each other out.

- This means no parity rebuild, just downtime during the 3tb disk-to-disk clone.
- Here's my plan in Maintenance Mode. We'll pretend I'm using /dev/sda and /dev/sdb as disks to be removed, for simplicity.

1 - Enter Maint. Mode (obviously)

2 - Wipe filesystem and partition table identifiers (same uuid on two filesystems can cause problems) and clone drives

wipefs -a /dev/sd{a,b}{1,}
dd if=/dev/sda of=/dev/sdb bs=32k

3 - Stop array, New Config, trust old Parity.

 

 

Option Three: "The White Unicorn" -- same as Option Two, but while the array is fully running, NOT in Maintenance Mode.

My sole idea on how to accomplish this is --

- Enter Maint. Mode

- wipefs -a so drives can't be identified

- Stop Array, Start Array

-- Will the drives present as two new disks? Maybe? Unmountable and prompting to format? Or will Unraid refuse to start?

- Run dd between disks while Unraid runs.

Is this a thing?

 

Is there any other way to bring up shfs with the two drives unmounted, but keeping them in md for parity? Thanks in advance.

I would add whatever disks you want to add all in one shot.  Migrate the data to the disks you want to keep.  Remove the old disks, then do the New Config bit with your remaining new/large disks.  Then eat the parity build that follows.

4 hours ago, Veah said:

I would add whatever disks you want to add all in one shot.  Migrate the data to the disks you want to keep.  Remove the old disks, then do the New Config bit with your remaining new/large disks.  Then eat the parity build that follows.

This is not helpful. I would love to have over two thousand dollars to burn on hard drives and just make it done right now. I'm not talking about days for these upgrades. I'm talking about years. It takes several months for me to afford a spare drive right now.

 

I need for people willing to discuss the details of the options I presented.

 

I'm looking for discussion of the inner workings of Unraid, its technical and presentational response to the actions I considered. Deep knowledge. Experience with the things I discussed.

 

The literal most obvious solution has been considered and discarded, as well as the next few.

 

If you can't provide that thing, please don't waste your time trying to answer this. If I've put you off and you can provide that information, I accept that but I was pretty clear about my ask.

? Unassign disk plugin. Format(Pre-clear) to xfs and copy data over ?

Edited May 8, 20242 yr by bmartino1

Just now, bmartino1 said:

? Unassign disk plugin. Format to xfs and copy data over ?

I assume you mean the Unassigned Devices plugin by dlandon? I'm familiar with it, but that doesn't prevent me from needing to do a full parity rebuild, which is my primary goal.

5 minutes ago, codefaux said:

I assume you mean the Unassigned Devices plugin by dlandon? I'm familiar with it, but that doesn't prevent me from needing to do a full parity rebuild, which is my primary goal.

Yes that plugin..

Regarding option 3 using dd while live is a thing and does work. but I have tried that in the past with a DD Unraid disk offline and had errors. The config folder has a few scattered files that dictate the "new config" options of the disk layout. Even with the data dd unraid still had me parity write.

The unassigned plugin is the only plugin i know to interact and use a disk while the array is live. if formatted as zfs and not the main array dealing with disk 1 I would gladly go through zfs commands to import replace copy etc. this won't work as we are dealing with the main unraid disk 1 array systems. there's another unassigned plugin with pre-clear that does most if not all of the parity build.

@itimpi and @JorgeB
Thoughts and ideas?

2 minutes ago, bmartino1 said:

Yes that plugin..

Regarding option 3 using dd while live is a thing and does work. but I have tried that in the past with a DD Unraid disk offline and had errors. The config folder has a few scattered files that dictate the "new config" options of the disk layout. Even with the data dd unraid still had me parity write.

The unassigned plugin is the only plugin i know to interact and use a disk while the array is live. if formatted as zfs and not the main array dealing with disk 1 I would gladly go through zfs commands to import replace copy etc. this won't work as we are dealing with the main unraid disk 1 array systems. there's another unassigned plugin with pre-clear that does most if not all of the parity build.

@itimpi and @JorgeB
Thoughts and ideas?

 

My primary concern with using dd on a live system is that I need for Unraid to not have those drives mounted. I can remove them from shares, but I suspect shfs still mounts the underlying filesystems. With a filesystem mounted, a dd on the underlying partition can leave the kernel [writing to | reading from] the disk still before it realizes inconsistency, and that can cause panic and/or just random bytes written on the disk I'm trying to mirror.

 

The secret sauce to option 3 is, I think, figuring out how to prevent it from being mounted when starting the array, which `wipefs -a` will accomplish, but I'm not clear on wether Unraid will allow the array to start in this state -- it's an odd mix of "new disk" and "unexpected configuration" that will probably cause issues.

 

I may just need to build a VM with a ton of empty disks and see what happens when I poke it while it's running, rather than theorizing.

The doc may be of some use her: https://docs.unraid.net/legacy/FAQ/replacing-multiple-data-drives/

I'm not 100% Do at your own risk.

I'm not sure of the outcome, but it seems like you would need to stop the array and unassign the disk. to remove them from unraid.

Start the array in maintenance mode to remove disk assignments, but not data.

?stop the array

unassigned disk or terminal command to manual mount a disk to access its files. Do a rsync not DD as we don't want to change UUID/Break Unraids disk config / super.dat data for dick configurations.

Quote

Remember, you've to make a directory first like this:

sudo mkdir /media/Name_of_directory

The above command will create a directory (folder) in media folder by replacing "Name_of_directory" with your providing folder name.

You can see drives numbers or id by:

sudo fdisk -l

Then mount the drive through:

sudo mount /dev/sda# /media/Name_of_directory

later umount /media/Name... to start/stop array...

Then assign the 16 GB disk as drive run stat array in maintenance mode to verify data and parity checks. to stop, start array normally.

^This process still gives you a mounted disk to access files but no vm/docker/other services touch the disk for the fear of a mis copy/write. 

My problem has always been getting Unraid to accept the disk without having to pre-cear/format it. Been a good 10 years on an older version but i did have success of removing a unraid disk 1 and other drives in Ubuntu format my new HD as xfs and copy the data i wanted on that drive. Then replaced the hd in unraid disk 1 without any pre changes. this worked once for me another time later tried the same unraid refused to accept the disk without format or days of a parity write. Regardless of what option you chose, there is still quite a bit of down time.

Review the docs: https://docs.unraid.net/legacy/FAQ/Parity/
on another post:

On 2/26/2020 at 10:00 AM, JorgeB said:

Parity is calculated sector by sector, no data or empty doesn't mean anything, only if all the disks were cleared (or precleared) and all zeros can you safely use parity is already valid.

 

Edited May 8, 20242 yr by bmartino1

6 hours ago, bmartino1 said:

Do a rsync not DD as we don't want to change UUID/Break Unraids disk config / super.dat data for dick configurations.

 

OK but I need you to understand that the dd is specifically done in my suggestion for a very direct and considered purpose. When doing parity calculation, two exactly identical drives are nullified, and can be remove from any parity-protected array while preserving parity data. That's why I'm seeking people with experience in what I'm trying to do.

 

The dd is critical to my intent; not having to do another parity sweep, after replacing my drives and migrating my data.

 

Second, I stated that my intent is to migrate the data off of the drives before beginning this entire process. The drives will have no important data on them. I just want them out of the array without doing a parity sync which is entirely possible, if I can figure out how to get the array to start or bring up shfs without those two drives mountable.

 

I don't need someone to refine my intent, or correct what I want to do.

 

I don't need advice on how to proceed. I need advice from someone who understood everything I've said, and knows the answer.

 

I need someone who can tell me if there's any way to start the array or bring up shfs without those two drives mounted.

 

I sincerely and greatly appreciate the efforts of anyone willing to so much as read this all, but if you don't have the answer to my question, you're wasting your time. My time isn't worth anything, I'm the one asking the favor.