Replacing small ReiserFS drive with larger drive (XFS) in the quickest time - not worried about parity

[belorion]

I have an unRAID 6.9.1 instance that is full of ReiserFS drives (as I migrated from unRAID 5) and I have an 18 TB parity drive.

• I learned the "hard-way" that ReiserFS doesn't support data drives that large (18 TB or larger)
• I need to convert existing drives to XFS to get more array capacity
• the array is close to capacity (FULL) perhaps 2-3 TB free on > 100 TB
•  I have no spare drive slots (all bays and ports are full)
• i have copied the content from the "DISK" share (legacy unRAID 4/5) to another host using FreeFileSync - so I'm ok that the content is safe (it's another unRAID host)

 

Parity computation takes about 40  hrs given the present size - so I'm *keen* to NOT to have to recompute it any more times than necessary (ie 2 days for every action makes for slow progress)

 

My understanding is that if I "forget" the drive - it will result in parity being recomputed ... BEFORE I put in a new drive .. and again it will do the preclear / parity computation when I actually put in the new drive ... that'll be 4 days before i actually start coping the content (which will be another 2 days).

 

i have content "safe" elsewhere as i said ...

 

i saw this link

in which ssd indicated that i could create a new config in from the "tools menu" if i didnt care about having parity protection 100% of the time.

 

i'm not certain how to use the new config - as i'm *keen* not to lose all my content

 

ie if I have 12 data drives and 1 parity drive ... how do I do it?

 

1).

can I just stop unRAID

pull out the undesired drive

put in the new unformatted 18TB drive

start up unRAID (it will complain drive is missing ... ignore the error as it's intentional)

create new config and select the preserve current assignments as well as adding the new drive?

(I would have to also select the existing parity drive as the new parity drive)

and then starting my copying back ...which would then take 2 days

 

2). create  a new config, preserve existing assignments but EXCLUDING the drive to be replaced, specifying the parity as before

recompute the parity (2 days)

turn off

removed unwanted drive

add 18TB drive

format, add to array yadda yadda  (2 day hit)

then start copying files (which will take about 2 days)

 

3). Something else?

 

Thanks for "listening" : )

 

(edited due to saying i was using BTRFS when its actually ReiserFS 😐)

Edited November 15, 2021 by belorion
listed the wrong filesystem type BTRFS instead of ReiserFS

[JonathanM]

Maybe I'm just not getting it, but first you say

1 hour ago, belorion said:

• I need to convert existing drives to XFS to get more array capacity

Then you say

1 hour ago, belorion said:

put in the new unformatted 18TB drive

 

 

Perhaps it would be clearer to outline your current state and then your desired end goal

Like this

 

Current

18TB parity

disk1 4TB BTRFS 3.9TB

disk2 4TB BTRFS 3.5TB

 

Desired

18TB parity

disk1 18TB XFS with data from former disk1 and disk2

 

Then maybe we can help formulate the fastest way to get things done.

[belorion]

My current state is:

• 18 TB parity drive
• 12 data drives mix of 10 & 12 TB data drives (all formatted as ReiserFS)
• total array capacity about 122 TB
• available capacity approx 2 TB
• all physical slots/ports used on motherboard and drive controllers)
• parity sync / verification takes approx 40 hours
• MB is 4 core xeon about 8 years old, 16 GB RAM, no docker containers, just a simple NAS

The use of BTRFS is from migrating from an older unRAID setup (5.x)

 

existing constraints:

• cant do in place upgrade of drives as ReiserFS (max file size limit)
• replacing an existing drive in ReiserFS format with a larger drive doesn't work as the larger drive can't be formatted larger than the max ReiserFS size (ie the rebuild from parity reconstructs the drive contents as ReiserFS)
• can't do in place upgrade as I have insufficient available array storage to move all content off any drive)
• no additional bays/ports etc to add a new drive

 

I want:

• to be able to expand the total drive array size
• I can't just replace the existing ReiserFS drives with bigger drives as the max drive size of ReiserFS is < 16TB or so (I tried that previously - and found a link to a max filesystem somewhere indicating as such)
• to replace all the 10,12 TB drives with 18TB drives or larger, which means XFS formatted drives (replacing the drives one at a time due to cost and effort)

 

I do have an additional unRAID host that I can copy off the content of one drive to ... so that it can be put back later

 

I want to be hit with the minimum  2 day parity hits for any activities eg

- wiping a drive

- adding drive

- reconstructing array

 

I am not concerned about the parity integrity as I have the content available elsewhere - which is why I thought the New Config may be the fastest option to keep most of my data (minus one drive I can remove and put in a new 18TB drive - and then copy back from the other unRAID host)

 

does that help? @JonathanM

 

Edited November 15, 2021 by belorion
listed wrong filesystem type BTRFS instead of ResiserFS

[trurl]

3 minutes ago, belorion said:

The use of BTRFS is from migrating from an older unRAID setup (5.x)

 

3 minutes ago, belorion said:

max drive size of BTFRS is < 16TB

 

Are you sure you have btrfs? Unraid 5 only had ReiserFS, and btrfs supports much larger volumes than 16TB

 

 

The title of this thread says you don't care about parity, so why not use the parity disk as an XFS data disk, copy all files from the disk with the most contents, reformat that disk as XFS, copy all files from the remaining disk with the most contents, reformat that disk as XFS, ...

 

[belorion]

Ooops ... I was reading something about BTRFS the other day and I have it on the brain ... yes... it's actually ReiserFS

 

I need to replace all the ReiserFS with XFS

 

thanks @trurl

[JonathanM]

Are you clear on how to follow trurl's instructions? That is by far the fastest way to get it done, assuming you do exactly as he said and not interpret his instructions to something else because you think it's better for some reason.

 

I'll restate step by step.

1. stop array

2. unassign parity drive

3. start array (may not be necessary, can't remember)

4. stop array

5. assign former parity drive as data drive in new slot

6. start array

7. format newly assigned (or copied) drive as XFS, make sure it's the only drive showing as unmountable before you hit format, and be sure it says it's using XFS

8. copy (NOT MOVE) all the data from the largest remaining ReiserFS drive to the newly formatted XFS drive

9. verify the copy completed successfully, full compare of files if you wish

10. stop array

11. select drive that was the source of the last copy and change it to XFS

12. are there any ReiserFS drives left? If yes, go to step 6. if no, continue

13. assuming everything went well, you should be left with 12 data drives with XFS content, and the smallest drive should be fully copied and ready to be removed

14. Power off and physically swap the last copy source drive with a new 18TB drive, be sure to note the serial number

15. go to tools and set a new config, preserve all

16. assign newly inserted 18TB drive as parity, rearrange data drives however you want, just don't put a data drive in a parity slot by mistake. BE VERY CAREFUL ABOUT THAT!!!!

17. start array and build parity

18. do a correcting parity check to be sure everything is happy in the new config. zero errors is the only acceptable answer.

 

The copy and verify steps are up to you how to accomplish them. I personally use rsync at the local console. If you insist on using a move instead of a copy action, 1. there is no way to verify the data moved properly 2. it will take roughly twice as long, possibly 3x or longer depending on the state of your ReiserFS file system.

[trurl]

Might be useful to also change your default filesystem to XFS in Disk Settings. Do that before the other steps.

[belorion]

12 hours ago, JonathanM said:

Are you clear on how to follow trurl's instructions? That is by far the fastest way to get it done, assuming you do exactly as he said and not interpret his instructions to something else because you think it's better for some reason.

 

I'll restate step by step.

1. stop array

2. unassign parity drive

3. start array (may not be necessary, can't remember)

4. stop array

5. assign former parity drive as data drive in new slot

6. start array

7. format newly assigned (or copied) drive as XFS, make sure it's the only drive showing as unmountable before you hit format, and be sure it says it's using XFS

8. copy (NOT MOVE) all the data from the largest remaining ReiserFS drive to the newly formatted XFS drive

9. verify the copy completed successfully, full compare of files if you wish

10. stop array

11. select drive that was the source of the last copy and change it to XFS

12. are there any ReiserFS drives left? If yes, go to step 6. if no, continue

13. assuming everything went well, you should be left with 12 data drives with XFS content, and the smallest drive should be fully copied and ready to be removed

14. Power off and physically swap the last copy source drive with a new 18TB drive, be sure to note the serial number

15. go to tools and set a new config, preserve all

16. assign newly inserted 18TB drive as parity, rearrange data drives however you want, just don't put a data drive in a parity slot by mistake. BE VERY CAREFUL ABOUT THAT!!!!

17. start array and build parity

18. do a correcting parity check to be sure everything is happy in the new config. zero errors is the only acceptable answer.

 

The copy and verify steps are up to you how to accomplish them. I personally use rsync at the local console. If you insist on using a move instead of a copy action, 1. there is no way to verify the data moved properly 2. it will take roughly twice as long, possibly 3x or longer depending on the state of your ReiserFS file system.

Thanks for the detailed instructions 🙂 I was going to ask a little more after I had finished "syncing" the content between the hosts "one more time" (just to be sure - the only reason I can be carefree about parity).

 

I think the above will make a massive performance increase  - ie I was looking at something that would take weeks .. and now I'm thinking it will be days (due to being able to copy the files at the full hardware speed rather than Network constrained).

 

 

[belorion]

On 11/15/2021 at 10:44 AM, trurl said:

 

 

Are you sure you have btrfs? Unraid 5 only had ReiserFS, and btrfs supports much larger volumes than 16TB

 

 

The title of this thread says you don't care about parity, so why not use the parity disk as an XFS data disk, copy all files from the disk with the most contents, reformat that disk as XFS, copy all files from the remaining disk with the most contents, reformat that disk as XFS, ...

 

Thanks very much for this "thinking outside the box solution" ... I hadn't even considered "temporarily" repurposing the parity drive like this .. but it totally makes sense .. .and it will speed up the process massively : )

[belorion]

On 11/16/2021 at 1:13 AM, JonathanM said:

Are you clear on how to follow trurl's instructions? That is by far the fastest way to get it done, assuming you do exactly as he said and not interpret his instructions to something else because you think it's better for some reason.

 

I'll restate step by step.

1. stop array

2. unassign parity drive

3. start array (may not be necessary, can't remember)

4. stop array

5. assign former parity drive as data drive in new slot

6. start array

7. format newly assigned (or copied) drive as XFS, make sure it's the only drive showing as unmountable before you hit format, and be sure it says it's using XFS

8. copy (NOT MOVE) all the data from the largest remaining ReiserFS drive to the newly formatted XFS drive

9. verify the copy completed successfully, full compare of files if you wish

10. stop array

11. select drive that was the source of the last copy and change it to XFS

12. are there any ReiserFS drives left? If yes, go to step 6. if no, continue

13. assuming everything went well, you should be left with 12 data drives with XFS content, and the smallest drive should be fully copied and ready to be removed

14. Power off and physically swap the last copy source drive with a new 18TB drive, be sure to note the serial number

15. go to tools and set a new config, preserve all

16. assign newly inserted 18TB drive as parity, rearrange data drives however you want, just don't put a data drive in a parity slot by mistake. BE VERY CAREFUL ABOUT THAT!!!!

17. start array and build parity

18. do a correcting parity check to be sure everything is happy in the new config. zero errors is the only acceptable answer.

 

The copy and verify steps are up to you how to accomplish them. I personally use rsync at the local console. If you insist on using a move instead of a copy action, 1. there is no way to verify the data moved properly 2. it will take roughly twice as long, possibly 3x or longer depending on the state of your ReiserFS file system.

It's all going along great ... I'm actually processing an entire drive a day - when I was originally expecting it to take a week each : )

 

So ... I'm thinking ... given I have no parity drive right now ... I'm running with no parity right now .. until the process is almost complete (ie I put in the 18TB drive) ...

 

Can I just put the new drive in and assign it to be a parity drive WITHOUT creating a new configuration?? Wouldn't that just construct parity with the array layout as it was at that instant?

 

My (limited?) understanding is that creating a new configuration will have the effect of scrubbing all the config (configured shares, users etc) but maintaining the data on the drives.

 

Or am I missing something?

 

[itimpi]

1 hour ago, belorion said:

Can I just put the new drive in and assign it to be a parity drive WITHOUT creating a new configuration?? Wouldn't that just construct parity with the array layout as it was at that instant?

Yes

1 hour ago, belorion said:

My (limited?) understanding is that creating a new configuration will have the effect of scrubbing all the config (configured shares, users etc) but maintaining the data on the drives.

configuration information that is not drive specific is not affected by the New Config option. 

 

In addition all top level folders are automatically treated as User shares so they should remain intact as well although you may need to adjust any settings that refer to specific drives.