Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)

[iZeus83]

Don't mess with the server until you have backups of everything you don't want to lose. Copying data from drive to drive and changing formats is risky, there is a chance of typing a command wrong or not understanding the directions and erasing stuff by accident. Add to that the fact you want to eliminate the single drive failure protection by invalidating parity in order to move stuff, and you have a recipe for disaster unless everything works perfectly.

 

Changing formats as described in this thread is not risky - except for the risk of human error. Which is something under each person's control to mitigate.

 

I do not recommend running a non-parity protected array unless the data of value is separately backed up. Having a parity protected array protected by another parity protected array provides a lot of protection. In this configuration, I could not argue that removing parity from one of them would add a lot of risk. So long as the backup is occurring very frequently to avoid losing newly added data.

 

---UPDATE---

 

Now, I'm in the process of copying files from one disk to another because I've backed up the most of the most important things for me. So I'm just using the parity disk as a disk4 and at the end I'd back the parity disk3 to the original slot.

 

When I put back the disk4 to the original slot of parity disk then I need to pleclear the disk again?.

[itimpi]

When I put back the disk4 to the original slot of parity disk then I need to pleclear the disk again?.

No. 

 

What you need to do at this stage is Tools->New Config and then assign the drives as you want them to finish up (including parity).  Make sure you do not accidentally assign a data disk as parity as this would lead to data loss.  When you start the array then unRAID will start building parity from the data disks.  It is NOT necessary to have pre-cleared the parity disk in this scenario as the process of building parity overwrites what is there anyway.

[iZeus83]

No. 

 

What you need to do at this stage is Tools->New Config and then assign the drives as you want them to finish up (including parity).  Make sure you do not accidentally assign a data disk as parity as this would lead to data loss.  When you start the array then unRAID will start building parity from the data disks.  It is NOT necessary to have pre-cleared the parity disk in this scenario as the process of building parity overwrites what is there anyway.

 

Thanks to all for help!!!, the process finally successful with parity back and without losing data but a few files has been diferent checksum. At least these files weren't critical.

[goinsnoopin]

So I just started my conversion to xfs.  Below is the output of my rsync verify.txt file...it lists a bunch of files that were skipped as they were non regular files.  These files appear to be data from my plex docker. Before I proceed with erasing the source disk, I thought I would post to get some feedback.

 

Here is the link to the file on my dropbox account as it exceeds the size permitted by the forum.

 

https://www.dropbox.com/s/gawnbm26ydwyewl/verifydisk3.zip?dl=0

 

Thanks for looking.

Dan

 

 

[goinsnoopin]

In looking at this a little closer...this folder refers to an appdata subfolder for an old instance of plex...I switched to another users docker...so I think I feel comfortable moving forward to the next disk. 

 

Dan

[goinsnoopin]

I have two WD Green disks in my array that have the jumper on pins 5-6 to force the 1.5 GB PHY setting...these drives go back to the time when unraid did not support 4K aligned disks.  Can I convert these drives to 4k aligned?  It seems like this would be the time to do so. Any recommendations on how to accomplish this as part of my conversion from reiserfs to xfs?

 

Dan

[Gizmotoy]

Anyone have any ideas why rsync would seem to copy a file at about 150Mbps for the first few seconds, then drop to a stable 25Mbps?  Both the source and destination drives could sustain transfers faster than 25Mbps and they're on fast controllers.  Is it normal, or do I have a problem somewhere?  Cloning around 3.5TB seems to take about 24hours.

 

Edit: Should probably note the exact command I'm using, which is    rsync -avPX /mnt/disk15/ /mnt/disk5/

[trurl]

Anyone have any ideas why rsync would seem to copy a file at about 150Mbps for the first few seconds, then drop to a stable 25Mbps?  Both the source and destination drives could sustain transfers faster than 25Mbps and they're on fast controllers.  Is it normal, or do I have a problem somewhere?  Cloning around 3.5TB seems to take about 24hours.

 

Edit: Should probably note the exact command I'm using, which is    rsync -avPX /mnt/disk15/ /mnt/disk5/

I/O is cached in memory until memory is used up, then it has to wait on the disk. That accounts for the burst at the beginning. Writing to the parity array is always slower than the drives because it must first read the disk to be written and read parity, calculate the change that would be made to parity by the data to be written, write parity and write data. So, 2 reads and 2 writes when writing to the parity array.

[Gizmotoy]

Anyone have any ideas why rsync would seem to copy a file at about 150Mbps for the first few seconds, then drop to a stable 25Mbps?  Both the source and destination drives could sustain transfers faster than 25Mbps and they're on fast controllers.  Is it normal, or do I have a problem somewhere?  Cloning around 3.5TB seems to take about 24hours.

 

Edit: Should probably note the exact command I'm using, which is    rsync -avPX /mnt/disk15/ /mnt/disk5/

I/O is cached in memory until memory is used up, then it has to wait on the disk. That accounts for the burst at the beginning. Writing to the parity array is always slower than the drives because it must first read the disk to be written and read parity, calculate the change that would be made to parity by the data to be written, write parity and write data. So, 2 reads and 2 writes when writing to the parity array.

Dah.  For some reason it didn't click that they're both parity-protected drives.  Makes sense.  Thanks for the explanation.

[cyan]

can I follow post no #3 if I want to do 4K alignment + XFS ?

WD green 2TB was previously "512 jumpered" (unraid V4)

[trurl]

can I follow post no #3 if I want to do 4K alignment + XFS ?

WD green 2TB was previously "512 jumpered" (unraid V4)

Don't see what exactly post #3 has to do with 4K alignment, since "align" doesn't appear anywhere on the first page of this thread. I can see how you might use the ideas in post #3 after you have aligned a disk. You need to spell out in much greater detail exactly what you are thinking about doing.

[cyan]

Don't see what exactly post #3 has to do with 4K alignment, since "align" doesn't appear anywhere on the first page of this thread. I can see how you might use the ideas in post #3 after you have aligned a disk. You need to spell out in much greater detail exactly what you are thinking about doing.

from example in reply #3 let's just say if disk 4 not aligned with 4K

when I finish round 4 and do round 5

"4 - Format the unformatted with unRAID. XFS disk formats."

since the default MBR is 4k-aligned in settings will it format XFS along with MBR 4K ?

or it will still use previous MBR ?

 

[RobJ]

Don't see what exactly post #3 has to do with 4K alignment, since "align" doesn't appear anywhere on the first page of this thread. I can see how you might use the ideas in post #3 after you have aligned a disk. You need to spell out in much greater detail exactly what you are thinking about doing.

from example in reply #3 let's just say if disk 4 not aligned with 4K

when I finish round 4 and do round 5

"4 - Format the unformatted with unRAID. XFS disk formats."

since the default MBR is 4k-aligned in settings will it format XFS along with MBR 4K ?

or it will still use previous MBR ?

 

Alignment is only relevant at drive initialization, has nothing at all to do with formatting and file systems.  Alignment is only done at initial setup, with the initial GPT and/or MBR and the partitions.  Once partitions are created, then you can format a partition with a particular file system.

 

Currently, questions about alignment only come up with a previously jumpered drive.  The old advice used to be - if it's not jumpered, don't add one; if it's jumpered, don't remove it, because some drives never worked correctly after the jumper was removed.  Recently, some have removed the jumper, but I'm not sure how well it is working for them.

 

It might be interesting for someone to try it the different ways, and test the disk speed (perhaps using the diskspeed script), see if it makes much difference, and if it does, which way is best.

[bonienl]

... Recently, some have removed the jumper, but I'm not sure how well it is working for them.

 

In the process of converting from RFS to XFS I also 'unjumpered' my older 2 TB WD drives and changed the alignment to 4K aligned. All is humming along 

[RobJ]

... Recently, some have removed the jumper, but I'm not sure how well it is working for them.

 

In the process of converting from RFS to XFS I also 'unjumpered' my older 2 TB WD drives and changed the alignment to 4K aligned. All is humming along 

That's good to hear, good to know!

[cyan]

... Recently, some have removed the jumper, but I'm not sure how well it is working for them.

 

In the process of converting from RFS to XFS I also 'unjumpered' my older 2 TB WD drives and changed the alignment to 4K aligned. All is humming along 

 

awesome news, thanks

I'll 'unjumpered' my WD green when I convert to XFS.

[spanky]

Hi,

 

I have been following these instructions to migrate two disks from reiserfs to XFS:

 

 

So how do you move from RFS to XFS? First, you need to buy (or free up) a disk as large as your largest data drive. If it is a new disk, you have to preclear it and add it to your array like any other new disk.

 

For these instructions, I will assume your array looks like this:

parity - 4T

disk1 - 4T - RFS

disk2 - 3T - RFS

disk3 - 4T - RFS

disk4 - 2T - RFS

disk5 - 4T - new/empty disk, FS doesn't matter we're going to set it in a minute

 

1 - Define your disk conversion order. Below is an example for this sample array. Make sure your order takes into account the drive sizes. If you copy a 2T drive to your new 4T drive, the 2T drive will not be big enough to convert your next 4T drive. You'll see I start with the biggest disk, and then move to the smaller ones. In the end, the smallest disk is empty. Substitute [source] and [dest] in the instructions below based on the round you are in. I don't address cache conversion. That's another topic.

 

round [source] [dest]    notes

------- ----------- -------- ------------

  1      disk1    disk5

  2      disk3    disk1  (4T to 4T)

  3      disk2    disk3  (3T to 4T)

  4      disk4    disk2  (2T to 3T)

  5      ---        disk4  (empty the 2T)

 

2 - With array stopped, change the filesystem on [dest] to XFS (or BTRFS is that is your preference. Most users are going with XFS).

 

3 - Start the array, the new XFS disk should show unformatted, all the other disks should be formatted!! (IF NOT, DO NOT PROCEED)

 

4 - Format the unformatted with unRAID. XFS disk formats.

 

(If you are on the last round and just reformatting the last disk to XFS - round 5 for the example - you are done!!!)

 

5 - Open or resume a screen session

 

(Aside - we are about to copy all of the files from one of your RFS disks to the XFS disk. If the disk is involved in a user share, this will result in creating duplicate files out the ying yang! For that reason, I create a temporary directory (I call it "t") on the XFS disk and copy everything there. This will eliminate the possibility of your user share dealing with and possibly logging all the duplicates. Once the copy is completed and verified, we will move the files from the "t" directory to the root, and get rid of the "t" directory.)

 

6 - Make the "t" directory on [dest]. You can easily do from Windows Explorer or with this command:

  mkdir /mnt/[dest]/t

 

7 - Copy the data from the [source] to [dest]. Note this will take a long time (this is why doing it in a screen session so it will not timeout). Do overnight or at a convenient time when the array will not be heavily utilized. You definitely don't want to be doing a parity check while this is happening! Note that -rpv means do it recursively, preserve the permissions, and be verbose (it lists each file it copies). Make sure they are all in lower case.

  cp -rpv /mnt/[source]/* /mnt/[dest]/t

 

Note if you happen to have the [dest] disk red-ball in the middle of the copy, you will have to do a drive replacement. I would recommend stopping the copy once you notice this has happened. You can resume it later. Then follow the normal disk replacement procedure. If this happens it is likely a file will get have been corrupted and the comparison below becomes very important so you know which one it is so you can re-copy it from the [source] before the source gets overwritten in the next round.

 

8 - It is now highly recommended to compare the checksums of the source to the dest. (If not, at least spot check several files). There are multiple ways to do that including md5sum, bitrot (a user contributed script). There is also a way to use rsync (perhaps Weebo will post or link instructions - sorry, i couldn't find them) to do the copy and verify in one rsync step. If there are questions here, please ask. My method is a bit of a Rube Goldberg, but will try to clean it up and post if there is interest.

 

UPDATE: Here is one way to complete step 8 (see later post in this thread) (thanks to danioj):

 

From a screen session, run this command:

  rsync -nrcv /mnt/[source]/ /mnt/[dest]/t >/boot/verify.txt

 

This will put the output of the comparison in a file on the flash disk. It will likely have little or no output, but this will preserve all output in case it is lengthy and would scroll off the screen.

 

You can view the output with this command in another session:

  cat /boot/verify.txt

 

9 - If there are any comparison failures, stop here and ask for help

 

10 - Now is the good time to move the files in the "t" directory to the root on [dest]. I do this with cut and paste from Windows explorer.

 

11 - Stop the array (no need to delete anything from the [source])

 

12 - Go back to step 2.  Note that this isn't a race - you can do it at your leisure of the course of days, weeks, or months. I do one or two a week or so.

 

Good luck. Post any questions.

 

There are 3 disks in the mix - (all 4TB), an empty drive formatted as XFS (disk3) + two disks (disk1 and disk2) to be moved over.

All has gone to plan, with step 7 running now from disk1 to disk3 (to a 't' directory on the new drive as suggested).

When I have a look at the contents of disk3 from the GUI I see the files I expect + a corresponding 4.10KB file for each that has it's name beginning with '._'

 

Hoping someone can tell me if this is expected while the process is underway - are these temp files created as part of the copy process ?

 

Thanks.

 

**UPDATE**

I actually went and had a look at disk1 (source disk) and all those 4.10KB files are there, which explains why they're being copied d'oh.

I still have no idea what they are though.

 

**UPDATE 2**

Did some searching and apparently these '._' files are a legacy mac thing that I don't see when using finder on the mac.

 

Cheers.

[psychodad1000]

Don't think I saw this scenario:

 

Disk 1 - 2TB (1TB free)

Disk 2 - 2TB (1TB free)

 

I want to move the files from Disk 1 to Disk 2 to create an empty drive.

Format Disk 1 to XFS.

Replace Disk 1 with a new 4TB drive and rebuild.  The result will be a larger drive formatted XFS.  Correct?  I assume the rebuild would go quickly since the drive is empty?

 

My actual scenario involves (5) 2TB drives and a 3TB drive but still results in replacing an emptied 2TB with a 4TB.  I know it would be easier to just add the 4TB drive but I would like to start replacing the smaller 2TB drives with 4TB drives.  Is there an easier way?

[trurl]

Don't think I saw this scenario:

 

Disk 1 - 2TB (1TB free)

Disk 2 - 2TB (1TB free)

 

I want to move the files from Disk 1 to Disk 2 to create an empty drive.

Format Disk 1 to XFS.

Replace Disk 1 with a new 4TB drive and rebuild.  The result will be a larger drive formatted XFS.  Correct?  I assume the rebuild would go quickly since the drive is empty?

 

My actual scenario involves (5) 2TB drives and a 3TB drive but still results in replacing an emptied 2TB with a 4TB.  I know it would be easier to just add the 4TB drive but I would like to start replacing the smaller 2TB drives with 4TB drives.  Is there an easier way?

Rebuilding an empty filesystem (freshly formatted drive) is no quicker than a full drive. Every bit of the entire disk will be rebuilt regardless. Parity knows nothing of filesytems.

 

In order to change filesystems, you are going to have to copy from an old filesystem disk to a new filesystem disk anyway you do it, so if you have enough slots, I think it would be easier to just preclear the new disks, add them to new slots and format them to the new filesystem, copy from the old filesystem disks to the new filesystem disks, then New Config without the old disks and rebuild parity.

[psychodad1000]

Don't think I saw this scenario:

 

Disk 1 - 2TB (1TB free)

Disk 2 - 2TB (1TB free)

 

I want to move the files from Disk 1 to Disk 2 to create an empty drive.

Format Disk 1 to XFS.

Replace Disk 1 with a new 4TB drive and rebuild.  The result will be a larger drive formatted XFS.  Correct?  I assume the rebuild would go quickly since the drive is empty?

 

My actual scenario involves (5) 2TB drives and a 3TB drive but still results in replacing an emptied 2TB with a 4TB.  I know it would be easier to just add the 4TB drive but I would like to start replacing the smaller 2TB drives with 4TB drives.  Is there an easier way?

Rebuilding an empty filesystem (freshly formatted drive) is no quicker than a full drive. Every bit of the entire disk will be rebuilt regardless. Parity knows nothing of filesytems.

 

In order to change filesystems, you are going to have to copy from an old filesystem disk to a new filesystem disk anyway you do it, so if you have enough slots, I think it would be easier to just preclear the new disks, add them to new slots and format them to the new filesystem, copy from the old filesystem disks to the new filesystem disks, then New Config without the old disks and rebuild parity.

 

Thanks for the reply.  I thought it was too good to be true.  I guess there really is "no free lunch".

[psychodad1000]

Sorry, I guess I'm going to be the problem child....

Popped in a new 4TB drive and formatted XFS.  Everything looked good.

Created t folder on new drive.

Started Putty, executed command: cp -rpv /mnt/disk1/* /mnt/disk7/t.

System started copying 2TB from disk1 to disk7, files names flying by at the speed of light (almost).

Came back some hrs later and find this msg: "Network Error:software caused connection abort" .

Only 1.2 of the 2TB was copied.  I left it overnite to see if it was just the putty connection to my computer that failed and hopefully the copy process was still running.

No joy.  The copy did stop at 1.2TB.  All drives are still green balled.  I visually checked and many files were copied to the t folder of the new drive.

Ran rsync and got a long list of files which didn't copy.

 

Anyone else have the copy command bomb midway thru the process?

What should I do now?  Reformat the drive and try again?

[trurl]

Sorry, I guess I'm going to be the problem child....

Popped in a new 4TB drive and formatted XFS.  Everything looked good.

Created t folder on new drive.

Started Putty, executed command: cp -rpv /mnt/disk1/* /mnt/disk7/t.

System started copying 2TB from disk1 to disk7, files names flying by at the speed of light (almost).

Came back some hrs later and find this msg: "Network Error:software caused connection abort" .

Only 1.2 of the 2TB was copied.  I left it overnite to see if it was just the putty connection to my computer that failed and hopefully the copy process was still running.

No joy.  The copy did stop at 1.2TB.  All drives are still green balled.  I visually checked and many files were copied to the t folder of the new drive.

Ran rsync and got a long list of files which didn't copy.

 

Anyone else have the copy command bomb midway thru the process?

What should I do now?  Reformat the drive and try again?

Probably putty connection. Any process you expect to take a long time should be done in a screen session so you can disconnect and resume as needed. Screen can be installed from NerdTools plugin. Google Linux Screen Command.

[psychodad1000]

Sorry, I guess I'm going to be the problem child....

Popped in a new 4TB drive and formatted XFS.  Everything looked good.

Created t folder on new drive.

Started Putty, executed command: cp -rpv /mnt/disk1/* /mnt/disk7/t.

System started copying 2TB from disk1 to disk7, files names flying by at the speed of light (almost).

Came back some hrs later and find this msg: "Network Error:software caused connection abort" .

Only 1.2 of the 2TB was copied.  I left it overnite to see if it was just the putty connection to my computer that failed and hopefully the copy process was still running.

No joy.  The copy did stop at 1.2TB.  All drives are still green balled.  I visually checked and many files were copied to the t folder of the new drive.

Ran rsync and got a long list of files which didn't copy.

 

Anyone else have the copy command bomb midway thru the process?

What should I do now?  Reformat the drive and try again?

Probably putty connection. Any process you expect to take a long time should be done in a screen session so you can disconnect and resume as needed. Screen can be installed from NerdTools plugin. Google Linux Screen Command.

 

Noob still at it.  Bear with me....

Forgot about screen.  Used screen to get the files copied to /t on destination.

Ran rsync, results were good.

 

10 - Now is the good time to move the files in the "t" directory to the root on [dest]. I do this with cut and paste from Windows explorer.

 

All I see in explorer are user shares.  How do I see the disk contents so I can move /t to the root.  I was going to try Midnight Commander but chickened out.

 

 

[itimpi]

10 - Now is the good time to move the files in the "t" directory to the root on [dest]. I do this with cut and paste from Windows explorer.

 

All I see in explorer are user shares.  How do I see the disk contents so I can move /t to the root.  I was going to try Midnight Commander but chickened out.

If you want to be able to see shares for the individual disks then you need to turn on disk shares under Settings->Global Share settings.  By default they are disabled to reduce the chance of users experiencing data loss by mixing disk shares and user shares in the same copy command.

[gundamguy]

All I see in explorer are user shares.  How do I see the disk contents so I can move /t to the root.  I was going to try Midnight Commander but chickened out.

 

You don't have to do it though Windows Explorer, if you are comfortable with Linux you can move the "t" folder using mv (or another command that works) as well.

 

Going to say this again, because it can't be said too many times.

 

NEVER copy from /mnt/user/Share to /mnt/disk/Share or vice versa. In windows explorer this would be copying from a "Share" to "DiskX" (Where X is the disk number). This causes a file system error that will result in data loss.

 

mnt/user0/share is a special case but I am 99% sure it suffers from the same user share to disk problem.