Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)


Recommended Posts

Hi all,

So I'm not sure if this is expected behavior but I am running U6 b12 and just swapped DISK11 for a bigger disk (1.5TB -> 4.0TB). For some background, I moved from 5.0.5 straight to 6.0 b12 at the beg of December. I have my settings set to format all new drives as XFS. I precleared this drive in a temporary unRAID box and it had no FS on it. The way I removed it was setting the array not to start on next boot, shutting down, swapping the drive, starting back up, choose the new drive from the drop down and start the array.

 

http://d.pr/i/1b8Sl

 

However I am looking at the array this morning and it formatted as reiserfs. Is this expected behavior since it replaced a reiserfs drive? If not, how can I reformat to XFS? After the recent ReiserFS FS corruption bug, I don't feel comfortable keeping new drives as ReiserFS and planned on moving data to this new drive from other drives one at a time and formatting all 23 of my data drives as XFS.

Link to comment

Hi all,

So I'm not sure if this is expected behavior but I am running U6 b12 and just swapped DISK11 for a bigger disk (1.5TB -> 4.0TB). For some background, I moved from 5.0.5 straight to 6.0 b12 at the beg of December. I have my settings set to format all new drives as XFS. I precleared this drive in a temporary unRAID box and it had no FS on it. The way I removed it was setting the array not to start on next boot, shutting down, swapping the drive, starting back up, choose the new drive from the drop down and start the array.

 

http://d.pr/i/1b8Sl

 

However I am looking at the array this morning and it formatted as reiserfs. Is this expected behavior since it replaced a reiserfs drive? If not, how can I reformat to XFS? After the recent ReiserFS FS corruption bug, I don't feel comfortable keeping new drives as ReiserFS and planned on moving data to this new drive from other drives one at a time and formatting all 23 of my data drives as XFS.

A drive rebuild can only recreate what was there originally, and expand from there. If you reformat the drive, all data on it will be gone. If you would have added the drive instead of swapping, it would have been set up as XFS since that is your setting for new drives. Do you have enough space on the rest of the array to clear off that new drive?

 

BTW, there is a forum section for the 6 beta stuff, this section is for 5.

Link to comment

It is very important to understand the way parity works. When you swap / reconstruct a drive, unRAID is going to restore it as a bit for bit copy. It DOES not copy each file individually. If the disk was fragmented, the restored disk is fragmented. Every sector of the new disk will be identical to every sector on the old disk.

 

So if the old disk is formatted with RFS, when you rebuild it, the new disk will be formatted with RFS. If you tell unRAID that the FS on the new disk is XFS, and then you restore an RFS disk on it, unRAID will still recover it sector by sector, but in the end unRAID will not recognize it as having a valid XFS format. If will show "unformatted" because it not contain a valid XFS disk format. You would have to switch it over to be a reiserfs disk, and then unRAID should (hiopefully, I've never tried it) recognize it and mount it. But if you do a recovery and the wrong FS is selected, and then click "format" because you are confused as to why the disk didn't recover, your recovery would be LOST and you'd be having to use reiserfsck (a last resort) to try to salvage some of your data!

 

So hopefully the message is clear. You cannot use unRAID's disk rebuild process to switch file systems. It does not work that way. If you do not understand why after this explanation (OP and other users), please ask for clarification. This goes to the heart of understanding what unRAID is and does, and understanding this very basic concept will help you recover from a variety of situations.

 

So how do you move from RFS to XFS? First, you need to buy (or free up) a disk as large as your largest data drive. If it is a new disk, you have to preclear it and add it to your array like any other new disk.

 

For these instructions, I will assume your array looks like this:

parity - 4T

disk1 - 4T - RFS

disk2 - 3T - RFS

disk3 - 4T - RFS

disk4 - 2T - RFS

disk5 - 4T - new/empty disk, FS doesn't matter we're going to set it in a minute

 

1 - Define your disk conversion order. Below is an example for this sample array. Make sure your order takes into account the drive sizes. If you copy a 2T drive to your new 4T drive, the 2T drive will not be big enough to convert your next 4T drive. You'll see I start with the biggest disk, and then move to the smaller ones. In the end, the smallest disk is empty. Substitute [source] and [dest] in the instructions below based on the round you are in. I don't address cache conversion. That's another topic.

 

round [source] [dest]    notes

------- ----------- -------- ------------

  1      disk1    disk5

  2      disk3    disk1  (4T to 4T)

  3      disk2    disk3  (3T to 4T)

  4      disk4    disk2  (2T to 3T)

  5      ---        disk4  (empty the 2T)

 

2 - With array stopped, change the filesystem on [dest] to XFS (or BTRFS is that is your preference. Most users are going with XFS).

 

3 - Start the array, the new XFS disk should show unformatted, all the other disks should be formatted!! (IF NOT, DO NOT PROCEED)

 

4 - Format the unformatted with unRAID. XFS disk formats.

 

(If you are on the last round and just reformatting the last disk to XFS - round 5 for the example - you are done!!!)

 

5 - Open or resume a screen session

 

(Aside - we are about to copy all of the files from one of your RFS disks to the XFS disk. If the disk is involved in a user share, this will result in creating duplicate files out the ying yang! For that reason, I create a temporary directory (I call it "t") on the XFS disk and copy everything there. This will eliminate the possibility of your user share dealing with and possibly logging all the duplicates. Once the copy is completed and verified, we will move the files from the "t" directory to the root, and get rid of the "t" directory.)

 

6 - Make the "t" directory on [dest]. You can easily do from Windows Explorer or with this command:

  mkdir /mnt/[dest]/t

 

7 - Copy the data from the [source] to [dest]. Note this will take a long time (this is why doing it in a screen session so it will not timeout). Do overnight or at a convenient time when the array will not be heavily utilized. You definitely don't want to be doing a parity check while this is happening! Note that -rpv means do it recursively, preserve the permissions, and be verbose (it lists each file it copies). Make sure they are all in lower case.

  cp -rpv /mnt/[source]/* /mnt/[dest]/t

 

Note if you happen to have the [dest] disk red-ball in the middle of the copy, you will have to do a drive replacement. I would recommend stopping the copy once you notice this has happened. You can resume it later. Then follow the normal disk replacement procedure. If this happens it is likely a file will get have been corrupted and the comparison below becomes very important so you know which one it is so you can re-copy it from the [source] before the source gets overwritten in the next round.

 

8 - It is now highly recommended to compare the checksums of the source to the dest. (If not, at least spot check several files). There are multiple ways to do that including md5sum, bitrot (a user contributed script). There is also a way to use rsync (perhaps Weebo will post or link instructions - sorry, i couldn't find them) to do the copy and verify in one rsync step. If there are questions here, please ask. My method is a bit of a Rube Goldberg, but will try to clean it up and post if there is interest.

 

UPDATE: Here is one way to complete step 8 (see later post in this thread) (thanks to danioj):

 

From a screen session, run this command:

  rsync -nrcv /mnt/[source]/ /mnt/[dest]/t >/boot/verify.txt

 

This will put the output of the comparison in a file on the flash disk. It will likely have little or no output, but this will preserve all output in case it is lengthy and would scroll off the screen.

 

You can view the output with this command in another session:

  cat /boot/verify.txt

 

9 - If there are any comparison failures, stop here and ask for help

 

10 - Now is the good time to move the files in the "t" directory to the root on [dest]. I do this with cut and paste from Windows explorer.

 

11 - Stop the array (no need to delete anything from the [source])

 

12 - Go back to step 2.  Note that this isn't a race - you can do it at your leisure of the course of days, weeks, or months. I do one or two a week or so.

 

Good luck. Post any questions.

  • Like 1
Link to comment

Pls, let me also know how you would proceed in this case: 1 parity drive 3tb, disk1 750gb _rfs (with data that I don't need), new added empty disk2 2tb_xfs. I want to delete all the data in disk1 and format it to xfs. Just that.

A second scenario would be to assign the 2tb drive as disk1 and set the 750gb disk as cache drive.

Rgds.

 

Link to comment

Pls, let me also know how you would proceed in this case: 1 parity drive 3tb, disk1 750gb _rfs (with data that I don't need), new added empty disk2 2tb_xfs. I want to delete all the data in disk1 and format it to xfs. Just that.

A second scenario would be to assign the 2tb drive as disk1 and set the 750gb disk as cache drive.

Rgds.

Stop array, select xfs filesystem for disk1, start array, let if format disk1. No deletion needed since it will be an empty xfs filesystem after the format, and since this all happens with the array started, parity is kept in sync during the format.

 

For the 2nd scenario do new config and make sure that you don't assign a data drive to the parity slot.

Link to comment

I agree. For the second scenario, you'd want to assign the 750g to the cache slot, and change it to use XFS. You can format it once the array is started.

 

Changing disk1 to XFS would be very quick. Removing the 750g drive will require rebuilding parity which will take a number of hours. Plan accordingly.

Link to comment

There is also a way to use rsync (perhaps Weebo will post or link instructions - sorry, i couldn't find them) to do the copy and verify in one rsync step.

 

I don't know the entire procedure for copy verify in one step but I think the procedure to verify is to get rsync to do a simulated sync and display the difference in files it has found:

 

rsync -nrcv /path/to/sourcefolder /path/to/destinationfolder

 

The output from this is a list of files in the source folder that are different.

Link to comment

So, for step 8, you could use ...

  rsync -nrcv /mnt/[source] /mnt[dest]/t >/boot/verify.txt

 

It should be done from a screen session or it will time out and stop.

 

This will put the output in a file on the flash disk. It will likely have little or no output, but this will preserve it in case it is lengthy and would scroll off the screen.

 

You can view the output with this command in another session:

  cat /boot/verify.txt

 

Link to comment

I've been reformatting/copying my RFS drives to XFS for several days now and I've been doing it on a windows box in the basement connected to my server.  It does a few other things for me so I just have a telnet window open to the server and do everything from there.  No timeouts to worry about and if I need to reboot my main box in the office, no big deal.  I just VNC into it when I need to.

 

For step 7, I use the following:  (assuming I'm copying from disk1 to a new formatted and empty disk2)

 

  rsync -arv /mnt/disk1/ /mnt/disk2  (I use -a because for me preserving time/date stamps is important.)

 

Then, I open two command prompts on a windows machine.  On the first one, I execute:

 

  dir \\tower\disk1 /s

 

and on the second one:

 

  dir \\tower\disk2 /s

 

When the directory listings are done, at the bottom of each command line window I should see an identical number of files and directories and the byte count should be identical.

 

Now I know everything has at least copied over

 

Back to my telnet session I then do this to verify the copy: (step 8)

 

  diff -rq /mnt/disk1 /mnt/disk2

 

But before I do that, I put a couple of files with the same name that I know will be different into to the same directories on both disks.  This way, I know I should see my "seed files" show up and generate an error.  If anything, it give me some confidence that the process is working.

 

Yes, this will take longer but for me, the extra checks and balances are worth it.

 

Link to comment
  • 4 weeks later...

I just started this process last night. I want to say thanks for the intro in how to make this change. With out your post I doubt I would have taken the effort to make the change to xfs.

 

I asked a few questions in another thread and someone made a suggestion which I want to share because I believe it reduces the steps, and is a more powerful approach.

 

To my knowledge the following code represents a simplified method to accomplish steps 6, 7 & 8 in one pass while also maintaining permissions, timestamps, owners, groups, symlinks, and device files using rsync. 

 

rsync -av --progress --remove-source-files /mnt/diskX/ /mnt/diskY/

 

Where disk X is the rfs disk you are copying to the newly formated xfs disk (disk Y).  For example I ran this last night on as /mnt/disk1/ /mnt/disk4/ copying disk1 to disk4

 

-a means it's in "archive mode" which equals -rlptgoD (Recursive, Symlinks, Permissions, Timestamps, Groups, Owners, Device Files) Note if you want to Preserve extended attributes add a capital X to your call of rsync.

-v puts it in verbose mode (verbose is not required)

--progress shows you the progress of the copies as files are moving (Progress is not required)

--remove-source-files : After the copy is done rsync will verify and then remove the source files. IMPORTANT do not use --remove-source-files if you have files currently being writen to the disk, this will result in corruption of the that file. You should do this anyway since your moving data from one disk to the other, but disable any plugins - dockers which write to that disk, and it's best to not write to shares that could be put on that disk while rsync is running.

 

The end result should be empty directories on diskX and a duplication of diskX on diskY.

 

Rsync is very powerful and there are way more things that can be done with it, so if you have questions or want to know more about what it can and can't do you can read more here.

 

I think this is a much more elegant solution.

 

Link to comment

Will this prevent corruption if a drive red balls during the copy?

 

After a bit of reading I think the answer is yes.

 

Quotes from the Rsync Documentation.

 

Under the -c (use Checksum) it says.

Note that rsync always verifies that each transferred file was correctly reconstructed on the receiving side by checking a whole-file checksum that is generated as the file is transferred, but that automatic after-the-transfer verification has nothing to do with this option's before-the-transfer "Does this file need to be updated?" check.

 

And under --remove-source-files it says.

This tells rsync to remove from the sending side the files (meaning non-directories) that are a part of the transfer and have been successfully duplicated on the receiving side.

 

So it seems that corruption would set off a redflag and the source file will not be deleted, unless I am missing something.

 

 

Link to comment

I just finished my process of converting from rfs to xfs last night using your described process and rsync to move the data. I had no problems.

 

I do want to say after a bit more reading about rsync I added to my prior post to add a note about not using --remove-source-files if there are active writes to the directories your moving.

Link to comment

I just finished my process of converting from rfs to xfs last night using your described process and rsync to move the data. I had no problems.

 

I do want to say after a bit more reading about rsync I added to my prior post to add a note about not using --remove-source-files if there are active writes to the directories your moving.

I never thought that corruption was possible.  I would have thought rsync would be smart enough to know not to copy / remove the source file if that file was currently being written to.

 

When I did this,  I disabled mover and my backup program while I was converting.  I just didn't want any files to be possibly fragmented to death by multiple writes to the drive at the same time.

Link to comment

I just finished my process of converting from rfs to xfs last night using your described process and rsync to move the data. I had no problems.

 

I do want to say after a bit more reading about rsync I added to my prior post to add a note about not using --remove-source-files if there are active writes to the directories your moving.

I never thought that corruption was possible.  I would have thought rsync would be smart enough to know not to copy / remove the source file if that file was currently being written to.

 

When I did this,  I disabled mover and my backup program while I was converting.  I just didn't want any files to be possibly fragmented to death by multiple writes to the drive at the same time.

 

I would have thought so too, but the documentation says to be warry of this. This might be a disqualifier and why you shouldn't use rsync as I originally suggested.

 

Perhaps a two step process is in order. First

rsync -avP /mnt/diskX/ /mnt/diskY/

then

rsync -avP --remove-source-files /mnt/diskX/ /mnt/diskY/

You could also throw in a checksum comparison on the second one if your concenred by sending rsync the -c attribute.  Also I'm pretty sure that -P is the same as --progress.

 

 

Link to comment

I've a question.

 

After I have copied my dest to source, then move the the files from t to root and stop my array.

 

We then go back to step 2 and format the original source disk.  Doesn't this format change the parity?  Do I have to recalculate parity after restarting the array?

 

thanks

david

Not entirely sure I understand what you have said.

 

When unRAID formats a drive in a parity-protected array, the parity disk is automatically updated. It is fairly quick when this happens because the entire drive is not erased when you format a drive. Many people seem to think formatting a drive means erasing it. What formatting really means is creating an empty filesystem. This is true in any OS. For example, when Windows does a quick format, all it is really doing is setting up some "data" on the disk that the OS uses to start building the directories from.

Link to comment

I've never formatted an already formatted drive.  How do I do this?

 

thanks,

david

 

When you change the disk format (step 2), the disk will appear unformatted when you start the array. You would then format it in the usual way. Parity is maintained through the format operation.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.