Jump to content
savestheday

Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)

900 posts in this topic Last Reply

Recommended Posts

Hi,

I recently revived a very old version (4.something) of my unraid to the new age (6.1.6). My server has numerous old hdds, with 500GB each and reisrfs. Now I  have some 3TB drives, that I would like use instead. If I understood correctly, just exchanging one drive at a time would bring me reiserfs 3TB drives(after rebuild). If I expand my system with a new 3TB xfs drive, I need to copy the data. What happens with the user shares?

 

e.g disk 1 is 500GB and has partial files of 4 user shares

If I copy all that to a new 3TB drive, then stop the array, take out the old 500GB, restart the array. Would I need to rebuild parity for everything? Would the user shares be intact?

 

Thanks a lot for your help!

Dominik

 

Share this post


Link to post

Hi,

I recently revived a very old version (4.something) of my unraid to the new age (6.1.6). My server has numerous old hdds, with 500GB each and reisrfs. Now I  have some 3TB drives, that I would like use instead. If I understood correctly, just exchanging one drive at a time would bring me reiserfs 3TB drives(after rebuild). If I expand my system with a new 3TB xfs drive, I need to copy the data. What happens with the user shares?

 

e.g disk 1 is 500GB and has partial files of 4 user shares

If I copy all that to a new 3TB drive, then stop the array, take out the old 500GB, restart the array. Would I need to rebuild parity for everything? Would the user shares be intact?

 

Thanks a lot for your help!

Dominik

 

You need to read at least the early part of this thread. If you rebuild your 500G rfs drive onto a 3T drive, you will wind up with a 3T rfs formatted drive. You cannot rebuild an rfs drive and have it be xfs when it us done. To unRaid the entire disk is just a long string of 1s and 0s. It cannot tell the difference between which ones related to file system versus data files.  Switching to a different file system requires copying the data off of an rfs drive and onto an xfs formatted drive.

 

This thread is full of people who have used various tweaks to the method I documented to accomplish this in a reliable way.

Share this post


Link to post

This has been discussed at length. There are good reasons - rfs is an aging filesystem and not being enhanced. We've seen a couple of file corruption bugs creep into rfs that have caused silent data corruption in some 6.0 betas - even for files that are not updated!  Many have had performance problems with larger drives with rfs, especially as the drive gets fuller. Xfs is widely used. It is definitely what you should use for any new disks. Some people here have argued for leaving rfs in place for old drives but I don't agree. A bunch of annoying hangs and timeouts have gone away since I switched. I am glad it is off my system. And one more reason - the author is in jail for brutally murdering his wife. I am happy not contributing to any popularity of his invention.

Share this post


Link to post

This has been discussed at length. There are good reasons - rfs is an aging filesystem and not being enhanced. We've seen a couple of file corruption bugs creep into rfs that have caused silent data corruption in some 6.0 betas - even for files that are not updated!  Many have had performance problems with larger drives with rfs, especially as the drive gets fuller. Xfs is widely used. It is definitely what you should use for any new disks. Some people here have argued for leaving rfs in place for old drives but I don't agree. A bunch of annoying hangs and timeouts have gone away since I switched. I am glad it is off my system. And one more reason - the author is in jail for brutally murdering his wife. I am happy not contributing to any popularity of his invention.

 

All that is good, but one additional reason is that rfs doesn't really support the Dynamix File Integrity plugin which many people are interested in running. (This is a design flaw in that plugin... but the fact remains that it's the way it is...)

Share this post


Link to post

...

If I copy all that to a new 3TB drive, then stop the array, take out the old 500GB, restart the array. Would I need to rebuild parity for everything? Would the user shares be intact?

yes and yes

Share this post


Link to post

Hi guys, this is a pretty daunting task. I've just spent the last week replacing most drives (including parity) from the array only to realize that when I finally added a new drive it was formatted to XFS. Then on search came here. Anyway, this is my scenario.

 

Parity - 5TB

Disk 1 - 5TB - RFS

Disk 2 - 5TB - RFS

Disk 3 - 2TB - RFS

Disk 4 - 2TB - RFS

Disk 5 - 5TB - XFS (brand new)

 

The contents on Disk 1 and 2 could fit in Disk 5, so the plan is:

 

Round 1: Disk 1 & 2 copy & verify to Disk 5 (already XFS)

- Format Disk 1 & 2 to XFS

- Copy back from Disk 5  to Disk 1 and 2, verify

- Empty Disk 5

 

Round 2: Disk 3 and 4  copy and verify to Disk 5

- Format Disk 3 and 4 to XFS

- Copy back from Disk 5 to Disk 3 and 4, verify

 

Does that seem acceptable?

 

Also, forgive me, I've tried to search and it's been a while since I payed attention to unRAID, I'm confused about how to format an existing drive RFS drive to XFS. Once I've copied all the files from the source to dest, do I then stop the array then unmount the source drives followed by formatting to XFS, then copy the data back (after verify)? Thank you.

Share this post


Link to post

... I'm confused about how to format an existing drive RFS drive to XFS.

 

This is VERY simple.    Simply Stop the array;  go to the Main tab;  click on the disk you want to change;  and then change the "File System Type" to what you want.  (e.g. XFS)

 

NOTE:  This will result in this disk being formatted when you then Start the array, which will DESTROY all data currently on the disk ... so be sure you are only doing it to a disk that's either empty or that you've already copied all the data off of and don't need anymore.

 

You can change multiple disks at once before starting the array (e.g. if you've copied all the data from two disks and are ready to format them both).

 

Share this post


Link to post

...  the plan is:

 

Round 1: Disk 1 & 2 copy & verify to Disk 5 (already XFS)

- Format Disk 1 & 2 to XFS

- Copy back from Disk 5  to Disk 1 and 2, verify

- Empty Disk 5

 

Round 2: Disk 3 and 4  copy and verify to Disk 5

- Format Disk 3 and 4 to XFS

- Copy back from Disk 5 to Disk 3 and 4, verify

 

Does that seem acceptable?

 

Works fine, but more copying than you really need.

 

You could simply do this:

(a)  Copy all of disk1 and disk2 to disk5 (& verify)

(b)  Reformat disk1 and disk2 to XFS

©  Copy disk3 to disk1 and disk4 to disk2 (& verify)

(d)  Reformat disk3 and disk4 to XFS

 

Done  :)

 

Unless you have some reason you want your data on specific disks, that's all you need to do.

 

 

Share this post


Link to post

... I'm confused about how to format an existing drive RFS drive to XFS.

 

This is VERY simple.    Simply Stop the array;  go to the Main tab;  click on the disk you want to change;  and then change the "File System Type" to what you want.  (e.g. XFS)

 

NOTE:  This will result in this disk being formatted when you then Start the array, which will DESTROY all data currently on the disk ... so be sure you are only doing it to a disk that's either empty or that you've already copied all the data off of and don't need anymore.

 

You can change multiple disks at once before starting the array (e.g. if you've copied all the data from two disks and are ready to format them both).

 

Thank you so much! I'll quadruple check before each step!

 

...  the plan is:

 

Round 1: Disk 1 & 2 copy & verify to Disk 5 (already XFS)

- Format Disk 1 & 2 to XFS

- Copy back from Disk 5  to Disk 1 and 2, verify

- Empty Disk 5

 

Round 2: Disk 3 and 4  copy and verify to Disk 5

- Format Disk 3 and 4 to XFS

- Copy back from Disk 5 to Disk 3 and 4, verify

 

Does that seem acceptable?

 

Works fine, but more copying than you really need.

 

You could simply do this:

(a)  Copy all of disk1 and disk2 to disk5 (& verify)

(b)  Reformat disk1 and disk2 to XFS

©  Copy disk3 to disk1 and disk4 to disk2 (& verify)

(d)  Reformat disk3 and disk4 to XFS

 

Done  :)

 

Unless you have some reason you want your data on specific disks, that's all you need to do.

 

I prefer to keep my movies (5GB - 25GB) on the faster and larger disks, they will be doing Direct Play through Plex. Disk 4, for Apps and Backup, while more media on Disk 5 that don't get used as often, ie. Anime and Sport.

Share this post


Link to post

All I see in explorer are user shares.  How do I see the disk contents so I can move /t to the root.  I was going to try Midnight Commander but chickened out.

 

You don't have to do it though Windows Explorer, if you are comfortable with Linux you can move the "t" folder using mv (or another command that works) as well.

 

Going to say this again, because it can't be said too many times.

 

NEVER copy from /mnt/user/Share to /mnt/disk/Share or vice versa. In windows explorer this would be copying from a "Share" to "DiskX" (Where X is the disk number). This causes a file system error that will result in data loss.

 

mnt/user0/share is a special case but I am 99% sure it suffers from the same user share to disk problem.

 

Just to be clear, though, I should be able to move the contents of one share (ex: TV-Shows spread across 5 RFs formatted disks) to another share (ex TV Shows on 5 XFS formatted disks) with no problem right?

 

The shares, are they just directories that I can cd into and do something like a mv * to \(destination share)

 

 

Share this post


Link to post

As long as the shares don't have the same name (and you set the excludes and includes for your RFS vs XFS correctly) you should be ok.

 

A user share say "TV-Shows" is simply an aggregation of all the "TV-Shows" directories on all of your disks including the cache. From a Linux prospective it's taking all the /mnt/disk1/TV-Shows (for each disk on which a TV-Shows directory exists and the cache drive) and creating a location /mnt/user/TV-Shows that is the sum of all the contents in the TV-Shows directory on each disk.

 

Caution: NEVER MIX your user paths with disk paths when using commands that move, write, or sync data between two directories. This will result in data corruption.

 

Note: When you start looking under /mnt/ you'll see Disk1-X, User, and User0.  User0 is the same as User but excludes the cache drive.

 

So to answer your actual question. Yes if the shares have different names you could do a command like mv /mnt/user/TVShows /mnt/user/TV-Shows

Share this post


Link to post
So to answer your actual question. Yes if the shares have different names you could do a command like mv /mnt/user/TVShows /mnt/user/TV-Shows

One thing to note is that I am reasonably sure that if you use the mv command then files and folders remain on the same disks they are currently located on (I think that under the covers mv will just do a rename).  This will only be an issue if you thought they might be redistributed by such an action.

Share this post


Link to post

As long as the shares don't have the same name (and you set the excludes and includes for your RFS vs XFS correctly) you should be ok.

 

A user share say "TV-Shows" is simply an aggregation of all the "TV-Shows" directories on all of your disks including the cache. From a Linux prospective it's taking all the /mnt/disk1/TV-Shows (for each disk on which a TV-Shows directory exists and the cache drive) and creating a location /mnt/user/TV-Shows that is the sum of all the contents in the TV-Shows directory on each disk.

 

Caution: NEVER MIX your user paths with disk paths when using commands that move, write, or sync data between two directories. This will result in data corruption.

 

Note: When you start looking under /mnt/ you'll see Disk1-X, User, and User0.  User0 is the same as User but excludes the cache drive.

 

So to answer your actual question. Yes if the shares have different names you could do a command like mv /mnt/user/TVShows /mnt/user/TV-Shows

 

And a wild card is Kosher?

Share this post


Link to post

So to answer your actual question. Yes if the shares have different names you could do a command like mv /mnt/user/TVShows /mnt/user/TV-Shows

One thing to note is that I am reasonably sure that if you use the mv command then files and folders remain on the same disks they are currently located on (I think that under the covers mv will just do a rename).  This will only be an issue if you thought they might be redistributed by such an action.

 

 

???HUH???

 

If the share I'm copying from is restricted to disks 4, 5, 6 and 7 and the share I'm copying to is restricted to drives 10, 11, 12 and 13 that would move the files from the first set of drives to the send ... right?

 

That is after all my objective ... to get the files off the RFS formatted drives and onto the XFS drives so I can format the former.

Share this post


Link to post
my objective ... to get the files off the RFS formatted drives and onto the XFS drives so I can format the former.
Take a look at the unbalance plugin. It is designed to automatically and safely clear off a drive by moving the contents to the other array drives.

Share this post


Link to post

So to answer your actual question. Yes if the shares have different names you could do a command like mv /mnt/user/TVShows /mnt/user/TV-Shows

One thing to note is that I am reasonably sure that if you use the mv command then files and folders remain on the same disks they are currently located on (I think that under the covers mv will just do a rename).  This will only be an issue if you thought they might be redistributed by such an action.

 

 

???HUH???

 

If the share I'm copying from is restricted to disks 4, 5, 6 and 7 and the share I'm copying to is restricted to drives 10, 11, 12 and 13 that would move the files from the first set of drives to the send ... right?

 

It should, as far as I am aware... MV is move, but since the path is a different share it should actually change the file location not just path... right?

 

You could do a copy and delete just to be safe. Or use Rsync.

 

Wildcards should be ok, but I'm not sure why you need them when you can (Should?) be moving directories and all sub-directories.

Share this post


Link to post

I still haven't started my move because the new drive is still preclearing. Just reading around including the several suggestions here, it looks like the plugin unBalance might be a good tool to use moving drive contents.

Share this post


Link to post

Over the weekend I decided to move one of my disks to the xfs filesystem. I have a total of 5 disks (3 data disks, 1 parity, 1 cache). Two of those 3 data disks were is RFS format(disk1, disk2) and 1 was in the XFS format (disk3). Disk3 (XFS) has been sitting empty in my array for several months waiting to be used to convert my RFS disks. So I moved disk1 into disk3 and it seems to have worked.

 

The problem I am running into is that my user shares are not showing the files after the move. For example if I open up Movies share in the unraid GUI I shows the files that were on disk2 are still there. But all the files that were on disk1 are missing and there is no mention of disk3. I know the files exist because if I use the command line to view files direct on disk3 they are there. I am not sure what to do next.

 

Below are the steps I followed.

 

1. Installed new empty drive (disk3(XFS) months ago)

2. Turned off all dockers, turned off mover and parity checker/builder thing

3. Opened up screen session

4. Started copy using rsync (Took ~20 hours total - No errors that I know of)

rsync -av --progress /mnt/disk1/ /mnt/disk3/

(~15 hours)

rsync -avc --progress --remove-source-files /mnt/disk1/ /mnt/disk3/

(~5 hours)

5. Checked a handful of files on disk3 (Seemed good). Disk1 was left with empty folders (Just rsync)

6. Checked unraid main page GUI. Disk1 showed empty (bascially (33mb - folders)). Disk3 showed about the size that disk1 was before copy (~2.5TB)

7. Turned off array

8. Changed filesystem on Disk1 to XFS

9. Started array and disk1 showed unformatted. So I formatted it. (Disk1 was the only one the showed)

10. After the format I looked at the main page GUI. I had two disks XFS now (Disk1 - empty, Disk3 - ~2.5TB)

11. I looked in the different shares that use to be on disk1 and they still exist but the files that were in disk1 were missing

12. Logged in command line and disk3 still has the files even though they are missing from the shares.

 

Will this fix itself or do I need to do something to let the share know of the move? Right now my array is in the off state. I don't want unraid wiping disk3 for some odd reason because it looks like it still has all the files.

 

I am running unraid version 6.1.7

Share this post


Link to post

Make sure disk3 is included or not excluded in the Global Shares Settings, and also that disk3 is included or not excluded in the settings for that specific user share.

 

Include means "only these disks", Exclude means "except for these disks". If both are blank then none are excluded and there is no limit to the disks that are included. So both blank means include all.

 

If that doesn't clear things up try stopping and starting the array since array start is when the user shares are actually instantiated.

Share this post


Link to post

@trurl thanks, I guess months ago when I setup disk3 I guess I excluded it from the global settings so nothing would get put on it. I removed the excluded and started up the array and it looks like my files are showing up in the share again and say they are on disk3.

 

Glad that was a simple fix.

Share this post


Link to post

Hi all!

 

I have been busy moving things around in my array as I leave ReiserFS for XFS.  No problems.

 

I have noticed that I still have a mix of 4k-aligned and unaligned MBRs.

 

My solo 4TB is 4k; the others (a mix of 2, 1, and 0.75TB drives) are showing a mix of 4k aligned and not.

 

I perhaps erroneously assumed that the reformat to XFS would by default 4k align things but it appears not.

 

Questions are...

 

1) Any benefit to having them 4k or not?  Peformance?  I am more than sure my older 2TBs and before are non-AF drives.

2) To make them all 4k, I must emtpy them, preclear them, format again as XFS and it will force 4k?

 

Is there a tool to run that shows conclusively which drives are AF and which are not?

 

Thx!

 

Kev.

Share this post


Link to post

 

The procedure in this post is only for releases of unRAID v6.0 and v6.1.  For unRAID v6.2 or later, go to File System Conversion on the wiki.

 

 

A few days ago, I finally began the process of converting old drives to XFS, and I found a few ideas that make the process simpler and less of a disruption.  Parity protection is always preserved, and normal server operation can continue at almost all times, except for brief moments of swapping drive assignments.  I use the rsync command with the -avPX options, as it accurately performs copies of all data, metadata, and extended attributes of all files and folders.  In addition, from statements online, I understand that rsync ALWAYS checksums every bit of data transferred all the way to its reconstruction at the destination.  That makes the additional lengthy checksum verification almost redundant, so I include that verification only as an option, for the most paranoid.  I've come to feel that an rsync transfer is the only option most users need, and that speeds this process up.

 

Then I perform a simple swap procedure, that preserves all shares and data, and doesn't require any lengthy parity checks or builds.  You can if you wish perform parity checks at any time, but they are optional, if you are sure there has been a successful recent one before you start.  I exclude the swapping drive from User Shares, so you never have to worry about files duplicated within shares, and therefore don't need a temporary copy location, and never need to move any files.

 

Important Update!  The swap trick below does not work in unRAID v6.2] (steps 10 through 13).  This is really unfortunate as the swap trick is a key part, makes the procedure much simpler and more convenient to perform.  I've left this post unchanged, but it is only for releases of unRAID v6.0 and v6.1.  For v6.2 or later, go to File System Conversion on the wiki.

 

Update and Important Warning!  My system is static, and my array only changes when I manually copy files and make changes.  Which is why I completely forgot that other user's arrays are dynamic, constantly changing in the background due to Dockers, plugins, VM's, scheduled backups from other networked machines, and the Mover copying and moving files around.  That's a big problem when you are converting a drive, because you cannot allow any changes to it once the copying begins.  As the copy sweeps through the folders of the original drive, any new files added to folders already processed will be lost!  It is your responsibility to make sure that NOTHING can make changes to a drive being copied and converted!  That may mean stopping your Dockers, plugins, VM's, and the Mover, if any of those could possibly add files or make changes to the drive being converted.  And you should check for and temporarily disable any scheduled backups from other machines.  Yes, this can require some careful planning!  You may want to consider rebooting into Safe Mode (stops plugins from loading), and disabling any VM's and the Docker service and the Mover.

 

Procedure to convert drives to XFS format

================================

System assumptions before you begin:

  - You have a given number of data drives you wish to convert (we will use an example of 10 data drives)

  - You have prepared a new empty drive that is as large or larger than the largest data drive; preferably you have Precleared it or otherwise tested it

  - Note: the new drive should not be formatted with XFS; if it is, then you will need to clear it or format it to something else first

 

1. If you haven't recently run a successful Parity Check, do so now; you want to be sure the array is perfect before you start

2. Prepare a strategy for the order of drive conversion.  Because you can't replace a larger drive with a smaller drive (unless the total file space used will fit on the smaller drive), you will have to order the conversions so that your largest data drive is first, then the next largest, then the next, with the smallest data drive being last.  Obviously, it doesn't matter for drives that are the same size.

3. With your empty drive installed and array stopped, assign it to the next empty drive slot (for our example, we will assign it to Disk 11)

4. Click on the disk name of your empty drive (e.g. 'Disk 11') and change the format to XFS if it isn't already, then click Apply and Done

5. If you have enabled User Shares (and most users have), go to Settings -> Global Share Settings and add your empty disk to 'Excluded disk(s)' (for our example, we would put disk11)

6. Start the array; your empty drive should show as 'Unmountable', and a Format button will be present

7. Click the check box for formatting, then click the Format button; it takes a few minutes, says it's formatting; when done, array should show an additional drive, almost completely empty, formatted with XFS

8. At the console or within a screen session, copy all data from your drive to be converted to the new and empty drive; use an rsync command based on the following, except change the drive numbers as appropriate for your system; type it exactly with the same slashes, upper and lower case matter; this command will take a long time but parity will be fully preserved; when complete, prompt should return with no errors showing; your array now has 2 drives that are identical except for format (one is excluded from shares)

  rsync -avPX  /mnt/disk10/ /mnt/disk11/  (using our example, copying our large disk10 to the new empty drive)

9. This step is optional, as the previous rsync automatically checksums each transfer.  But if you would like to verify that the end-to-end transfer was perfect, perform the next rsync command below; it will take a long time, and probably nothing will be copied unless the drive has been updated (see warning below!) since the full copy above; there's no progress info, it's over when the prompt returns

  rsync -rcvPX  /mnt/disk10/ /mnt/disk11/

10. Stop the array; we are now going to swap their assignments; when stopped, click on the dropdown for the new drive (e.g. Disk 11) and unassign it; it should notify you of 'Missing drive'

11. Click on the dropdown for the other drive (e.g. Disk 10), the one being converted, and reassign it as the physical drive that was just added, the new drive that was empty; you will be notified of 'Wrong drive'

12. Click on the dropdown for the slot of the new drive (e.g. Disk 11) and reassign it to the physical drive that was being converted (e.g. Disk 10); you will get more notifications of wrong or missing drives; you have just swapped the 2 drives, which is fine as they are identical (except for file system format)

13. Important! Click on each drive name (e.g. Disk 10 and Disk 11) and change the format of the drive; if it's ReiserFS change it to XFS, if it's XFS change it to ReiserFS; it's important to swap the disk formats as well as the physical drive assignments

14. You should see both drives listed with errors, possibly as unmountable, and a check box; click it and click the Start button to start the array; it should start up without issue and look almost identical to what it looked like before the swap, with no parity check needed; however the XFS disk is now online and its files are now being shared as they normally would; check it all if you are in doubt

15. If you are sure it's all fine, stop the array and click the empty disk slot (e.g. still Disk 11), and change the format to XFS, then click Apply and Done

16. Start the array; the Format button should be available, format it now; when done, your empty disk slot now has a fresh and empty disk formatted with XFS and ready to fill again; your data drive has completed the conversion process and is already back online, with all files and shares intact, but formatted with XFS

17. You are now ready to convert the next drive, so circle back to Step 8 and repeat these steps (Step 8 through Step 16), substituting your next drive to be converted; the empty and excluded disk slot will always be the same (e.g. always Disk 11 in our example), the other will change as you convert different data drives

 

When done, you have an empty XFS drive appended to your system, probably your smallest drive, and still excluded.  It's up to you what you want to do with it.  You can leave it as is, or you can unassign it and rebuild parity, or you can use the parity preserving remove-a-drive procedure, instructions elsewhere.  Remember, it's probably still globally excluded from shares.

 

I do recommend that if you are going to try this procedure, you read through them carefully until you fully understand them, and understand the importance of each detail.  Missing a step or typing the wrong disk number could be disastrous!

 

If you wish, you can perform parity checks at any point during and after.  I don't believe they are necessary, I only did one before starting, and I may do one after the last.

 

Warning!  If you run the verification copy in Step 9, and it actually copies files, then it is likely you have a process still changing the drive!  These newly copied files were not there for the Step 8 copy!  You need to determine what process (Docker, plugin, VM, an external backup, or the Mover) made the changes to this drive, and stop it.  Then you may need to run Step 9 again, because the process may have made even more changes to folders, after the Step 9 rsync process had moved past those folders.  In summary, if the Step 9 copy actually copies any files, then you should probably repeat Step 9 until nothing is copied.

 

I've checked the above pretty carefully, if you see any errors, PLEASE let me know ASAP!  I'm sure it can be improved.  Steps 15 and 16 are a repeat of 4, 6, and 7, but it seemed safer this way.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.