Jump to content
savestheday

Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)

900 posts in this topic Last Reply

Recommended Posts

I have a quick question.  I'm sorry if it has already been covered, but I'm still on Unraid 5.  I want to move to 6 and convert all my disks to XFS gradually.  If I run the array with mostly RFS disks and one or two XFS disks, will my array still be protected?  Can I recover from a disk loss, either RFS or XFS disk losses?  I just want to be sure as I have all 2tb drives and want to slowly convert all of them to 5TB drives over the period of a year.  Any clarification would be greatly appreciated.

 

V6 completely supports 3 different file formats, and the array can mix them up, all of one type or some of each type.  Parity protection does not care about what's on a disk, including what file format it is in.

Share this post


Link to post

I have a quick question.  I'm sorry if it has already been covered, but I'm still on Unraid 5.  I want to move to 6 and convert all my disks to XFS gradually.  If I run the array with mostly RFS disks and one or two XFS disks, will my array still be protected?  Can I recover from a disk loss, either RFS or XFS disk losses?  I just want to be sure as I have all 2tb drives and want to slowly convert all of them to 5TB drives over the period of a year.  Any clarification would be greatly appreciated.

 

V6 completely supports 3 different file formats, and the array can mix them up, all of one type or some of each type.  Parity protection does not care about what's on a disk, including what file format it is in.

 

Awesome.  Thanks for the info.

Share this post


Link to post

I'm considering changing my cache drive to an array drive, which would offer a convenient way to move to XFS.

 

Would I just re-format the (emptied) cache drive to XFS, and start then the transfer of files from each data drive to it?

Share this post


Link to post

I'm considering changing my cache drive to an array drive, which would offer a convenient way to move to XFS.

 

Would I just re-format the (emptied) cache drive to XFS, and start then the transfer of files from each data drive to it?

Not clear at what point you intend to add it to the array. You should do this before you format it to XFS since unRAID will want a clear drive if adding it as an additional array drive. A clear drive is not at all the same as an empty filesystem drive.

Share this post


Link to post

I'm considering changing my cache drive to an array drive, which would offer a convenient way to move to XFS.

 

Would I just re-format the (emptied) cache drive to XFS, and start then the transfer of files from each data drive to it?

Not clear at what point you intend to add it to the array. You should do this before you format it to XFS since unRAID will want a clear drive if adding it as an additional array drive. A clear drive is not at all the same as an empty filesystem drive.

 

So I should

1. empty cache drive

2. format it to XFS

3. add to array

4. start moving files from existing array drives to the XFS drive formerly known as cache

Share this post


Link to post

I'm considering changing my cache drive to an array drive, which would offer a convenient way to move to XFS.

 

Would I just re-format the (emptied) cache drive to XFS, and start then the transfer of files from each data drive to it?

Not clear at what point you intend to add it to the array. You should do this before you format it to XFS since unRAID will want a clear drive if adding it as an additional array drive. A clear drive is not at all the same as an empty filesystem drive.

 

So I should

1. empty cache drive

2. format it to XFS

3. add to array

4. start moving files from existing array drives to the XFS drive formerly known as cache

Assuming you don't want to do a new config and rebuild parity, you should preclear the drive, then add it to the array, then format it.

 

If you try to add a drive that is not clear to the array, unRAID will clear it, taking the array offline until it's done, then you will have to format it.

 

The only way to maintain valid parity when adding a drive is to add a clear drive. A clear drive is all zeroes, so has no effect on parity. A formatted drive is not all zeroes, because it has a filesystem written to it.

 

So, you should:

 

1. preclear drive

 

2. add to array

 

3. format

Share this post


Link to post

I'm considering changing my cache drive to an array drive, which would offer a convenient way to move to XFS.

 

Would I just re-format the (emptied) cache drive to XFS, and start then the transfer of files from each data drive to it?

Not clear at what point you intend to add it to the array. You should do this before you format it to XFS since unRAID will want a clear drive if adding it as an additional array drive. A clear drive is not at all the same as an empty filesystem drive.

 

So I should

1. empty cache drive

2. format it to XFS

3. add to array

4. start moving files from existing array drives to the XFS drive formerly known as cache

Assuming you don't want to do a new config and rebuild parity, you should preclear the drive, then add it to the array, then format it.

 

If you try to add a drive that is not clear to the array, unRAID will clear it, taking the array offline until it's done, then you will have to format it.

 

The only way to maintain valid parity when adding a drive is to add a clear drive. A clear drive is all zeroes, so has no effect on parity. A formatted drive is not all zeroes, because it has a filesystem written to it.

 

So, you should:

 

1. preclear drive

 

2. add to array

 

3. format

 

Wasn't aware of the part in italics. The cache drive has been pre-cleared but of course has data, and has a file system, so it needs to be pre-cleared again from what you said.

 

Thank you, now it's clear!

Share this post


Link to post

i use this command to transfer all my data

 

rsync -av --progress --remove-source-files /mnt/diskX/ /mnt/diskY

 

i didnt get any red balls and all went well

i transferred 28TB in a span of 15days. About 2tb a day.

all my drive are XFS now :)

 

thanks for this post! made my conversion easy!

Share this post


Link to post

I am looking to do the transfer but want to check something.

 

I have a 4tb drive that I just put into the array, it was my parity disk that was replaced with an 8tb drive.  As I have other 8tb drives in the array with about 2.5tb on them and more free space than an empty 4tb drive the 4tb drive has no data on it at all.  Is it just a simple stop array change file system bring back online then format to have the same empty disk change the filesystem it uses while keeping parity intact??  Then I can do the data moves between the disks??

 

 

Share this post


Link to post

I am looking to do the transfer but want to check something.

 

I have a 4tb drive that I just put into the array, it was my parity disk that was replaced with an 8tb drive.  As I have other 8tb drives in the array with about 2.5tb on them and more free space than an empty 4tb drive the 4tb drive has no data on it at all.  Is it just a simple stop array change file system bring back online then format to have the same empty disk change the filesystem it uses while keeping parity intact??  Then I can do the data moves between the disks??

Yes that will work fine.  After you have stopped the array and changed the format then on restarting the disk will be shown as unformatted and give you the option to format it.

Share this post


Link to post

rsync -av --progress --remove-source-files /mnt/diskX/ /mnt/diskY

 

Can this command be used to add files to a disk that already has data on it

Ie diskx has a folder call movies with files and disky has a folder called movies with files will it merge or overwrite??

 

Share this post


Link to post

rsync -av --progress --remove-source-files /mnt/diskX/ /mnt/diskY

 

Can this command be used to add files to a disk that already has data on it

Ie diskx has a folder call movies with files and disky has a folder called movies with files will it merge or overwrite??

 

Both?

 

If the file doesn't exist on diskY it'll copy the file, if the file does exist and when checked has the same time stamp and size, it won't transfer that file as it already exists, if the file exists but the time stamp or size are different it will update the file on diskY to match.

 

 

Share this post


Link to post

I moved one disk to another using an rsync command. Now I want to change the FS for the first disk. What are the steps for that? How do I avoid a rebuild?

Share this post


Link to post

I moved one disk to another using an rsync command. Now I want to change the FS for the first disk. What are the steps for that? How do I avoid a rebuild?

  • Stop the array
  • Click on the drive in the Main tab and change the format to the one you want
  • Start the array.  It will now say that an unformatted disk is present.
  • Select the option to format unformatted disks.  Before doing so check that the serial number displayed is what you expect

Make sure that the disk is REALLY empty (or at least only contains data you are happy to lose) before doing the above.

Share this post


Link to post

did so, but the array wants to rebuild the data. how do I prevent this, or the the array something like 'leave it like that'. Just run a parity check?

Share this post


Link to post

did so, but the array wants to rebuild the data. how do I prevent this, or the the array something like 'leave it like that'. Just run a parity check?

Was the disk disabled, showing a red X next to it? If so, then what you did was copy the data from the emulated drive. unRAID will not let you format a disabled disk.

Share this post


Link to post

formating did go well, but i think i startet the array once without the disk. so now it wants to rebuild the array.

current status for the disk is emulated data.

Share this post


Link to post

Great thread! Sorry for people having problems but they help everyone else tremendously.

 

I am trying to migrate 800GB from a 2TB drive to 2-500GB drives. For XFS formatting purposes.

 

What does the rsync command for that look like?

 

Tried unBalance which I assume does this in the background if you select the correct checkboxes but it hangs up on a couple files.

 

Share this post


Link to post

I am trying to migrate 800GB from a 2TB drive to 2-500GB drives. For XFS formatting purposes.

 

What does the rsync command for that look like?

 

In other words, you want to selectively move 400GB to one drive, and the rest to the other.  I always prefer commander-like tools for selective copying/moving, tools like Total Commander, or the built-in MC, or the Krusader Docker.  Like Total Commander, Krusader should have copying with verification.

Share this post


Link to post

Yeah pretty much that scenario. Once I clear this 2TB and format it, I'll have enough room to relocate my other 2-2TB drives' data.

 

I'll look into those tonight. Thanks!!

Share this post


Link to post

Moved my entire 32TB array to XFS over two weeks. Parity check was error-free, and I switched my smallest data drive to become cache in the process.

 

I tried to speed things up by moving drives from different SATA cards, not sure how big of a difference that makes. Didn't really help anyway, since it was about a drive a day as I didn't sit at the computer waiting for each transfer to finish to fire up the next one.

 

I recommend moving drives around in order, or if not, making a list of the order so you don't have to start figuring out what you moved, what to format, etc. Had to triple check a few ones just to be sure I wasn't about to format a non-duplicated drive :P

 

Has anyone done any tests on speed improvement over RFS? Forgot to test that... My parity check certainly went faster, was blazing fast at only 26 hours @ 42.0 MB/sec when it used to be 30+ hours.

 

Thank you to all the contributors in this thread, was very helpful!

Share this post


Link to post

... My parity check certainly went faster, was blazing fast at only 26 hours @ 42.0 MB/sec when it used to be 30+ hours...

Parity check is not something that would be improved by changing filesystems, since parity operations deal only with the bits and not the files.

Share this post


Link to post

... My parity check certainly went faster, was blazing fast at only 26 hours @ 42.0 MB/sec when it used to be 30+ hours...

Parity check is not something that would be improved by changing filesystems, since parity operations deal only with the bits and not the files.

I'm going to play devil's advocate here, because of things I've seen recently. What if the code that runs the file access is working more efficiently because of the XFS file system change? If there are more disk I/O interruptions for the same tasks using RFS than there are using XFS, I could argue that unless you run the parity check in maintenance mode so no files can be read from the shares, it's possible that for a server that is in use during a parity check that file system change may effect the average speed.

 

Or, it could be some other factor that was changed.

Share this post


Link to post

I'm sure the answer is buried somewhere here in this thread....  but should I bother to change all my drives to XFS?

I just finished replacing a 2TB with a spare (it was showing signs of becoming weak) so that became XFS and I consolidated 2 1.5TB drives to a new 4TB drive which also is xfs now.  I have one more 4TB (spare for now) drive that I can used to convert the rest of my array to xfs...

 

Is there a compelling reason to do so? What do I gain other than getting rid of any fragmented files?

 

Jim

Share this post


Link to post

... Is there a compelling reason to do so? What do I gain other than getting rid of any fragmented files?

 

You'll likely get other opinions, but IMHO there is NO compelling reason to change your current drives to XFS.  I'm certainly NOT doing that with any of my old Reiser-formatted drives.

 

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.