Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)


Recommended Posts

4 minutes ago, Frank1940 said:

require more housekeeping overhead

 

Thanks, yes, it felt that way to me too, I do have many small files :o

 

One more question, I formatted the drive to xfs and it shows it's using 4Gb. I checked on the command line and it's empty.

 

Is that filesystem overhead ??

 

root@wopr:/mnt/disk13# du -bs /mnt/disk13
6	/mnt/disk13

 

Screen Shot 2017-06-14 at 08.57.11.png

Link to comment
20 hours ago, jbrodriguez said:

So, I converted my first reiserfs to xfs.

 

A full 4tb, took approx 26 hours.

 

bjp999, I used turbo write and write speeds went up about 10Mb/s (roughly 44Mb/s from 34Mb/s).

 

Is that an ok gain from turbo write ?

 

If you look back in the thread several posts, you'll see @Frank1940 ran some comparison benchmarks on this system. His speeds were somewhat faster than these. But there are a lot of variables that contribute to turbo write speed. You are seeing nearly 30% improvement, which is quite significant.

 

 

Link to comment
  • 1 month later...

Could you convert a disk like this:

Turn off array, go to a disk in array, change filesystem to XFS, format disk, start array, data will then be re-built on that disk? It's that last step I'm foggy on, will it be re-built (similar to if a new drive was installed due to a failure) or will it be lost? Also, I know the array would be vulnerable during the re-build, but I'm okay with that.

 

Nate

Link to comment
30 minutes ago, nate1749 said:

Could you convert a disk like this:

Turn off array, go to a disk in array, change filesystem to XFS, format disk, start array, data will then be re-built on that disk? It's that last step I'm foggy on, will it be re-built (similar to if a new drive was installed due to a failure) or will it be lost? Also, I know the array would be vulnerable during the re-build, but I'm okay with that.

 

NO!!!!  Because the rebuild of a replacement is a bit-by-bit restoration of the contents of the original disk-- Not a file-by-file copy.  (Even though , you formatted the disk as XFS, the bit-by-bit restoration will result in it being a reiserfs disk when done.)

  • Upvote 1
Link to comment

Ironically, now I'm hoping a rebuild will be possible as I just realized that I missed moving a directory after just now formatting it to XFS. I started the array and it's empty, so doing a parity check now, but if formatting it wrote those changes to the parity drive (which I'm guessing it did) then I guess that data is gone forever. Oh well ::shrug:: I needed a good reason to audit all the data on my server anyways.

Link to comment
8 minutes ago, nate1749 said:

Ironically, now I'm hoping a rebuild will be possible as I just realized that I missed moving a directory after just now formatting it to XFS. I started the array and it's empty, so doing a parity check now, but if formatting it wrote those changes to the parity drive (which I'm guessing it did) then I guess that data is gone forever. Oh well ::shrug:: I needed a good reason to audit all the data on my server anyways.

Were you using one of the procedures in this WIKI?

 

  https://wiki.lime-technology.com/File_System_Conversion

 

If so, which one were you using? 

 

I used the "Mirror each disk with rsync, preserving parity" method which copies the entire contents of each disk as it proceeds.  (I also printed out this method on paper and setup a table to show each disk as it was completed.)  While one disk was being copied, I was carefully laying out the steps for the next disk.  I actually marked rsync command (in pencil) with the proper disk number and using the arrow keys to get the rsync command back so that I could edited it.  Using these built-in editing features of the BASH shell (which unRAID uses), this only required changing the disk number of the next disk to be changed in the rsync command line.  I, personally, found that steps 13 and 14 can be confusing and that is where the table really came in handy.  I identified the physical drive in these instructions by the last four numbers of the drives' serial numbers so that I could double check that I had everything assigned correctly before I went to step 15!

Link to comment

Hi, I just got a new drive so thought it would be a good time to make the move to XFS.

 

I followed the wiki and took my next largest drive out of the global share settings and set of a rsync using nohup.

 

It'd been killed after copying about 500GB of the 3TB drive;

 

rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(636) [sender=3.1.2]
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at io.c(504) [generator=3.1.2]

 

Any ideas what could have caused this to stop?  Is using nohup likely to cause issues with all the output being written out?

Link to comment
On 8/4/2017 at 1:46 PM, Frank1940 said:

Were you using one of the procedures in this WIKI?

 

  https://wiki.lime-technology.com/File_System_Conversion

 

If so, which one were you using? 

 

I used the "Mirror each disk with rsync, preserving parity" method which copies the entire contents of each disk as it proceeds.  (I also printed out this method on paper and setup a table to show each disk as it was completed.)  While one disk was being copied, I was carefully laying out the steps for the next disk.  I actually marked rsync command (in pencil) with the proper disk number and using the arrow keys to get the rsync command back so that I could edited it.  Using these built-in editing features of the BASH shell (which unRAID uses), this only required changing the disk number of the next disk to be changed in the rsync command line.  I, personally, found that steps 13 and 14 can be confusing and that is where the table really came in handy.  I identified the physical drive in these instructions by the last four numbers of the drives' serial numbers so that I could double check that I had everything assigned correctly before I went to step 15!

I wasn't using a procedure, I was just being sloppy and doing it my own myself, and I would have been fine except forgetting that formatting a disk will write that change to parity so I couldn't back out of it. I don't use rsync, I just move files with midnight commander because it's not worth it to me to be that careful with my data as the stuff I have just isn't valuable and I can always retrieve it again and after using unraid for 9 years I've only lost data once (and that was just now), so ::shrug::. I could have tried getting it back with reiserfsck, but I honestly just don't care. My server has terabytes of nonsense on it and I just don't need to hoard so much so this is forcing me to audit and cleanup.

 

You should note though that if your disks aren't 4k aligned, the procedures that I saw don't address changing that if you want them to be. Formatting them to XFS doesn't change the alignment in unraid (some confusion in posts about this from people assuming it does), so you'll need to either preclear the disk or blow out the partition table w/gparted (dd'ing would probably do it as well) if you want them aligned, but on the other hand it's a pedantic measure and not really needed if everything is working fine.

Link to comment
38 minutes ago, nate1749 said:

upthetoon: did you preclear the new disk first? Wondering if you got a bad one and it just died. Try a full SMART test and see results.

 

Yeah, did a preclear on the disk which went fine.

 

Something is up now though, I can't get smartdisk info, just says  "Please wait... retrieving S.M.A.R.T. information!" .

 

The disk is still mounted and I can copy files from it.

 

The identity tab looks ok;

SMART support: Available - device has SMART capability.
SMART support: Enabled
SMART overall-health: Passed

 

Other disks are fine and a reboot hasn't fixed it.

Edited by upthetoon
Link to comment
1 hour ago, upthetoon said:

 

Yeah, did a preclear on the disk which went fine.

 

Something is up now though, I can't get smartdisk info, just says  "Please wait... retrieving S.M.A.R.T. information!" .

 

The disk is still mounted and I can copy files from it.

 

The identity tab looks ok;

SMART support: Available - device has SMART capability.
SMART support: Enabled
SMART overall-health: Passed

 

Other disks are fine and a reboot hasn't fixed it.

Attach a diagnostics file with your next post.  Tools   >>>  Diagnostics   

 

Can you spin the disk up and get the temperature reading of it?

 

When you are reading files off of it, are all of the other disks spun up?    (Spin all the disks down before you do this check)

 

Link to comment
2 minutes ago, Frank1940 said:

Attach a diagnostics file with your next post.  Tools   >>>  Diagnostics   

 

Can you spin the disk up and get the temperature reading of it?

 

When you are reading files off of it, are all of the other disks spun up?    (Spin all the disks down before you do this check)

 

 

That diagnostic tool is great.  Thanks for offering to take a look.  The new disk is WDC_WD40EZRZ-00WN9B0_WD-WCC4E7RXXNR1 - 4 TB (sdg).

 

It's 32C.  No, some of the other disks were spun down.

 

I had a look at the troubleshooting section before and have tried setting off a long smart test from the terminal.  Not 100% sure it is running.

ridcully-diagnostics-20170806-1257.zip

Link to comment
2 hours ago, nate1749 said:

I would have been fine except forgetting that formatting a disk will write that change to parity so I couldn't back out of it.

This is the most important thing, the very reason this 29 page thread exists, and the reason people took the trouble to write up the procedure.

Link to comment
9 minutes ago, upthetoon said:

That diagnostic tool is great.  Thanks for offering to take a look.  The new disk is WDC_WD40EZRZ-00WN9B0_WD-WCC4E7RXXNR1 - 4 TB (sdg).

 

The SMART report on that Disk looks fine.  A very quick look at the syslog--- didn't see anything unusual.  No time today to do more looking.

Link to comment
6 hours ago, trurl said:

This is the most important thing, the very reason this 29 page thread exists, and the reason people took the trouble to write up the procedure.

Completely agree, but I'm not going thru a 29 page thread and I skim the procedures and am grateful they exist, but my data isn't that important and the procedures generally treat data like it's more important than mine, so knowing that mine isn't, I intentionally skip some steps (e.g. I move rather than copy data). I also prefer learning and while following a guided step tour is great for completing the task, it doesn't help me (personally) with learning. Using a tiny bit of guidance and then tripping and stumbling is the best way that I learn, so for my hobbies like (UnRaid), that's what I do. Not saying it's the best method, but that's just the way I'm wired. I do appreciate all the efforts everyone puts into this thing, and I completely get RTFM, but I also am aware of the risks I'm taking by cowboy'ing it a little bit and not following all the prescribed steps.

Link to comment
1 minute ago, jonathanm said:

Moving is significantly slower, as it requires writes to the source drive and parity drive, while a copy only reads from the source and writes to the destination and parity.

Yeah it is! The last time I did a major move years ago I think I was also lazy about copying so I just unassigned the parity drive until I had everything where I wanted it then re-built it. This time though, I'm opting to maintain parity during the process. Certainly not the best way, but I know myself and that I will get busy with other things and not look at the server for a few days/week and then I'll forget what step I was on or what I was even doing, so that's why I personal prefer moving in my specific scenario.

Link to comment

You don't really have to follow any procedure written by someone else. You can definitely wing it, as many of us did before there was a wiki about this.

 

You just have to understand that unRAID parity is realtime, so that anytime a data disk is written, parity is updated. The thing that maybe trips up some people is that anything that isn't a read is actually a write. Writing files, deleting files, moving files, and formatting a disk are all writes.

Link to comment

If you want to manually rearrange data as you move it off of a drive to other drives, I'd recommend moving. Then you know what you have already copied off and what is remaining. If you are just copying all off one disk onto another, copying is faster.

 

Among its other "features", RFS deletes are particularly slow for big files. So avoiding the delete is worthwhile!

Link to comment
  • 1 month later...
1 hour ago, bombz said:

Hello,

I would LOVE to change FS as I am using  ReiserFS Still

 

What would be my best plan of attack for this array? I feel overwhelmed thinking about it

 

Thanks

Capture-raid-sync.PNG

Your screenshot indicates it is rebuilding a data disk. Let it finish obviously. Without any idea of how much free space each of your drives has, I would say add a new empty 6TB disk formatted as XFS, then copy your largest drive to it, reformat the drive to XFS, rinse, repeat.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.