Re: Format XFS on replacement drive / Convert from RFS to XFS (discussion only)


Recommended Posts

I do not have a spare monitor and keyboard to connect directly to the machine for console. I attempted to use Screen software application but it did not work after rebooting into Safe Mode as its part of a plugin and I rebooted with the plugins disabled, I did this as its noted in the guide itself.

"That may mean stopping your Dockers, plugins, VM's, and the Mover, if any of those could possibly add files or make changes to the drive being converted. And you should check for and temporarily disable any scheduled backups from other machines. Yes, this can require some careful planning! You may want to consider rebooting into Safe Mode (stops plugins from loading), and disabling any VM's and the Docker service and the Mover."

Link to comment
51 minutes ago, trurl said:

Could you give us a screenshot of this?

 

Sounds like it. Don't do anything without further advice.

Thank you for your time and help sir!  I transferred the contents of disk 4 to disk 10 and a little to another disk using Unbalance.  From what I can tell there are no files remaining on disk 4 but the share directories are still there.  Thank you very much for your guidance and I apologize for not being up on this process.  

Array.JPG

Disk 4 dir.JPG

Pictures.JPG

Link to comment
5 hours ago, allischalmersman said:

Do I un-assign the disk and let parity rebuild with the missing disk?  Then format the 4TB disk?  I would like to pre-clear stress test the disk before adding it back to the array.  I'm sorry but I'm confused.

I have crossed out everything you absolutely must not do. If the disk is empty you just need to change its filesystem to XFS by making Unraid reformat it.

 

Since you are so confused though, before I go into detail about how exactly you make Unraid reformat that disk to XFS, go to Tools - Diagnostics and attach the complete diagnostics zip to your next post so I can take a look and make sure you aren't overlooking something important.

Link to comment
9 minutes ago, trurl said:

I have crossed out everything you absolutely must not do. If the disk is empty you just need to change its filesystem to XFS by making Unraid reformat it.

 

Since you are so confused though, before I go into detail about how exactly you make Unraid reformat that disk to XFS, go to Tools - Diagnostics and attach the complete diagnostics zip to your next post so I can take a look and make sure you aren't overlooking something important.

Haha Glad I stopped and asked. Are you also saying I can't or shouldn't stress test that drive while it's out?  Only reason I ask is because those 3TB drives that still need converted are ones that do not have a great reputation and I had 1 fail about 2 years ago.  Thanks again for your help.

tower-diagnostics-20181126-1517.zip

Link to comment
2 minutes ago, trurl said:

SMART for Disk4 looks fine. Nothing to be gained by stress testing it, and you will greatly complicate your life if you try to.

Ok I will heed your advise.  Sorry bad choice of wording. No the disk has not been removed.  I guess I mistakenly thought at some point throughout the process I would have that disk no longer a part of the parity protected array.  

Link to comment
1 minute ago, allischalmersman said:

thought at some point throughout the process I would have that disk no longer a part of the parity protected array.  

No, in fact it is critical that the disk remain in the array while it is reformatted or you will invalidate parity. You likely have some misunderstandings about formatting and how parity works.

 

Do you know exactly what "format" means?

 

Link to comment



 
Do you know exactly what "format" means?
 


I thought I did but I ASSumed it could only be done on a disk that was not a member of the array. I thought formatting was when a file system format was written to the disc and will invalidate anything previously written to the disk. I've only ever done it when adding a new disk. Perhaps I'm way off base here.

Sent from my SM-G965U using Tapatalk

Link to comment
5 minutes ago, allischalmersman said:

I thought I did but I ASSumed it could only be done on a disk that was not a member of the array. I thought formatting was when a file system format was written to the disc and will invalidate anything previously written to the disk. I've only ever done it when adding a new disk. Perhaps I'm way off base here.

Actually that IS what format means. Many people have a pretty vague idea about formatting, thinking it is just something you do before you can use a disk.

 

But the key part of that is "a file system format was written to the disc". Format is a write operation. Unraid updates parity in realtime on ANY write operation. So if you format the disk when it isn't in the array obviously parity will be invalid.

 

And, in fact, Unraid won't let you do that without going to an awful lot of trouble, which is why I said trying to stress that disk would greatly complicate your life

Link to comment
  • 6 months later...

I'm in the process of a massive upgrade to my Atlas clone. I've already upgraded from 5.0.5 to 6.7.0 and replaced a couple failing drives in the nick of time (I've been really lucky). Now I'm in the process of converting all my disks from rfs to xfs. I've done 2 of 10 so far using the mirroring method described here.

 

As with the original Atlas, I have UnRAID running as a guest on an esxi 5.1.0 host along with an ubuntu guest that runs plex. This ubuntu guest along with a local printer (for scans) and an IP cam are the only users that have access to the UnRAID shares. 

 

When I did the first 2 disks I shut down the ubuntu guest, turned off the printer and unplugged the cam to make sure nothing was writing to the array during the rsync operation. Obviously this means I can't use plex during this process. My question is, rather than only adding the swap disk to the excluded disk(s) list under 'global share settings' as instructed to do in the wiki, could I also add the source disk and be assured that nothing is written to the source disk during the rsync operation?

 

I have no VMs or dockers running on UnRAID (yet).

Link to comment

I've been proceeding with the conversion and decided to just add all the remaining rfs drives to the excluded list. This seems to be working well and I don't need to worry about the disks being written to during the rsync copy to the swap disk.

 

One other thing I am curious about is Step 17. Is there a good and QUICK way to check that everything is fine? Should I just pick a few random files and compare checksums?

Link to comment

I think what step 17 is referring to double check/verify that you have requested the format changes to the proper disks on the setup screen BEFORE you click 'Apply'.  Because if you didn't, you will lose all of the data on an 'innocent' drive as soon as it starts the format procedure.  As I recall, this is a point at which it is actually quite easy to make a mistake.  That was why I had a table prepared with what the settings were to be on each step and I checked-off each step as I did it! 

  • Like 1
Link to comment
4 hours ago, Frank1940 said:

I think what step 17 is referring to double check/verify that you have requested the format changes to the proper disks on the setup screen BEFORE you click 'Apply'.  Because if you didn't, you will lose all of the data on an 'innocent' drive as soon as it starts the format procedure.  As I recall, this is a point at which it is actually quite easy to make a mistake.  That was why I had a table prepared with what the settings were to be on each step and I checked-off each step as I did it! 

Thanks for clarifying.

 

So I'm totally panicked right now. Just opened Plex and noticed a ton of my files are missing. Is this because I switched a bunch of the disks to excluded? I assumed excluded just prevented new files from being written to those disks. Does it also not allow them to be read? Will they show up again when I 'un-exclude' them?

Link to comment
59 minutes ago, jonathanm said:

If you mean global exclude list, yes. Those disks are globally excluded from participating in user shares.

So when I uncheck them from being excluded will all the files on those disks show up in the share again?

Edited by lazant
Link to comment

Open up the shares in Windows using File Explorer and see if the 'missing' files are there.  It is my understanding that excluding a share from a disk only prevents writing new files to that disk.  Existing files can still be read.  What could be happening is that Plex is attempting to access those files in a read-write mode and that might not be allowed.

 

EDIT: you might want to read through this thread:

 

 

Edited by Frank1940
  • Like 1
Link to comment
44 minutes ago, Frank1940 said:

Open up the shares in Windows using File Explorer and see if the 'missing' files are there.  It is my understanding that excluding a share from a disk only prevents writing new files to that disk.  Existing files can still be read.  What could be happening is that Plex is attempting to access those files in a read-write mode and that might not be allowed.

I looked in Finder on my mac and there are a bunch of folders missing. However, when I tried browsing the the ‘Disk Shares’ from the excluded disks using UnRAID’s browser interface I can see the missing files. Phew!

 

Anything special I need to do? Or just reenable the disks when I’m done with all the fs conversion?

Link to comment
13 hours ago, Frank1940 said:

EDIT: you might want to read through this thread:

 

 

Cheers. That cleared it up. It seems then that as I un-exclude the disks after conversion to xfs, the files will become available again in the share. I'm about halfway through a 3TB copy right now so I'll confirm tomorrow afternoon.

 

Edit:

Confirmed. Un-excluding the next disk converted to xfs did make the files on the disk available again.

Edited by lazant
Link to comment
  • 3 months later...
On 8/1/2018 at 4:08 PM, sketchy said:

I've just started the 'mirror' procedure on my first disk to convert from btrfs to XFS. I don't know any better than bjp999 / SSD, but i included the '--info=progress2' in my rsync command to give the overall rsync time remaining as opposed to the time remaining on the current file being rsync'd...i think. Seems helpful.

I'm in the middle of copying contents and saw this post. Where exactly do I add the '--info=progress2' to the 'rsync -avPX /mnt/disk10/ /mnt/disk11/' command?

I'd rather ask than guess.

Thanks in advance :)

Link to comment
10 minutes ago, zOnk.oNe said:

I'm in the middle of copying contents and saw this post. Where exactly do I add the '--info=progress2' to the 'rsync -avPX /mnt/disk10/ /mnt/disk11/' command?

I'd rather ask than guess.

Thanks in advance :)

A bit of googling (    rsync --info=progress2    )  found these two links which should give what you need:

         https://www.cyberciti.biz/faq/show-progress-during-file-transfer/

and 

         https://unix.stackexchange.com/questions/215271/understanding-the-output-of-info-progress2-from-rsync

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.