File System Conversion

[Syco54645]

So I am sitting here with an server that was setup when ReiserFS was still the recommended file system and am looking for a simple way to convert these drives to XFS and am coming up empty. It seems that ReiserFS will eventually be fully abandoned in favor of other file systems. The issue is all of the current processes require the somewhat dangerous tasks of creating a new config. Would it be possible to create a feature to aid in this process?

Edited September 19, 2021 by Syco54645

[trurl]

1 hour ago, Syco54645 said:

all of the current processes require the somewhat dangerous tasks of creating a new config

 

New Config isn't particularly dangerous, but it also isn't required (or necessarily even useful) for converting. Not sure where you got that idea.

 

Do you need some advice on converting?

[Syco54645]

1 hour ago, trurl said:

New Config isn't particularly dangerous, but it also isn't required (or necessarily even useful) for converting. Not sure where you got that idea.

 

Wouldn't using new config and placing a drive in the wrong location cause data loss? I do not trust myself to not mess up somewhere along the line with 10 drives to convert.

 

1 hour ago, trurl said:

Do you need some advice on converting?

 

Yes please if you can give advice I would greatly appreciate it (I am on 6.9.0 if it matters). I have posted on the reddit community multiple times and was told that I basically must use unbalance to move all of the files off of the drive then use new config to change the format of the empty drive. This video was given to me as the basic method, just not following the encryption bit.

 

 

[trurl]

34 minutes ago, Syco54645 said:

give advice

Moved this thread to General Support.

 

Attach Diagnostics to your NEXT post

[trurl]

36 minutes ago, Syco54645 said:

Wouldn't using new config and placing a drive in the wrong location cause data loss?

Only if you assigned a data drive to a parity slot.

[Syco54645]

3 minutes ago, trurl said:

Moved this thread to General Support.

 

Attach Diagnostics to your NEXT post

Sure

essex-diagnostics-20210919-1729.zip

[trurl]

Changing filesystem of a disk requires formatting the disk to the new filesystem. So, you have to put its files somewhere else before formatting.

 

All of your disks are very full. The simplest way to make space would be to add a disk to a new slot, or replace a disk with a larger disk.

[Syco54645]

24 minutes ago, trurl said:

Changing filesystem of a disk requires formatting the disk to the new filesystem. So, you have to put its files somewhere else before formatting.

 

All of your disks are very full. The simplest way to make space would be to add a disk to a new slot, or replace a disk with a larger disk.

Yes I am in the process of waiting for an 8tb to add to the array for space and in hopes of starting conversions.

 

What would the process look like for this? I would assume I have to stop anything that writes to the array and then use unbalance to move the files and this is where I get fuzzy. When I move the files are they moved as far as the array is concerned or will I have to rebuild parity?

[trurl]

Didn't notice before but your disk6 is new, though not showing up as having any capacity yet. Looks like it must be clearing

Sep 19 13:17:32 Essex kernel: md: recovery thread: clear ...

 

16 minutes ago, Syco54645 said:

will I have to rebuild parity?

All write operations to the array update parity at the same time so parity remains in sync.

In addition to writing a file, move, copy, delete, and even format, are all write operations that update parity.

[Syco54645]

4 minutes ago, trurl said:

Didn't notice before but your disk6 is new, though not showing up as having any capacity yet. Looks like it must be clearing

Sep 19 13:17:32 Essex kernel: md: recovery thread: clear ...

 

All write operations to the array update parity at the same time so parity remains in sync.

In addition to writing a file, move, copy, delete, and even format, are all write operations that update parity.

 

So how can I empty a drive and not have any new files being added to it at the same time? I guess that is the only stumbling block at this point.

[trurl]

22 minutes ago, Syco54645 said:

not have any new files being added to it

I don't use unBALANCE so I don't know exactly how it plays with disks and user shares.

 

If you don't want any new files written to a disk while writing to user shares you can exclude the disk from all user shares in Global Share Settings. That would also mean no files from that disk would be included when reading user shares.

[Syco54645]

32 minutes ago, trurl said:

I don't use unBALANCE so I don't know exactly how it plays with disks and user shares.

 

If you don't want any new files written to a disk while writing to user shares you can exclude the disk from all user shares in Global Share Settings. That would also mean no files from that disk would be included when reading user shares.

 

What method do you recommend for moving files from one drive to another within the array? I thought doing a "mv /mnt/disk1/* /mnt/disk11/" would not update the array. If I am incorrect then I have been given wrong information and this will be quite an easy process.

 

[trurl]

29 minutes ago, Syco54645 said:

I thought doing a "mv /mnt/disk1/* /mnt/disk11/" would not update the array.

That is incorrect. Writing any disk in the parity array updates parity at the same time.

[Syco54645]

13 minutes ago, trurl said:

That is incorrect. Writing any disk in the parity array updates parity at the same time.

 

In that case the process seems to be 

1. mv /mnt/diskX/* /mnt/diskY/
   1. Once complete make sure all files are moved
2. Stop array
3. Go to Disk Settings and change the File system type to xfs
4. Start array
5. Format the drive that is needed

Would that be a good assessment of what I need to do? Is there a better way to move the files than just mv in a screen session?

[itimpi]

2 hours ago, Syco54645 said:

 

In that case the process seems to be 

1. mv /mnt/diskX/* /mnt/diskY/
   1. Once complete make sure all files are moved
2. Stop array
3. Go to Disk Settings and change the File system type to xfs
4. Start array
5. Format the drive that is needed

Would that be a good assessment of what I need to do? Is there a better way to move the files than just mv in a screen session?

That would work fine..

 

as to whether there is a ‘better’ method that really depends on what tools you are most familiar with.  Using mv in a screen session is fine if you know how to do that.  You could achieve similar results using any file manipulation tool such as rsync, mc (midnight commander), Krusader running as a Docket containet, file manager on another machine etc.

[Syco54645]

9 hours ago, itimpi said:

That would work fine..

 

as to whether there is a ‘better’ method that really depends on what tools you are most familiar with.  Using mv in a screen session is fine if you know how to do that.  You could achieve similar results using any file manipulation tool such as rsync, mc (midnight commander), Krusader running as a Docket containet, file manager on another machine etc.

Thanks! This greatly simplifies things for me.

 

Also thanks trurl for helping as well.

[abhi.ko]

Apologies for dusting up this old thread - but I am trying to change FS on multiple data disks from reiserfs to xfs.

 

So, I believe the above steps should work for me, I just have to do them multiple times for each of the reiserfs disks. Is that accurate?

 

Questions -

1. Would I have to stop all dockers and VM's while doing this?
2. Do I need to copy the contents of the disk back to the original disk after formatting it to xfs. If I don't, is there anything in config that i need to change?

Steps to follow for multiple disks:

1. Shutdown VM and docker
2. From Disk 1 (reiserfs) move contents to empty disk 20 (xfs)
3. Stop Array
4. Change Disk 1 to xfs 
5. Restart array
6. Format Disk 1
7. Repeat Steps 2-6 for next disk, until all are xfs 
8. Restart Docker and VM

One of my VM's is Home Assistant - so this would mean that all the smart home related stuff will go down during this process, is there another way?

Edited April 4 by abhi.ko
added step #6

[Syco54645]

11 minutes ago, abhi.ko said:

Apologies for dusting up this old thread - but I am trying to change FS on multiple data disks from reiserfs to xfs.

 

So, I believe the above steps should work for me, I just have to do them multiple times for each of the reiserfs disks. Is that accurate?

 

Questions -

1. Would I have to stop all dockers and VM's while doing this?
2. Do I need to copy the contents of the disk back to the original disk after formatting it to xfs. If I don't, is there anything in config that i need to change?

Steps to follow for multiple disks:

1. Shutdown VM and docker
2. From Disk 1 (reiserfs) move contents to empty disk 20 (xfs)
3. Stop Array
4. Change Disk 1 to xfs 
5. Restart array
6. Repeat Steps 2-5 for next disk, until all are xfs 
7. Restart Docker and VM

One of my VM's is Home Assistant - so this would mean that all the smart home related stuff will go down during this process, is there another way?

If your containers write to the array you'd have to turn those off. Or if you store your docker image on the array. Stuff that only reads and stores data on the cache drive (like Plex) can be left running. 

 

You don't need to copy the contents back to the now empty drive. 

 

You are missing the formatting of the xfs disk but otherwise your steps seem correct to me. 

[abhi.ko]

Thank you. Edited steps to add the format disk step.

 

My docker image & libvirt image, vm domains (vdisk location) and isos (boot iso's for VM's) are all cache primary and always on the cache disk and not on the array. As far as containers that write to the array yes I do have a few like Nextcloud (data location).

 

I am okay with shutting docker off. Would it be okay to leave the VM running (just HA) while I do this. 

 

Also with not copying the content back to original disk - as per the docs this is a step, assume you did not do this in your original case was there any issues with shares and files etc.

Edited April 4 by abhi.ko

[Syco54645]

There was no issue leaving them "unbalanced".

 

I do not have any VM but I would think home assistant will be ok. I am running my Home Assistant in a docker container but want to switch to home assistant os at some point. 

[abhi.ko]

Thank you - appreciate the quick responses @Syco54645.

 

If anyone else feels that I should not attempt this with the VM running please let me know - otherwise I plan to proceed with this plan. 

[Syco54645]

19 minutes ago, abhi.ko said:

Thank you - appreciate the quick responses @Syco54645.

 

If anyone else feels that I should not attempt this with the VM running please let me know - otherwise I plan to proceed with this plan. 

No problem. I wish you the best of luck. Before I did the conversion I was super anxious but after I started it really wasn't that bad. Just takes forever. 

[abhi.ko]

Finally all done. Left most of the docker apps running except for the ones that wrote to the disk, VM (HA) was never stopped. Used unbalanced to move.

 

Took forever - started with the largest disk and worked my way down. Largest disk with 7TB of data to move took ~42hrs and the smallest with 1.2TB of data took 8 hrs.

 

All disks are xfs now.

[Syco54645]

7 hours ago, abhi.ko said:

Finally all done. Left most of the docker apps running except for the ones that wrote to the disk, VM (HA) was never stopped. Used unbalanced to move.

 

Took forever - started with the largest disk and worked my way down. Largest disk with 7TB of data to move took ~42hrs and the smallest with 1.2TB of data took 8 hrs.

 

All disks are xfs now.

Nice, glad it worked out for you!