Reiserfs to XFS conversion (Solved)

[marlin]

Preparing to do Reiserfs to XFS conversion on some drives. Running 6.9.2 PRO version.

4TB parity, all assigned data drives 2TB – 5 reiserfs, 3 xfs (2 of these are empty), 2 – 3TB drives precleared/unassigned.

 

Question using the 'Mirroring procedure' https://wiki.unraid.net/index.php/File_System_Conversion ,

If using an empty 2TB disk as the swap disc, which is already assigned and formatted xfs, can/should I still do step 4 settings/global shares settings/excluded disks" with the array stopped?

 

An additional confusion point is the "NOTE" above the procedure which says "Note: the swap drive should not be formatted with XFS (the resultant file system); if it is, then you will need to clear it or format it to something else first"

Any clarification will be appreciated.

[Frank1940]

3 hours ago, marlin said:

"Note: the swap drive should not be formatted with XFS (the resultant file system); if it is, then you will need to clear it or format it to something else first"

 

I am going to ping @SSD as I think he prepared this portion of the Documentation for this procedure.  If he has not responded in a couple of days, post back up (I will be following the activity on this thread) and I will tell you how I would proceed with your starting point.  (It will take a bit longer to do it the way that I would propose...)

[marlin]

Thank you for looking into this. Look forward to hearing back. No loss of data more important than speed. Data is replaceable but who wants to go that route if avoidable. Thanks again. 

[Frank1940]

25 minutes ago, marlin said:

No loss of data more important than speed.

I just checked and @SSD has not be on the forum since April.   Give your above condition, I would suggest that you run a parity check to verify that parity is still correct before proceeding further.  Once completed without an error...

 

18 hours ago, marlin said:

2 – 3TB drives precleared/unassigned.

I have re-read your first post and do you have two 3TB drives that are not assigned to the array at this time?   If so, are they cold spares or hot spares?

 

18 hours ago, marlin said:

all assigned data drives 2TB – 5 reiserfs, 3 xfs (2 of these are empty)

You have --eight-- 2TB data drives currently in the array-- correct?  Do you want to keep all these drives in the array and convert the --five-- reiserfs drives to XFS?  

 

Is it acceptable/possible to have --Nine-- data drives in the array when you finish the conversion? 

[trurl]

attach diagnostics to your NEXT post in this thread

[marlin]

Will run Parity check before proceeding with conversion. Unit is not left powered on for much time so I run Parity check somewhat infrequently. Date count somewhat deceiving. Electricity here comes and goes so even though I have a UPS no reason to tempt fate. 

 

2 - 3TB drives are unassigned and are basically ready to be assigned as needed. Not sure on the definition of 'cold/hot spare' is but set up these 2 drives in case an assigned drive was to fail, so I'd have something ready to go. There are also 2 - 2TB drives assigned to unRaid which I know are empty since I fill 1 drive at a time to 90%. I like to know what media is on what drive.

 

Yes 8 data drives in array. I think it will be simpler to convert --five-- reiserfs drives to XFS and leave them in the array. Please correct me if I'm wrong. Would still like to have at least 1 drive ready as replacement in case an assigned drive goes red ball at some point. 

PRO license, so disk count not an issue.  I realize just getting 2 big drives (1 for parity) would be simpler but even though these drives are old not many hours on them. 

 

Ran diagnostics last time I had unit powered up, see attached. I seem to always be running to catch up and since newer O/S will stop supporting reiserfs trying to get ahead. Just updated to x64 system after years of box being dormant. 

Thank you for any assistance.

tower-diagnostics-20220714-0834.zip

[trurl]

4 minutes ago, marlin said:

the definition of 'cold/hot spare'

"cold" spare is not connected, sounds like you are saying yours are "hot".

[trurl]

Diagnostics look good (but old as you explained). Didn't really examine all SMART reports. Do any disks have SMART warnings on the Dashboard page?

 

And my other standard lectures:

 

Do you have Notifications setup to alert you immediately be email or other agent as soon as a problem is detected.

 

Do you have another copy of anything important and irreplaceable? Parity is not a substitute for backups. Probably you have heard that one before since you have been on the forum longer than me.

[Frank1940]

2 hours ago, marlin said:

Electricity here comes and goes so even though I have a UPS no reason to tempt fate. 

 

Clarify with more details.  Outages hourly, daily, weekly, monthly...   This conversion process is going to take a while.  Like 2-3 hours per TB of data as I recall.  

[marlin]

@trurl Thanks for reviewing my diagnostics. No SMART warnings on dashboard page, thankfully.  I appreciate your standard warnings but I have never run UnRaid when I'm away from it and I have never had full time internet connection up until very recently when I finally got STARLINK. So in light of now having real internet and not barbwire telephone dial up I will implement your reminder and setup alerting. No such thing as too many lerts. 😁 Nothing irreplaceable. Just lots of media that would take a lot of time to reload. I've been on the forum a long time but still a newbie since I don't pull on all the wonderful bells and whistles available. the 3TB drives are powered up so I guess = hot.

 

@Frank1940 Rural electric with very random outages thanks to PG&E hazard reduction outages, fires, idiots running into power poles, drought stress and beetle bored trees falling onto power lines as they feel required or winter storms blowing things down, etc. Country living is great. Nice to have my own 12TB of movies to watch by the hum of a generator. 🤣 UPS gives me about 20 minutes run time to get generator fired up if I'm in the middle of something important on the computer or server. The various UPS's I have start screaming as soon as they go on battery to let me know.  Plan is to convert 2TB at a time while babysitting the process. So a few days of conversion and I should be catching up to the 2020's. 

 

Thanks both for putting some time into this to help me along. 

Edited August 29, 2022 by marlin
add hot drive info

[trurl]

2 minutes ago, marlin said:

UPS gives me about 20 minutes run time

Normally I recommend not running on UPS any longer than it takes to shutdown cleanly. In your case, I'll modify that to don't start your system back up until UPS has recharged completely.

[JonathanM]

1 hour ago, marlin said:

Nice to have my own 12TB of movies to watch by the hum of a generator. 🤣 UPS gives me about 20 minutes run time to get generator fired up

Lower quality UPS's can have issues running on some generators, but with high quality equipment, both UPS and gensets, it can be ok. Have you run your IT stack for any significant time on genny? I'm asking just to make sure you aren't making assumptions that because the genny salesman said it would be fine but you haven't actually tried it, if that's the case I'd do dry runs with all your actual computers and servers unplugged and dummy loads on the UPS's to see how they behave. I've seen some nasty bangs as a result of unhappy electrics.

[marlin]

I know about the battery drain and recharge time issue, thanks. UPS front panel has graph/readout of % charge. New battery set last year.

 

My UPS is a APC Back-UPS RS 1500 which has line conditioning capability. During grid power outage I try to keep my computer use at a minimum but have used my desktop successfully.  Do not recall running Unraid during grid down time but if I did it was the old x32 hardware which is gone now. I do have the server connected to the UPS for clean shutdown after 2-3 minutes or some % of battery left if needed.  Also just upgrade my generator to a nice quiet Honda 7500 which has full sine wave output so the power in should be as good or better than the utilities provide.

 

Any further input on the originally posted questions about the Reiserfs to XPS conversion process?

[JonathanM]

1 hour ago, marlin said:

 

Any further input on the originally posted questions about the Reiserfs to XPS conversion process?

Do you care which drive number slots have specific shares on them? If not, you don't have to do nearly as much work.

 

In a nutshell, copy one of the ReiserFS drives to one of your empty XFS drives. Compare the content, as thoroughly as you wish, cursory examination or binary file by file, whichever fits your risk profile. Format the source of the copy to XFS. Repeat until all ReiserFS drives have been formatted to XFS.

 

Parity stays valid the entire time, no need to mess with new config and possibly rebuilding parity.

[Frank1940]

1 hour ago, JonathanM said:

 

In a nutshell, copy one of the ReiserFS drives to one of your empty XFS drives. Compare the content, as thoroughly as you wish, cursory examination or binary file by file, whichever fits your risk profile. Format the source of the copy to XFS. Repeat until all ReiserFS drives have been formatted to XFS.

 

 

The only problem that I see is that if both drives are a part of the Global Share Setting (Step 4 of the instructions), then there will be two copies of the files (Source disk and target disk) copied on the array until the source disk is reformatted.  This should not cause any problem unless someone is doing any type of file operation (except for strict read operation) on those files.   I think that is why the Author had the conversion disk outside of the Shares.   (To be honest, as I recall, back in about 2018 when I did the conversion, I had the server totally offline at the time so that nothing could possibility access a file via SMB or NFS share.  If this is done, there should not be a problem...)   

 

One more thing.  While it may seem obvious,  the ordering of the copy operation should start with the largest disk and work toward the smallest disk.

 

PS----  I almost forgot, Unraid is now (apparently) defaiulting to a version of the XFS formatting that uses a lot more disk space for the storage of the file system than reiserfs.  This could present a problem if the source disk is virtually full and the target disk has the same capacity as the source disk.  (However, this XFS version provides for better recovery of data in cases of disk corruption.)

Edited August 30, 2022 by Frank1940

[marlin]

@JonathanM I don't really care which drive slot has which share on it. As I understand that is the whole strength of shares. What media is on which drive is more of a concern but that can be sorted out.  Your suggestion of process is understandable, thanks.

 

@Frank1940 Your point of the 'global shares setting' was actually part of my original post. So to re ask the question,  can I set the target drive which is already formatted and assigned to 'excluded disk' and then proceed? I assume this is done with array not started? And this would avoid the problem of the same file being in 2 places at the same time. The 'check for error' plug-in will show this duplicate of files I've found.  Of course doing the 'global shares setting' will require me to go through the 'new configuration' steps which is a bit confusing also. I like the idea of being offline while doing the process, will pull the ethernet cable. Basically all drives will be 2TB size but appreciate the suggestion of largest to smallest progress.  Per PS I always left 10% of the disk empty so this should not be an issue, but thanks for the thought and it may be important to someone else who stumbles across this thread in the future. 

 

I'm going to assume the NOTE question is something residual that nobody remembers why it is included in the process. Please let me know if this is incorrect. 

Thanks as always

[Frank1940]

57 minutes ago, marlin said:

So to re ask the question,  can I set the target drive which is already formatted and assigned to 'excluded disk' and then proceed?

 

A qualified 'Yes'---  I can't recall reading that anyone else had actually doe it that way.   You have to make sure that the disk is actually empty.  Then give it a try and see what the OS tells you when you restart the array.  That was why I wanted @SSD to have a chance to reply...

 

1 hour ago, marlin said:

Of course doing the 'global shares setting' will require me to go through the 'new configuration' steps which is a bit confusing also.

 

I made a table of each step in the instruction and before I started each cycle for a pair of disks, I put in the disk numbers so that I didn't make a mistake.  I also marked off each step as I went along so I always know where I was.  And it is confusing as one reads it but it is not so bad once you work through it with your exact configuration of drives to do your first two drives.

 

1 hour ago, marlin said:

Per PS I always left 10% of the disk empty so this should not be an issue,

 

Smart thinking a head of time.   That should be more than enough room for the extra space for the XFS file system.  With all the disks being 2TB (My situation was virtually identical) in size, that makes this one a none issue. 

 

 

On 8/27/2022 at 6:29 PM, marlin said:

An additional confusion point is the "NOTE" above the procedure which says "Note: the swap drive should not be formatted with XFS (the resultant file system); if it is, then you will need to clear it or format it to something else first"

I am at a total loss here.  The only thing I can seem to recall hearing about is that Unraid does not like drives formatted with XFS on another Linux distribution.  Again, this a somewhat foggy recollection...  You could change it to reiserfs (or btrfs) and change it back again--- this would take less then ten minutes--- just to be safe.  

 

One more check I would make after the parity check is complete is that counts for SMART attributes 197 and 198 are zero for all drives.  (I, personally, don't like potential problem disks in my array!)  

[Frank1940]

One more thing when you are using the command line (for the rsync command), there is a built-in editor in the BASH shell which is allows you to walk up and down the list of previously entered command lines and edit them.  You use the arrow keys (up-down-left-right) for navigation.   The up and down arrow keys will move up and down the list of previous commands.  The left and right arrow keys will position the cursor in the command line for editing.  From this brief description, you should be able to figure out what the other keys will do.

[marlin]

Thanks Frank confidence level improved. Thanks also for the editor information, helpful news for me.
Going to wait for the present west coast heatwave to pass since grid power is questionable apparently. Power problems in both summer and winter, good grief. 

Will report back when complete or if gremlins show up. Thanks again 

[marlin]

Just some notes for anyone traveling down this path. The first 2 of 5 disk conversions are complete.

 

Recommended to take a screen shot/print out of MAIN page to have a record of disk serial numbers and position in the array for reference.

The transfers take almost forever. The write speed runs between ~10-23MB/s. This is probably a function of keeping parity updated, disk speed of 5400 rpm (old WD green), and location of head on disk. This worked out to be about 24 hrs/TB, which is painfully stressful for the first disk. And about 30 hours for the second disk of 1.8TB on a 2TB drive. But looking on the forum not unusual speed for having parity disk still engaged.

 

Rebooting before copying into SAFE MODE eliminates the worry and warnings about disks missing data due to data being placed on disks after the copy function, “background writing”. Highly recommended.

 

The ‘NOTE’ in the procedure under “system assumptions” page 5/7, referencing not starting with a disk already with XFS file system seems to be irrelevant since the first disk I copied to (swap disk) was previously formatted to XFS. Not sure what’s up with this?

 

STEP 4; Stopping the array and going to Settings/Global Shares Setting/ADD/Excluded Disk # ‘swap disk’ (whatever that is for you) seems to work fine even if the disk had already been assigned to ARRAY previously.

 

The newer XFS file system looks to take up more space on the disk than the REISERFS did. My reserve space dropped from 204GB 10% (of 2TB) to 191GB. Really glad I didn’t fill the disk up during the original loading of media.

 

My generator is on standby since a storm is due in a month earlier than most years, UPS at 100%. Weather is obviously broken.

Will update with further information as it arises. Hope this is helpful for anyone else slow to keep up with all the latest changes.

[marlin]

Update.

During 4th drive of 5 conversions the inevitable happened and the power went out in the middle of the night. Perfect weather but a branch someplace supposedly fell on wires, 890 households out.  Once I heard alarm the Unraid had already shut down the UPS (as it is suppose to) but apparently there was still a 'unclean shutdown'.

Can the server NOT do a clean shutdown during rsync read/write? Not sure what caused the unclean shutdown. Anyway ran a Parity check (with correction) and finished with Zero sync errors.

Thought at this point is to reformat my 'swap' disk and restart the conversion of disk 4 from the beginning since it was someplace around 414GB into the 1.8TB conversion.

To do this can I just change the format on swap disk to RFS then back to XFS, 2 format steps, and then restart the conversion? Don't want to have to rerun the 11.5 hour parity again if possible. Any suggestions welcome. Thanks.

tower-diagnostics-20220925-2238.zip

[JonathanM]

5 minutes ago, marlin said:

start the conversion of disk 4 from the beginning since it was someplace around 414GB into the 1.8TB conversion.

If you are using my preferred command line for rsync (one line for the initial copy, a second run through to verify) then you could to a file system check on the destination drive to be sure it's clean, then issue the same copy command again and it will pick up where it left off, then the rsync verify command will catch any inconsistencies.

 

You didn't say what exact commands you were using, so it's tough to advise on best options.

[marlin]

Thanks for the reply.

Using: rsync -avPX /mnt/disk'#'/ /mnt/disk'swap'  from the "mirror procedure" in the sticky.  It is 1 run through so I think it is different than your process. 

Do you think my idea for reformatting the swap disk will work?

[JonathanM]

26 minutes ago, marlin said:

Do you think my idea for reformatting the swap disk will work?

Probably, but why throw away the progress you made? The command line you posted is specifically tailored to resuming a transfer. Just run the same command and it should pick up where it left off.

 

After that rsync finishes, you can check the transfer by issuing rsync with the same source and destination, but substitute -narcv for the -avPX part. It will compare the source and destination, and list any differences. If it returns to the command line without listing any files, it means the source and destination are identical.

[marlin]

Ah, live and learn as usual. Thanks for the help. Will gladly follow your suggestions. Will return with results. 

Thanks again!