[SOLVED] Wrong Parity Config After Back

[thedudezor]

I recently had a issue where my Unraid USB lost my configuration. I suspect that the USB drive is going bad and needs to be swapped out. To get myself back up and running (unraid acts as my main workstation VM), I copied the config from a backup I created about a month ago. This got the server back "online", however I have since changed the disk configuration by adding a new 4TB Parity drive and reusing one of the older smaller drives as just a data disk.

 

The problem is that I can not tell Unraid to use the correct Parity drive (error is wrong disk). The other issue is that the Array has now wrote minimal changes to this data disk it now thinks is the parity drive due to the old config being used. I assume that IF I can get Unraid to accept what should be the correct Parity drive and Data Disk#3, a rebuild of the Array should correct any data that has been written to that drive. (attached is a image to help visualize the problem).

 

Any help here would be greatly appreciated, I'm still new to Unraid and I've now learned to make a backup before and after making any config changes.. Homerdoh...

 

 

[JorgeB]

5 minutes ago, thedudezor said:

I assume that IF I can get Unraid to accept what should be the correct Parity drive and Data Disk#3, a rebuild of the Array should correct any data that has been written to that drive.

Not necessarily since there were also writes to other data disks that made old parity out of sync, still a filesystem check might be enough to recover the old or a rebuilt disk, in any case you need to do a new config first:

 

-Tools -> New Config -> Retain current configuration: All -> Apply

-Assign correct parity drive in place of current disk3

-Leave the array like that for now, don't start it.

 

Now before re-assigning old disk3 you want to make sure it's still mounting, see if it mounts with UD (use read-only option) then post results so we can decide best course of action.

 

 

 

 

 

 

 

[thedudezor]

I did the new config and was able to set the correct parity drive. When I mount what should be disk3 using UD, it contains no data:

 

root@Universe:/mnt# ls -l /mnt/disks/
total 0
root@Universe:/mnt#

Edited April 25, 2020 by thedudezor

[JorgeB]

That suggests it didn't mount, please post the diagnostics: Tools -> Diagnostics

 

P.S. I'll be leaving for the day in a few minutes so if you can reply fast.

[thedudezor]

Diagnostics attached.

 

Looks like this is the issue:

 

Apr 25 12:04:56 Universe unassigned.devices: Adding disk '/dev/sde1'...
Apr 25 12:04:56 Universe unassigned.devices: Mount drive command: /sbin/mount -t xfs -o rw,noatime,nodiratime '/dev/sde1' '/mnt/disks/WDC_WD20EZRZ-00Z5HB0_WD-WMC4N0N5FTHH'
Apr 25 12:04:56 Universe kernel: XFS (sde1): Mounting V5 Filesystem
Apr 25 12:04:57 Universe kernel: XFS (sde1): Corruption warning: Metadata has LSN (1:2803) ahead of current LSN (1:2795). Please unmount and run xfs_repair (>= v4.3) to resolve.
Apr 25 12:04:57 Universe kernel: XFS (sde1): Metadata corruption detected at xfs_agi_verify+0x63/0x130 [xfs], xfs_agi block 0x2
Apr 25 12:04:57 Universe kernel: XFS (sde1): Unmount and run xfs_repair
Apr 25 12:04:57 Universe kernel: XFS (sde1): First 128 bytes of corrupted metadata buffer:
Apr 25 12:04:57 Universe kernel: 00000000: 58 41 47 49 00 00 00 01 00 00 00 00 07 47 04 44  XAGI.........G.D
Apr 25 12:04:57 Universe kernel: 00000010: 00 00 00 80 00 00 00 03 00 00 00 01 00 00 00 32  ...............2
Apr 25 12:04:57 Universe kernel: 00000020: 03 b0 1d 40 ff ff ff ff ff ff ff ff ff ff ff ff  ...@............
Apr 25 12:04:57 Universe kernel: 00000030: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................
Apr 25 12:04:57 Universe kernel: 00000040: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................
Apr 25 12:04:57 Universe kernel: 00000050: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................
Apr 25 12:04:57 Universe kernel: 00000060: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................
Apr 25 12:04:57 Universe kernel: 00000070: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff  ................
Apr 25 12:04:57 Universe kernel: XFS (sde1): metadata I/O error in "xfs_trans_read_buf_map" at daddr 0x2 len 1 error 117
Apr 25 12:04:57 Universe kernel: XFS (sde1): xfs_imap_lookup: xfs_ialloc_read_agi() returned error -117, agno 0
Apr 25 12:04:57 Universe kernel: XFS (sde1): Failed to read root inode 0x80, error 117
Apr 25 12:04:57 Universe unassigned.devices: Mount of '/dev/sde1' failed. Error message: mount: /mnt/disks/WDC_WD20EZRZ-00Z5HB0_WD-WMC4N0N5FTHH: mount(2) system call failed: Structure needs cleaning.
Apr 25 12:04:57 Universe unassigned.devices: Partition 'WDC_WD20EZRZ-00Z5HB0_WD-WMC4N0N5FTHH' could not be mounted.

 

 

Thanks

universe-diagnostics-20200425-1215.zip

Edited April 25, 2020 by thedudezor

[JorgeB]

On the console/terminal type:

 

xfs_repair -v /dev/sde1

 

If it asks for -L use it:

 

xfs_repair -vL /dev/sde1

When done try to mount it again.

[JorgeB]

When attempting to mount again no need for read-only mode anymore.

[thedudezor]

xfs_repair -v /dev/sde1

 

That worked and was able to mount and view files / folders on that drive.

[JorgeB]

OK, check contents and make sure data looks correct, look for any lost files/folders inside a lost+found folder, if everything looks good assign disk 3 to the array, all disks should still be new (blue icon), if yes check "parity is already valid" and start the array, then run a correcting parity check since a few sync errors are expected.

[thedudezor]

OK. The data not in lost and found looks to be OK from what I can tell. I do have some files in the lost+found, but nothing important. Will give it a try.

 

Thank you again for your help here. Greatly appreciated and have a good rest of your weekend!

[thedudezor]

Ohh.. I get a error "Unmountable: No file system" on the disk #3 now only option is to format it?

[JorgeB]

NEVER format, did you unmount it from UD before starting the array? If not do that (and re-start the array), if yes post new diags.

[thedudezor]

Yes I unmounted from UD, selected disk 3 (all in blue as you expected), then check the "parity is already valid" and started the array.

 

New daig attached.

universe-diagnostics-20200425-1236.zip

[JorgeB]

It's showing as duplicate UUID, which suggest it didn't unmount successfully, reboot the server and all should be OK, then run the parity check.

[JorgeB]

I have to go now, someone else should be able to help you if still needed, but I'll check back tomorrow.

 

 

[thedudezor]

Hmm. Yes, a reboot allowed the disk to mount and am running a check now. Thank you again!

[JorgeB]

You're welcome, just for completion and having more time to look at the logs the UD disk didn't unmount because it was busy:

Apr 25 12:30:06 Universe unassigned.devices: Unmount of '/dev/sde1' failed. Error message: umount: /mnt/disks/WDC_WD20EZRZ-00Z5HB0_WD-WMC4N0N5FTHH: target is busy.
Apr 25 12:30:07 Universe unassigned.devices: Since there aren't any open files, will force unmount.

 

You likely had a terminal windows open on it, in those cases UD forces an unmount to prevent an unclean shutdown, so it appears unmounted in the GUI but it really isn't, hence why rebooting fix it.

 

Also, if you don't mind I'm going to tag this solved.