Crashing server setup has led to unmountable drive, please help!

[John4559]

I'm running Unraid 6.12.3, with two 12 TB drives - one used for shares and the other for parity.  I've been having a lot of random freezeups/crashes and coudln't figure out what was causing it.  I thought it was faulty RAM, but when I replaced that the problem persisted.  I changed the Docker custom network setting from "macvlan" to "ipvlan," but then I couldn't start or remove one of the containers.  Ultimately, I noticed a bunch of errors and log entries suggesting I do an xfs_repair, which I attempted "xfs_repair -v /dev/sdd1" but it gave me an error about a log problem.  Following another thread, I repeated the command with an additional parameter "-L" which appear to complete successfully.  BUT then my drive shows up as unmountable and the space where I'd normally put in my passphrase to decrypt the drive was asking for a new passphrase.  

 

Now, I'm concerned I won't be able to recover the data on my drive or restore my server setup.  I'm so hoping someone here can point me in the right direction.  I've been using Unraid for years and have never had this kind of problem.

[itimpi]

You should post your system's diagnostics zip file in your next post in this thread to get more informed feedback.  It is always a good idea to post this if your question might involve us seeing how you have things set up or to look at recent logs.  At the moment it is not at all clear what the state of your system is.

 

BTW:  The xfs_repair command you ran is not the ideal one as at the very least it would invalidate parity.   You should always run xfs_repair via the GUI if at all possible as it uses a variant that maintains parity.

[John4559]

Thanks so much itimpi.  I'm attaching my diagnostics file.  I wish I had used the other repair utility, but now I'm hoping there's a way to get back on track! 

elitedesk-diagnostics-20231206-1027.zip

[JorgeB]

Post the output of

blkid

[John4559]

1 hour ago, JorgeB said:

Post the output of

blkid

/dev/sda1: LABEL_FATBOOT="UNRAID" LABEL="UNRAID" UUID="2732-64F5" BLOCK_SIZE="512" TYPE="vfat"
/dev/loop1: TYPE="squashfs"
/dev/sdd1: UUID="b2aa9d32-c183-4326-aacf-5e08acb1d737" BLOCK_SIZE="512" TYPE="xfs" PARTUUID="c19cae91-3a78-48e4-abd4-3bd5f4230135"
/dev/sdb1: UUID="521c4bfc-01d0-4c32-a5a7-812a532cf80a" UUID_SUB="3f9e0276-c0a3-4e32-9590-fbcd0ce996b5" BLOCK_SIZE="4096" TYPE="btrfs"
/dev/loop0: TYPE="squashfs"
/dev/sdc1: UUID="3765e3e1-c1fc-466e-9b29-43ac5cf7bdd8" TYPE="crypto_LUKS" PARTUUID="25a605ed-bcaf-4dbd-a1e0-e3509b9d0854"

[JorgeB]

Disk1 is showing as unencrypted which is not a good sign, parity shows as encrypted, since you only had one data disk, parity is a mirror of disk1, you can do a new config and assign old parity as disk1, leave parity unassigned, start array and see if it mounts, don't touch old disk1 for now.

[John4559]

1 hour ago, JorgeB said:

Disk1 is showing as unencrypted which is not a good sign, parity shows as encrypted, since you only had one data disk, parity is a mirror of disk1, you can do a new config and assign old parity as disk1, leave parity unassigned, start array and see if it mounts, don't touch old disk1 for now.

When I change parity to unassigned and assign the old parity drive to be Disk1, it's says it's "Wrong".  I'm not exactly sure what that means, but it won't let me start the array.  It says it's an invalid configuration.

Edited December 6, 2023 by John4559

[itimpi]

3 hours ago, John4559 said:

When I change parity to unassigned and assign the old parity drive to be Disk1, it's says it's "Wrong".  I'm not exactly sure what that means, but it won't let me start the array.  It says it's an invalid configuration.

Did you use the New Config tool as was suggested to change the assignments?

[trurl]

4 hours ago, JorgeB said:

do a new config

https://docs.unraid.net/unraid-os/manual/storage-management/#reset-the-array-configuration

[John4559]

6 hours ago, trurl said:

https://docs.unraid.net/unraid-os/manual/storage-management/#reset-the-array-configuration

Sorry.  I didn't understand to do this, but I've now done it and it mounted using my passphrase for the older drive.  What next??  I'm hopeful.....

[JorgeB]

If old parity is mounting and showing the correct data you can assign old disk1 to parity and re-sync.

[John4559]

So far so good!  Thank you all sooooooo much.  I am so relieved.  Do we think it's safe to use now or should I wait until the parity-sync finishes just to be sure?

[JorgeB]

It's safe to use but the array is not protected, also avoid heavy i/o since it will slow don the parity sync.

[John4559]

I just noticed that Unraid says my parity drive is "invalid," but it's still doing the parity sync so far as I can tell.  Is that normal?  Is there something I should do?

[itimpi]

12 minutes ago, John4559 said:

I just noticed that Unraid says my parity drive is "invalid," but it's still doing the parity sync so far as I can tell.  Is that normal?  Is there something I should do?

That is expected until the sync completes successfully.