I recently had a disk error that offlined a drive in an encrypted array (4*3tb inc a parity).  I've replaced the drive (twice now) having deleted all partitions off the replacements, and it has rebuilt (twice with different drives) apparently successfully each time.  I'm pretty sure the parity was good, as I checked it recently, but I have moved the disks into a new server since (note the drive did not offline immediately but a day later).  However, when completed, the drive status is "Unmountable: Volume not encrypted" and the icon is orange unlocked.  I can't understand how it can't be encrypted after a block by block rebuild?

 

 

I tried a check filesytem status...

 

 

Help - How do I rebuild my array and get it back online?

 

4-28medina-diagnostics-20191109-0814.zip

Edited November 9, 20196 yr by TheTinMan
Added screenshot

I just thought - if I pull the "umnountable" drive, will unraid emulate the old drive from parity so I can copy off the share data?  If I can find a temporary home and copy it off then i can junk the array and rebuild from scratch.  Currently I'm copying all the data I can off the "good" disks in the array.  Would that work? 

 

56 minutes ago, TheTinMan said:

if I pull the "umnountable" drive, will unraid emulate the old drive from parity so I can copy off the share data?

 Yes, unRaid will emulate the drive, but the emulated version is also going to be unmountable.  Wait for the experts ( @johnnie.black ) before going further, as I would have expected the file system check to do something other than just exiting, even with the -n flag set.

System became unresponsive so I had to reboot.  After booting, got an error at the server console:

 

Metadata CRC error detected at xfs_agf_read_verify 

XFS (sdg1): Unmount and run xfs_repair

 

And now it's treating the disk as new again (Start will start Parity-Sync and/or Data-Rebuild) instead of unmountable.

 

Interestingly though, the orange unlock icon is there ("Device to be encrypted") beside the blue "new disk" circle.  Default file system is xfs_encrypted though.

 

This is a sick puppy... 😞

 

I'm away for the weekend and only on phone, so not easy to check diags, but looks like the problem is encryption related, and I don't use that, do you get the same result if no disk is assigned to that slot?

Thanks anyway Johnnie; yes I get same result.

 

 

But for a disk that's not installed it seems to describe it!

 

Overnight it decided to do a read check on the mounted volumes; all okay.

 

And when I unmount the array, we get the same thing; device to be encrypted.  

 

 

 

I'm really scratching my head here; config corruption?

 

Appreciate the help.

Looks to me like the LUKS mbr is damaged on the emulated disk, and if that's the case it needs to be fixed first, but like mentioned I don't use encryption. 

So as a last throw of the dice, while the array was stopped, I changed the Disk 1 settings file type to Auto.  This magically got rid of the orange unlock icon, hurrah!  Started a rebuild and went to bed; surely this would produce the result?

 

Nope.

 

The rebuild completed, but now we had a different error.  Unmountable:  No File System.

 

Stopped the array, removed the disk and looked at the underlying partition on the disk.  It was formatted to xfs, but 0 bytes.  So somehow the rebuild has just written zeroes to it.

 

 

 

It looks like this is beyond any hope of recovery; I believe I have archive copies of 95% of my data, but it is on unprotected (ha!) disks.  I need to blow away this whole config and start from scratch, as I just don't trust it now.  We may never know how it got to this stage, but it IS possible to lose a disk in an Unraid array and not be able to get it back.  Think I'll choose a second parity disk this time...