Need Help - Disk says Unmountable No file system

[badboyz]

I've been having issues for a while now and haven't figured out the cause.  It seems anytime I have an unclean shutdown from a power failure during the parity check one of my drives will start reporting errors and eventually get disabled.  At first I thought maybe it was a power issue but now I believe it may be my HBA card or the controller in the disk shelf. 

 

My current issue is during a parity rebuild another disk starting having I/O errors and then showed up under "Unassigned Devices" even though the rebuild was still running.  After the rebuild finished this disk was now disabled and like an idiot I click on mount in "Unassigned Devices" but it did not mount.  I checked the actual disk shelf to make sure the drive was properly connected and rebooted unraid.  After rebooting this disk now says "disabled/emulated" but also says Unmountable No file system.  Is there a way to recover from this within unraid or should I remove the disk and copy the data off, then allow unraid to reformat it?  Below is the log info for the disk and attach is the diagnostics.  

 

Aug 27 16:14:00 nas emhttpd: shcmd (150): mkdir -p /mnt/disk19
Aug 27 16:14:00 nas emhttpd: shcmd (151): mount -t btrfs -o noatime,nodiratime /dev/md19 /mnt/disk19
Aug 27 16:14:00 nas kernel: BTRFS info (device md19): disk space caching is enabled
Aug 27 16:14:00 nas kernel: BTRFS info (device md19): has skinny extents
Aug 27 16:14:01 nas kernel: BTRFS error (device md19): bad tree block start 9784718863715718926 2751066292224
Aug 27 16:14:01 nas kernel: BTRFS error (device md19): failed to recover balance: -5
Aug 27 16:14:01 nas root: mount: /mnt/disk19: wrong fs type, bad option, bad superblock on /dev/md19, missing codepage or helper program, or other error.
Aug 27 16:14:01 nas kernel: BTRFS error (device md19): open_ctree failed
Aug 27 16:14:01 nas emhttpd: shcmd (151): exit status: 32
Aug 27 16:14:01 nas emhttpd: /mnt/disk19 mount error: No file system
Aug 27 16:14:01 nas emhttpd: shcmd (152): umount /mnt/disk19
Aug 27 16:14:01 nas root: umount: /mnt/disk19: not mounted.
Aug 27 16:14:01 nas emhttpd: shcmd (152): exit status: 32
Aug 27 16:14:01 nas emhttpd: shcmd (153): rmdir /mnt/disk19

nas-diagnostics-20180827-2318.zip

[JorgeB]

If you still need help please repost your diagnostics as there seems to be a problem where the old forum diags can't be downloaded.

[badboyz]

I'm able to read the drive outside of unraid, so I'm in process of copying the data to an external drive before letting unraid wipe the drive.  I attached another diagnostic file from earlier.  If you have any insight into what my cause my i/o time out issues? I just don't know if its the 24 bay enclosure or one of its components or the hba card in my r710 server...  

nas-diagnostics-20180827-1620.zip

[JorgeB]

Looks like there's metadata corruption, best course of action is what you're already doing, i.e., copy data and reformat, but for the future make sure you're using DUP metadata on the data disks, it should be able to fix these issues, unless it caused by memory error or similar, since that could damage both metadata copies.

 

To convert metadata to DUP you can use:

 

btrfs balance start -mconvert=dup /mnt/diskX

 

It needs to be done on all disks.

 

 

[badboyz]

On 9/3/2018 at 9:41 AM, johnnie.black said:

Looks like there's metadata corruption, best course of action is what you're already doing, i.e., copy data and reformat, but for the future make sure you're using DUP metadata on the data disks, it should be able to fix these issues, unless it caused by memory error or similar, since that could damage both metadata copies.

 

To convert metadata to DUP you can use:

 

btrfs balance start -mconvert=dup /mnt/diskX

 

It needs to be done on all disks.

 

 

I'm in the process of converting the the metadata now before I copy my data back over.  I just wanted to check if this looks correct now after I run that command. 

 

[JorgeB]

Yes, this way all metadata is duplicated, it uses very little extra space but if it get's damaged by an I/O error or similar the other copy will automatically be used.