My unraid was dead and unresponsive, found that the USB stick had died, grabbed another and restored my backup, things were back up and running. Everything was good until I decided I may as well replace a failing data drive and kick off a rebuild, at the same time my app data cache pool decided to die. I dont know if this is recoverable or did the multiple reboots kill my SSD? Had no errors for this SSD previously.

 

The cache pool initially showed "Cache drive unmountable: Wrong or no file system"

 

But I then made it worse by trying to remove one drive from the cache pool to see if I encountered a different error but on starting the array unraid decided it was a new device and blew the config away for the drive.

 

Now showing "Cache Pool: Unmountable no file system"

 

Screen shot and logs below

 

 

Jul 29 20:49:46 tower kernel: sde: sde1
Jul 29 20:51:40 tower  emhttpd: read SMART /dev/sde
Jul 29 20:53:17 tower kernel: BTRFS info (device sde1): turning on async discard
Jul 29 20:53:17 tower kernel: BTRFS info (device sde1): using free space tree
Jul 29 20:53:17 tower kernel: BTRFS info (device sde1): has skinny extents
Jul 29 20:53:17 tower kernel: BTRFS info (device sde1): bdev /dev/sde1 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Jul 29 20:53:17 tower kernel: BTRFS error (device sde1): parent transid verify failed on 1252245504 wanted 494259 found 494255
Jul 29 20:53:17 tower kernel: BTRFS warning (device sde1): checksum verify failed on 1252245504 wanted 0x16801d8b found 0x8f3bb060 level 1
Jul 29 20:53:17 tower kernel: BTRFS error (device sde1): failed to read block groups: -5
Jul 29 20:53:17 tower root: mount: /mnt/appcache: wrong fs type, bad option, bad superblock on /dev/sde1, missing codepage or helper program, or other error.
Jul 29 20:53:17 tower kernel: BTRFS error (device sde1): open_ctree failed
Jul 29 20:58:39 tower  emhttpd: read SMART /dev/sde
Jul 29 20:59:02 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R (sde) 512 976773168
Jul 29 20:59:02 tower  emhttpd: read SMART /dev/sde
Jul 29 20:59:14 tower  emhttpd: shcmd (493): /usr/sbin/cryptsetup luksErase /dev/sde1
Jul 29 20:59:14 tower root: Device /dev/sde1 is not a valid LUKS device.
Jul 29 20:59:14 tower  emhttpd: shcmd (494): /sbin/wipefs -a /dev/sde1
Jul 29 20:59:14 tower root: /dev/sde1: 8 bytes were erased at offset 0x00010040 (btrfs): 5f 42 48 52 66 53 5f 4d
Jul 29 20:59:17 tower unassigned.devices: Partition 'sde1' does not have a file system and cannot be mounted.
Jul 29 21:00:30 tower  emhttpd: read SMART /dev/sde
Jul 29 21:00:51 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R (sde) 512 976773168
Jul 29 21:00:51 tower  emhttpd: import 30 cache device: (sde) Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R
Jul 29 21:00:51 tower  emhttpd: read SMART /dev/sde
Jul 29 21:01:25 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R (sde) 512 976773168
Jul 29 21:01:25 tower  emhttpd: read SMART /dev/sde
Jul 29 21:01:39 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R (sde) 512 976773168
Jul 29 21:01:39 tower  emhttpd: import 30 cache device: (sde) Samsung_SSD_850_EVO_500GB_S2R9NX0J522893R
Jul 29 21:01:39 tower  emhttpd: read SMART /dev/sde

 

 

Jul 29 20:49:40 tower kernel: sdf: sdf1
Jul 29 20:51:40 tower  emhttpd: read SMART /dev/sdf
Jul 29 20:58:39 tower  emhttpd: read SMART /dev/sdf
Jul 29 20:59:02 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z (sdf) 512 976773168
Jul 29 20:59:02 tower  emhttpd: import 31 cache device: (sdf) Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z
Jul 29 20:59:02 tower  emhttpd: read SMART /dev/sdf
Jul 29 20:59:16 tower kernel: BTRFS info (device sdf1): turning on async discard
Jul 29 20:59:16 tower kernel: BTRFS info (device sdf1): allowing degraded mounts
Jul 29 20:59:16 tower kernel: BTRFS info (device sdf1): using free space tree
Jul 29 20:59:16 tower kernel: BTRFS info (device sdf1): has skinny extents
Jul 29 20:59:16 tower kernel: BTRFS warning (device sdf1): devid 1 uuid f74f0fcf-9a14-4e4f-b889-3e4e39c34dbf is missing
Jul 29 20:59:16 tower kernel: BTRFS warning (device sdf1): devid 1 uuid f74f0fcf-9a14-4e4f-b889-3e4e39c34dbf is missing
Jul 29 20:59:16 tower kernel: BTRFS info (device sdf1): bdev (efault) errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Jul 29 20:59:16 tower root: mount: /mnt/appcache: wrong fs type, bad option, bad superblock on /dev/sdf1, missing codepage or helper program, or other error.
Jul 29 20:59:16 tower kernel: BTRFS warning (device sdf1): checksum verify failed on 1252245504 wanted 0x16801d8b found 0x8f3bb060 level 1
Jul 29 20:59:16 tower kernel: BTRFS error (device sdf1): failed to read block groups: -5
Jul 29 20:59:16 tower kernel: BTRFS error (device sdf1): open_ctree failed
Jul 29 21:00:30 tower  emhttpd: read SMART /dev/sdf
Jul 29 21:00:51 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z (sdf) 512 976773168
Jul 29 21:00:51 tower  emhttpd: import 31 cache device: (sdf) Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z
Jul 29 21:00:51 tower  emhttpd: read SMART /dev/sdf
Jul 29 21:01:25 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z (sdf) 512 976773168
Jul 29 21:01:25 tower  emhttpd: import 31 cache device: (sdf) Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z
Jul 29 21:01:25 tower  emhttpd: read SMART /dev/sdf
Jul 29 21:01:39 tower  emhttpd: Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z (sdf) 512 976773168
Jul 29 21:01:39 tower  emhttpd: import 31 cache device: (sdf) Samsung_SSD_850_EVO_500GB_S2R9NX0J522319Z
Jul 29 21:01:39 tower  emhttpd: read SMART /dev/sdf
Jul 29 21:02:07 tower kernel: BTRFS info (device sdf1): turning on async discard
Jul 29 21:02:07 tower kernel: BTRFS info (device sdf1): allowing degraded mounts
Jul 29 21:02:07 tower kernel: BTRFS info (device sdf1): using free space tree
Jul 29 21:02:07 tower kernel: BTRFS info (device sdf1): has skinny extents
Jul 29 21:02:07 tower kernel: BTRFS warning (device sdf1): devid 1 uuid f74f0fcf-9a14-4e4f-b889-3e4e39c34dbf is missing
Jul 29 21:02:07 tower kernel: BTRFS info (device sdf1): bdev (efault) errs: wr 0, rd 0, flush 0, corrupt 4, gen 0
Jul 29 21:02:07 tower kernel: BTRFS warning (device sdf1): checksum verify failed on 1252245504 wanted 0x16801d8b found 0x8f3bb060 level 1
Jul 29 21:02:07 tower kernel: BTRFS error (device sdf1): failed to read block groups: -5
Jul 29 21:02:07 tower kernel: BTRFS error (device sdf1): open_ctree failed
Jul 29 21:02:07 tower root: mount: /mnt/appcache: wrong fs type, bad option, bad superblock on /dev/sdf1, missing codepage or helper program, or other error.

 

 

Edited July 29, 20232 yr by Distorter

Please post the diagnostics.

I've been able to restore from one of the cache pool disks that has survived with this command from the FAQ page.

btrfs restore -v /dev/sdf1 /mnt/disk3/restore

 

Have also ran extended smart tests across both drives with success.

 

My thinking is that I'll turn off the docker service, delete the pool create a new one and move the files back?

 

Diag attached.

 

 

ragnarok-diagnostics-20230801-2003.zip

16 minutes ago, Distorter said:

My thinking is that I'll turn off the docker service, delete the pool create a new one and move the files back?

If the data is backed up you can do that.