March 15, 20251 yr I have a BTRFS pool with two SDD, i was seeing a lot of errors on one of the disks, so I replaced it with a new disk. (shutdown, replaced disk, added new disk to pool). I let the pool rebuild and the next day i was getting those same errors, so i shut it back down and replaced the sata cable. Now i get the following errors when i start the array. I have tried just about everything from all the existing posts on here about this kind of issue and have not made any progress at all. Does anyone have any suggestions of other things i could try? Syslog Output Mar 14 20:58:03 HYATTS emhttpd: import 32 pool device: (sdl) SAMSUNG_MZ7LM1T9HMJP-00005_S2TVNX0JB08456 Mar 14 20:58:03 HYATTS emhttpd: import 33 pool device: (sdk) Samsung_SSD_860_EVO_500GB_S3Z1NB0K814596V Mar 14 20:58:03 HYATTS emhttpd: ssd_cache: recover volume Mar 14 20:58:03 HYATTS emhttpd: /sbin/blkid /dev/sdl1 2>&1 Mar 14 20:58:03 HYATTS emhttpd: /dev/sdl1: UUID="ac0daf63-c73c-4130-90a7-65bb7beeb45c" UUID_SUB="c912ab9f-d56d-4890-83f7-f2621615d435" BLOCK_SIZE="4096" TYPE="btrfs" Mar 14 20:58:03 HYATTS emhttpd: ssd_cache: btrfs recover pool Mar 14 20:58:03 HYATTS emhttpd: ssd_cache: btrfs verify devices Mar 14 20:58:03 HYATTS emhttpd: /sbin/btrfs filesystem show ac0daf63-c73c-4130-90a7-65bb7beeb45c 2>&1 Mar 14 20:58:03 HYATTS emhttpd: Label: none uuid: ac0daf63-c73c-4130-90a7-65bb7beeb45c Mar 14 20:58:03 HYATTS emhttpd: Total devices 2 FS bytes used 210.78GiB Mar 14 20:58:03 HYATTS emhttpd: devid 0 size 476.94GiB used 48.03GiB path /dev/sdl1 Mar 14 20:58:03 HYATTS emhttpd: devid 2 size 465.76GiB used 216.03GiB path /dev/sdk1 Mar 14 20:58:03 HYATTS emhttpd: shcmd (778): mount -t btrfs -o ro,degraded -U ac0daf63-c73c-4130-90a7-65bb7beeb45c /tmp/emhttp/mnt Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): first mount of filesystem ac0daf63-c73c-4130-90a7-65bb7beeb45c Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): using crc32c (crc32c-intel) checksum algorithm Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): allowing degraded mounts Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): using free space tree Mar 14 20:58:03 HYATTS kernel: BTRFS warning (device sdl1): devid 1 uuid b7d94211-9749-40bd-adb2-f44797b585e6 is missing Mar 14 20:58:03 HYATTS root: mount: /tmp/emhttp/mnt: mount system call failed: Structure needs cleaning. Mar 14 20:58:03 HYATTS root: dmesg(1) may have more information after failed mount system call. Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): bdev /dev/sdl1 errs: wr 591963, rd 0, flush 208, corrupt 0, gen 0 Mar 14 20:58:03 HYATTS kernel: BTRFS info (device sdl1): bdev <missing disk> errs: wr 12196108, rd 2443, flush 8144, corrupt 0, gen 0 Mar 14 20:58:03 HYATTS kernel: BTRFS error (device sdl1): replace without active item, run 'device scan --forget' on the target device Mar 14 20:58:03 HYATTS kernel: BTRFS error (device sdl1): failed to init dev_replace: -117 Mar 14 20:58:03 HYATTS kernel: BTRFS error (device sdl1): open_ctree failed When i do a btrfs fi show, the new disk which is a 2TB disk (i bought two to upgrade that pool since i though the drive was failing) Label: none uuid: ac0daf63-c73c-4130-90a7-65bb7beeb45c Total devices 2 FS bytes used 210.78GiB devid 0 size 476.94GiB used 48.03GiB path /dev/sdl1 <--- this is the new disk, that should be a 2TB disk. devid 2 size 465.76GiB used 216.03GiB path /dev/sdk1 If i attach the old disk via a usb adapter the drive size is the same as the new 2tb disk Label: none uuid: ac0daf63-c73c-4130-90a7-65bb7beeb45c Total devices 2 FS bytes used 210.78GiB devid 0 size 476.94GiB used 48.03GiB path /dev/sdl1 devid 1 size 476.94GiB used 216.03GiB path /dev/sda1 <-- old disk devid 2 size 465.76GiB used 216.03GiB path /dev/sdk1 The PARTUUID is missing from the pool that wont mount. # blkid /dev/sdb1: LABEL_FATBOOT="UNRAID" LABEL="UNRAID" UUID="272C-EBE2" BLOCK_SIZE="512" TYPE="vfat" /dev/loop1: BLOCK_SIZE="131072" TYPE="squashfs" /dev/sdk1: UUID="ac0daf63-c73c-4130-90a7-65bb7beeb45c" UUID_SUB="642602af-b04f-416a-aa0b-0e3fd1d31175" BLOCK_SIZE="4096" TYPE="btrfs" /dev/sdg1: UUID="1b6b0121-607d-40da-abba-3066b1df6eb4" UUID_SUB="75fa5d29-33f9-4ed9-bfef-ccee577abdc4" BLOCK_SIZE="4096" TYPE="btrfs" PARTUUID="136c4b ad-a7e5-4237-a238-2e72c0f3701c" /dev/loop0: BLOCK_SIZE="131072" TYPE="squashfs" /dev/sdl1: UUID="ac0daf63-c73c-4130-90a7-65bb7beeb45c" UUID_SUB="c912ab9f-d56d-4890-83f7-f2621615d435" BLOCK_SIZE="4096" TYPE="btrfs" /dev/sdh1: UUID="1b6b0121-607d-40da-abba-3066b1df6eb4" UUID_SUB="98f7ee37-7c2e-4f37-ab16-8b1639cc88e2" BLOCK_SIZE="4096" TYPE="btrfs" /dev/md2p1: UUID="a9433759-3dd6-4469-bf20-5b12a84c0f01" BLOCK_SIZE="512" TYPE="xfs" /dev/md5p1: UUID="9398a7c0-202a-4469-8d84-ae3b45b0c2e9" BLOCK_SIZE="512" TYPE="xfs" /dev/md1p1: UUID="e694944e-4ce4-4e5b-8520-8d0e2a4e057c" BLOCK_SIZE="512" TYPE="xfs" /dev/md4p1: UUID="fb32da95-1beb-4fbb-ade4-d69cdd4643ee" BLOCK_SIZE="512" TYPE="xfs" /dev/md3p1: UUID="41abf08d-1e8e-4b13-9435-18e0619b608b" BLOCK_SIZE="512" TYPE="xfs" /dev/loop2: UUID="0be7b92c-f210-4892-8b94-c6bc67aeaa2f" UUID_SUB="7cf8ed06-8d54-41a3-984f-9b1e3610faed" BLOCK_SIZE="4096" TYPE="btrfs" /dev/loop3: UUID="3f086c01-7c91-489e-b831-c2a8eb9c6f6a" UUID_SUB="177f97d1-d783-4088-af05-f7a5debaa0d6" BLOCK_SIZE="4096" TYPE="btrfs" btrfs check /dev/sdl1 Opening filesystem to check... warning, device 1 is missing Checking filesystem on /dev/sdl1 UUID: ac0daf63-c73c-4130-90a7-65bb7beeb45c [1/8] checking log skipped (none written) [2/8] checking root items [3/8] checking extents [4/8] checking free space tree [5/8] checking fs roots [6/8] checking only csums items (without verifying data) [7/8] checking root refs [8/8] checking quota groups skipped (not enabled on this FS) found 226325237760 bytes used, no error found total csum bytes: 220170504 total tree bytes: 336019456 total fs tree bytes: 58064896 total extent tree bytes: 48136192 btree space waste bytes: 37366285 file data blocks allocated: 243617456128 referenced 224019578880 # btrfs check /dev/sdk1 Opening filesystem to check... warning, device 1 is missing Checking filesystem on /dev/sdk1 UUID: ac0daf63-c73c-4130-90a7-65bb7beeb45c [1/8] checking log skipped (none written) [2/8] checking root items [3/8] checking extents [4/8] checking free space tree [5/8] checking fs roots [6/8] checking only csums items (without verifying data) [7/8] checking root refs [8/8] checking quota groups skipped (not enabled on this FS) found 226325237760 bytes used, no error found total csum bytes: 220170504 total tree bytes: 336019456 total fs tree bytes: 58064896 total extent tree bytes: 48136192 btree space waste bytes: 37366285 file data blocks allocated: 243617456128 referenced 224019578880 When i try to mount it with rescue, ro # mount -o rescue=all,ro /dev/sdk1 /mnt/ssd_cache mount: /mnt/ssd_cache: mount system call failed: Structure needs cleaning. dmesg(1) may have more information after failed mount system call. diagnostics-20250315-0824.zip Edited March 15, 20251 yr by jhyatt1271 adding more info
March 15, 20251 yr Community Expert Solution 1 hour ago, jhyatt1271 said: size 476.94GiB Note that this is the total allocated fs size, not the device capacity. That looks confusing, especially the new device being ID 0, a normal replacement would inherit the same ID. Try importing the pool with all 3 devices assigned, if that doesn't work, try the two old devices, if you are not sure how to re-import a pool let us know if you are on 6.12 or 7.0
March 15, 20251 yr Author 6 hours ago, JorgeB said: Note that this is the total allocated fs size, not the device capacity. OK that makes sense. Importing with all three, failed with the same errors, of structure needs cleaning. Mar 15 16:31:38 HYATTS kernel: BTRFS error (device sdl1): failed to init dev_replace: -117 Mar 15 16:31:38 HYATTS root: mount: /tmp/emhttp/mnt: mount system call failed: Structure needs cleaning. Importing with the original two failed with (cannot import with misplaced drives) Mar 15 16:36:17 HYATTS emhttpd: ssd_cache_tmp: cannot import with misplaced devices Edited March 15, 20251 yr by jhyatt1271
March 15, 20251 yr Author @JorgeB Well it didnt exactly work, but it did lead to a solution. I deleted the pool, put in the original drives and it let me mount the original disk (devid 2) from unassigned disks. That allowed me to copy all the data over to another pool. Then I just formated, and created a new pool with the two new drives. Thank you so much for your help on this! Edited March 15, 20251 yr by jhyatt1271
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.