Went to bed, all was fine. Woke to this.

Cache is used only for main array, containers is where I have appdata and such (so filling cache wouldn't take down VMs, as happened once before. Logs attached. I tried a reboot just to see, but nothing changed.

akanas-diagnostics-20250706-0945.zip

Edit: Steps taken reading forums:

root@akanas:~# blkid

/dev/sda1: LABEL_FATBOOT="UNRAID" LABEL="UNRAID" UUID="16FC-3C51" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="95171951-01"

/dev/loop1: BLOCK_SIZE="131072" TYPE="squashfs"

/dev/loop0: BLOCK_SIZE="131072" TYPE="squashfs"

/dev/md2p1: UUID="845e86ed-3072-42dd-911a-578457dbc4c8" BLOCK_SIZE="512" TYPE="xfs"

/dev/md5p1: UUID="50699b8d-2d8f-4f93-b588-df0c3870669e" BLOCK_SIZE="512" TYPE="xfs"

/dev/md1p1: UUID="e90ecf46-6718-41f5-9f09-e914d82d994c" BLOCK_SIZE="512" TYPE="xfs"

/dev/md4p1: UUID="40718d4f-2b92-49cf-a2a3-ac5a965e7cbd" BLOCK_SIZE="512" TYPE="xfs"

/dev/md3p1: UUID="7c513fa3-3022-4f47-88d3-4433cd2f62af" BLOCK_SIZE="512" TYPE="xfs"

/dev/md6p1: UUID="3d5988f7-656e-449e-acca-d010b0bdf7fa" BLOCK_SIZE="512" TYPE="xfs"

Running filesystem check on any of the disks:
ERROR: not a regular file or block device: /dev/ ERROR: cannot open file system Opening filesystem to check...

Worried to try anything else...

EDIT 2: SMART extended self-tests on all 3 drives result in 0 errors. No commands are returning a filesystem. Am I hosed? This is the 3rd time I would have to rebuild the cache, first when an SSD suddenly died on me and corrupted everything, and second the aforementioned disk full issue...

Edited July 7, 2025Jul 7 by kahlif
solved

Post the output from

btrfs fi show

4 hours ago, JorgeB said:

Post the output from

btrfs fi show

Nothing, no output, no error.

EDIT: I had started the array in maintenance mode with the above. When started normally, I get:

root@akanas:~# btrfs fi show

Label: none uuid: 444cab5a-d33c-4a5c-810d-80753362c808

Total devices 1 FS bytes used 372.00KiB

devid 1 size 20.00GiB used 536.00MiB path /dev/loop2

Label: none uuid: b6a29e09-4796-492e-81fd-fa22626f0335

Total devices 1 FS bytes used 416.00KiB

devid 1 size 1.00GiB used 126.38MiB path /dev/loop3

Edited July 7, 2025Jul 7 by kahlif

All 3 pool devices are missing the partition, any idea how that happened?

Jul 6 09:34:42 akanas emhttpd: /bin/lsblk -lnbo TYPE,NAME,START,SIZE /dev/nvme0n1 2>&1

Jul 6 09:34:42 akanas emhttpd: disk nvme0n1 1024209543168

Jul 6 09:34:42 akanas emhttpd: device nvme0n1: no partitions

Jul 6 09:34:42 akanas emhttpd: import 30 pool device: (nvme0n1) SP010TIMEC3M5EV0_24028387060011

Jul 6 09:34:42 akanas emhttpd: /bin/lsblk -lnbo TYPE,NAME,START,SIZE /dev/sdb 2>&1

Jul 6 09:34:42 akanas emhttpd: disk sdb 2048408248320

Jul 6 09:34:42 akanas emhttpd: device sdb: no partitions

Jul 6 09:34:42 akanas emhttpd: import 31 pool device: (sdb) P3-2TB_9P50427012406

Jul 6 09:34:42 akanas emhttpd: /bin/lsblk -lnbo TYPE,NAME,START,SIZE /dev/sdc 2>&1

Jul 6 09:34:42 akanas emhttpd: disk sdc 2048408248320

Jul 6 09:34:42 akanas emhttpd: device sdc: no partitions

Absolutely not - went to bed with nothing happening. That is the most painful part...

Type sfdisk /dev/nvme0n1 and enter, then type 2048 and hit enter again, and post the results, don't do anything else on sfdisk

2 hours ago, JorgeB said:

Type sfdisk /dev/nvme0n1 and enter, then type 2048 and hit enter again, and post the results, don't do anything else on sfdisk

root@akanas:~# sfdisk /dev/nvme0n1

Welcome to sfdisk (util-linux 2.41).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Checking that no-one is using this disk right now ... OK

Disk /dev/nvme0n1: 953.87 GiB, 1024209543168 bytes, 2000409264 sectors

Disk model: SP010TIMEC3M5EV0

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

sfdisk is going to create a new 'dos' disk label.

Use 'label: <name>' before you define a first partition

to override the default.

Type 'help' to get more information.

>>> 2048

Created a new DOS (MBR) disklabel with disk identifier 0x1d27bfe1.

Created a new partition 1 of type 'Linux' and of size 953.9 GiB.

Partition #1 contains a btrfs signature.

Do you want to remove the signature? [Y]es/[N]o:

I chose no for now...

Yes, choose N to keep the signature, then type write and enter and post new btrfs fi show output

1 hour ago, JorgeB said:

Yes, choose N to keep the signature, then type write and enter and post new btrfs fi show output

Looking good!

Created a new DOS (MBR) disklabel with disk identifier 0x50d4d50d.

Created a new partition 1 of type 'Linux' and of size 953.9 GiB.

Partition #1 contains a btrfs signature.

Do you want to remove the signature? [Y]es/[N]o: n

/dev/nvme0n1p1 : 2048 2000409263 (953.9G) Linux

/dev/nvme0n1p2: write

New situation:

Disklabel type: dos

Disk identifier: 0x50d4d50d

Device Boot Start End Sectors Size Id Type

/dev/nvme0n1p1 2048 2000409263 2000407216 953.9G 83 Linux

The partition table has been altered.

Calling ioctl() to re-read partition table.

Syncing disks.

root@akanas:~# btrfs fi show

Label: none uuid: 444cab5a-d33c-4a5c-810d-80753362c808

Total devices 1 FS bytes used 372.00KiB

devid 1 size 20.00GiB used 536.00MiB path /dev/loop2

Label: none uuid: b6a29e09-4796-492e-81fd-fa22626f0335

Total devices 1 FS bytes used 416.00KiB

devid 1 size 1.00GiB used 126.38MiB path /dev/loop3

Label: none uuid: fa55b70a-6dca-4a0e-86e3-229e4f7755ae

Total devices 1 FS bytes used 669.72GiB

devid 1 size 953.87GiB used 674.02GiB path /dev/nvme0n1p1

Holy cow!!! Did the same to the others, unmounted then remounted, and it looks like its all back!!! TYSVM!!!!!

One thing - the list of docker containers is empty...

Any way to get that config back?

looks like adding the containers back 1 at a time using the profiles has everything saved, so we are all good!