OK, I might have screwed myself here, I tried to figure things out on my own, have done a few reboots, followed some commands from other threads, may or may not be boned but I'm shooting up the signal flare for help here

Unraid 6.12.6, been pretty rock solid until now

Long story short I have direct passthrough of 2 SSD's to my VM and it was giving me IO errors (disk has a smart warning, was in the process of starting to empty it to replace it but just started getting a ton of IO errors) so I figured a reboot of the VM might help. Rebooted the VM and the disk vanished, I still saw it in my unassigned devices (that's how it's mapped in to the VM) so I just rebooted the whole unraid box thinking maybe something would come back after a reboot.

On reboot nothing worked and it was because my cache was shown as unsupported or no file system. Here's where I might have screwed myself. I tried to start stop, nothing, unassign and reassign, no dice. Now when I try to assign the second disk back its saying all will be wiped. Unfortunately my dumb a$$ didn't screenshot settings so I don't remember exactly which disk combo it was and what the exact order it was. I know one of them was the Micron 2TB (currently showing sdk) and the other was one of my 2 NVME drives, plan was eventually to get them both on NVME and get rid of the Micron but I didn't get to swapping out the last Micron yet, was still in use. NVME drives were fine from a zpool in another unraid box I got rid of previously so they should be good and pass the smart tests fine.

I can't remember which one I had in there, and in what order, and now when I try to put them in it warns me whatever goes in slot 2 will be wiped

Tried running this from other posts, didn't work on either NVME drive or the Micron for that matter

root@UNRAIDMAIN:~# btrfs rescue zero-log /dev/nvme1n1

No valid Btrfs found on /dev/nvme1n1

ERROR: could not open ctree

root@UNRAIDMAIN:~# btrfs rescue zero-log /dev/nvme0n1

No valid Btrfs found on /dev/nvme0n1

ERROR: could not open ctree

root@UNRAIDMAIN:~# btrfs rescue zero-log /dev/sdk

No valid Btrfs found on /dev/sdk

ERROR: could not open ctree

root@UNRAIDMAIN:~# mount -o degraded,rescue=all,ro /dev/nvme0n1 /temp

mount: /temp: wrong fs type, bad option, bad superblock on /dev/nvme0n1, missing codepage or helper program, or other error.

dmesg(1) may have more information after failed mount system call.

root@UNRAIDMAIN:~# mount -o degraded,rescue=all,ro /dev/nvme1n1 /temp

mount: /temp: wrong fs type, bad option, bad superblock on /dev/nvme1n1, missing codepage or helper program, or other error.

dmesg(1) may have more information after failed mount system call.

root@UNRAIDMAIN:~# mount -o degraded,rescue=all,ro /dev/sdk /temp

mount: /temp: wrong fs type, bad option, bad superblock on /dev/sdk, missing codepage or helper program, or other error.

dmesg(1) may have more information after failed mount system call.

Have I screwed myself here? Any chance I could fix this, copy off my data, and then format back to a fresh start on the pair of NVME's? Like I said I have rebooted a couple times hoping that would help, followed other threads but nothing worked so far, didn't find the thread saying don't reboot until well after I already had lol

unraidmain-diagnostics-20260503-2204.zip

Edited May 4May 4 by Vaulter98c

for reference the 1 TB WD Blue SSD (not getting a disk identifier right now for some reason) is the one that I knew had smart errors, that and the 2 TB TEAM SSD (sdl) are the ones passed into the VM so I know it wasn't one of those 2 for cache, I can fight with those later but main priority would be getting cache back so maybe I can save the VM and the one good disk and still dump all that data, plus all my dockers

The 8x 8 TB drives in the main array seem to be fine and unaffected as far as I can tell, all shares on that are good and still work, it's only my cache that seems to be the issue

Post the output from btrfs fi show

I get no output from that command whether the array is running or not , that can't be good lol

It's not a good sign, but it may still be recoverable. For all pool members, post the output from

fdisk -l /dev/sdX (or /dev/nvmeXn0)

blkid /dev/sdX1 (or /dev/nvmeXn0p1)

root@UNRAIDMAIN:~# fdisk -l /dev/sdk

Disk /dev/sdk: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors

Disk model: Micron_1100_MTFD

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

root@UNRAIDMAIN:~# fdisk -l /dev/nvme0n1

Disk /dev/nvme0n1: 953.87 GiB, 1024209543168 bytes, 2000409264 sectors

Disk model: SPCC M.2 PCIe SSD

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

root@UNRAIDMAIN:~# fdisk -l /dev/nvme1n1

Disk /dev/nvme1n1: 953.87 GiB, 1024209543168 bytes, 2000409264 sectors

Disk model: SPCC M.2 PCIe SSD

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

root@UNRAIDMAIN:~# blkid /dev/sdk

root@UNRAIDMAIN:~# blkid /dev/nvme0n1

root@UNRAIDMAIN:~# blkid /dev/nvme1n1

Nothing comes from those bottom 3

Currently the Micron (sdk) is the one sitting in the pool config

Above commands were done with arrays started, get the same info with it not running, still no output from that second set of commands

None of the drives has a partition. Any idea how that happened?

Type sfdisk /dev/sdk, then type 2048 and hit enter and post the result; don't type anything else for now.

No clue, VM was getting IO issues as I tried to empty that passed through disk, thought a reboot might help, then nothing was showing on those cache drives after the reboot. only changes I ever made was assigning and unassigning disks trying to get them to pop back in, never formatted or wiped anything, only other commands I tried were the above ones in my OP. Never seen unraid act like this before over the years (on my 3rd unraid box now), I'm no linux guru but this one obviously stumped me lol

root@UNRAIDMAIN:~# fdisk /dev/sdk

Welcome to fdisk (util-linux 2.38.1).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Device does not contain a recognized partition table.

Created a new DOS disklabel with disk identifier 0x3d49aff2.

Command (m for help): 2048

2: unknown command

5 minutes ago, Vaulter98c said:

fdisk /dev/sdk

It's not fdisk

56 minutes ago, JorgeB said:

Type sfdisk /dev/sdk, then type 2048 and hit enter and post the result; don't type anything else for now.

right... sorry fighting the Monday BS at work lol

root@UNRAIDMAIN:~# sfdisk /dev/sdk

Welcome to sfdisk (util-linux 2.38.1).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Checking that no-one is using this disk right now ... OK

Disk /dev/sdk: 1.86 TiB, 2048408248320 bytes, 4000797360 sectors

Disk model: Micron_1100_MTFD

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

sfdisk is going to create a new 'dos' disk label.

Use 'label: <name>' before you define a first partition

to override the default.

Type 'help' to get more information.

>>> 2048

Created a new DOS disklabel with disk identifier 0x95cf08d3.

Created a new partition 1 of type 'Linux' and of size 1.9 TiB.

Partition #1 contains a btrfs signature.

Do you want to remove the signature? [Y]es/[N]o:

Type N and enter to keep the signature, then type write and enter and post a new output from btrfs fi show

Do you want to remove the signature? [Y]es/[N]o: n

/dev/sdk1 : 2048 4000797359 (1.9T) Linux

/dev/sdk2: write

New situation:

Disklabel type: dos

Disk identifier: 0x95cf08d3

Device Boot Start End Sectors Size Id Type

/dev/sdk1 2048 4000797359 4000795312 1.9T 83 Linux

The partition table has been altered.

Calling ioctl() to re-read partition table.

Syncing disks.

root@UNRAIDMAIN:~# btrfs fi show

warning, device 2 is missing

Label: none uuid: 45e8152a-9352-4226-a1c2-776cc68d5c45

Total devices 2 FS bytes used 251.08GiB

devid 1 size 1.86TiB used 286.03GiB path /dev/sdk1

*** Some devices missing

hey looks like we're moving somewhere, want me to start the array again now?

If the pool was redundant (raid1), you can see if it can be imported with just that device. If it wasn't redundant, we can try to find the other member.

it was setup as raid1, started the array and it looks like I'm back in business, or at least the cache mounts and I can see my crap again so I'm like 99% the way there barring any other disk issues, way better than I was before!

Since it was setup as raid 1 though before, and the 2 TB micron has some smart warnings now (maybe what triggered this all?), what would you suggest as the best way to migrate over to dual NVME's and just get rid of the Micron all together?

A) Should I add one of the NVME's, let it wipe then rebuild the raid, then pull the micron and repeat with the other nvme until it's rebuilt again on 2x NVME?

B) Or should I just manually back it up (doing that right now anyways - appdata, domain, system), pull the micron, start both NVME's together as a fresh and wiped new start, and copy the backups back, leaving the MIcron as sort of an offline back at that point?

Post new diags with the array started to get the pool status.

diags, not sure if usage shows in these but I'm currently copying out my appdata so if you see higher than idle thats why

unraidmain-diagnostics-20260505-0948.zip

The pool has been converted to single, so you can just stop the array, add a second device, and it should create a mirror.

Also recommend upgrading Unraid; you are on a very old release.

Did the mirror and then re-mirrored again to get both NVME's in play and finally pull the last Micron out, seems to have solved everything cache related now and I just need to figure out my passthrough issue I was having but that's not related, I should be able to handle that as worst case I can just pull that entirely and recover manually if needed

Also did push the latest update, I was hesitant on 7 for the longest time from horror stories I saw in the start of 7, but now that I have manual copies of all my stuff even if something goes wrong I can recover now at least. Guess I need to look into good ways to automate backups of my cache but that's it's own rabbit hole I guess

Sent ya a couple bucks for the trouble, saved me from having to start over from scratch so next drink is on me! Thanks man!

Thanks!

7 hours ago, Vaulter98c said:

Guess I need to look into good ways to automate backups of my cache but that's it's own rabbit hole I guess

Typically the Appdata Backup and VM Backup plugins are used for this purpose. If you have special requirements these do not meet then a custom script run via User Scripts plugin would be the way to go.