So I just upgraded to 6.7.0 and when I rebooted after adding the new vfio-pci.cfg to passthrough my GPU's unraid somehow thought I passthroughed my HBA (and my dumb ass didnt notice) which in turn caused me to start the BTRFS cache array with just 2 SSD's (instead of the normal 10, the missing 8 disks are all connected to the "passthroughed" HBA).

 

Now my cache array is in the "Unmountable: No file system" state and I'm scared.

 

I've rolled back the vfio-pci.cfg and unraid can see all SSD's but I need help to repair my BTRFS RAID10 cache filesystem. I havent started the array after my first fuckup and if I try assign the disks to cache now I get the "All existing data on this device will be OVERWRITTEN when array is Started."

 

Technically the data should still be left on the disks, but metadata and logs are probably bonkers.

 

I've encountered the "Unmountable: No file system" on the main array and at that time I repaired it with xfs_repair. But that was just 1 disk.

 

Any guidance on how to proceed to repair the cache array would be very much appreciated.

Edited May 16, 20197 yr by sorano

I felt the need to clarify the reason how I ended up vfio-bind'ing the HBA. Might save someone else from suffering the same fate.

At first when I upgraded to 6.7.0 I had problems with booting unraid properly, while safe mode worked.

 

After troubleshooting the non-boot I found the cause to be something installed from devpack plugin which I removed and then could boot normally. But by then I had already made a clean unraid install on the USB and just copied over my config folder (but I forgot the syslinux folder). The forgotten syslinux folder caused the pcie_acs_override=downstream to dissapear in my syslinux configuration append line which in turn cause my HBA and GPU to end up in the same IOMMU group. The new vfio-pci.cfg will by default passthrough everything in the IOMMU group, and that is what screwed me over when I started my array.

 

So all in all bad luck mixed with some laziness screwed me over. Don't be lazy people and double check your configurations!

 

Now I'm hoping some btrfs guru will drop some magic repair command on me to save the day.

 

IOMMU group 1:	[8086:1901] 00:01.0 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) (rev 05)
[8086:1905] 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) (rev 05)
[10de:1b81] 01:00.0 VGA compatible controller: NVIDIA Corporation GP104 [GeForce GTX 1070] (rev a1)
[10de:10f0] 01:00.1 Audio device: NVIDIA Corporation GP104 High Definition Audio Controller (rev a1)
[1000:0072] 02:00.0 Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)

 

Pool should be fine, you just need to make Unraid forget it, there are 2 options, either do a new config and reassign all devices or start the array without any cache devices assigned, stop the array, assigned them all (there can't be the overwrite warning for any cache device) and start again, for the latter best to temporarily disable any services using the cache pool, like docker and/or VMs.

I did the:

 

"... start the array without any cache devices assigned, stop the array, assigned them all (there can't be the overwrite warning for any cache device) and start again, for the latter best to temporarily disable any services using the cache pool, like docker and/or VMs."

 

And everything went well.

 

Big up @johnnie.black