March 30Mar 30 Yesterday i installed the new Folderview Plus and after setup and creating a new folder the UI suddenly was stuck.After a force reload in the browser I also rebooted the server but was unable to start the array as the cache pool (2 Samsung 990 EVO Plus) was giving me a config failed message when i tried to start the array.When checking the forums (this thread for example) i removed the cache pool and recreate it, but now it says 'Unmountable: wrong or no file system'.Hope no data is lost from the NVMe drives, but how can i get to it when the pool cannot be mounted?blkid output gives me this/dev/sdb1: LABEL="data" UUID="8685187093077842655" UUID_SUB="10709511807475224983" BLOCK_SIZE="4096" TYPE="zfs_member" PARTUUID="e0008686-01"/dev/sda1: LABEL_FATBOOT="UNRAID" LABEL="UNRAID" UUID="B485-160D" BLOCK_SIZE="512" TYPE="vfat"/dev/loop1: BLOCK_SIZE="131072" TYPE="squashfs"/dev/loop0: BLOCK_SIZE="131072" TYPE="squashfs"Diagnostics are attached.Also i was already able to make an image with the dd commanddd if=/dev/nvme1n1 of=/mnt/disks/data1-2tb/nvme0.img bs=1M status=progressdd if=/dev/nvme1n1 of=/mnt/disks/data2-2tb/nvme1.img bs=1M status=progressand that for both poolszdb -l /dev/nvme0n1 gives me no labelszdb -l /dev/nvme1n1 gives the following output------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'cache' state: 1 txg: 3491005 pool_guid: 10896664973895392852 errata: 0 hostname: 'Serenity' top_guid: 6220366269392501499 guid: 13938023003294794310 vdev_children: 1 vdev_tree: type: 'mirror' id: 0 guid: 6220366269392501499 whole_disk: 0 metaslab_array: 132 metaslab_shift: 33 ashift: 12 asize: 1000198897664 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 13938023003294794310 path: '/dev/nvme0n1p1' whole_disk: 0 DTL: 395 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 12792291676762730391 path: '/dev/nvme1n1p1' whole_disk: 0 DTL: 519 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 failed to unpack label 2failed to unpack label 3How can i continue to recover data from the nvme1.img file and format the cache pool again on the server and then copy the data back over.Guess trying to recover the ZFS pool itself is not an immediate option, correct? serenity-diagnostics-20260330-1806.zip
March 30Mar 30 Community Expert Post the output from zpool import and alsofdisk -l /dev/nvme1n1fdisk -l /dev/nvme0n1
March 30Mar 30 Author zpool importno pools available to importwhich makes sense since it currently shows 'Unmountable: wrong or no file system'fdisk -l /dev/nvme1n1Disk /dev/nvme1n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectorsDisk model: Samsung SSD 990 EVO Plus 1TB Units: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesfdisk -l /dev/nvme0n1Disk /dev/nvme0n1: 931.51 GiB, 1000204886016 bytes, 1953525168 sectorsDisk model: Samsung SSD 990 EVO Plus 1TB Units: sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytes
March 30Mar 30 Community Expert Both devices are missing the partition. Was the pool created in Unraid?
March 31Mar 31 Community Expert Type sfdisk /dev/nvme0n1, then type 2048 and hit enter and post the output; don't type anything else for now
March 31Mar 31 Author Got this as output and have not typed anything else yet.Created a new DOS (MBR) disklabel with disk identifier 0xdcb90663.Created a new partition 1 of type 'Linux' and of size 931.5 GiB.Partition #1 contains a zfs_member signature.Do you want to remove the signature? [Y]es/[N]o:
March 31Mar 31 Community Expert Type N to keep the signature, then type write and enterRepeat the same for the other device; if a signature is found, keep it as well, then once both partitions have been recreated, try zpool import again
March 31Mar 31 Author Thanks so far @JorgeB ,Second one just gives me thisCreated a new DOS (MBR) disklabel with disk identifier 0xa4782ddd.Created a new partition 1 of type 'Linux' and of size 931.5 GiB./dev/nvme1n1p1 : 2048 1953525167 (931.5G) LinuxShould I also just type write here?Also what is the preferred command for zpool import, have little experience using ZFS commands through the terminal.
March 31Mar 31 Community Expert 30 minutes ago, Stiibun said:Should I also just type write here?Note if it doesn't find the zfs signature31 minutes ago, Stiibun said:Also what is the preferred command for zpool importJust open a terminal window and type that command, then paste here the output, do it now, with just the partition restored on one pool device
March 31Mar 31 Author Will come back on this later today and post the output of the zpool import command, at work now. One has to work to pay for this hobby 😉For the second NVMe it indeed didn't find a zfs signature.
March 31Mar 31 Author @JorgeB , output of the zpool import command is as follows; pool: cache id: 10896664973895392852state: DEGRADEDstatus: One or more devices contains corrupted data.action: The pool can be imported despite missing or damaged devices. The fault tolerance of the pool may be compromised if imported. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-4Jconfig: cache DEGRADED mirror-0 DEGRADED nvme0n1p1 ONLINE nvme1n1p1 UNAVAIL
March 31Mar 31 Community Expert You should be able to import the pool degraded using the device that had the partition restored, then add the other device to recreate the mirror.If the pool is still assigned, first go to Main click on the first device for that pool and then "remove pool"back on main, create a new pool with the same name and 1 slotassign just nvme0n1, leave the filesystem set to autostart the array to import the poolPool should import degraded, if it doesn't post new diags; if it does, stop the array, assign the other device to the disabled slot, start array to resilver.
March 31Mar 31 Author Well that seemed to have ended in a panic after following your instructions, so removed pool after stopping array. Created a new pool with the same name and 1 slot and assigned nvme01n1 to it and file system on auto.After starting the array I see the following happening (terminal window was still opened)Message from syslogd@Serenity at Mar 31 18:53:54 ...kernel:VERIFY0(zap_lookup(spa->spa_meta_objset, spa->spa_feat_enabled_txg_obj, feature->fi_guid, sizeof (uint64_t), 1, res)) failed (52)Message from syslogd@Serenity at Mar 31 18:53:54 ...kernel:PANIC at zfeature.c:294:feature_get_enabled_txg()
March 31Mar 31 Author Now I see this after the pool was imported, but in the UI it's not immediately clear that the pool is degraded? Edited March 31Mar 31 by Stiibun
April 1Apr 1 Author @JorgeB , server keeps at Array Starting after adding the pool.And taking diagnostics doesn't seem to work either as it hangs at smartctl -x '/dev/sda' 2>/dev/null|todos >'/serenity-diagnostics-20260401-0756/smart/Samsung_Flash_Drive_0355025010003213-0-0-2025-12-18 flash (sda).txt' /usr/sbin/zpool status 2>/dev/null|todos >>'/serenity-diagnostics-20260401-0756/system/zfs-info.txt'
April 1Apr 1 Community Expert The pool is possibly panininc on import.Instead of starting the array, see if you can import the pool manually read-onlyzpool import -o readonly=on cacheIf that works, look for the data in /mnt/cache and copy it somewhere else, then redo the pool.If the pool still fails to import read-only, see if you can get the output from this tail -n 100 /proc/spl/kstat/zfs/dbgmsg
April 1Apr 1 Author @JorgeB , it panics again when executing the import manuallyzfs dbgmsg logs the following;timestamp message 1775041272 ffff88811c58b300 metaslab.c:1703:spa_set_allocator(): spa allocator: dynamic1775041272 ffff88811c58b300 spa.c:6921:spa_tryimport(): spa_tryimport: importing cache1775041272 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load($import-ffff88811c58b300-cache, config trusted): LOADING1775041272 ffff8881a77fa200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': probe done, cant_read=0 cant_write=11775041273 ffff88810696c400 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme1n1p1': open error=2 timeout=1007543090/10000000001775041273 ffff88811c58b300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': best uberblock found for spa $import-ffff88811c58b300-cache. txg 34910051775041273 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load($import-ffff88811c58b300-cache, config untrusted): using uberblock with txg=34910051775041273 ffff8881a77fa200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': probe done, cant_read=0 cant_write=11775041274 ffff88819b3e0000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme1n1p1': open error=2 timeout=1008705958/10000000001775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading checkpoint txg1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading indirect vdev metadata1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Checking feature flags1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading special MOS directories1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading properties1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading AUX vdevs1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading vdev metadata1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading dedup tables1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading BRT1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Verifying Log Devices1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Verifying pool data1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Calculating deflated space1775041274 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Starting import1775041274 ffff88811c58b300 spa.c:9320:spa_async_request(): spa=$import-ffff88811c58b300-cache async request task=20481775041274 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load($import-ffff88811c58b300-cache, config trusted): LOADED1775041274 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load($import-ffff88811c58b300-cache, config trusted): UNLOADING1775041274 ffff88811c58b300 metaslab.c:1703:spa_set_allocator(): spa allocator: dynamic1775041274 ffff88811c58b300 spa.c:6766:spa_import(): spa_import: importing cache1775041274 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load(cache, config trusted): LOADING1775041274 ffff88819d183300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': probe done, cant_read=0 cant_write=11775041275 ffff888118a58000 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme1n1p1': open error=2 timeout=1004162104/10000000001775041275 ffff88811c58b300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': best uberblock found for spa cache. txg 34910051775041275 ffff88811c58b300 spa_misc.c:430:spa_load_note(): spa_load(cache, config untrusted): using uberblock with txg=34910051775041275 ffff88819d183300 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme0n1p1': probe done, cant_read=0 cant_write=11775041276 ffff8881027a6600 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/nvme1n1p1': open error=2 timeout=1000928757/10000000001775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading checkpoint txg1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading indirect vdev metadata1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Checking feature flags1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading special MOS directories1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading properties1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading AUX vdevs1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading vdev metadata1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading dedup tables1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Loading BRT1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Verifying Log Devices1775041276 ffff88811c58b300 spa_misc.c:2405:spa_import_progress_set_notes_impl(): 'cache' Verifying pool data
April 1Apr 1 Community Expert Pool may have other issues besides the missing device. You can try this to see if it can recover:zpool import -o readonly=on -f cacheIf that still fails:zpool import -o readonly=on -F -f cacheand as a last resort:zpool import -o readonly=on -X -f cache
April 4Apr 4 Author @JorgeB , to conclude this thread none of the commands you provided a solution I each time got a ZFS panic warning and needed to reboot each time. Also tried with a Ubuntu Live USB stick but to no avail.So I refreshed my USB stick as I wanted to start a fresh but now I have troubles with the registration. Will create a new post for this.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.