April 3, 20251 yr I have a ZFS pool that migrated first from TrueNAS to Proxmox, and now to Unraid, problem is that despite having same hardware, I can import the zfs pool just fine manually and with webui, but if I try to export it, that export hangs forever until rebooted. When WebUI is trying to do that, the array start hangs indefinitely. There is an odd error in console, but it references devices I can't trace to specific disks... Apr 3 02:58:56 SERVER emhttpd: shcmd (145): /usr/sbin/zpool import -f -m -N -d /dev/sdg1 -d /dev/sdi1 -d /dev/sdm1 -d /dev/sdk1 -d /dev/sdf1 -d /dev/sdl1 -d /dev/sdh1 -d /dev/sdj1 5996407494583481397 main Apr 3 02:59:05 SERVER kernel: zd32: p1 p2 p3 Apr 3 02:59:05 SERVER kernel: zd48: p1 p2 p3 Apr 3 02:59:05 SERVER kernel: device-mapper: ioctl: 4.48.0-ioctl (2023-03-01) initialised: [email protected] Apr 3 02:59:06 SERVER kernel: zd64: p1 p2 p3 p4 Apr 3 02:59:07 SERVER kernel: zd112: p1 p2 p3 p4 Apr 3 02:59:07 SERVER kernel: zd128: p1 p2 p3 p4 Apr 3 02:59:07 SERVER kernel: zd144: p1 p2 p3 Apr 3 02:59:07 SERVER emhttpd: main: zfs verify devices Apr 3 02:59:07 SERVER emhttpd: /usr/sbin/zpool status -P main 2>&1 Apr 3 02:59:07 SERVER emhttpd: pool: main Apr 3 02:59:07 SERVER emhttpd: state: ONLINE Apr 3 02:59:07 SERVER emhttpd: status: Some supported and requested features are not enabled on the pool. Apr 3 02:59:07 SERVER emhttpd: #011The pool can still be used, but some features are unavailable. Apr 3 02:59:07 SERVER emhttpd: action: Enable all features using 'zpool upgrade'. Once this is done, Apr 3 02:59:07 SERVER emhttpd: #011the pool may no longer be accessible by software that does not support Apr 3 02:59:07 SERVER emhttpd: #011the features. See zpool-features(7) for details. Apr 3 02:59:07 SERVER emhttpd: scan: scrub repaired 0B in 13:00:47 with 0 errors on Sun Mar 9 13:24:49 2025 Apr 3 02:59:07 SERVER emhttpd: config: Apr 3 02:59:07 SERVER emhttpd: #011NAME STATE READ WRITE CKSUM Apr 3 02:59:07 SERVER emhttpd: #011main ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 raidz1-0 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdg1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdi1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdm1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdk1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 raidz1-1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdf1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdl1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdh1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: #011 /dev/sdj1 ONLINE 0 0 0 Apr 3 02:59:07 SERVER emhttpd: errors: No known data errors Apr 3 02:59:07 SERVER emhttpd: main: zfs assign devices Apr 3 02:59:07 SERVER emhttpd: shcmd (146): /usr/sbin/zpool export -f main Apr 3 02:59:07 SERVER kernel: Buffer I/O error on dev zd144p3, logical block 128295120, async page read Apr 3 02:59:07 SERVER kernel: Buffer I/O error on dev zd144p1, logical block 256, async page read Apr 3 02:59:07 SERVER kernel: Buffer I/O error on dev zd144p2, logical block 32, async page read That zd144 is definitely related to the zpool, but it seems that since it tried to export it, the mapping is already removed by the time I want to see what it is. p.s. I also see a lot of udma crc errors, so probably cabling, but I've already swapped cables from backplane expander to jbod edge, out of jbod to server, and hba in the server, still looking into this... I also run a memtest with no errors. server-diagnostics-20250403-0305.zip Edited April 3, 20251 yr by Auxilium fixed log formatting
April 3, 20251 yr Author For zd144, mounted the pool outside of webui, and I can see it looks like a 512GB size volume, zd144 230:144 0 512G 0 disk ├─zd144p1 230:145 0 244M 0 part ├─zd144p2 230:146 0 22.4G 0 part └─zd144p3 230:147 0 489.4G 0 part I have only one volume of that size /mnt/disks/main/system filesystem none - - volume - 120G - volume - 1M - volume - 512G - volume - 240G - volume - 1M main/system 405G 8.79T 126G /mnt/disks/main/system main/system/Windows10-sz7kgd 159G 8.92T 26.8G - main/system/vm-101-disk-0 320K 8.79T 250K - main/system/vm-101-disk-1 74.5G 8.79T 74.5G - main/system/vm-106-disk-0 45.2G 8.79T 35.7G - main/system/vm-106-disk-1 291K 8.79T 221K - Which seems to be vm-101-disk-1 one, I haven't used it so deleted it now, will see if it avoids that last error. the zfs export still hangs, will leave it overnight...
April 3, 20251 yr Community Expert Those are zvols, but not they should interfere with a pool mount, if they are OK.
April 3, 20251 yr Author As of now, `/usr/sbin/zpool export -f main` is the issue, whether it's done manually or by unraid, it hangs at this point and there's no errors at `/proc/spl/kstat/zfs/dbgmsg` 1743673194 ffff8890506cc100 spa_history.c:293:spa_history_log_sync(): command: zpool export -f main 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1536, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817053 ms ago, max_size 4020535296 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1345, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 674740 ms ago, max_size 4294909952 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1538, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817113 ms ago, max_size 2162425856 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1540, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817082 ms ago, max_size 2922012672 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1542, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817048 ms ago, max_size 2204794880 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1544, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817041 ms ago, max_size 2284568576 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1545, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817060 ms ago, max_size 4294901760 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1548, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 817034 ms ago, max_size 2344198144 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1551, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 816986 ms ago, max_size 2158493696 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1553, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 816940 ms ago, max_size 2284036096 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 0, ms_id 1554, weight 7c0000000000001, selected txg 16729142 (604251 ms ago), alloc_txg 0, loaded 719064 ms ago, max_size 2163531776 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1333, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817176 ms ago, max_size 4294909952 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1334, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817193 ms ago, max_size 4294033408 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1337, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817058 ms ago, max_size 4294934528 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1338, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817041 ms ago, max_size 4294868992 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1339, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817131 ms ago, max_size 4294942720 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1340, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817079 ms ago, max_size 4294959104 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1341, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817044 ms ago, max_size 4294868992 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1342, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817130 ms ago, max_size 4294803456 1743673793 ffff8890506cc100 metaslab.c:2615:metaslab_unload(): metaslab_unload: txg 16729261, spa main, vdev_id 1, ms_id 1343, weight 7c0000000000001, selected txg 16729142 (604252 ms ago), alloc_txg 0, loaded 817142 ms ago, max_size 4294950912
April 3, 20251 yr Community Expert If it also hangs when doing a manual zpool export, it suggests it's not an Unraid issue, which Openzfs release were you using with TrueNAS? Looks like the pool was not upgrade to 2.3, so might be worth a try with Unraid 7.0 (openzfs 2.2), but try only importing/exporting using the CLI for now.
May 2, 20251 yr Author So far I've experimented in Ubuntu livecd, and it has similar behavior where I can't export that pool (resource busy), if I import it as readonly=on, I can export it just fine, if I import and export the zfs cache pool that I created from within unraid, it works without any issues. Pretty sure something is wrong on ZFS side at this point, the only questions would be what exactly for unraid does the export as part of the import routine, and if it is doing the export, can it perhaps do the initial import with -o readonly=on? Edited May 2, 20251 yr by Auxilium
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.