April 17Apr 17 Im not sure what's going on here. Not sure why it has 6 groups now and i only have 15 drives and why 1-5 only has 2 drives. Im thinking the 1-5 drive is supposed to be in 1-4. Not sure if its possible to retrieve the data at this point. I was using this setup for testing and forgot it was setup like this. Any help would be appreciated. Thanks in advance.
April 18Apr 18 I assume the pool was not created using the GUI? The last vdev is raidz1 with two faulted devices, so it won't likely be possible to recover, post the output of zdb -l /dev/sdX1 | grep -P "^\s*txg" for all pool members and also the diagnostics.
April 18Apr 18 Author It was made with the GUI. The last vdev shouldn't be possible though, correct? It only has 2 drives and you can't mix and match vdev setups? I should have 5 groups of 3 drives, i had 5 groups of 3 at one point. I only have 15 drives and those weird long numbers are not drives. Talk to me like im an idiot lol im still learning. I'm an idiot but somehow the last vdev took a drive from 1-4 idk?Maybe im doing something wrong but nothing happens with zdb -l /dev/sdX1 | grep -P "^\s*txg"I appreciate the help, thank you very much!
April 18Apr 18 3 hours ago, Abuseddoughboy said:It was made with the GUI. The last vdev shouldn't be possible though, correct?Not using the GUI, hence my question: it should never accept a smaller vdev.3 hours ago, Abuseddoughboy said:zdb -l /dev/sdX1 | grep -P "^\s*txg"You need to replace X with the correct identifier for each pool member, e.g., /edv/sdan1
April 18Apr 18 Author This was a test zfs setup when they first added it natively to unraid. It was definitely made using the gui. Not sure it is possible but is there any way to add sdaq1 back into 1-4 and get rid of 1-5. Thank you for being patient with me. This all happened after i had a power outage and i didn't realize my ups batteries were screwed so it did not shut down clean. i was also transferring a large amount of files around.
April 19Apr 19 Type zpool import -F download; it will likely still fail to import, but then grab and post the output from tail -n 100 /proc/spl/kstat/zfs/dbgmsgAlso post the diagnostics.
April 19Apr 19 Author root@Tower:~# zpool import -F downloadcannot import 'download': one or more devices is currently unavailableroot@Tower:~# tail -n 100 /proc/spl/kstat/zfs/dbgmsg1776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdao1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdak1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff88912010d280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdap1': probe done, cant_read=0 cant_write=11776608277 ffff88897d7d4200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': vdev_validate: config guid doesn't match label guid1776608277 ffff88897d7d4200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': CONFIG: guid 6473561883912491977, top_guid 85083793290528370261776608277 ffff88897d7d4200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': LABEL: guid 4236473859647311292, top_guid 7301492552058973520, aux_guid 01776608277 ffff8888ee029080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': vdev_validate: config guid doesn't match label guid1776608277 ffff8888ee029080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': CONFIG: guid 14154127591005351988, top_guid 109267246342844295831776608277 ffff8888ee029080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': LABEL: guid 1984276560438213106, top_guid 3639066115140394353, aux_guid 01776608277 ffff8888ee163180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': vdev_validate: config guid doesn't match label guid1776608277 ffff8888ee163180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': CONFIG: guid 18057109111749112359, top_guid 85083793290528370261776608277 ffff8888ee163180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': LABEL: guid 1984276560438213106, top_guid 3639066115140394353, aux_guid 01776608277 ffff88902315a100 spa_misc.c:416:spa_load_failed(): spa_load($import-ffff88902315a100-download, config trusted): FAILED: cannot open vdev tree after invalidating some vdevs1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 8980900103222356967, path: N/A, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: raidz, guid: 3639066115140394353, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 15606560977127719128, path: /dev/sdan1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 1984276560438213106, path: /dev/sdaq1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 7529842671122589782, path: /dev/sdap1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: raidz, guid: 13841255773640351550, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 12982454225254910215, path: /dev/sdac1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 16208377627330825322, path: /dev/sdal1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 11727081736508910261, path: /dev/sdah1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: raidz, guid: 7788840969503653268, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 15650727483572047387, path: /dev/sdaf1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 2657739808070973024, path: /dev/sdak1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 3061768830033961780, path: /dev/sdai1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 3: raidz, guid: 7301492552058973520, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 5133868157694348226, path: /dev/sdad1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 4842366315372101292, path: /dev/sdao1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 4236473859647311292, path: /dev/sdag1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 4: raidz, guid: 10926724634284429583, path: N/A, degraded1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 12224681384942138511, path: /dev/sdaj1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 11660946513604001104, path: /dev/sdam1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 14154127591005351988, path: /dev/sdaq1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 5: raidz, guid: 8508379329052837026, path: N/A, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 6473561883912491977, path: /dev/sdag1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 18057109111749112359, path: /dev/sdaq1, can't open1776608277 ffff88902315a100 spa_misc.c:430:spa_load_note(): spa_load($import-ffff88902315a100-download, config trusted): UNLOADING1776608277 ffff88902315a100 metaslab.c:1703:spa_set_allocator(): spa allocator: dynamic1776608277 ffff88902315a100 spa.c:6768:spa_import(): spa_import: importing download, max_txg=-1 (RECOVERY MODE)1776608277 ffff88902315a100 spa_misc.c:430:spa_load_note(): spa_load(download, config trusted): LOADING1776608277 ffff88882342c200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdac1': open error=30 timeout=13810/10000000001776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdal1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdah1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdam1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaj1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdak1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdai1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaf1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdad1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdan1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdao1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdap1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff8888ee02b180 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': probe done, cant_read=0 cant_write=11776608277 ffff889821a9c200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': vdev_validate: config guid doesn't match label guid1776608277 ffff889821a9c200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': CONFIG: guid 6473561883912491977, top_guid 85083793290528370261776608277 ffff889821a9c200 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdag1': LABEL: guid 4236473859647311292, top_guid 7301492552058973520, aux_guid 01776608277 ffff8890fef15280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': vdev_validate: config guid doesn't match label guid1776608277 ffff8890fef15280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': CONFIG: guid 18057109111749112359, top_guid 85083793290528370261776608277 ffff8890fef15280 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': LABEL: guid 1984276560438213106, top_guid 3639066115140394353, aux_guid 01776608277 ffff88904a551080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': vdev_validate: config guid doesn't match label guid1776608277 ffff88904a551080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': CONFIG: guid 14154127591005351988, top_guid 109267246342844295831776608277 ffff88904a551080 vdev.c:184:vdev_dbgmsg(): disk vdev '/dev/sdaq1': LABEL: guid 1984276560438213106, top_guid 3639066115140394353, aux_guid 01776608277 ffff88902315a100 spa_misc.c:416:spa_load_failed(): spa_load(download, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 8980900103222356967, path: N/A, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: raidz, guid: 3639066115140394353, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 15606560977127719128, path: /dev/sdan1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 1984276560438213106, path: /dev/sdaq1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 7529842671122589782, path: /dev/sdap1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: raidz, guid: 13841255773640351550, path: N/A, degraded1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 12982454225254910215, path: /dev/sdac1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 16208377627330825322, path: /dev/sdal1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 11727081736508910261, path: /dev/sdah1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: raidz, guid: 7788840969503653268, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 15650727483572047387, path: /dev/sdaf1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 2657739808070973024, path: /dev/sdak1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 3061768830033961780, path: /dev/sdai1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 3: raidz, guid: 7301492552058973520, path: N/A, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 5133868157694348226, path: /dev/sdad1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 4842366315372101292, path: /dev/sdao1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 4236473859647311292, path: /dev/sdag1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 4: raidz, guid: 10926724634284429583, path: N/A, degraded1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 12224681384942138511, path: /dev/sdaj1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 11660946513604001104, path: /dev/sdam1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 14154127591005351988, path: /dev/sdaq1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 5: raidz, guid: 8508379329052837026, path: N/A, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 6473561883912491977, path: /dev/sdag1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 18057109111749112359, path: /dev/sdaq1, can't open1776608277 ffff88902315a100 spa_misc.c:430:spa_load_note(): spa_load(download, config untrusted): UNLOADING1776608296 ffff88886ee42100 spa_history.c:330:spa_history_log_sync(): ioctl channel_program1776608296 ffff88886ee42100 spa_history.c:294:spa_history_log_sync(): command: zfs program -jn -m 20971520 zfs_10tb /usr/local/emhttp/plugins/zfs.master/scripts/zfs_pool_get_datasets_snapshots.lua zfs_10tb root@Tower:~#
April 20Apr 20 18 hours ago, Abuseddoughboy said:1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 4236473859647311292, path: /dev/sdag1, healthy1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 6473561883912491977, path: /dev/sdag1, can't open18 hours ago, Abuseddoughboy said:1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 2: disk, guid: 14154127591005351988, path: /dev/sdaq1, can't open1776608277 ffff88902315a100 vdev.c:236:vdev_dbgmsg_print_tree(): vdev 1: disk, guid: 18057109111749112359, path: /dev/sdaq1, can't openThis pool is a mess, you have multiple devices being reported as members of multiple VDEVs, post the full oputput fromzdb -l /dev/sdaq1zdb -l /dev/sdag1And On 4/19/2026 at 9:34 AM, JorgeB said:Also post the diagnostics.
April 20Apr 20 Author root@Tower:~# zdb -l /dev/sdaq1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 3639066115140394353 guid: 1984276560438213106 vdev_children: 6 vdev_tree: type: 'raidz' id: 0 guid: 3639066115140394353 nparity: 1 metaslab_array: 170 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15606560977127719128 path: '/dev/sdac1' whole_disk: 0 DTL: 1329 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 1984276560438213106 path: '/dev/sdae1' whole_disk: 0 DTL: 1328 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 7529842671122589782 path: '/dev/sdad1' whole_disk: 0 DTL: 1327 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l /dev/sdag1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7301492552058973520 guid: 4236473859647311292 vdev_children: 6 vdev_tree: type: 'raidz' id: 3 guid: 7301492552058973520 nparity: 1 metaslab_array: 155 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 5133868157694348226 path: '/dev/sdap1' whole_disk: 0 DTL: 1325 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4842366315372101292 path: '/dev/sdaf1' whole_disk: 0 DTL: 1324 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 4236473859647311292 path: '/dev/sdam1' whole_disk: 0 DTL: 1323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# tower-diagnostics-20260420-1546.zip
April 21Apr 21 There may be an issue with the pool cached config, try the follwoing:On main click on the download pool and then "remove pool"Reboot UnraidBefore trying to reimport that pool post once again, output from zpool import
April 21Apr 21 Author root@Tower:~# zpool import pool: download id: 8980900103222356967 state: UNAVAILstatus: One or more devices contains corrupted data.action: The pool cannot be imported due to damaged devices or data. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5Econfig: download UNAVAIL insufficient replicas raidz1-0 ONLINE sdan1 ONLINE sdaq1 ONLINE sdap1 ONLINE raidz1-1 ONLINE sdac1 ONLINE sdal1 ONLINE sdah1 ONLINE raidz1-2 ONLINE sdaf1 ONLINE sdak1 ONLINE sdai1 ONLINE raidz1-3 ONLINE sdad1 ONLINE sdao1 ONLINE sdag1 ONLINE raidz1-4 DEGRADED sdaj1 ONLINE sdam1 ONLINE 14154127591005351988 FAULTED corrupted data raidz1-5 UNAVAIL insufficient replicas 6473561883912491977 FAULTED corrupted data sdaq1 FAULTED corrupted data Edited April 21Apr 21 by Abuseddoughboy
April 22Apr 22 It wasn't a cache problem, post the full output from zdb -l for all the pool devices (no need to post again for sdaq1 and sdag1)
April 23Apr 23 Author root@Tower:~# zdb -l sdan1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 3639066115140394353 guid: 15606560977127719128 vdev_children: 6 vdev_tree: type: 'raidz' id: 0 guid: 3639066115140394353 nparity: 1 metaslab_array: 170 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15606560977127719128 path: '/dev/sdac1' whole_disk: 0 DTL: 1329 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 1984276560438213106 path: '/dev/sdae1' whole_disk: 0 DTL: 1328 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 7529842671122589782 path: '/dev/sdad1' whole_disk: 0 DTL: 1327 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdaq1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 3639066115140394353 guid: 1984276560438213106 vdev_children: 6 vdev_tree: type: 'raidz' id: 0 guid: 3639066115140394353 nparity: 1 metaslab_array: 170 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15606560977127719128 path: '/dev/sdac1' whole_disk: 0 DTL: 1329 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 1984276560438213106 path: '/dev/sdae1' whole_disk: 0 DTL: 1328 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 7529842671122589782 path: '/dev/sdad1' whole_disk: 0 DTL: 1327 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdap1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 3639066115140394353 guid: 7529842671122589782 vdev_children: 6 vdev_tree: type: 'raidz' id: 0 guid: 3639066115140394353 nparity: 1 metaslab_array: 170 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15606560977127719128 path: '/dev/sdac1' whole_disk: 0 DTL: 1329 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 1984276560438213106 path: '/dev/sdae1' whole_disk: 0 DTL: 1328 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 7529842671122589782 path: '/dev/sdad1' whole_disk: 0 DTL: 1327 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdac1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8425413 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 13841255773640351550 guid: 12982454225254910215 vdev_children: 5 vdev_tree: type: 'raidz' id: 1 guid: 13841255773640351550 nparity: 1 metaslab_array: 164 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12982454225254910215 path: '/dev/sdan1' whole_disk: 0 DTL: 1573 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 16208377627330825322 path: '/dev/sdah1' whole_disk: 0 DTL: 1318 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 11727081736508910261 path: '/dev/sdal1' whole_disk: 0 DTL: 1317 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdal1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 13841255773640351550 guid: 16208377627330825322 vdev_children: 6 vdev_tree: type: 'raidz' id: 1 guid: 13841255773640351550 nparity: 1 metaslab_array: 164 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12982454225254910215 path: '/dev/sdan1' whole_disk: 0 not_present: 1 DTL: 1573 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 16208377627330825322 path: '/dev/sdah1' whole_disk: 0 DTL: 1318 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 11727081736508910261 path: '/dev/sdal1' whole_disk: 0 DTL: 1317 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdah1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 13841255773640351550 guid: 11727081736508910261 vdev_children: 6 vdev_tree: type: 'raidz' id: 1 guid: 13841255773640351550 nparity: 1 metaslab_array: 164 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12982454225254910215 path: '/dev/sdan1' whole_disk: 0 not_present: 1 DTL: 1573 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 16208377627330825322 path: '/dev/sdah1' whole_disk: 0 DTL: 1318 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 11727081736508910261 path: '/dev/sdal1' whole_disk: 0 DTL: 1317 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdaf1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7788840969503653268 guid: 15650727483572047387 vdev_children: 6 vdev_tree: type: 'raidz' id: 2 guid: 7788840969503653268 nparity: 1 metaslab_array: 159 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15650727483572047387 path: '/dev/sdao1' whole_disk: 0 DTL: 1322 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2657739808070973024 path: '/dev/sdai1' whole_disk: 0 DTL: 1320 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 3061768830033961780 path: '/dev/sdak1' whole_disk: 0 DTL: 1319 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdak1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7788840969503653268 guid: 2657739808070973024 vdev_children: 6 vdev_tree: type: 'raidz' id: 2 guid: 7788840969503653268 nparity: 1 metaslab_array: 159 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15650727483572047387 path: '/dev/sdao1' whole_disk: 0 DTL: 1322 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2657739808070973024 path: '/dev/sdai1' whole_disk: 0 DTL: 1320 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 3061768830033961780 path: '/dev/sdak1' whole_disk: 0 DTL: 1319 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdai1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7788840969503653268 guid: 3061768830033961780 vdev_children: 6 vdev_tree: type: 'raidz' id: 2 guid: 7788840969503653268 nparity: 1 metaslab_array: 159 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15650727483572047387 path: '/dev/sdao1' whole_disk: 0 DTL: 1322 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 2657739808070973024 path: '/dev/sdai1' whole_disk: 0 DTL: 1320 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 3061768830033961780 path: '/dev/sdak1' whole_disk: 0 DTL: 1319 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdad1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7301492552058973520 guid: 5133868157694348226 vdev_children: 6 vdev_tree: type: 'raidz' id: 3 guid: 7301492552058973520 nparity: 1 metaslab_array: 155 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 5133868157694348226 path: '/dev/sdap1' whole_disk: 0 DTL: 1325 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4842366315372101292 path: '/dev/sdaf1' whole_disk: 0 DTL: 1324 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 4236473859647311292 path: '/dev/sdam1' whole_disk: 0 DTL: 1323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdao1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7301492552058973520 guid: 4842366315372101292 vdev_children: 6 vdev_tree: type: 'raidz' id: 3 guid: 7301492552058973520 nparity: 1 metaslab_array: 155 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 5133868157694348226 path: '/dev/sdap1' whole_disk: 0 DTL: 1325 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4842366315372101292 path: '/dev/sdaf1' whole_disk: 0 DTL: 1324 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 4236473859647311292 path: '/dev/sdam1' whole_disk: 0 DTL: 1323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdag1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 7301492552058973520 guid: 4236473859647311292 vdev_children: 6 vdev_tree: type: 'raidz' id: 3 guid: 7301492552058973520 nparity: 1 metaslab_array: 155 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 5133868157694348226 path: '/dev/sdap1' whole_disk: 0 DTL: 1325 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 4842366315372101292 path: '/dev/sdaf1' whole_disk: 0 DTL: 1324 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 4236473859647311292 path: '/dev/sdam1' whole_disk: 0 DTL: 1323 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdaj1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 10926724634284429583 guid: 12224681384942138511 vdev_children: 6 vdev_tree: type: 'raidz' id: 4 guid: 10926724634284429583 nparity: 1 metaslab_array: 149 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12224681384942138511 path: '/dev/sdaj1' whole_disk: 0 DTL: 1332 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 11660946513604001104 path: '/dev/sdag1' whole_disk: 0 DTL: 1331 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 14154127591005351988 path: '/dev/sdaq1' whole_disk: 0 not_present: 1 DTL: 1330 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdam1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 10926724634284429583 guid: 11660946513604001104 vdev_children: 6 vdev_tree: type: 'raidz' id: 4 guid: 10926724634284429583 nparity: 1 metaslab_array: 149 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 12224681384942138511 path: '/dev/sdaj1' whole_disk: 0 DTL: 1332 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 11660946513604001104 path: '/dev/sdag1' whole_disk: 0 DTL: 1331 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 14154127591005351988 path: '/dev/sdaq1' whole_disk: 0 not_present: 1 DTL: 1330 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3 root@Tower:~# zdb -l sdaq1------------------------------------LABEL 0 ------------------------------------ version: 5000 name: 'download' state: 0 txg: 8628388 pool_guid: 8980900103222356967 errata: 0 hostname: 'Tower' top_guid: 3639066115140394353 guid: 1984276560438213106 vdev_children: 6 vdev_tree: type: 'raidz' id: 0 guid: 3639066115140394353 nparity: 1 metaslab_array: 170 metaslab_shift: 34 ashift: 12 asize: 9001764126720 is_log: 0 create_txg: 4 children[0]: type: 'disk' id: 0 guid: 15606560977127719128 path: '/dev/sdac1' whole_disk: 0 DTL: 1329 create_txg: 4 children[1]: type: 'disk' id: 1 guid: 1984276560438213106 path: '/dev/sdae1' whole_disk: 0 DTL: 1328 create_txg: 4 children[2]: type: 'disk' id: 2 guid: 7529842671122589782 path: '/dev/sdad1' whole_disk: 0 DTL: 1327 create_txg: 4 features_for_read: com.delphix:hole_birth com.delphix:embedded_data com.klarasystems:vdev_zaps_v2 labels = 0 1 2 3
April 23Apr 23 Not sure how you ended up with a pool in this state, but you have a six-vdev pool structure where several physical disks are being claimed by multiple VDEVs simultaneously. This is why the standard import fails; ZFS cannot verify which VDEV a disk truly belongs to when the labels provide conflicting information.raidz1-0 (ID 0): Claims /dev/sdan1, /dev/sdaq1, and /dev/sdap1raidz1-1 (ID 1): Claims /dev/sdac1, /dev/sdal1, and /dev/sdah1raidz1-2 (ID 2): Claims /dev/sdaf1, /dev/sdak1, and /dev/sdai1raidz1-3 (ID 3): Claims /dev/sdad1, /dev/sdao1, and /dev/sdag1raidz1-4 (ID 4): Claims /dev/sdaj1 and /dev/sdam1. It specifically lists its third child (GUID 1415...) as not_present.Missing Disk: sdaq1 is the biggest point of failure. Its label says it belongs to VDEV 0, but the pool config (as seen in your first screenshot) is trying to find it in VDEV 5 (the phantom VDEV) and VDEV 4.I doubt the opoio can be imported as is, but try this as a last resort:Delete the ZFS state cache first just in case it's being used:rm /etc/zfs/zpool.cacheThen try the "Extreme" recovery flagzpool import -d /dev/disk/by-id -f -F -X downloadThis may take a long time (minutes to hours) as it traverses every disk to find a valid Uberblock.If that fails, you can try a read-only import, to see if you can get the data off the pool:zpool import -d /dev/disk/by-id -o readonly=on -f -F -X downloadIf none of those work, I'm afraid the data may be lost.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.