Unraid 7.3.1. ZFS array, 13 single-disk pools (one disk per pool) + 22TB parity.

Background: My original motherboard (MSI PRO Z790-P) died. After a long saga I ended up fully rebuilding onto a new platform (Gigabyte Z890 Aorus Elite, Intel Core Ultra 7 265K, DDR5) with the same flash drive and all 13 original drives + parity. There were several forced/unclean shutdowns during troubleshooting. VMD is disabled / SATA in AHCI. Drives are on two LSI SAS9211-8i HBAs (IT mode) plus one motherboard SATA port.

Current state: The server boots fine and all 13 disks are detected (green, 0 errors). 11 of 13 data disks mount normally with their data intact. Two data disks — disk1 (8TB) and disk11 (20TB) — show "Unmountable: wrong or no file system" in the GUI.

What I've ruled out: I suspected a cable/HBA/port issue, so I swapped the SFF-8087 breakout cables between the two HBAs. After reboot, the same two pools (disk1, disk11) still fail — the fault followed the disks, not the cables/ports. So it's not a connection issue.

zpool import shows disk1 and disk11 as:

disk1   UNAVAIL  insufficient replicas
  sdf2  UNAVAIL  invalid label
disk11  UNAVAIL  insufficient replicas
  sdl2  UNAVAIL  invalid label

(all other 11 pools show ONLINE and import fine)

Importing by pool ID fails:

zpool import 3396799116815999594   → cannot import 'disk1': one or more devices is currently unavailable
zpool import -o readonly=on -d /dev/sdf2 3396799116815999594   → same

But zdb -l shows the labels are actually intact — all 4 label copies present, full vdev tree, correct pool name/GUID/size. The only issue is "Bad label cksum," and the path points to the md device:

zdb -l /dev/sdf2
LABEL 0 (Bad label cksum)
    version: 5000
    name: 'disk1'
    pool_guid: 3396799116815999594
    ...
    path: '/dev/md1p1'
    labels = 0 1 2 3

(disk11/sdl2 shows the same pattern: intact label, "Bad label cksum," path: '/dev/md11p1')

My read (please correct me): the labels look intact but have a checksum mismatch, and they reference /dev/mdXp1 — i.e. these want to be assembled through the Unraid md layer, not imported as raw partitions, which is why raw zpool import says "unavailable." I suspect the bad cksum is from the unclean shutdowns.

My questions:

1. Given the labels are intact per zdb -l and only the checksum is bad, what's the correct/safe way to bring disk1 and disk11 back through the array without triggering a format or a rebuild?

2. I have single parity and two affected disks — I know single parity can't rebuild two at once, so I want to avoid any path that treats these as failed and tries to format/rebuild them. What's the right sequence?

3. Would starting the array normally assemble these via /dev/mdXp1 and resolve the checksum (or let a scrub repair it from the 3 good label copies), or is there a specific import step first?

I have NOT started the array, formatted, or run any -f / labelclear / create — holding until I get the right procedure. Diagnostics zip attached. Thanks in advance.

atlas-diagnostics-20260701-2317.zip

3 hours ago, BigGreek19 said:

the labels look intact but have a checksum mismatch, and they reference /dev/mdXp1 — i.e. these want to be assembled through the Unraid md layer, not imported as raw partitions,

This should not make any difference, but you can try starting the array in maintenance mode and importing using the pool using the md device, if it still fails, and most likely will, post the output from

tail -n 50 /proc/spl/kstat/zfs/dbgmsg

5 hours ago, JorgeB said:

This should not make any difference, but you can try starting the array in maintenance mode and importing using the pool using the md device, if it still fails, and most likely will, post the output from

tail -n 50 /proc/spl/kstat/zfs/dbgmsg

root@Atlas:~#

root@Atlas:~# zpool import -d /dev/md1p1 disk1

cannot import 'disk1': one or more devices is currently unavailable

root@Atlas:~# zpool import -d /dev/md11p1 disk11

cannot import 'disk11': one or more devices is currently unavailable

root@Atlas:~# tail -n 50 /proc/spl/kstat/zfs/dbgmsg

1782992795 ffff888201672180 spa_misc.c:431:spa_load_note(): spa_load(disk4, config trusted): UNLOADING

1782992795 ffff888201672180 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk4 gethrtime=264161162596

1782992795 ffff8882042dd3c0 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk6

1782992796 ffff888219bf90c0 spa_misc.c:431:spa_load_note(): spa_load(disk6, config trusted): UNLOADING

1782992796 ffff888219bf90c0 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk6 gethrtime=264830350006

1782992796 ffff88811368c300 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk7

1782992796 ffff8881cc266480 spa_misc.c:431:spa_load_note(): spa_load(disk7, config trusted): UNLOADING

1782992796 ffff8881cc266480 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk7 gethrtime=265397318977

1782992796 ffff888256aa10c0 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk8

1782992797 ffff88820c966480 spa_misc.c:431:spa_load_note(): spa_load(disk8, config trusted): UNLOADING

1782992797 ffff88820c966480 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk8 gethrtime=266059340427

1782992797 ffff888107c7a180 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk9

1782992798 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk9, config trusted): UNLOADING

1782992798 ffff888219550000 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk9 gethrtime=267226663557

1782992924 ffff8882195553c0 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic

1782992924 ffff8882195553c0 spa.c:7553:spa_tryimport(): spa_tryimport: importing disk1

1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config trusted): LOADING

1782992924 ffff8881141353c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': probe done, cant_read=0 cant_write=1

1782992924 ffff888246c8c300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': vdev_validate: failed reading config for txg 18446744073709551615

1782992924 ffff8882195553c0 spa_misc.c:417:spa_load_failed(): spa_load(disk1, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs

1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 3396799116815999594, path: N/A, can't open

1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 16671010031764798182, path: /dev/md1p1, can't open

1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config untrusted): UNLOADING

1782992924 ffff8882195553c0 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic

1782992924 ffff8882195553c0 spa.c:7396:spa_import(): spa_import: importing disk1

1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config trusted): LOADING

1782992924 ffff8881141c53c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': probe done, cant_read=0 cant_write=1

1782992924 ffff888106fb0000 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': vdev_validate: failed reading config for txg 18446744073709551615

1782992924 ffff8882195553c0 spa_misc.c:417:spa_load_failed(): spa_load(disk1, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs

1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 3396799116815999594, path: N/A, can't open

1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 16671010031764798182, path: /dev/md1p1, can't open

1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config untrusted): UNLOADING

1782992941 ffff888219550000 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic

1782992941 ffff888219550000 spa.c:7553:spa_tryimport(): spa_tryimport: importing disk11

1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config trusted): LOADING

1782992941 ffff888106fb4300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': probe done, cant_read=0 cant_write=1

1782992941 ffff8881c5976480 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': vdev_validate: failed reading config for txg 18446744073709551615

1782992941 ffff888219550000 spa_misc.c:417:spa_load_failed(): spa_load(disk11, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs

1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 16557732136013927194, path: N/A, can't open

1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 8680501602283973622, path: /dev/md11p1, can't open

1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config untrusted): UNLOADING

1782992941 ffff888219550000 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic

1782992941 ffff888219550000 spa.c:7396:spa_import(): spa_import: importing disk11

1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config trusted): LOADING

1782992941 ffff8881c5974300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': probe done, cant_read=0 cant_write=1

1782992941 ffff8881c4afd3c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': vdev_validate: failed reading config for txg 18446744073709551615

1782992941 ffff888219550000 spa_misc.c:417:spa_load_failed(): spa_load(disk11, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs

1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 16557732136013927194, path: N/A, can't open

1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 8680501602283973622, path: /dev/md11p1, can't open

1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config untrusted): UNLOADING

root@Atlas:~#

2 hours ago, BigGreek19 said:

cant_write=1

Both show this, mening zfs cannpt write to them, try a read-only import

zpool import -o readonly=on -d /dev/md1p1 disk1

zpool import -o readonly=on -d /dev/md11p1 disk11

2 hours ago, JorgeB said:

Both show this, mening zfs cannpt write to them, try a read-only import

zpool import -o readonly=on -d /dev/md1p1 disk1

zpool import -o readonly=on -d /dev/md11p1 disk11

Thanks Jorge! Ran both read-only imports via the md devices (confirmed /dev/md1p1 and /dev/md11p1 exist, array is in maintenance mode):

zpool import -o readonly=on -d /dev/md1p1 disk1     → cannot import 'disk1': one or more devices is currently unavailable
zpool import -o readonly=on -d /dev/md11p1 disk11   → cannot import 'disk11': one or more devices is currently unavailable
zpool status → no pools available

Still fails even read-only. Given the earlier dbgmsg showed cant_read=0 but vdev_validate: failed reading config for txg 18446744073709551615 (all-ones txg = no valid uberblock found?), it looks like the labels are intact but the config/uberblock won't validate.
But zdb -lu on both md devices shows all 4 labels present and 32 valid uberblocks (magic 0xbab10c on all), with a clean txg history:

• disk1: uberblocks up to txg 5094784, Thu Jun 25 09:22:43 2026

• disk11: uberblocks up to txg 4770970, Sat Jun 13 02:51:18 2026

So labels + uberblocks read fine via zdb, but the live import rejects them on the "bad label cksum" / config validation. Full zdb -lu output for both attached.

Given the uberblocks are intact, is this a candidate for an extended/rewind import (-FX or a specific -T <txg>), and if so what's the safe invocation? Single parity, both are data disks with real content

disk1_zdb-lu_md1p1.txt disk11_zdb-lu_md11p1.txt

22 minutes ago, BigGreek19 said:

is this a candidate for an extended/rewind import (-FX or a specific -T <txg>),

It's worth a try, but in my experience, if it doesn't mount read-only, most times this requires a pool destroy and restore.

If you don't have backups, I recommend cloning the disk with dd before proceeding so that you also have the option to use a file recovery app like UFS Explorer. For a single disk zfs pool, the standard cheaper version may be able to recover the data, and the free trial should show if it can recover.

47 minutes ago, JorgeB said:

It's worth a try, but in my experience, if it doesn't mount read-only, most times this requires a pool destroy and restore.

If you don't have backups, I recommend cloning the disk with dd before proceeding so that you also have the option to use a file recovery app like UFS Explorer. For a single disk zfs pool, the standard cheaper version may be able to recover the data, and the free trial should show if it can recover.

Thanks Jorge — makes sense. I unfortunately don't have 30TB of spare drives to clone to, and the data is re-acquirable media (Movies/Shows/Anime), so I've decided to accept the loss on disk1 and disk11 and re-download the content rather than go the dd/UFS Explorer route.

What's the correct/safe procedure to bring these two disks back as fresh empty ZFS disks and get parity valid again, without disturbing the other 11 healthy disks? Do I just start the array and format the two unmountable disks, or is there a specific New Config / parity step I should follow given it's single parity and two disks?

Started the Array normally (outside of maintenance mode). I was going to just format the 2 bad drives (Drive 1 & 11) however format wants to format all 5 unmountable disks: disk1, disk5, disk10, disk11, disk12. But zpool import (scan) shows disk5, disk10, disk12 as ONLINE/healthy — only disk1 and disk11 are UNAVAIL/invalid label.

When I try to import the healthy three via their md devices, I get an I/O error:

root@Atlas:~# zpool import
  pool: disk5
    id: 10072013541632102913
 state: ONLINE
status: Some supported features are not enabled on the pool.
	(Note that they may be intentionally disabled if the
	'compatibility' property is set.)
action: The pool can be imported using its name or numeric identifier, though
	some features will not be available without an explicit 'zpool upgrade'.
config:

	disk5       ONLINE
	  md5p1     ONLINE

  pool: disk1
    id: 3396799116815999594
 state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

	disk1       UNAVAIL  insufficient replicas
	  md1p1     UNAVAIL  invalid label

  pool: disk12
    id: 3952630588497545706
 state: ONLINE
status: Some supported features are not enabled on the pool.
	(Note that they may be intentionally disabled if the
	'compatibility' property is set.)
action: The pool can be imported using its name or numeric identifier, though
	some features will not be available without an explicit 'zpool upgrade'.
config:

	disk12      ONLINE
	  md12p1    ONLINE

  pool: disk11
    id: 16557732136013927194
 state: UNAVAIL
status: One or more devices contains corrupted data.
action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
config:

	disk11      UNAVAIL  insufficient replicas
	  md11p1    UNAVAIL  invalid label

  pool: disk10
    id: 12786734985925565630
 state: ONLINE
status: Some supported features are not enabled on the pool.
	(Note that they may be intentionally disabled if the
	'compatibility' property is set.)
action: The pool can be imported using its name or numeric identifier, though
	some features will not be available without an explicit 'zpool upgrade'.
config:

	disk10      ONLINE
	  md10p1    ONLINE
root@Atlas:~# 
root@Atlas:~# 
root@Atlas:~# zpool import -d /dev/md5p1 disk5
cannot import 'disk5': I/O error
	Destroy and re-create the pool from
	a backup source.
root@Atlas:~# zpool import -d /dev/md10p1 disk10
cannot import 'disk10': I/O error
	Destroy and re-create the pool from
	a backup source.
root@Atlas:~# zpool import -d /dev/md12p1 disk12
cannot import 'disk12': I/O error
	Destroy and re-create the pool from
	a backup source.
root@Atlas:~# 

So the scan says these 3 are ONLINE but importing via md throws I/O error. Is there a way to get disk5/10/12 to mount properly so they leave the format list, and I only format the 2 actually-dead disks (disk1, disk11)? I'm worried the array's md layer is the issue, not the disks.

I truly appreciate any assistance here, this is the first time I've run into a major issue with Unraid since using it.

Something weird happened there to corrupt so many filesystems, likely a hardware issue.

You can try the read-only option, or even -FX as a last resort, but possibly the only option is also using a file recovery app or destroy and recreate.

On 7/3/2026 at 2:47 AM, JorgeB said:

Something weird happened there to corrupt so many filesystems, likely a hardware issue.

You can try the read-only option, or even -FX as a last resort, but possibly the only option is also using a file recovery app or destroy and recreate.

Thank you for your advice, I really appreciate it. I just decided to take the loss as I didn't have ~60TB of empty space to copy everything to off the drives with a file recovery app.