BigGreek19

Thank you for your advice, I really appreciate it. I just decided to take the loss as I didn't have ~60TB of empty space to copy everything to off the drives with a file recovery app.

Started the Array normally (outside of maintenance mode). I was going to just format the 2 bad drives (Drive 1 & 11) however format wants to format all 5 unmountable disks: disk1, disk5, disk10, disk11, disk12. But zpool import (scan) shows disk5, disk10, disk12 as ONLINE/healthy — only disk1 and disk11 are UNAVAIL/invalid label. When I try to import the healthy three via their md devices, I get an I/O error: root@Atlas:~# zpool import pool: disk5 id: 10072013541632102913 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk5 ONLINE md5p1 ONLINE pool: disk1 id: 3396799116815999594 state: UNAVAIL status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E config: disk1 UNAVAIL insufficient replicas md1p1 UNAVAIL invalid label pool: disk12 id: 3952630588497545706 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk12 ONLINE md12p1 ONLINE pool: disk11 id: 16557732136013927194 state: UNAVAIL status: One or more devices contains corrupted data. action: The pool cannot be imported due to damaged devices or data. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E config: disk11 UNAVAIL insufficient replicas md11p1 UNAVAIL invalid label pool: disk10 id: 12786734985925565630 state: ONLINE status: Some supported features are not enabled on the pool. (Note that they may be intentionally disabled if the 'compatibility' property is set.) action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: disk10 ONLINE md10p1 ONLINE root@Atlas:~# root@Atlas:~# root@Atlas:~# zpool import -d /dev/md5p1 disk5 cannot import 'disk5': I/O error Destroy and re-create the pool from a backup source. root@Atlas:~# zpool import -d /dev/md10p1 disk10 cannot import 'disk10': I/O error Destroy and re-create the pool from a backup source. root@Atlas:~# zpool import -d /dev/md12p1 disk12 cannot import 'disk12': I/O error Destroy and re-create the pool from a backup source. root@Atlas:~# So the scan says these 3 are ONLINE but importing via md throws I/O error. Is there a way to get disk5/10/12 to mount properly so they leave the format list, and I only format the 2 actually-dead disks (disk1, disk11)? I'm worried the array's md layer is the issue, not the disks. I truly appreciate any assistance here, this is the first time I've run into a major issue with Unraid since using it.

Thanks Jorge — makes sense. I unfortunately don't have 30TB of spare drives to clone to, and the data is re-acquirable media (Movies/Shows/Anime), so I've decided to accept the loss on disk1 and disk11 and re-download the content rather than go the dd/UFS Explorer route. What's the correct/safe procedure to bring these two disks back as fresh empty ZFS disks and get parity valid again, without disturbing the other 11 healthy disks? Do I just start the array and format the two unmountable disks, or is there a specific New Config / parity step I should follow given it's single parity and two disks?

Thanks Jorge! Ran both read-only imports via the md devices (confirmed /dev/md1p1 and /dev/md11p1 exist, array is in maintenance mode): zpool import -o readonly=on -d /dev/md1p1 disk1 → cannot import 'disk1': one or more devices is currently unavailable zpool import -o readonly=on -d /dev/md11p1 disk11 → cannot import 'disk11': one or more devices is currently unavailable zpool status → no pools availableStill fails even read-only. Given the earlier dbgmsg showed cant_read=0 but vdev_validate: failed reading config for txg 18446744073709551615 (all-ones txg = no valid uberblock found?), it looks like the labels are intact but the config/uberblock won't validate. But zdb -lu on both md devices shows all 4 labels present and 32 valid uberblocks (magic 0xbab10c on all), with a clean txg history: disk1: uberblocks up to txg 5094784, Thu Jun 25 09:22:43 2026 disk11: uberblocks up to txg 4770970, Sat Jun 13 02:51:18 2026 So labels + uberblocks read fine via zdb, but the live import rejects them on the "bad label cksum" / config validation. Full zdb -lu output for both attached. Given the uberblocks are intact, is this a candidate for an extended/rewind import (-FX or a specific -T <txg>), and if so what's the safe invocation? Single parity, both are data disks with real content disk1_zdb-lu_md1p1.txt disk11_zdb-lu_md11p1.txt

root@Atlas:~# root@Atlas:~# zpool import -d /dev/md1p1 disk1 cannot import 'disk1': one or more devices is currently unavailable root@Atlas:~# zpool import -d /dev/md11p1 disk11 cannot import 'disk11': one or more devices is currently unavailable root@Atlas:~# tail -n 50 /proc/spl/kstat/zfs/dbgmsg 1782992795 ffff888201672180 spa_misc.c:431:spa_load_note(): spa_load(disk4, config trusted): UNLOADING 1782992795 ffff888201672180 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk4 gethrtime=264161162596 1782992795 ffff8882042dd3c0 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk6 1782992796 ffff888219bf90c0 spa_misc.c:431:spa_load_note(): spa_load(disk6, config trusted): UNLOADING 1782992796 ffff888219bf90c0 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk6 gethrtime=264830350006 1782992796 ffff88811368c300 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk7 1782992796 ffff8881cc266480 spa_misc.c:431:spa_load_note(): spa_load(disk7, config trusted): UNLOADING 1782992796 ffff8881cc266480 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk7 gethrtime=265397318977 1782992796 ffff888256aa10c0 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk8 1782992797 ffff88820c966480 spa_misc.c:431:spa_load_note(): spa_load(disk8, config trusted): UNLOADING 1782992797 ffff88820c966480 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk8 gethrtime=266059340427 1782992797 ffff888107c7a180 spa_history.c:295:spa_history_log_sync(): command: zpool export -f disk9 1782992798 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk9, config trusted): UNLOADING 1782992798 ffff888219550000 mmp.c:269:mmp_thread_stop(): mmp: mmp thread stopped spa=disk9 gethrtime=267226663557 1782992924 ffff8882195553c0 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic 1782992924 ffff8882195553c0 spa.c:7553:spa_tryimport(): spa_tryimport: importing disk1 1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config trusted): LOADING 1782992924 ffff8881141353c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': probe done, cant_read=0 cant_write=1 1782992924 ffff888246c8c300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': vdev_validate: failed reading config for txg 18446744073709551615 1782992924 ffff8882195553c0 spa_misc.c:417:spa_load_failed(): spa_load(disk1, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs 1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 3396799116815999594, path: N/A, can't open 1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 16671010031764798182, path: /dev/md1p1, can't open 1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config untrusted): UNLOADING 1782992924 ffff8882195553c0 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic 1782992924 ffff8882195553c0 spa.c:7396:spa_import(): spa_import: importing disk1 1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config trusted): LOADING 1782992924 ffff8881141c53c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': probe done, cant_read=0 cant_write=1 1782992924 ffff888106fb0000 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md1p1': vdev_validate: failed reading config for txg 18446744073709551615 1782992924 ffff8882195553c0 spa_misc.c:417:spa_load_failed(): spa_load(disk1, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs 1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 3396799116815999594, path: N/A, can't open 1782992924 ffff8882195553c0 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 16671010031764798182, path: /dev/md1p1, can't open 1782992924 ffff8882195553c0 spa_misc.c:431:spa_load_note(): spa_load(disk1, config untrusted): UNLOADING 1782992941 ffff888219550000 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic 1782992941 ffff888219550000 spa.c:7553:spa_tryimport(): spa_tryimport: importing disk11 1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config trusted): LOADING 1782992941 ffff888106fb4300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': probe done, cant_read=0 cant_write=1 1782992941 ffff8881c5976480 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': vdev_validate: failed reading config for txg 18446744073709551615 1782992941 ffff888219550000 spa_misc.c:417:spa_load_failed(): spa_load(disk11, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs 1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 16557732136013927194, path: N/A, can't open 1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 8680501602283973622, path: /dev/md11p1, can't open 1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config untrusted): UNLOADING 1782992941 ffff888219550000 metaslab.c:1837:spa_set_allocator(): spa allocator: dynamic 1782992941 ffff888219550000 spa.c:7396:spa_import(): spa_import: importing disk11 1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config trusted): LOADING 1782992941 ffff8881c5974300 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': probe done, cant_read=0 cant_write=1 1782992941 ffff8881c4afd3c0 vdev.c:185:vdev_dbgmsg(): disk vdev '/dev/md11p1': vdev_validate: failed reading config for txg 18446744073709551615 1782992941 ffff888219550000 spa_misc.c:417:spa_load_failed(): spa_load(disk11, config untrusted): FAILED: cannot open vdev tree after invalidating some vdevs 1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: root, guid: 16557732136013927194, path: N/A, can't open 1782992941 ffff888219550000 vdev.c:237:vdev_dbgmsg_print_tree(): vdev 0: disk, guid: 8680501602283973622, path: /dev/md11p1, can't open 1782992941 ffff888219550000 spa_misc.c:431:spa_load_note(): spa_load(disk11, config untrusted): UNLOADING root@Atlas:~#

Unraid 7.3.1. ZFS array, 13 single-disk pools (one disk per pool) + 22TB parity. Background: My original motherboard (MSI PRO Z790-P) died. After a long saga I ended up fully rebuilding onto a new platform (Gigabyte Z890 Aorus Elite, Intel Core Ultra 7 265K, DDR5) with the same flash drive and all 13 original drives + parity. There were several forced/unclean shutdowns during troubleshooting. VMD is disabled / SATA in AHCI. Drives are on two LSI SAS9211-8i HBAs (IT mode) plus one motherboard SATA port. Current state: The server boots fine and all 13 disks are detected (green, 0 errors). 11 of 13 data disks mount normally with their data intact. Two data disks — disk1 (8TB) and disk11 (20TB) — show "Unmountable: wrong or no file system" in the GUI. What I've ruled out: I suspected a cable/HBA/port issue, so I swapped the SFF-8087 breakout cables between the two HBAs. After reboot, the same two pools (disk1, disk11) still fail — the fault followed the disks, not the cables/ports. So it's not a connection issue. zpool import shows disk1 and disk11 as: disk1 UNAVAIL insufficient replicas sdf2 UNAVAIL invalid label disk11 UNAVAIL insufficient replicas sdl2 UNAVAIL invalid label(all other 11 pools show ONLINE and import fine) Importing by pool ID fails: zpool import 3396799116815999594 → cannot import 'disk1': one or more devices is currently unavailable zpool import -o readonly=on -d /dev/sdf2 3396799116815999594 → sameBut zdb -l shows the labels are actually intact — all 4 label copies present, full vdev tree, correct pool name/GUID/size. The only issue is "Bad label cksum," and the path points to the md device: zdb -l /dev/sdf2 LABEL 0 (Bad label cksum) version: 5000 name: 'disk1' pool_guid: 3396799116815999594 ... path: '/dev/md1p1' labels = 0 1 2 3(disk11/sdl2 shows the same pattern: intact label, "Bad label cksum," path: '/dev/md11p1') My read (please correct me): the labels look intact but have a checksum mismatch, and they reference /dev/mdXp1 — i.e. these want to be assembled through the Unraid md layer, not imported as raw partitions, which is why raw zpool import says "unavailable." I suspect the bad cksum is from the unclean shutdowns. My questions: Given the labels are intact per zdb -l and only the checksum is bad, what's the correct/safe way to bring disk1 and disk11 back through the array without triggering a format or a rebuild? I have single parity and two affected disks — I know single parity can't rebuild two at once, so I want to avoid any path that treats these as failed and tries to format/rebuild them. What's the right sequence? Would starting the array normally assemble these via /dev/mdXp1 and resolve the checksum (or let a scrub repair it from the 3 good label copies), or is there a specific import step first? I have NOT started the array, formatted, or run any -f / labelclear / create — holding until I get the right procedure. Diagnostics zip attached. Thanks in advance. atlas-diagnostics-20260701-2317.zip

Hello! So the server is deployed, but when anybody tries to connect to it they get an error with version mismatch: [20:31:48] [Server thread/INFO] [minecraft/ServerConfigurationPacketListenerImpl]: Big_Greek19 () lost connection: Incompatible client! Please use NeoForge 26.1.2.30-beta [20:31:48] [Server thread/WARN] [minecraft/Connection]: handleDisconnection() called twice [20:32:51] [User Authenticator #2/INFO] [minecraft/ServerLoginPacketListenerImpl]: UUID of player eyeopening is [20:32:51] [Server thread/INFO] [minecraft/ServerConfigurationPacketListenerImpl]: eyeopening () lost connection: Incompatible client! Please use NeoForge 26.1.2.30-beta [20:32:51] [Server thread/WARN] [minecraft/Connection]: handleDisconnection() called twice Latest from Curseforge is 26.1.2.42. When trying to install the old .30 version it still hits the same mismatch error because it is not a '-beta'. Any idea why this is happening? Is the server not pulling the latest from Curseforge?