Skip to content
View in the app

A better way to browse. Learn more.

Unraid

A full-screen app on your home screen with push notifications, badges and more.

To install this app on iOS and iPadOS
  1. Tap the Share icon in Safari
  2. Scroll the menu and tap Add to Home Screen.
  3. Tap Add in the top-right corner.
To install this app on Android
  1. Tap the 3-dot menu (⋮) in the top-right corner of the browser.
  2. Tap Add to Home screen or Install app.
  3. Confirm by tapping Install.

Unraid 7.3.4 - Wrong Pool State - Invalid Config

Featured Replies

Hello,

I'm hoping someone here can help.

So I have a couple of zfs pools. One in particular (named Mediafun) is made up of 13 devices. I had some errors show up on one of them, Device 12 (sdn). I kept using it until the system removed the disk.

I ran a quick SMART test on that disk that came back with no errors. I tried to reintroduce it, and the resilver process started but was moving extremely slow (0.3%, 70 days to go) and didn't run for too long before the drive was shown as removed again by the system and seemed to stop the resilver process.

In the meantime I had a second drive, Device 5 (sdt), that started stating it had errors. I decided to shut down the server and check to see if the same breakout cable was connected to both drives as I have had loose/bad cables in the past. Device 12 (that was previously removed by the system) is connected to a breakout cable and Device 5 has a sata cable connected directly to a Sata port on the motherboard. I re-seated only those two sata cables and powered up the server.

Now all drives are showing as offline and when I try to start the array I'm receiving "Wrong Pool State mediafun - invalid config". Device 12 (sde) still shows as unassigned.

I've tried rebooting as well as assigning the drive that was removed (Device 12) back into that same slot. I'm wondering if stopping it during the resilver process was problematic as I'm seeing "sdn1/old" as OFFLINE and "sdn1" as UNAVAIL when I run zpool import.

Any suggestions would be welcome.

*******************************

I ran lsblk -o name,size,model,serial

-Result was all the drives are seen.

When I run "zpool import" I see the below:

pool: mediafun

id: 2954754274543482954

state: DEGRADED

status: One or more devices were being resilvered.

action: The pool can be imported despite missing or damaged devices. The

fault tolerance of the pool may be compromised if imported.

config:

mediafun DEGRADED

raidz2-0 DEGRADED

sde1 ONLINE

sdf1 ONLINE

sda1 ONLINE

sds1 ONLINE

sdt1 ONLINE

sdb1 ONLINE

sdd1 ONLINE

sdj1 ONLINE

sdh1 ONLINE

sdk1 ONLINE

sdo1 ONLINE

replacing-11 UNAVAIL insufficient replicas

sdn1/old OFFLINE

sdn1 UNAVAIL

sdm1 FAULTED corrupted data

When I run zpool status -v mediafun I receive the following:

pool: mediafun

state: DEGRADED

status: One or more devices is currently being resilvered. The pool will

continue to function, possibly in a degraded state.

action: Wait for the resilver to complete.

scan: resilver in progress since Mon Oct 26 07:52:13 2026

0B / 108T scanned, 0B / 108T issued

0B resilvered, 0.00% done, no estimated completion time

config:

NAME STATE READ WRITE CKSUM

mediafun DEGRADED 0 0 0

raidz2-0 DEGRADED 0 0 0

sde1 ONLINE 0 0 4

sdf1 ONLINE 0 0 4

sda1 ONLINE 0 0 4

sds1 ONLINE 0 0 4

sdt1 ONLINE 0 0 4

sdb1 ONLINE 0 0 4

sdd1 ONLINE 0 0 4

sdj1 ONLINE 0 0 4

sdh1 ONLINE 0 0 4

sdk1 ONLINE 0 0 4

sdo1 ONLINE 0 0 4

replacing-11 UNAVAIL 0 0 0 insufficient replicas

sdn1/old OFFLINE 0 0 0

sdn1 REMOVED 0 0 0

7355020672882961856 FAULTED 0 0 0 was /dev/sdm1

I've been trying to work through Claude but things are getting pretty in the weeds for me. Part of what its come back with is it thinks that Device 13 (sdm) somehow had it's label changed so it's GUID is different than the GUID ZFS is expecting. I'm still unsure why anything for that drive may have been changed.

In it's words:

Two different records are involved.

  • The pool’s config (zdb -e -C mediafun) says position 12 is a disk with guid 7355020672882961856 at /dev/sdm1. This is what ZFS expects, and it’s the value we want. The config is read from the labels of the healthy disks, so it is intact. It also shows that the attempted replace on sdm1 never made it into the pool’s config.

  • The label on the disk itself (zdb -l /dev/sdm1) says guid: 8450324955016200314. That is the stub label that overwrote the original.

When ZFS opens sdm1, it compares the guid in the disk’s own label to the guid the config expects. 8450324955016200314 isn’t 7355020672882961856, so ZFS decides this isn’t the right disk and marks it FAULTED with “corrupted data”. The disk is fine. Its label is simply wrong.

This is the mismatch the reconstruction would fix. The goal is to make sdm1’s label say 7355020672882961856, so it matches what the config already says.

I realize this is a dump of info, let me know if I can clarify anything.

Diagnostics are attached.

Screenshot 2026-10-26 125858.png

Screenshot 2026-10-26 125740.png

mediaunraid-diagnostics-20261026-1226.zip

Edited by khior
More information and clarification

Solved by JorgeB

  • khior changed the title to Unraid 7.3.4 - Wrong Pool State - Invalid Config

The pool should still be recoverable. 11 of the 13 devices are fine and raidz2 can survive two missing devices, but to reimport the the pool using the GUI is best to clear the repoacling line.

With the pool imported using the CLI, post the ouptu from zpool status -g mediafun

  • Author

Running zpool status -g mediafun returns the following:

pool: mediafun

state: DEGRADED

status: One or more devices is currently being resilvered. The pool will

continue to function, possibly in a degraded state.

action: Wait for the resilver to complete.

scan: resilver in progress since Mon Oct 26 07:52:13 2026

0B / 108T scanned, 0B / 108T issued

0B resilvered, 0.00% done, no estimated completion time

config:

NAME STATE READ WRITE CKSUM

mediafun DEGRADED 0 0 0

7917958360641739530 DEGRADED 0 0 0

12441589783024071164 ONLINE 0 0 4

6719155247144394418 ONLINE 0 0 4

7709659236461100366 ONLINE 0 0 4

11888992234242972831 ONLINE 0 0 4

456132895116674593 ONLINE 0 0 4

1778687606147317910 ONLINE 0 0 4

3065735978438110781 ONLINE 0 0 4

8405909615551463313 ONLINE 0 0 4

15925004999378470692 ONLINE 0 0 4

14978431327348962353 ONLINE 0 0 4

5776364110155512343 ONLINE 0 0 4

7997955580701256598 UNAVAIL 0 0 0 insufficient replicas

4734395572079804081 OFFLINE 0 0 0

16037769206436712332 REMOVED 0 0 0

7355020672882961856 FAULTED 0 0 0 was /dev/sdm1

  • Author

I had previously tried the following command based on some earlier searches:
zpool import -o readonly=on -R /mnt/rescue mediafun

My understanding is it imported it in a read-only state to an alternate mount point. I'm guessing I would need to reverse that as well?

Also, when I run "zpool list" I only see mediafun although I have a couple other pools. To see those other pools I'd need to import those also?:

NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT

mediafun 213T 0 213T - - 0% 0% 1.00x DEGRADED /mnt/rescue

Thanks. In the replacing vdev, 16037769206436712332 is the new device that was being resilvered, and it has no data yet. Detaching it cancels the replacement. Device 12 will then show as one missing member instead of the "replacing" entry, which should let the GUI better import the pool.

The pool is currently imported read-only, and a detach needs write access. So first export it, then import it again without the read-only option. The -N option means the datasets are not mounted:

zpool export mediafun

zpool import -N mediafun

zpool detach mediafun 16037769206436712332

If any of these commands gives an error, stop and post the full output. Otherwise post the output of:

zpool status mediafun

Use the normal command this time, not -g. Leave the pool as it is after that, and don't run any other commands on it. I'll post the next steps for the GUI import.

About your other questions:

- Yes, the read-only import to /mnt/rescue needs to be undone. The zpool export above does that.

- zpool list only shows pools that are imported. Your other pools are normally imported by Unraid when the array starts, and the array hasn't started because of this pool. Don't import them manually. They will be imported when the array starts.

- Don't try to rebuild or edit the label on Device 13. Writing labels by hand can make things much worse. The data on that disk will be rebuilt by a resilver later, the same as Device 12.

- Until both devices are resilvered, the pool has no redundancy left. If you have important data on it that isn't backed up anywhere, copy it off as soon as the pool is mounted again.

  • Author

Thank you very much. All commands ran successfully.

The output of zpool status mediafun is as follows:

pool: mediafun

state: DEGRADED

status: One or more devices could not be used because the label is missing or

invalid. Sufficient replicas exist for the pool to continue

functioning in a degraded state.

action: Replace the device using 'zpool replace'.

see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-4J

scan: scrub in progress since Tue Oct 27 10:54:23 2026

2.41T / 108T scanned at 154G/s, 0B / 108T issued

0B repaired, 0.00% done, no estimated completion time

config:

NAME STATE READ WRITE CKSUM

mediafun DEGRADED 0 0 0

raidz2-0 DEGRADED 0 0 0

sde1 ONLINE 0 0 0

sdf1 ONLINE 0 0 0

sda1 ONLINE 0 0 0

sds1 ONLINE 0 0 0

sdt1 ONLINE 0 0 0

sdb1 ONLINE 0 0 0

sdd1 ONLINE 0 0 0

sdj1 ONLINE 0 0 0

sdh1 ONLINE 0 0 0

sdk1 ONLINE 0 0 0

sdo1 ONLINE 0 0 0

sdn1 OFFLINE 0 0 0

7355020672882961856 FAULTED 0 0 0 was /dev/sdm1

Good, the replacing entry is gone. The pool now shows 11 online members plus the two missing ones, which is what we wanted.

First, export the pool again:

zpool export mediafun

Then import it using the GUI with only the 11 online devices:

- on Main click on the first device for that pool and then "Remove pool"

- back on Main, create a new pool with the same name (mediafun) and 11 slots

- assign the 11 online devices, and leave the filesystem set to auto

- start the array to import the pool

The pool will import degraded, and that is expected. If you see a warning that data on a device will be overwritten, or any other error, don't start the array. Post a screenshot instead.

Once the array is started, post new diagnostics so we can confirm all looks good before the next step, same if the poo doesn't import.

Until the two missing devices are resilvered the pool has no redundancy. If there is anything important on it that isn't backed up, it's a good idea to copy it somewhere else now.

If the diagnostics confirm everything is OK, stop the array, assign both missing devices at the same time, and start the array to begin the resilver. Before that, replace or reseat the cable for Device 12. Its SMART report shows CRC errors, and those usually point to a connection problem.

  • Author

Ok, I guess the only questions I have are:

  1. The original pool had 13 slots with a zfs filesystem, Raid Z-2. I thought the amount of disks chosen for a ZFS RAID pool are set and I couldn't add slots afterward once the pool is created. If I create a new pool with the same name and only 11 slots, will I be able to add both devices?

  2. I had an issue last year where I had to recreate a pool due to an invalid configuration. In that scenario I believe it was important to keep the disks in exactly the same slots they were in previously for each pool as I was using the "New Config" option. Is that important here?

26 minutes ago, khior said:

The original pool had 13 slots with a zfs filesystem, Raid Z-2. I thought the amount of disks chosen for a ZFS RAID pool are set and I couldn't add slots afterward once the pool is created. If I create a new pool with the same name and only 11 slots, will I be able to add both devices?

Yes, the pool should be imported degraded the two missing devices will show with a red x, so still 13 in total, also, and FYI, raidz pools can now be expanded, but that's a different issue.

27 minutes ago, khior said:

I had an issue last year where I had to recreate a pool due to an invalid configuration. In that scenario I believe it was important to keep the disks in exactly the same slots they were in previously for each pool as I was using the "New Config" option. Is that important here?

Not necessary with pools and the current releases; Unraid auto sorts the devices during import.

  • Author

Ok, it seems to have started, got a message while it was starting that "Mediafun 13 is in an error state". It now looks like this:

image.png

Diags attached

mediaunraid-diagnostics-20261027-1224.zip

Edited by khior

Post new diags please, but I'm about to go offline, so I can probably only check them tomorrow.

  • Author

Thanks for all the help - edited last post to include diags

Errors are running up, not sure why:
image.png

Edited by khior

  • Author

Just to update this thread:

So the good news is that I was able to backup the data from the problematic pool. Rsync reported some files with I/O errors while reading and others that didn’t pass verification, but those were in the minority. I'm happy (and relieved).

It was really weird as I watched the error count of all drives in this pool climb up to a high of 152,000 per drive by the time all of the files were copied over (~3 hours). One drive in particular also got very hot (560 C).

I’m not sure what your thoughts are but my guess is maybe a cable issue based on the exact same synchronized error count on all these drives. I’m guessing the HBA might not be the problem as all of these problems are on my “mediafun” pool and I also have a separate pool of 4 drives that have no problems and are being served by this same HBA (a Broadcom LSI SAS 9305-16i). I think a couple of drives are connected directly to the motherboard’s Sata port but otherwise all of the other drives are being served by this HBA and are via mini SAS to Sata breakout cables.

After all the data was copied, I ended up stopping the array and shutting down my server to try to stop the error count and let the drives cool. I ordered all new breakout cables and, depending on what you recommend based on the diags, I would like to replace these cables before powering up and bringing the array online again.

Good to hear you were able to copy most of the data off.

The errors are not caused by the cables. When every disk in a raidz pool shows the same checksum error count, ZFS found blocks it could not rebuild. With two devices missing, a raidz2 pool has no redundancy left, so when a block fails its checksum ZFS cannot tell which disk returned the bad data, and it adds the error to all of them. A cable or disk problem would show READ or WRITE errors only on the affected disks, and I/O errors for those disks in the syslog. The diagnostics have none, for the whole boot. The files rsync could not read or verify are those same damaged blocks. The count went up each time they were read, by rsync and by the scrub that was also running.

Those blocks are permanently damaged. Resilvering the two missing devices cannot fix them, because the resilver rebuilds the missing disks from that same data.

Also, when the array started, one dataset failed to mount:

cannot mount 'mediafun/photos_immich': Input/output error

So nothing in that dataset was copied, because it was not mounted. Your photos_immich share is set to the backupandiso pool, so that dataset on mediafun may only be an old copy. Please check that nothing in it is missing from your backup.

Since you now have a copy of the data, I recommend that you recreate the pool and restore the data, not resilver. Then you start with a clean pool, and not one with damaged files and a dataset that does not mount.

Before that:

- Replace the cables as planned. Device 12 has CRC errors and command timeouts in its SMART report, so its connection needs attention.

- Check the cooling for the disk that reached 56 C. That is hot for a hard drive.

- After you replace the cables, boot the server, start the array, and post the output of:

zpool status -v mediafun

Also post new diagnostics. The output lists the damaged files, so you can check them against what rsync reported. The diagnostics show if any disk still logs connection errors with the new cables.

  • Author

Thanks again. At the moment I'm waiting for the cables to be delivered. Then I'll install them, check zpool status as well as grab new diags. I'm poking around the last set of diags now...noob question, where can I find a list of the damaged files in the diags? Also, I'm erring on the side of super caution at the moment, would it be safe to proceed to delete and recreate that pool immediately after grabbing and posting the latest diagnostics or should I wait for word back in this forum before deleting and recreating that pool in case there is something that jumps out on your end, something I'm missing?

20 minutes ago, khior said:

where can I find a list of the damaged files in the diags?

Full list is not in the diags, pool needs to be imported, and you can then check with

zpool status -v mediafun

It should be OK to recreate and reformat the pool immediately, but you can post diags after that to check for any ATA or other errors.

  • Author

Ok, cables are replaced. I started the array. This is the output of "zpool status -v mediafun" (Not including the file names listed with errors, they were just a few and replaceable). I'm also attaching a set of Diagnostics.

I was reading a few different ways to correctly remove this pool to repurpose these drives. I stopped the array and erased the pool, however, I still see mediafun listed. Before starting the array again to reform the pool should I just choose "Remove Pool" or do they need to be unassigned before restarting the array?

root@MediaUnraid:~# zpool status -v mediafun

  pool: mediafun

 state: DEGRADED

status: One or more devices has experienced an error resulting in data

        corruption.  Applications may be affected.

action: Restore the file in question if possible.  Otherwise restore the

        entire pool from backup.

   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A

  scan: scrub in progress since Tue Oct 27 10:54:23 2026

        24.4T / 108T scanned at 30.2G/s, 20.1T / 108T issued at 8.68M/s

        8K repaired, 18.62% done, no estimated completion time

config:

 

        NAME                     STATE     READ WRITE CKSUM

        mediafun                 DEGRADED     0     0     0

          raidz2-0               DEGRADED     0     0     0

            sda1                 ONLINE       0     0     4

            sdd1                 ONLINE       0     0     4

            sde1                 ONLINE       0     0     4

            sdr1                 ONLINE       0     0     4

            sdn1                 ONLINE       0     0     4

            sdl1                 ONLINE       0     0     4

            sdh1                 ONLINE       0     0     4

            sdj1                 ONLINE       0     0     4

            sdk1                 ONLINE       0     0     4

            sdg1                 ONLINE       0     0     4

            sdb1                 ONLINE       0     0     4

            sdn1                 OFFLINE      0     0     0

            7355020672882961856  FAULTED      0     0     0  was /dev/sdm1

 

errors: Permanent errors have been detected in the following files:

mediaunraid-diagnostics-20261028-1856.zip

Screenshot 2026-10-28 193422.png

Edited by khior
More information

The new diagnostics look good. There are no disk or controller errors in the log for the whole boot, and the CRC count on Device 12 has not gone up since you replaced the cables. The pool status showed 8147 data errors before the erase, so recreating the pool was the right decision.

You don't need to unassign the devices first. "Remove pool" is enough:

- With the array stopped, click the first device of the mediafun pool and click "Remove pool".

- On Main, add a new pool with the same name and 13 slots.

- Assign all 13 disks. This includes the 24TB disk and the disk that was Device 13. The order is not important.

- Click the first pool device, set the file system to ZFS, confirm allocation profile to raidz2, and 1 group of 13 devices. Click Apply.

- Start the array. The pool will show as unmountable, which is normal for new disks. At the bottom of the Main page, check that the Format list shows only the 13 mediafun disks, then confirm and click Format.

When the format is complete, restore your data and post new diagnostics, so we can confirm that the pool is healthy and that there are still no disk errors.

  • Author

I went through the above but after the formatting was complete the pool shows the message "Unmountable: wrong or no file system". I shut down the array and tried once more to recreate the pool with the same result.

Screenshot 2026-10-29 080118.png

Screenshot 2026-10-29 081027.png

Edited by khior
More detail

Oct 29 07:59:21 MediaUnraid root: mountpoint '/mnt/mediafun' exists and is not empty

Delete that folder or reboot and try again

The new diagnostics look good. The mediafun pool is online with all 13 devices, there are no errors, and the log for the whole boot shows no disk or controller errors. The CRC count on the 24TB disk has not changed since you replaced the cables.

While the restore is running, keep an eye on the disk temperatures, especially the one that got hot before. After the restore is complete, I suggest you run a scrub to confirm that all is good.

  • Author

Thanks again for all the assistance. I have another unraid box that I have been backing up to for some time and the restore was going well for about the last 24 hours using rsync until just about now, about 17TB copied. I had a disk in that pool that was displaying SMART errors but it seemed ok until now.

I noticed the speed of the restore just reduced sharply and the error count on that one drive has skyrocketed. Reads and writes are fluctuating between 0kb and a couple hundred KBs while the error count climbs and I'm getting notices that it's failing. Not sure if I should stop the sync, kick out that drive or let it run. I'm leaning toward stopping it as much as I don't want to. I do have a spare 16TB drive for that system as well I could swap out and let it resilver if needed.

(Edit): After letting it run for a bit more I think it's fairly obvious that I need to replace that drive...I'll:

  1. Stop the backup array (K-Unraid)

  2. Unassign that disk

  3. Start the backup array

  4. Shut down the backup array

  5. replace the disk

  6. start the backup array and let it rebuild.

I don't see this progressing otherwise. I'm a bit gun shy as what I thought was a simple disk replace before is what kicked off this whole mess, and now my data is only in one spot.

Diags from that other system are attached if you have the time...

k-unraid-diagnostics-20261030-0824.zip

Edited by khior

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

Account

Navigation

Search

Search

Configure browser push notifications

Chrome (Android)
  1. Tap the lock icon next to the address bar.
  2. Tap Permissions → Notifications.
  3. Adjust your preference.
Chrome (Desktop)
  1. Click the padlock icon in the address bar.
  2. Select Site settings.
  3. Find Notifications and adjust your preference.