Hello, after a recent reboot, I have been encountering an "Unmountable: wrong or no file system" on my ZFS pool. Before the reboot, the pool reported zero free space, which I found unusual. What was stranger was that I was unable to delete any files from the tool. I thought a reboot would resolve the issue, but it resulted in the "Unmountable: wrong or no file system" error. The issue is very concerning as the pull contains nearly 1 PB of data. Can someone provide some assistance?

Here is the log message upon reboot, right before the mount error. The log indicates that the space issue is likely the cause. But I have no idea how I can remedy that at this point:

```Dec 20 01:11:45 Nebula root: cannot import 'zmain' as 'zmain': out of space

Dec 20 01:11:45 Nebula root: Destroy and re-create the pool from

Dec 20 01:11:45 Nebula root: a backup source.

Dec 20 01:11:45 Nebula emhttpd: shcmd (417): exit status: 1

Dec 20 01:11:45 Nebula emhttpd: shcmd (418): rmdir /mnt/zmain

Dec 20 01:11:45 Nebula emhttpd: zmain: mount error: wrong or no file system

Dec 20 01:11:45 Nebula emhttpd: mounting /mnt/zmini

Dec 20 01:11:45 Nebula emhttpd: shcmd (419): mkdir -m 0666 -p /mnt/zmini

Dec 20 01:11:45 Nebula emhttpd: shcmd (420): /usr/sbin/zpool import -f -m -N -o autoexpand=on -d /dev/sdab1 -d /dev/sdaf1 -d /dev/sdai1 -d /dev/sdac1 -d /dev/sdaj1 -d /dev/sdak1 -d /dev/sdao1 -d /dev/sdbq1 18197799099838447052 zmini

Dec 20 01:11:57 Nebula emhttpd: zmini: zfs verify devices

Dec 20 01:11:57 Nebula emhttpd: /usr/sbin/zpool status -P zmini 2>&1```

Please post the diagnostics and the output from zpool import

I've attached the diagnostics. Importing the zpool via the command line didn't return any errors, but the pool remains inaccessible. Attempting to delete files manually via the command line in an attempt ti fix the space issue returns failed to remove '....': No space left on device.
nebula-diagnostics-20251220-0200.zip

Pool should be accessible under /mnt/zmain

Filesystem Size Used Avail Use% Mounted on

zmain 256K 256K 0 100% /mnt/zmain

zmain/plexshare 693T 693T 0 100% /mnt/zmain/plexshare

zfs pools should be kept under 80% usage, 90% max, and completely filling them up is a good way to lose the pool. Confirm if you can browse the pool contents under /mnt/zmain

I am able to browse the content under /mnt/zmain/ from the commandline. Is there anyway I can recover this?

If manually deleting data fails I suspect you will need to back up and recreate the pool, which, with 700TB of data, won't be fun.

Another option could be if you can add a new vdev to the pool, but you would need 10 additional disks.

Some possible options from ChatGPT that you can try:

1. Identify and delete snapshots first (most common fix)

Snapshots can prevent space from being reclaimed even after deletions.

zfs list -t snapshot -o name,used -S used

Delete large snapshots:

zfs destroy pool/dataset@snapname

If there are many:

zfs destroy -r pool/dataset@snap*

This often immediately frees space without touching live data.

---

2. Temporarily lower copies, recordsize, or compression side effects

If copies=2 or 3 is set, ZFS needs extra space to update metadata.

Check:

zfs get copies pool/dataset

Temporarily reduce:

zfs set copies=1 pool/dataset

Then retry deletion.

---

3. Destroy a dataset or zvol entirely (stronger than file deletes)

File-level deletes need metadata updates; dataset destruction is more atomic.

List datasets by usage:

zfs list -o name,used -S used

Destroy a whole dataset:

zfs destroy pool/large_dataset

If mounted and busy:

zfs unmount -f pool/large_dataset
zfs destroy pool/large_dataset

---

4. Set a temporary refreservation to create working space

This is a powerful workaround.

zfs set refreservation=10G pool/dataset

Then immediately remove it:

zfs set refreservation=none pool/dataset

This can force ZFS to carve out metadata headroom.

---

5. Export and re-import with recovery flags (last-resort soft reset)

This can relax certain allocation constraints.

zpool export pool
zpool import -f -o readonly=off pool

Then retry deletion.

---

6. Emergency: mount read-only, backup, and rebuild

If nothing works:

zfs set readonly=on pool/dataset

Back up critical data, destroy and recreate the pool.

Hmm, adding another vdev could be a viable option, though it would require some drive consolidation since my rack is currently full. I’ll go ahead and move forward with that option. Do you know if Unraid supports creating snapshots? This is a feature I’ll need to try and take advantage of in the future if it is available.

1 hour ago, Chukwuka13 said:

Do you know if Unraid supports creating snapshots?

It does, but there's no GUI option for it currently; they can be created with the CLI or a plugin, like ZFS Master.

Okay, that's good to know—one more question about the vdev extension. While the zpool was low on storage, I tried running the mover task to move data from the pool to the Unraid array. I saw that the unraid array storage decreased, indicating files were being copied, but the storage on the zpool remained untouched. This to me indicates that files were copied but never successfully deleted. Once I’m able to bring the pool back online, how will Unraid handle this data duplication?

Enable mover logging and run the mover, if there are any duplicates, they will not be moved but will show on the syslog.

I appreciate all the help. I try to check back in once the new drives are here and the pool has hopefully been expanded successfully.

@JorgeB I have just added the additional vdev, but Unraid still can't import the pool on start. I still see the following error on startup:

Dec 29 17:32:22 Nebula emhttpd: shcmd (7733): mkdir -m 0666 -p /mnt/zmain
Dec 29 17:32:22 Nebula emhttpd: shcmd (7734): /usr/sbin/zpool import -f -m -N -o autoexpand=on -d /dev/sdbf1 -d /dev/sdbg1 -d /dev/sdbh1 -d /dev/sdbi1 -d /dev/sdbj1 -d /dev/sdbk1 -d /dev/sdbl1 -d /dev/sdbm1 -d /dev/sdbn1 -d /dev/sdbo1 -d /dev/sdbu1 -d /dev/sdbv1 -d /dev/sdbw1 -d /dev/sdbx1 -d /dev/sdby1 -d /dev/sdbz1 -d /dev/sdca1 -d /dev/sdcb1 -d /dev/sdcc1 -d /dev/sdcd1 -d /dev/sdce1 -d /dev/sdcf1 -d /dev/sdcg1 -d /dev/sdch1 -d /dev/sdci1 -d /dev/sdcj1 -d /dev/sdck1 -d /dev/sdcl1 -d /dev/sdcm1 -d /dev/sdcn1 -d /dev/sdco1 -d /dev/sdcw1 -d /dev/sdcx1 -d /dev/sdcy1 -d /dev/sddc1 -d /dev/sddd1 -d /dev/sdde1 -d /dev/sddf1 -d /dev/sddg1 -d /dev/sddh1 -d /dev/sdcp1 -d /dev/sdcz1 -d /dev/sdcq1 -d /dev/sdcv1 -d /dev/sdcr1 -d /dev/sdct1 -d /dev/sdcu1 -d /dev/sdda1 -d /dev/sdbt1 -d /dev/sddb1 9455508329953933197 zmain
Dec 29 17:33:29 Nebula root: cannot import 'zmain' as 'zmain': out of space
Dec 29 17:33:29 Nebula root: Destroy and re-create the pool from
Dec 29 17:33:29 Nebula root: a backup source.
Dec 29 17:33:29 Nebula emhttpd: shcmd (7734): exit status: 1
Dec 29 17:33:29 Nebula emhttpd: shcmd (7735): rmdir /mnt/zmain
Dec 29 17:33:29 Nebula emhttpd: zmain: mount error: wrong or no file system

Any ideas what to do from here?

Edited December 29, 2025Dec 29 by Chukwuka13

I see that the allocation profile still shows that the array has 5 vdevs instead of 6. It sounds like unraid never attempted to actually expand the pool after adding the new 10 drives. Perhaps the expansion needs to be done manually? I'm just unsure of how to do this.

Edited December 29, 2025Dec 29 by Chukwuka13

From some research, I learned that I may be able to manually add the vdev via the CLI by importing the pool, then running zpool add zmain raidz2 /dev/sdab1 /dev/sdac1 /dev/sdad1 /dev/sdaf1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdao1 /dev/sdbq1 /dev/sdbs1. I'm hesitant to run this, though, as I'm unsure if it will cause synchronization issues with Unraid and its configuration. I'm awaiting more expert advice before taking further action.

It may be worth mentioning that I downloaded ZFS Master, which reports that the /mnt/zmain mount point has 105GiB of free space, while the /mnt/zmain/plexshare mount point has 0 B. This info may or may not be significant. I found it interesting, as I was originally under the impression that the entire pool had no space.

Unraid tries to import the pool first, then add the devices, not sure you are able to add devices with the pool offline, but give me a couple of hours to test and confirm.

Yep, I forgot that to add a new vdev to a pool, the pool must be imported, so it's not really a solution, if none of the options above worked to import the pool, I'm afraid you will not be able to add a new vdev.

What about the option of adding the vdev manually via the CLI?

It still requires the pool to be imported first, so if those options still cannot import the pool, like the read-only options and the other ones, I'm afraid it's not possible to add more devices.

I’ve never had issues importing the pool via the command line using the zpool import command. The issue has only been with Unraids import on startup. Is zpool import command not viable?

56 minutes ago, Chukwuka13 said:

I’ve never had issues importing the pool via the command line using the zpool import command.

Do you mean that zpool import zmain works?

Yes, the command works just fine, and ZFS master reports the pool correctly. The Unraid UI just won’t show the pool as mounted, even though it is.

Note that I’ve been chatting with you over the phone while still in bed, since it seems like your morning is my night, so I haven't taken any additional action on the server since my first post from this week. I should probably also mention that when you mentioned adding a new vdev as an option, I never bothered with the ChatGPT options you provided, since some of them could corrupt my pool.

33 minutes ago, Chukwuka13 said:

The Unraid UI just won’t show the pool as mounted, even though it is.

Note that the GUI will not show the correct status if the pool is imported manually.

33 minutes ago, Chukwuka13 said:

I never bothered with the ChatGPT options you provided, since some of them could corrupt my pool.

If the pool imports with just zpool import, no need to try the other options.

If the pool imports using the CLI, it should also import using the GUI, reboot to clear the logs, don't import the pool manually initially; post a screenshot from main showing the pool assignments before array start, and also the output from zpool import

Okay, let me go restart the array then I can get you the screenshot of the pool assignment. When I ran zpool import zmain in the past, via the command line, it never output anything. A subsequent status command would just show the pool was online.

I should mention that earlier, when I restarted the server after assigning 10 new drives to the pool, Unraid initiated a recovery process that effectively ejected the 10 new drive assignments from the pool. I imagine that will happen again.

I just stopped the array. Here is a quick screenshot of the pool with the 10 new drives. I'm going to restart now

8 minutes ago, Chukwuka13 said:

When I ran zpool import zmain in the past, via the command line, it never output anything.

That's normal if the pool import succeeds. I would like to see the output from just zpool import

Also, import the pool manually after that, with zpool import zmainand also post the output from zpool status