Chukwuka13

Okay, let me go restart the array then I can get you the screenshot of the pool assignment. When I ran zpool import zmain in the past, via the command line, it never output anything. A subsequent status command would just show the pool was online. I should mention that earlier, when I restarted the server after assigning 10 new drives to the pool, Unraid initiated a recovery process that effectively ejected the 10 new drive assignments from the pool. I imagine that will happen again.

Yes, the command works just fine, and ZFS master reports the pool correctly. The Unraid UI just won’t show the pool as mounted, even though it is. Note that I’ve been chatting with you over the phone while still in bed, since it seems like your morning is my night, so I haven't taken any additional action on the server since my first post from this week. I should probably also mention that when you mentioned adding a new vdev as an option, I never bothered with the ChatGPT options you provided, since some of them could corrupt my pool.

I’ve never had issues importing the pool via the command line using the zpool import command. The issue has only been with Unraids import on startup. Is zpool import command not viable?

What about the option of adding the vdev manually via the CLI?

From some research, I learned that I may be able to manually add the vdev via the CLI by importing the pool, then running zpool add zmain raidz2 /dev/sdab1 /dev/sdac1 /dev/sdad1 /dev/sdaf1 /dev/sdai1 /dev/sdaj1 /dev/sdak1 /dev/sdao1 /dev/sdbq1 /dev/sdbs1. I'm hesitant to run this, though, as I'm unsure if it will cause synchronization issues with Unraid and its configuration. I'm awaiting more expert advice before taking further action. It may be worth mentioning that I downloaded ZFS Master, which reports that the /mnt/zmain mount point has 105GiB of free space, while the /mnt/zmain/plexshare mount point has 0 B. This info may or may not be significant. I found it interesting, as I was originally under the impression that the entire pool had no space.

I see that the allocation profile still shows that the array has 5 vdevs instead of 6. It sounds like unraid never attempted to actually expand the pool after adding the new 10 drives. Perhaps the expansion needs to be done manually? I'm just unsure of how to do this.

@JorgeB I have just added the additional vdev, but Unraid still can't import the pool on start. I still see the following error on startup: Dec 29 17:32:22 Nebula emhttpd: shcmd (7733): mkdir -m 0666 -p /mnt/zmain Dec 29 17:32:22 Nebula emhttpd: shcmd (7734): /usr/sbin/zpool import -f -m -N -o autoexpand=on -d /dev/sdbf1 -d /dev/sdbg1 -d /dev/sdbh1 -d /dev/sdbi1 -d /dev/sdbj1 -d /dev/sdbk1 -d /dev/sdbl1 -d /dev/sdbm1 -d /dev/sdbn1 -d /dev/sdbo1 -d /dev/sdbu1 -d /dev/sdbv1 -d /dev/sdbw1 -d /dev/sdbx1 -d /dev/sdby1 -d /dev/sdbz1 -d /dev/sdca1 -d /dev/sdcb1 -d /dev/sdcc1 -d /dev/sdcd1 -d /dev/sdce1 -d /dev/sdcf1 -d /dev/sdcg1 -d /dev/sdch1 -d /dev/sdci1 -d /dev/sdcj1 -d /dev/sdck1 -d /dev/sdcl1 -d /dev/sdcm1 -d /dev/sdcn1 -d /dev/sdco1 -d /dev/sdcw1 -d /dev/sdcx1 -d /dev/sdcy1 -d /dev/sddc1 -d /dev/sddd1 -d /dev/sdde1 -d /dev/sddf1 -d /dev/sddg1 -d /dev/sddh1 -d /dev/sdcp1 -d /dev/sdcz1 -d /dev/sdcq1 -d /dev/sdcv1 -d /dev/sdcr1 -d /dev/sdct1 -d /dev/sdcu1 -d /dev/sdda1 -d /dev/sdbt1 -d /dev/sddb1 9455508329953933197 zmain Dec 29 17:33:29 Nebula root: cannot import 'zmain' as 'zmain': out of space Dec 29 17:33:29 Nebula root: Destroy and re-create the pool from Dec 29 17:33:29 Nebula root: a backup source. Dec 29 17:33:29 Nebula emhttpd: shcmd (7734): exit status: 1 Dec 29 17:33:29 Nebula emhttpd: shcmd (7735): rmdir /mnt/zmain Dec 29 17:33:29 Nebula emhttpd: zmain: mount error: wrong or no file system Any ideas what to do from here?

I appreciate all the help. I try to check back in once the new drives are here and the pool has hopefully been expanded successfully.

Okay, that's good to know—one more question about the vdev extension. While the zpool was low on storage, I tried running the mover task to move data from the pool to the Unraid array. I saw that the unraid array storage decreased, indicating files were being copied, but the storage on the zpool remained untouched. This to me indicates that files were copied but never successfully deleted. Once I’m able to bring the pool back online, how will Unraid handle this data duplication?

Hmm, adding another vdev could be a viable option, though it would require some drive consolidation since my rack is currently full. I’ll go ahead and move forward with that option. Do you know if Unraid supports creating snapshots? This is a feature I’ll need to try and take advantage of in the future if it is available.

I am able to browse the content under /mnt/zmain/ from the commandline. Is there anyway I can recover this?

I've attached the diagnostics. Importing the zpool via the command line didn't return any errors, but the pool remains inaccessible. Attempting to delete files manually via the command line in an attempt ti fix the space issue returns failed to remove '....': No space left on device. nebula-diagnostics-20251220-0200.zip

Hello, after a recent reboot, I have been encountering an "Unmountable: wrong or no file system" on my ZFS pool. Before the reboot, the pool reported zero free space, which I found unusual. What was stranger was that I was unable to delete any files from the tool. I thought a reboot would resolve the issue, but it resulted in the "Unmountable: wrong or no file system" error. The issue is very concerning as the pull contains nearly 1 PB of data. Can someone provide some assistance? Here is the log message upon reboot, right before the mount error. The log indicates that the space issue is likely the cause. But I have no idea how I can remedy that at this point: ```Dec 20 01:11:45 Nebula root: cannot import 'zmain' as 'zmain': out of space Dec 20 01:11:45 Nebula root: Destroy and re-create the pool from Dec 20 01:11:45 Nebula root: a backup source. Dec 20 01:11:45 Nebula emhttpd: shcmd (417): exit status: 1 Dec 20 01:11:45 Nebula emhttpd: shcmd (418): rmdir /mnt/zmain Dec 20 01:11:45 Nebula emhttpd: zmain: mount error: wrong or no file system Dec 20 01:11:45 Nebula emhttpd: mounting /mnt/zmini Dec 20 01:11:45 Nebula emhttpd: shcmd (419): mkdir -m 0666 -p /mnt/zmini Dec 20 01:11:45 Nebula emhttpd: shcmd (420): /usr/sbin/zpool import -f -m -N -o autoexpand=on -d /dev/sdab1 -d /dev/sdaf1 -d /dev/sdai1 -d /dev/sdac1 -d /dev/sdaj1 -d /dev/sdak1 -d /dev/sdao1 -d /dev/sdbq1 18197799099838447052 zmini Dec 20 01:11:57 Nebula emhttpd: zmini: zfs verify devices Dec 20 01:11:57 Nebula emhttpd: /usr/sbin/zpool status -P zmini 2>&1```

The share is configured to use the zpool as the primary storage and the Unraid Array as the secondary storage. I did actually notice that even though the Zpool was configured to have a minimum free space of 5%, the share itself was somehow configured to have a minimum free space of 50%. I don't know if that setting was recently changed since my server has been well over 50% capacity for a while now. Reducing the value on the share setting did address this issue.

I turned logging on during a mover scan, and I see a lot of errors about files not being able to be moved due to the No space left on device error. This is probably why nothing was moved after 2 days of leaving that task to run. My array is also not at full capacity.

It looks like IPv6 was never enabled. My network protocol is currently set to IPv4:

@JorgeB My scrub finished with a few permanent errors on some files, which I've gone ahead and deleted. After I paused the parity sync and rebooted my system, then attempted to write to my share, but I still got the insufficient storage error. Here is the diagnostic after the write attempt: diagnostics-20230705-0840.zip Keep in mind the ZFS pool is now at 86.4% capacity: and the minimum free space is set to 4%:

While I wait for the parity sync, can you recommend anything to prevent the nchan errors? I've been seeing these for a while and don't understand the cause. My server has 256 GBs of memory, so it should have more than enough to prevent any out-of-memory issues.

I'm not able to reboot at the moment. I'm currently running a parity check on the array and a scrub on the zpool to see if that helps anything. The parity check probably won't be done until about 48 more hours.

@JorgeB here you go: diagnostics-20230704-0825.zip

When my ZFS pool got to around 90% capacity, I started receiving the "[Errno 28] No space left on device" error when writing to my ZFS pool. I tried moving files off the pool, reducing the storage capacity to 88.5 %, yet this error persists. I later noticed the Minimum free space setting was set to 10%, which could explain the original error. However, I have since tried updating that setting to values less than 5%, but the No space left error persists. I'm not sure what could be the cause at this point. This pool is configured to my Array so that mover moves files from the ZFS pool into the Array. Here is a screenshot providing more info on my share configuration. I'm wondering if this error may have been a result of my moving content from my pool into the Array outside of the mover task. I don't know if that is supported. I did this because the mover task seems to either be broken since the 6.12 upgrade or is very slow when running against the ZFS pool. I had the task running for almost 2 days and saw no sign of anything actually being moved. Any suggestion on how to resolve this would be greatly appreciated.