I managed to get enough blank disks to migrate my server to ZFS pools (or at least try to)

I had 7x8TB drives, created the first pool of 4 drives, off to the races.

I added a single drive, expanded the pool. I "thought" it had finished adding the drive (I was mistaken) so I added another, and another.

Now I'm left with what I assume is asingle pool with 1 vdev of 5 drives, and 1 vdev of 2 drives. Unraid doesn't seem to acknowledge the second vdev for storage space though.

Just wondering how screwed am I, Do I have to add 3 more drives for the other vdev to be acknowledged/used?

 pool: mainzpool
 state: ONLINE
  scan: scrub repaired 1000K in 10:26:38 with 0 errors on Thu Oct 15 20:10:52 2026
expand: expanded raidz1-0 copied 20.5T in 2 days 03:15:39, on Thu Oct 15 09:44:14 2026
config:

	NAME           STATE     READ WRITE CKSUM
	mainzpool      ONLINE       0     0     0
	  raidz1-0     ONLINE       0     0     0
	    /dev/sda1  ONLINE       0     0     0
	    /dev/sdc1  ONLINE       0     0     0
	    /dev/sdk1  ONLINE       0     0     0
	    /dev/sdd1  ONLINE       0     0     0
	    /dev/sdb1  ONLINE       0     0     0
	  raidz1-1     ONLINE       0     0     0
	    /dev/sdj1  ONLINE       0     0     0
	    /dev/sdi1  ONLINE       0     0     0

errors: No known data errors

3 hours ago, mnovak said:

Do I have to add 3 more drives for the other vdev to be acknowledged/used?

Basically, yeah, Unraid doesn't support using vdevs with different widths. Also, it's not efficient to have a 2 device raidz1 vdev, since it will use 50% for parity. My recommendation would be to recreate the pool as a single vdev raidz2.

Well f*$@&

Really wish the documentation was clearer about expanding. Can't remove the drives once they're added, so I don't have enough spare space to clear off the drives to rebuild them into a single vdev.

16 hours ago, mnovak said:

Really wish the documentation was clearer about expanding.

I assume you expanded the array using the CLI, since the WebGUI should not allow that, or do you mean the ZFS documentation?

I didn't do anything from the command line, the gui didn't show any problems, I assumed it was queuing disks as I added, not creating a separate vdev. Honestly I would have expected a warning about stopping the array while it was expanding.

Regardless. I should have been slightly more patient and read more before I started.

I might be able to recover using some questionable drives I previously pulled and haven't gotten around to destroying yet.

If you still have the diags or remember the steps to get the pool in that state, please post here to see if I can reproduce it.

It would be easy to reproduce, simply stop the array, add a disk, let expansion start. Then 2 days later stop the array, add another disk, start, stop, add...

It was on the 3rd drive that I finally clicked into the pool properties to see that the first disk hadn't finished silvering.

If the gui had shown a degraded or resilvering etc state I might have hesitated before adding, but the drives didn't appear to be reading or writing when I added disks (I know now I should have checked the properties and not just looked at the IO on the dashboard)

Seems silly that the default behavior adding more disks is a second vdev though.

Also, definitely not blaming anyone but myself here. Just that the documentation isn't especially clear about how long adding a drive takes, or where to look to see the status. Or that adding another drive before it finishes creates a new vdev.

(I also should have known better than to copy data to the drives before I had finished expanding in the first place!)

Edited October 19Oct 19 by mnovak

11 minutes ago, mnovak said:

It would be easy to reproduce, simply stop the array, add a disk, let expansion start.

Right, I mean, did you start with a pool with 2 VDEVs? The normal Raidz expansion doesn't add another vdev.

No, I never actively created a second vdev, it was apparently automatic. Because the first expansion wasn't complete, the additional drives created their own separate vdev (as seen above) I'm sure if I deep dive on ZFS it's a default behavior, I just didn't expect it, or expected some kind of error or warning. My assumption is that because the first expansion wasn't complete ZFS couldn't change the first vdev to include the additional disks, so it created a second vdev to add them. That's my pain point because you can't remove the second vdev without destroying the first one as well even though the other 2 disks aren't actually in use until I assume both vdevs match (which would require adding 3 more disks, and in this economy I'm just not willing to spend that much lol)

13 minutes ago, mnovak said:

No, I never actively created a second vdev, it was apparently automatic

By default, a 4-device zfs pool is created as a single RAIDZ1 vdev, unless you changed that. How many disks had the pool initially?

4, then expanded 1 to 5, then the 2 additional drives created a new vdev.

mainzpool      ONLINE       0     0     0
	  raidz1-0     ONLINE       0     0     0
	    /dev/sda1  ONLINE       0     0     0
	    /dev/sdc1  ONLINE       0     0     0
	    /dev/sdk1  ONLINE       0     0     0
	    /dev/sdd1  ONLINE       0     0     0
	    /dev/sdb1  ONLINE       0     0     0
	  raidz1-1     ONLINE       0     0     0
	    /dev/sdj1  ONLINE       0     0     0
	    /dev/sdi1  ONLINE       0     0     0

So the last two were added at the same time?

If I understood correctly, I cannot reproduce this the way you describe, started with a 4-device raidz1 pool, expanded it to 5 devices, then tried to add 2 more, and it's not allowed as expected.

If you remember, please post step by step what you did to arrive at that result, there may be a bug, but only under some specific circumstances.

The last 2 were added at the same time yes, no error, just changed the drive count to 7 from 5, and added the 2, started the array, and then noticed the 2 new drives had no IO activity. When I investigated the pool properties from there I saw that the first drive hadn't completed resilvering, and the other 2 drives were in a second vdev, didn't think anything of it at the time, but when the first drive finally finished the other 2 remained separate, and I was missing 14-ish TB of usable space because of the mismatch of vdev sizes. (I'm almost done moving the data off the drives, I'll see if I can recreate it once I destroy the pool)

Edited October 20Oct 20 by mnovak

I tried that already but could not repro, started with a 4-drive raidz1, expanded the pool with a 5th drive, stopped the array before the expansion was complete, tried to add 2 other devices. The array doesn't start with an "invalid expansion" error, so please let me know if you can repro, and the steps to do it.