Hi All,

I have a single large ZFS pool of mirrored pairs. I did this, as I was building a media server, where all my data needs to be stored on a single share.

Currently I have used up the 120 slots, as that apparently seems to the max allowed. Is there a way to add more drives?

Thanks!

Edited November 13, 2025Nov 13 by shelfactor

Cli only not by gui. Your the first I hear to have hit the limit.

1 hour ago, shelfactor said:

Is there a way to add more drives?

For now only adding another pool or using the array, how many devices does your pool have?

@JorgeB I currently have 120 drives in the pool. I have more drives that are connected, not being used. I would like to add them to the current pool, so that they can be a part of the same share.

As far as I know, there are limitations to the software for hard-linking. that wont allow me to use the array or another pool.

@bmartino1 You can see in this screenshot that 120 is the max. *Prior to 120 assigned, I would see all the number up to 120, but never higher than 120.

30 minutes ago, shelfactor said:

I would like to add them to the current pool, so that they can be a part of the same share.

Understood, and how large would you want the limit to be? I believe this is not a hard limit, and it could be changed; it's been increased before.

@JorgeB Thank you as always for all the help.

Currently I have another 2 disk shelves connected, so I could see potentially another 48 drives. But immediate expansion needs would be to allow at least another 24.

Initially, pools were limited to 30 devices; that was doubled to 60 when someone asked, and then again to 120, and LT thought that would be enough for a few years 😀

That should be an easy change, so I'm going to ask LT to see if they can do that for 7.2.1, I'll post here when I have more info.

@JorgeB You're the best, I will look forward to the update.

I can offload some data for awhile until the change is implemented.

FYI, here is the current setup, (prior to adding the aforementioned 2 more shelves)

1.1PB of fully air-conditioned awesomeness

1 hour ago, shelfactor said:

@JorgeB You're the best, I will look forward to the update.

I can offload some data for awhile until the change is implemented.

FYI, here is the current setup, (prior to adding the aforementioned 2 more shelves)

1.1PB of fully air-conditioned awesomeness

You have piqued my curiosity.

What motherboard are you running and how are you connecting all these drives to your motherboard.

I understand you have multiple dish shelves per year. Your picture here.

And I understand one pcie lane can equal one sata disc.

I'm just curious how you're connecting them all.

Then I'm curious of how you're handling your ZFS arrayed if you're having them all in one pool.

Like what is your vdev setup? Are you running a raid Z5?

Edited November 13, 2025Nov 13 by bmartino1

@bmartino1

I love it when we all get excited about everyone's setups. I continually find myself intrigued about posts I see as well!

Recently, I was running this setup from a NUC13 Extreme (mini tower looking one). However, I have recently reverted back to an older setup, as I wanted to rack mount the server in my other rack, instead of having it standing on a shelf in the rack.

Current harward is an old Gigabyte MD60-SC0 dual Xeon motherboard. Allows for me to have dual 10g NIC coming off the Intel 82599ES connector.

I then have the NetApp DS4248 disk shelves connected to a LSI 9300-8e. 4 shelves daisy-chained per port on the card.

- I recently switched to using two of these cards, to test if any latency was experienced with 1 card... meaning 2 cards, 2 shelves per port. There wasnt a noticable difference that warrants the extra card right now.

I use an Intel Arc B380 for transcoding needs, as the Xeons do not have QuickSync.

As far as the layout, I am using mirrored pairs, during a lengthy discussion and an issue that I had with a prior layout (z1) I decided I would rather spend the extra money and have pairs. Which will allow me to remove pairs/drives if ever needed.

I am always open to suggestions if you feel I am doing something incorrectly :)

1 minute ago, shelfactor said:

@bmartino1

I love it when we all get excited about everyone's setups. I continually find myself intrigued about posts I see as well!

Recently, I was running this setup from a NUC13 Extreme (mini tower looking one). However, I have recently reverted back to an older setup, as I wanted to rack mount the server in my other rack, instead of having it standing on a shelf in the rack.

Current harward is an old Gigabyte MD60-SC0 dual Xeon motherboard. Allows for me to have dual 10g NIC coming off the Intel 82599ES connector.

I then have the NetApp DS4248 disk shelves connected to a LSI 9300-8e. 4 shelves daisy-chained per port on the card.

- I recently switched to using two of these cards, to test if any latency was experienced with 1 card... meaning 2 cards, 2 shelves per port. There wasnt a noticable difference that warrants the extra card right now.

I use an Intel Arc B380 for transcoding needs, as the Xeons do not have QuickSync.

As far as the layout, I am using mirrored pairs, during a lengthy discussion and an issue that I had with a prior layout (z1) I decided I would rather spend the extra money and have pairs. Which will allow me to remove pairs/drives if ever needed.

I am always open to suggestions if you feel I am doing something incorrectly :)

Thats awesome. My friend recently got himself a hand full of old decominisend telecom racks. He has a lot of weird compute all over. I get to test and be his tech support from time to time. :)

My only concern is on how your handling mirrored paris? as I would expect mutiple pool groupings not 1 sinlge group of disks.

Example: normaly you would do a grouping of disks between pools:

Example 1 zfs raidz2 with 6 disks. I can lose 2 disk and still have all my data. This stripes for speed and mirror for backup.
I'm also only using 1 vdev. (Separating vdevs would mater in your case across that many disks)

so when I saw you hit the slot limit, I was wondering what you were actually doing with 120+ disk in what setup and format.

TO me, it sounds like you went zfs mirror only 1 vdev across all 120?

When it sounds like your intention is to make multiple disk pool of 2 slots and mirror the 2 pools with each other. (user scripting later between the 2 pools to zfs send miro and maintain data across disk shelfs...)

SO I would have been implementing something like
disk shelf 1 row 1 disk shelf 1 called tank a and make a pool called tankarow1 to denote where in the rack...

this would result with tank a row 1 tank a row 2 etc etc
Then the next shelf tank b for next disk shelf etc etc... for help locating them and to have some form of raid parity across a row of disks in the shelfs

So you surprised me with going wiht all in one pool group.

but with all 120 in a zfs pool per the slot I was curious of what you ended up with?

3 minutes ago, bmartino1 said:

Thats awesome. My friend recently got himself a hand full of old decominisend telecom racks. He has a lot of weird compute all over. I get to test and be his tech support from time to time. :)

My only concern is on how your handling mirrored paris? as I would expect mutiple pool groupings not 1 sinlge group of disks.

Example: normaly you would do a grouping of disks between pools:

Example 1 zfs raidz2 with 6 disks. I can lose 2 disk and still have all my data. This stripes for speed and mirror for backup.
I'm also only using 1 vdev. (Separating vdevs would mater in your case across that many disks)

so when I saw you hit the slot limit, I was wondering what you were actually doing with 120+ disk in what setup and format.

TO me, it sounds like you went zfs mirror only 1 vdev across all 120?

When it sounds like your intention is to make multiple disk pool of 2 slots and mirror the 2 pools with each other. (user scripting later between the 2 pools to zfs send miro and maintain data across disk shelfs...)

SO I would have been implementing something like
disk shelf 1 row 1 disk shelf 1 called tank a and make a pool called tankarow1 to denote where in the rack...

this would result with tank a row 1 tank a row 2 etc etc
Then the next shelf tank b for next disk shelf etc etc... for help locating them and to have some form of raid parity across a row of disks in the shelfs

So you surprised me with going wiht all in one pool group.

but with all 120 in a zfs pool per the slot I was curious of what you ended up with?

I have 60 vdevs of 2 devices

As I mentioned, I was using zfs raidz1 6wide prior. But, I learned about the limitations of pool reduction (if ever necessary, and at the time i was for me). Thus, through a bunch of reading and comparison, I landed back to classic mirrored pairs.

The data is very static, and easily replaceable if i were to ever lose two drives from the same vdev. Which I feel is a low probability. I create a report of files to drive, so I would know what was lost if that were to ever happen.

As far as I know *and I could very well be wrong.... I need to have my data located on one pool, so that I can take advantage of hard-linking.

yes due to the nature of hard linking yes I can see that. Due to how the inode can spin and link... Personally, I would have to do some more research As I never thought nor expected hard linking in a disk layout to affect how I would manage storage.

I shared some videos with another users on the form going over symlink, soft link and hardlink on linux and unraid implementing with *arrs.

as I usually see implementation of the *Ars systems when hard linking is mentioned.

Some good videos to re-review in this forum post on hard links.
https://forums.unraid.net/topic/194824-question-about-hard-link-behavior-when-using-mntuser-for-media-scraping-nastools-multi-disk-array/

Your implementation makes sense to me now of what I would go one way of the other. Although I wouild suspect, needing the use of terminal CLI to manage that many vdevs in unraid.

with my example of tank a row 1
the hardlinks would be in that pool and the data is sent and shared to another pool on the same system so the hard links would still be tankarow1 in that example.

While rare for that many disk due to the number of vdevs, if a single vdev pair broke all data would be lost per my understanding of zfs implementation. (while mirrored, and easily recoverable to replace a disk to fix a vdev) a single vdev broken brings down the entiere system...

so I'm not saying your implementation is wrong or bad. I'm not sure that what you're wanting out of it, though...



Since the vdev is all in one pool. if a vdev fails, the entire pool is down... here some general zfs data info:

How multiple vdevs work in a pool

• Data distribution: Data is striped across the vdevs in the pool, which increases performance compared to a single vdev.

• Fault tolerance: Each vdev is a self-contained fault domain. If one vdev fails, the entire pool is lost. However, within a multi-vdev pool, a single drive can fail within a vdev (depending on the vdev type) without causing data loss for the pool as a whole.

• Adding vdevs: You can create a pool from multiple vdevs at once, or add new vdevs to an existing pool to increase capacity.

• Combining vdev types: It is possible to mix different vdev types in a single pool, such as combining RAID-Z vdevs with mirrored vdevs to get the benefits of each. 

How it works

• A pool is a collection of vdevs: A zpool is the top-level storage object and is composed of one or more vdevs. Each vdev is in turn made up of one or more physical devices.

• Data is distributed across vdevs: Data is spread across the different vdevs, but fault tolerance is maintained within each vdev, not across the pool itself. For example, if you have two mirror vdevs, losing a disk in one vdev will not affect the other vdev.

• Different vdev types can be combined: You can mix and match different vdev types (e.g., RAIDZ, mirror) in the same pool, but it is generally recommended to have vdevs of the same type and size for performance reasons.

• Support vdevs can be added: A pool can also include support vdevs like a cache vdev (L2ARC) for speeding up reads or a log vdev (SLOG) for speeding up synchronous writes. 

Edited November 13, 2025Nov 13 by bmartino1
spelling

@shelfactor the next release, which is expected in one or two weeks if all goes as planned, will support 200 devices per pool.

5 minutes ago, JorgeB said:

@shelfactor the next release, which is expected in one or two weeks if all goes as planned, will support 200 devices per pool.

@JorgeB Challenge Accepted. haha. Thanks again!

40 minutes ago, bmartino1 said:

yes due to the nature of hard linking yes I can see that. Due to how the inode can spin and link... Personally, I would have to do some more research As I never thought nor expected hard linking in a disk layout to affect how I would manage storage.

I shared some videos with another users on the form going over symlink, soft link and hardlink on linux and unraid implementing with *arrs.

as I usually see implementation of the *Ars systems when hard linking is mentioned.

Some good videos to re-review in this forum post on hard links.
https://forums.unraid.net/topic/194824-question-about-hard-link-behavior-when-using-mntuser-for-media-scraping-nastools-multi-disk-array/

Your implementation makes sense to me now of what I would go one way of the other. Although I wouild suspect, needing the use of terminal CLI to manage that many vdevs in unraid.

with my example of tank a row 1
the hardlinks would be in that pool and the data is sent and shared to another pool on the same system so the hard links would still be tankarow1 in that example.

While rare for that many disk due to the number of vdevs, if a single vdev pair broke all data would be lost per my understanding of zfs implementation. (while mirrored, and easily recoverable to replace a disk to fix a vdev) a single vdev broken brings down the entiere system...

so I'm not saying your implementation is wrong or bad. I'm not sure that what you're wanting out of it, though...



Since the vdev is all in one pool. if a vdev fails, the entire pool is down... here some general zfs data info:

How multiple vdevs work in a pool

• Data distribution: Data is striped across the vdevs in the pool, which increases performance compared to a single vdev.

• Fault tolerance: Each vdev is a self-contained fault domain. If one vdev fails, the entire pool is lost. However, within a multi-vdev pool, a single drive can fail within a vdev (depending on the vdev type) without causing data loss for the pool as a whole.

• Adding vdevs: You can create a pool from multiple vdevs at once, or add new vdevs to an existing pool to increase capacity.

• Combining vdev types: It is possible to mix different vdev types in a single pool, such as combining RAID-Z vdevs with mirrored vdevs to get the benefits of each. 

How it works

• A pool is a collection of vdevs: A zpool is the top-level storage object and is composed of one or more vdevs. Each vdev is in turn made up of one or more physical devices.

• Data is distributed across vdevs: Data is spread across the different vdevs, but fault tolerance is maintained within each vdev, not across the pool itself. For example, if you have two mirror vdevs, losing a disk in one vdev will not affect the other vdev.

• Different vdev types can be combined: You can mix and match different vdev types (e.g., RAIDZ, mirror) in the same pool, but it is generally recommended to have vdevs of the same type and size for performance reasons.

• Support vdevs can be added: A pool can also include support vdevs like a cache vdev (L2ARC) for speeding up reads or a log vdev (SLOG) for speeding up synchronous writes. 

Unfortunately, I prefer to use hardlinks.

And you are correct about if an entire vdev (both drives in one of the mirrored pairs) then yes, I would lose the pool. However, as I mentioned the one issue I had, taught me that I can recover from that loss if needed (time is the only thing lost really)

Thanks for the input!

On 11/13/2025 at 1:05 PM, JorgeB said:

@shelfactor the next release, which is expected in one or two weeks if all goes as planned, will support 200 devices per pool.

Thanks again for the help! Great to see this: