I have 4x 1TB SSDs and I am debating whether or not I should arrange them in a dual mirror vdev pool or raidz2 but I have some questions about how a dual mirror vdev pool distributes data and handles disk failure.

 

Config Option #1:

    NAME        STATE     READ WRITE CKSUM
    cache       ONLINE       0     0     0
      mirror-0  ONLINE       0     0     0
        disk1   ONLINE       0     0     0
        disk2   ONLINE       0     0     0
      mirror-1  ONLINE       0     0     0
        disk3   ONLINE       0     0     0
        disk4   ONLINE       0     0     0

 

1. What happens if only a single disk is lost from only one of the mirror vdevs?

 

2. Can lose one mirror vdev and still have a functioning pool?

 

3. What if disk1 and disk3 are lost but disk2 and disk4 are fine?

 

4. At which point is this entire cache pool toast?

 

Thanks.

 

For space and preformance 4 disk in a raid z1 with 1 vdev...

 

RecommendationL

If your priority is performance, choose the dual mirror vdev pool.

If your priority is maximum fault tolerance, RAIDZ2 is the better option.

Using ZFS with your proposed configuration (a dual mirror vdev pool) provides a robust level of fault tolerance and performance, but it comes with specific trade-offs and behaviors. Let's break down your questions

 

Quote

What happens if only a single disk is lost from only one of the mirror vdevs?

If a single disk is lost from one of the mirror vdevs (e.g., disk1 or disk2 from mirror-0, or disk3 or disk4 from mirror-1), the pool remains online and fully functional.

 

-The remaining disk in the affected mirror will continue to serve data for that vdev.

 

-The pool can tolerate this scenario because mirrors are designed to provide redundancy.

 

-It's crucial to replace the failed disk and rebuild the mirror as soon as possible to restore redundancy.

 

Quote

Can you lose one mirror vdev and still have a functioning pool?

No, you cannot lose an entire mirror vdev and still have a functioning pool.

 

-In ZFS, the pool is only as resilient as its individual vdevs. If a vdev (e.g., mirror-0 or mirror-1) is entirely lost, the pool becomes unusable because the data distribution across vdevs requires all vdevs to be available.

 

-To ensure pool integrity, at least one disk from each vdev must remain operational.

 

Quote

What if disk1 and disk3 are lost but disk2 and disk4 are fine?

 

In this scenario, the pool remains functional:

Each vdev still has one healthy disk (disk2 for mirror-0 and disk4 for mirror-1), so the pool can serve data without interruption.

However, redundancy is completely lost for both vdevs. If another disk fails before replacements are introduced and rebuilt, the pool will fail.

 

Quote

At which point is this entire cache pool toast?

The pool becomes unusable in these situations:

 

Complete failure of a single vdev:

If both disks in either mirror-0 (disk1 and disk2) or mirror-1 (disk3 and disk4) fail, the pool is unrecoverable.

 

Simultaneous loss of the last operational disk in each mirror:

For example, if disk1 and disk4 fail after disk2 and disk3 have already failed, the pool will be lost.

 

Corruption across multiple vdevs beyond what redundancy can handle:

While ZFS offers mechanisms to detect and correct errors (e.g., checksumming), if corruption impacts data across all vdevs, recovery may not be possible.

 

comparisons:

 

 

 

Dual Mirror Vdev Pool vs. RAIDZ2

If you're weighing a dual mirror vdev pool against RAIDZ2, here are some points to consider:

 

Performance:

Dual mirrors generally provide better random read and write performance than RAIDZ2 because of their simpler structure and increased IOPS (input/output operations per second).

 

Capacity:

Dual mirrors give you 2 TB of usable space (50% of the total capacity).

RAIDZ2 gives you 2 TB of usable space as well, but with the ability to survive any two disk failures, regardless of which disks fail.

 

Fault Tolerance:

RAIDZ2 offers better fault tolerance than dual mirrors because it can survive two simultaneous disk failures anywhere in the pool.

Dual mirrors can only survive a single disk failure per vdev.

 

Rebuild Times:

Rebuilding a mirror is faster and involves less stress on surviving drives compared to RAIDZ2, which requires recalculating parity for the entire stripe during a rebuild.