Hello

 

I have been running 12x12TB sas3 drives on sas2 enclosures for a while now, I have also been using xfs since I didn't have anywhere to offload the data to try zfs etc.   I now have delivery today of 12x12tb sas3 drives that I plan to setup zfs and then move the data to them, then add the existing drives to zfs to expand storage to 24x12tb drives.

 

In a week I will be receiving enclosures for sas3 compatability.

 

Now it's sas3 cards, sas3-sas2 cables, sas2 enclosures with sas3 drives.

 

It will be sas3 all the way to the drives next week.

 

Is zfs the way to go?  Is my plan the best path?  Anything I'm not thinking about?

 

Edited March 21, 20242 yr by DieFalse

OK.   I am feeling like this was a HORRIBLE idea.

 

My ZFS pool 1x12 RaidZ2 is only getting ~10MB/s write speeds.    This is way lower than I am used to, and ZFS was chosen / previously suggested for speed since I am using matching disks.

 

Is this a known and fixable issue?   

 

Also,  it appears "Sync Filesystem" sticks for a long time when stopping the array or rebooting.    I don't think it actually finishes.

blinky-diagnostics-20240320-2331.zip

Edited March 21, 20242 yr by DieFalse

7 hours ago, DieFalse said:

Is this a known and fixable issue? 

No, a single disk should perform much better than that, there may be a problem with one of the disks, or something else, I would suggest creating for a example a two device zfs mirror and retest, if the same create another one with different disks.

This worked,  I built 3x4 mirror pools, tested each until I found the slow down,  located in in one test group.   then isolated to two drives so far.   I pulled the two drives as I have two spares, built my RaidZ2 1x12 Pool and am now getting 250MB/s+ on transfer, much more acceptable given I have 101TB of data to transfer before I can format the original 12 drives and add as the second 12 pool making it 2x12 RaidZ2.

 

Once thats done, I will test the two drives individually and return the culprit(s) to ensure I have spares on hand.