Hi there,
I'm trying to find a sweet spot for a ZFS configuration with 15 disks and potentially a level 2 ARC, and thus, am benchmarking around a bit.
Configuration is a 16core XEON with 144GB of RAM, 15 WD RED 14TB drives, a 1TB EVO970 NVME and 10GBE ethernet connectivity.
I access via Samba with MacOSX.
While trying different RAID-Z configurations, with trying 16GB write once, read multiple times scenario:
Writing directly to the NVME as a single drive pool (for testing only) I can reach 880MB/s.
Reading from that same place multiple times I get 574MB/s.
Writing to a 2x7raidz1 I'm getting write speeds at about 760MB/s, but the exact same read speeds of 574MB/s.
Since the file is easily small enough to fit in the ARC, I would assume, latest with the second read it comes from RAM, so I doubt the read speeds on the box are the problem.
So, my question is, why is my read speed somehow capped at 574MB/s?
MTU-size is configured to 9000 and SMB-signing is switched off, I don't have any other ideas to try.
Thanks,
M