I've been searching for how to solve this problem for weeks to no solution, so hopefully this post will help.
I have a new server, about a month old now,
ASUS Z12PE-D16 WS, XEON E5-2697 v4 @ 2.3Ghz, 128Gb RAM @ 2133Mhz.... not exactly a budget build... 16X6TB Drives in Array so far.,...
During copies from my old server over the network everything will be going find 60+ MB/s (in and out limitation of PC doing the transfer). Then suddenly everything will drop to 1 MB/s and access all files on the array and the entire web interface comes to a hault. This occours and can go on for hours and hours at full speed before this happens and sometimes minutes...
I've just finished a party check at 130+ MB/s and I can pull off each drive at atleast 150+ MB/s ...
The only error I'm getting as such is in the glances docker (screenshot attached) that shows CUP_IOWAIT going through the roof.
With Cache Drive disabled for partition, 60+ MB/s copy normal, 1+MB/s copy during CPU_IOWAIT issues..
With Cache Drive Enabled for partition, 60+ MB/s Copy normal, but also 60+ MB/s copy during IO-WAIT but server is still very sluggish..... that makes me think this is happening all the time and I only notice it when copies are happening...
Most drives are WD RED 6TB connected directly to motherboard, including parity, with 4 drives being enterprise seagate SAS through a LSI 9311-8I.
Any thoughts? this is driving me crazy!!
Thanks
tower-diagnostics-20170207-1917.zip