Copy to disk capped at ~32 MB/s

[Apil]

Hi everyone!

 

I hope someone is able to help me figure out this problem i am having.

 

I recently bought a new server to setup with unraid.

Its an HP ProLiant DL380p gen 8, 2 x 12 core, 128gb 1833 mhz ecc ddr3 ram dual 10 gig nic, Smart Array P420i Controller setup in HBA mode and with 8 WD red 3 TB disks (HxV83Km9TeRi).

 

Everything seems to be working fine, expect for that i am trying to copy my data back on to this new server and there is a bottleneck at about 30-33 MB/s that i cannot figure out where is.

 

I have done full preclear on all disks successful.

 

I have tried to copy both from an external usb drive connected via usb3 directly to the server, then copying with MC in terminal on the server, and also copying from network to disks.

No matter how i copy to disks there is a shared 30 MB/s write speed, for example if i have the usb conected drive copying, it will start at 30 MB/s and stay there, until i start a network copy (even to a different drive), and the bandwith that the network runs at will be taken away from the usb copy, so it is clearly shared.

 

I have tried to run the diskspeed benchmark test, and all the drives seems fine and starts off at 150 MB/s and ends at 90 MB/s at the end of the test, as expected.

Same values is seen when preclear ran.

 

Benchmark of the P420i also showed full 150 MB/s speed when ran simultatiously across all drives.

 

I even tried resetting the array and running it without a parity drive, still 30 MB/s.

 

Does anyone have any idea why this is happening, and where the bottleneck could be ?

 

Best regard

Apil

[JorgeB]

https://forums.unraid.net/topic/50397-turbo-write/

[Apil]

Thank you JorgeB.

 

I have now tried to enable reconstruct write, and it did start off great with 100 MB/s on the transfer only to fall down slowly to 30 MB/s again furing the next 15 secounds.

 

Is 30 MB/s just what i can expect due to filesystems or how the array works ?

I dont even have a parity disk enabled at this moment.

Edited June 1, 2021 by Apil

[JorgeB]

14 minutes ago, Apil said:

Is 30 MB/s just what i can expect due to filesystems or how the array works ?

No, turbo write should be as fast as your slowest disk at that position, assuming there are no controller bottlenecks, I can write at 200MB/s+ to some of my servers.

[Apil]

Okay, then i am lost for reasons why i am seeing this bottleneck.

I also tried to use unbalance to move files inbetween disks, and was seeing the same bottleneck.

[JorgeB]

14 minutes ago, Apil said:

I also tried to use unbalance to move files inbetween disks, and was seeing the same bottleneck.

With Unbalance it's normal, since it's copying from array disk to another array disk, turbo write doesn't work for that.

[SimonAG]

There was some bad firmware that caused slow disks years ago on HP controller. https://community.hpe.com/t5/ProLiant-Servers-ML-DL-SL/Smart-Array-P420i-extremely-slow-writes/td-p/6497596

 

Might be the cause, try in MC over ssh between disks, that should cut out network as the cause.