Slow parity sync ,is 80mb/s right?

[dafa]

Hi, I am getting a 80mb/s sync

its quiet slow?or is it right?

Here is the Diagnostics

Please help me with this

tower-diagnostics-20230523-1100.zip

[Kilrah]

Seems about right, there could be a bottleneck somewhere with that many drives but since parity operations are limited to the slowest drive and these old 3TB ones are likely not able to do more it's probably more of the latter. 

Edited May 23, 2023 by Kilrah

[dafa]

1 hour ago, Kilrah said:

Seems about right, there could be a bottleneck somewhere with that many drives but since parity operations are limited to the slowest drive and these old 3TB ones are likely not able to do more it's probably more of the latter. 

Thanks a lot

But I have done the Disk Speed test in docker ,the result shows the slowest one is 90mb/s ,and its not the disk in the array but the unassigned device which even did not mounted...

So is there any other reason I can make some improvements.

[Kilrah]

You'd have to describe your whole setup with how disks are connected through what HBAs/expanders, what mobo slots are used etc so someone can figure out if there's a bottleneck somewhere.

[dafa]

5 hours ago, Kilrah said:

You'd have to describe your whole setup with how disks are connected through what HBAs/expanders, what mobo slots are used etc so someone can figure out if there's a bottleneck somewhere.

Thanks for your help 

here is the disk connection map

Host bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 DMI2 (rev 04)

PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 2c (rev 04)

RAID bus controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
     13 drives(left 13 in the picture ,main disks in parity sync situation)

PCI bridge: Intel Corporation Xeon E7 v2/Xeon E5 v2/Core i7 PCI Express Root Port 3a (rev 04)

Serial Attached SCSI controller: Broadcom / LSI SAS2008 PCI-Express Fusion-MPT SAS-2 [Falcon] (rev 03)
     34 drives

 

 

and it is going down to 14m/s ……

I cant figure out why this is happening ,all the 8 disk(2parity and 6 data disks) are in the first bus, and no background app is running .

 

 

[Kilrah]

This doesn't say how they're connected. So you have 13 drives connected to an 8-port card, and 32 drives connected to another 8-port card? That means you have expanders, and it's precisely what's between the controllers and drives in terms of hardware and cabling that's needed. Obviously there'll be some bandwidth sharing at play here.

 

Also what's that spam of something connecting/disconnecting via ssh 10 times per second?

Edited May 25, 2023 by Kilrah

[dafa]

Just now, Kilrah said:

This doesn't say how they're connected. So you have 13 drives connected to an 8-port card, and 32 drives connected to another 8-port card? That means you have expanders, and it's precisely what's between the controllers and drives in terms of hardware and cabling that's needed.

the 13 drive is in a Dell R720XD server rack ,and the another 34 is in a supermicro disk chaise ,connect with SFF-8088 cable and hba card .

[dafa]

4 minutes ago, Kilrah said:

This doesn't say how they're connected. So you have 13 drives connected to an 8-port card, and 32 drives connected to another 8-port card? That means you have expanders, and it's precisely what's between the controllers and drives in terms of hardware and cabling that's needed.

what kind of disk connection describ is better for understanding ,I am new…… sorry

[dafa]

[JorgeB]

Run a controller test with the diskspeed docker to confirm no controller bottlenecks.

[Kilrah]

1 hour ago, dafa said:

what kind of disk connection describ is better for understanding ,I am new…… sorry

Physically what's connected to what...

 

1 hour ago, dafa said:

the another 34 is in a supermicro disk chaise ,connect with SFF-8088 cable and hba card

Only one cable or 2? If one that's 4 SAS2 lanes, shared between 34 drives so yes when all are accessed simulataneously that would bean about 80MB/s to each of them.

 

Single drive disk speed test would not show anything since it's only when a significant number are accessed simulataneously that the bottleneck would be noticeable.

[dafa]

1 hour ago, Kilrah said:

Physically what's connected to what...

 

Only one cable or 2? If one that's 4 SAS2 lanes, shared between 34 drives so yes when all are accessed simulataneously that would bean about 80MB/s to each of them.

 

Single drive disk speed test would not show anything since it's only when a significant number are accessed simulataneously that the bottleneck would be noticeable.

I can get what you are takling about ...

But here is the situation ,all the 8 drives are syncing now is in the dell 720xd server itself .There should be no conection issue, and even there is one ,how could it slow down to 14m/s now……