Hey Guys,

 

I swapped to new hardware and was doing a pairty check but I seem to be getting much slower pairty check than before. This system's specs are much better than the previous hardware. Just curious as to what could be the issue. The previous hardware was going roughly 60-80 mb/s but this is barely hitting 20 mb/s.

 

Diagnostics are attached. 

 

Thanks for any help, 

zeus-diagnostics-20210721-1542.zip

7 hours ago, thezyth said:

I swapped to new hardware and was doing a pairty check but I seem to be getting much slower pairty check than before. This system's specs are much better than the previous hardware. Just curious as to what could be the issue. The previous hardware was going roughly 60-80 mb/s but this is barely hitting 20 mb/s.

Not sure for your Parity Check, but you do have issues on your cache drive. Should not be impacting the check though.

 

I'd suggest fixing that first, once someone with more experience on those issues can advise you.

 

Then, use the DiskSpeed docker to check for any issues with disk speed.

Yeah I swapped out the cache drive already. I realized that was causing some issues with dockers also. No sure what could possibily decrease the speed by 1/3. I used disk speed to check for issues and it seems like the sas 2tb drives are much much slower than the other drives maybe this could cause the issue. But I had these drives in the previous setup also.

 

 

3 minutes ago, thezyth said:

and it seems like the sas 2tb drives are much much slower than the other drives

20MB/s slow or just slower? Parity check can't go faster than the slowest disk at any point.

The slowest drive is at 73 mb/s. So disk 1-4 are my slowest drives and they were also used in my previous build but I was getting much better parity check speeds. Maybe it's due to the change in the interface for the drives?

That points to another issue, not the disks, is the controller the same you were using before? Also check which CPU scaling governor is in use.

The last build was using breakout cables direclty from the drives to the motherboard. The new build is using ASR 72405 raid controller with a BPN-sas-846a backplane. I will look into what the CPU scaling is enabled or not. 

I just looked at the CPU scaling governor and it is currently set to ON-Demand.

 

Ran a controller bench mark and I don't see anything out of the ordinary here either. Anyone have any suggestions to what else I could check?

That all looks good, though it could still be a controller issue that manifests itself only during a parity check, I have no experience with that controller, do you have another one you could test with, like an LSI HBA?

Nope I don't have a spare controller to test with. I did end up running the pairty check for longer and it seems like one of the drives did end up dying during the check....and another one of the sas drives is now throwing up major issues. This may have just been due to dying drives. Will update when I get new drives and get the system up and running.  WHile the parity check was running I was getting some regular speeds instead of the abysmal 20mb/s. 

I did end up swapping out disk2 for a new hdd but the rebuild seems to be going equally slow now. I'm more leaning towards maybe a controller issue? Any recommendations on a SAS controller that I can use with my Supermicro 848?

I think I may have fixed it. Details below.

 

The mother board that I have currently is a : Supermicro X9QRi-F+ with the following expansion slots

-Two (2) PCI Express 3.0 x16 slots (Slot2/Slot4), Slots 

-Two (2) PCI Express 3.0 x8 in x 16 slots (Slot1/Slot3)

 

The raid controller was plugged into SLOT1. Swapped over to slot 4 and now. I'm getting much better speeds. Will update if the speeds stay steady.

7 hours ago, thezyth said:

Swapped over to slot 4 and now. I'm getting much better speeds.

Strange, it shouldn't make any difference in this case.

I'm not sure what could be causing this issue then. Do you have any other possible suggestions to try?

My suggestion would be to test with a different controller, if you had one.

On 7/27/2021 at 11:56 PM, JorgeB said:

My suggestion would be to test with a different controller, if you had one.

I'll try to buy a different controller when I get a chance and see if that makes a difference. 

On 7/27/2021 at 11:56 PM, JorgeB said:

My suggestion would be to test with a different controller, if you had one.

Do you have different controller in mind that would work better? I'm currently doing a rebuild of disk 1 and it seems to be slowed down to a crawl again. Only getting 20mb/s on the rebuild right now. I have attached a new diagnostic. 

 

More info:

 

Rebuilding a 2tb drive that failed on me to a 8tb drive which was shucked. 

 

Full Specs of build:

 

PSUS upermicro PWS-1K62P-1R

MOBO X9QFI-F+

SAS BackplaneBPN-SAS-846A

Raid Controller ASR 72405

Ram 4gb DDR3 1333 MHZx16

CPU E5-4650 x4

 

 

zeus-diagnostics-20210817-1311.zip

18 minutes ago, thezyth said:

Do you have different controller in mind that would work better?

 

I ended up just canceling the rebuild because the data on the drive is not that important. I started a new config and did a new pairty sync. The pairty sync is going much much faster than the rebuild. I'm kind of confused as to what the issue could be. The sync is going at 100mb/s while the rebuild was maxing out at 30mb/s....

10 hours ago, thezyth said:

The sync is going at 100mb/s

That suggests it's not the controller.

16 hours ago, JorgeB said:

That suggests it's not the controller.

 

I moved data off the last two old drives onto a different drive. Pulled the old drives out and did a new config. The current speed of the parity sync is now back to 25-30 mb/s... :')  No docker containers running no VM's running. I'm so confused at what it could be.

I do have one SMR drive within the array. Could this be the cause of the slow downs?

22 hours ago, JorgeB said:

That suggests it's not the controller.

 

The speeds seem to increase for a few seconds and tapper of...is this normal behaviour? 

5 hours ago, thezyth said:

I do have one SMR drive within the array. Could this be the cause of the slow downs?

Possibly.

 

1 hour ago, thezyth said:

The speeds seem to increase for a few seconds and tapper of...is this normal behaviour? 

No, but also doesn't look like a controller related issue.