theonewithin Posted October 21, 2022 Share Posted October 21, 2022 Hello all! Trying to diagnose why the performance of my setup is slower than expected. Have 10x 8TB SAS drives connected through a SAS expander to a typical LSI 8 Channel HBA. Previously when using SATA based driver performance was stellar and during parity operations I could achieve well over 1.5GB/sec With these SAS drives I appear to be hitting some sort of bottleneck. Only maxing out at around 900MB/s total transfer rate. Each drive when tested alone can achieve over 200MB/s. Adding 4 drives directly to the HBA and bypassing the expander shows no change at all. If I do a scrub of a set of 3 SAS 15K cache drives running on the same setup the speed of this is very slow. Once pausing the parity check the performance shoots over 700MB/s which is the expected speed of these drives. Have run out of things to try and am now here to find help! I hope someone can figure out where the bottleneck of my system is. Thanks all! Have attached diagnostics. big-server-diagnostics-20221022-0038.zip Quote Link to comment
JorgeB Posted October 21, 2022 Share Posted October 21, 2022 Run the diskspeed docker, it will test disks individually and controller bandwidth with all together. Quote Link to comment
theonewithin Posted October 21, 2022 Author Share Posted October 21, 2022 It doesn't give a controller option. All disks run at expected speed as per my original post. Quote Link to comment
JorgeB Posted October 21, 2022 Share Posted October 21, 2022 Post the screenshot from the test. Quote Link to comment
theonewithin Posted October 21, 2022 Author Share Posted October 21, 2022 Not sure why you won't just accept that I have done it and per drive performance is fine. Quote Link to comment
JorgeB Posted October 21, 2022 Share Posted October 21, 2022 Not that one, the controller test. Quote Link to comment
theonewithin Posted October 21, 2022 Author Share Posted October 21, 2022 (edited) As I said, It gives no option for that. Wait... I just found it. Why is it hidden behind a button? Edited October 21, 2022 by theonewithin Quote Link to comment
theonewithin Posted October 21, 2022 Author Share Posted October 21, 2022 Ok. That has pointed out the issue. For some reason the card is running at 5GT/s over 2 lanes... Odd. Time to start unplugging stuff I guess. Quote Link to comment
theonewithin Posted October 21, 2022 Author Share Posted October 21, 2022 Well crap. HBA card appears to be the issue. Won't negotiate more than 2x lanes. Swapped in a GPU and it connects fine with 4-16 lanes. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.