mbc0 Posted March 17, 2020 Share Posted March 17, 2020 Hi, I am wondering if I am not using my devices at optimily as I could? I am getting such terrible speeds and really think things should be better! I am getting a parity build speed of approx 9-10 mb/s with the occasional 40-50mb/s I have the following hardware. M/B: Gigabyte Technology Co., Ltd. X399 DESIGNARE EX-CF Version x.x - s/n: Default string BIOS: American Megatrends Inc. Version F12g. Dated: 05/08/2019 CPU: AMD Ryzen Threadripper 2950X 16-Core @ 3500 MHz HVM: Enabled IOMMU: Enabled Cache: 1536 KiB, 8192 KiB, 32768 KiB Memory: 48 GiB DDR4 (max. installable capacity 512 GiB) Network: bond0: transmit load balancing, mtu 1500 eth0: 1000 Mbps, full duplex, mtu 1500 eth1: interface down eth2: 10000 Mbps, full duplex, mtu 9000 Kernel: Linux 4.19.98-Unraid x86_64 OpenSSL: 1.1.1d Port 1 of my SAS3008 12gb/s card goes to 4 drives (including Parity Disk) but ALL drives are connected via a 6gb/s Backplane The rest are either connected via the Dell RES2SV240 or the motherboard SATA But all drives are connected via the 6gb/s Backplane. [0:0:0:0]disk PNY Tech USB 2.0 FD 1100 /dev/sda 4.05GB ---- USB With unRAID [2:0:0:0]disk ATA ST3500312CS SC13 /dev/sdi 500GB ---- Motherboard SATA [4:0:0:0]disk ATA SAMSUNG HD204UI 0001 /dev/sdw 2.00TB ---- Motherboard SATA [6:0:0:0]disk ATA SAMSUNG HD204UI 0001 /dev/sdx 2.00TB ---- Motherboard SATA [8:0:0:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdy 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:0:0]disk HGST HUS726060AL5210 AD05 /dev/sdb 6.00TB ---- Dell RES2SV240 Extender Port 1 of SAS3008 [11:0:1:0]disk SEAGATE ST6000NM0095 E004 /dev/sdc 6.00TB ---- Dell RES2SV240 Extender Port 1 of SAS3008 [11:0:2:0]disk ATA ST6000NM0115-1YZ SN04 /dev/sdd 6.00TB ---- Dell RES2SV240 Extender Port 1 of SAS3008 [11:0:3:0]disk HGST HUS726060AL5210 AD05 /dev/sde 6.00TB ---- Dell RES2SV240 Extender Port 1 of SAS3008 [11:0:4:0]disk ATA ST6000VN0033-2EE SC60 /dev/sdf 6.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:5:0]disk ATA ST6000DM004-2EH1 DN03 /dev/sdg 6.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:6:0]disk ATA ST6000DM003-2CY1 0001 /dev/sdh 6.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:7:0]disk ATA ST2000DL004 HD20 0001 /dev/sdj 2.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:8:0]disk ATA ST3000DM008-2DM1 CC26 /dev/sdk 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:9:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdl 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:10:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdm 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:11:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdn 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:12:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdo 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:13:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdp 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:14:0]disk ATA TOSHIBA HDWD130 ACF0 /dev/sdq 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:15:0]disk ATA SAMSUNG HD204UI 0001 /dev/sdr 2.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:16:0]disk ATA WDC WD30EFRX-68E 0A82 /dev/sds 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [11:0:17:0]disk ATA ST1000DX001-1CM1 CC43 /dev/sdt 1.00TB -NOT PART OF ARRAY [11:0:18:0]disk ATA TOSHIBA HDWD110 A8J0 /dev/sdu 1.00TB -NOT PART OF ARRAY [11:0:19:0]disk ATA ST3000DM008-2DM1 CC26 /dev/sdv 3.00TB ---- Dell RES2SV240 Extender Port 2 of SAS3008 [N:0:4:1]disk Samsung SSD 970 EVO Plus 1TB__1 /dev/nvme0n1 1.00TB - CACHE DRIVE The parity disk is one of 4 drives connected to one of the ports of my sas3008 pci SAS controller Would I be better to connect all drives on 1 port of the sas card going through the dell extender? My thinking was that one port to host the parity and 3 other drives would help speed? Any advice greatly appreciated! Quote Link to comment
mbc0 Posted March 20, 2020 Author Share Posted March 20, 2020 Hi just thought I would bump this as the parity took 4 days to complete and this used to take a day at most! I have seen my speeds drop from 150mb/s to 9-15mb/s for parity. I have noticed my system is generally very sluggish, sometimes waiting 10 seconds or so for a response even thought the drives are all spun up and being on a 10gb/E fibre connection. Any recommendations please? Quote Link to comment
JorgeB Posted March 20, 2020 Share Posted March 20, 2020 Please post diags grabbed during a parity check. Quote Link to comment
mbc0 Posted March 20, 2020 Author Share Posted March 20, 2020 thanks for your reply, "unfortunately" the parity rebuild has finished, it made my system so slow I do not want to start another rebuild for another 4 days, is there any other way I can provide something without rebuilding parity or can I just start a parity check, grab the diags and cancel? Thank you Quote Link to comment
JorgeB Posted March 20, 2020 Share Posted March 20, 2020 You can start a parity check, wait 5 minutes for speed to stabilize, if still slow grab diags then cancel it. Quote Link to comment
mbc0 Posted March 20, 2020 Author Share Posted March 20, 2020 Hi @johnnie.black I started the check, waited 5 mins, it slowed from 100 mb/s to 20 mb/s I grabbed the diagnostic so hope you can see if something is going on? Can I ask what you think about the way I have connected things and if you think I can make any improvements there please? Thank you for your time! unraidserver-diagnostics-20200320-1614.zip Quote Link to comment
JorgeB Posted March 20, 2020 Share Posted March 20, 2020 Syslog is filled with HBA related errors, I don't even see the last parity check start, reboot first then repeat the procedure above, make sure nothing else is using the array, there were reads from multiple array disks when these diags were captured. Quote Link to comment
JorgeB Posted March 20, 2020 Share Posted March 20, 2020 1 hour ago, mbc0 said: Can I ask what you think about the way I have connected things and if you think I can make any improvements there please? Though that's not the issue here if you connected both cables from the HBA to the expander it would be better for performance, but it would limit you to 16 ports. 1 Quote Link to comment
mbc0 Posted March 20, 2020 Author Share Posted March 20, 2020 (edited) Thanks for the reply, I will reboot tonight but firstly connect both cables to the HBA as I do not require all 24 ports currently. Also I use the mainboard sata ports through a reverse cable to the backplane to run SSD Drives, would that be ok or even neccassary? Thank you for your advice! Edited March 20, 2020 by mbc0 spelling Quote Link to comment
mbc0 Posted March 21, 2020 Author Share Posted March 21, 2020 Hi @johnnie.black I have now connected both cables from the SAS3008 to the RES2SV240 as recommended and used 2 reverse breakout cables from the 8 on-board SATA motherboard ports to the other 8 bays on the backplane so all 24 bays are active 🙂 Is there a way of ensuring that both connections are active from the SAS3008 as I only had 1 extra cable to connect from the SAS3008 to the RES2SV240 and was not 100% sure if it is functioning correctly and would order a new one if I could find a way of telling? Many Thanks Again 🙂 Quote Link to comment
JorgeB Posted March 21, 2020 Share Posted March 21, 2020 Check output of: cat /sys/class/sas_host/host11/device/port-11\:0/sas_port/port-11\:0/num_phys 4 = single link 8 = dual link P.S.: If the command doesn't work post new diags since host# is based on the last ones and it can change. 1 Quote Link to comment
mbc0 Posted March 21, 2020 Author Share Posted March 21, 2020 Hi, Things certainly seem alot snappier! I can confirm that I am showing dual link, I will do a parity check soon and post the diags, thank you so much for your advise & information! Quote Link to comment
mbc0 Posted March 25, 2020 Author Share Posted March 25, 2020 Hi @johnnie.black I hope you, your friends and family are safe and well during this pandemic 😞 I like most of the world am in lockdown so finding more time to work on my server so I have attached some diagnostics when my server literally ground to a halt whilst copying some data and if you get a chance wonder if you can look through it to see what can be causing the issue. Many Many Thanks unraidserver-diagnostics-20200325-0116.zip Quote Link to comment
JorgeB Posted March 25, 2020 Share Posted March 25, 2020 Syslog is flooded with wrong csrf token errors, this means there's are multiple open browser windows, close them all and reboot, then post new diags after there are problems again. Quote Link to comment
mbc0 Posted March 25, 2020 Author Share Posted March 25, 2020 Do you mean Windows Explorer Windows? I have 4 open if that is the case Quote Link to comment
JorgeB Posted March 25, 2020 Share Posted March 25, 2020 No, open browser windows in the GUI. https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=545988 Quote Link to comment
mbc0 Posted March 25, 2020 Author Share Posted March 25, 2020 ah, I see! Thank you, I will reboot a bit later this afternoon 🙂 Quote Link to comment
mbc0 Posted April 1, 2020 Author Share Posted April 1, 2020 Hi @johnnie.black Hope you are safe & well! It's the 1st of the month and the parity rebuild has started and I still have whoefully slow speeds. I also notice I have sync errors already 😞 Diagnostics attached, I would really appreciate any help you can offer on this. Thank you unraidserver-diagnostics-20200401-0215.zip Quote Link to comment
JorgeB Posted April 1, 2020 Share Posted April 1, 2020 Did you just un-pause it? Elapsed time is less than a minute, stats need to stabilized, according to diags it's going close to 50MB/s, which seems about right since it's reaching the slower inner sectors of the 2TB disks. You can also try the diskspeed docker, there might be one more more disks under performing. Quote Link to comment
mbc0 Posted April 1, 2020 Author Share Posted April 1, 2020 Hi, yes I did just unpause it, I am keeping it paused as I started moving data from disk to disk yesterday morning to allow me to create some redundant disks but my disk to disk performance is painfully slow as well. once this has completed I will restart the parity leave it for an hour and take another diag. Thank you Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.