Rubydoo Posted October 24, 2019 Share Posted October 24, 2019 (edited) Hi all, Having a really weird problem with my Unraid array that I just can't put my finger on, and I'm hoping someone will be able to help out. The array runs perfectly for the first day or so, then begins slow down to a crawl, running at between 2 and 6mb/sec. The syslog is showing drive errors, which I realise is Unraid being incapable of spinning down SAS drives, and another feature of the SAS controller that Unraid cannot support. I was running FreeNAS when I first built the system hoping to use Bhyve for VMs, and it ran beautifully, no slowdowns, went for weeks at 100mb+/sec easy, but I switched to Unraid as the KVM VM is much better, and now I'm having a pretty bad time with the actual storage portion of the namesake's purpose. I am temporarily regretting the move, but I'm sure someone here will know the solution In the meantime, I'm stuck stopping the array, rebooting Unraid, then starting the array again, every time this happens. A bit of a bummer. I have tried just stopping the array and restarting it, but it does not fix the problem. I have attached photos of the speed before and after rebooting Unraid. The array is btrfs encrypted. The server specs are: Motherboard: ASUS Z9NA-D6 CPUs: 2 x Intel Xeon E5-2440 RAM: 24GB DDR3 ECC Registered Cache: 400GB Seagate ST400FM0073 SAS SSD Array: 3 x 3TB HP/Seagate MB3000FCWDH SAS - 1 in parity, 2 in array SAS controller: ASUS PIKE 2008 (expansion card) Any help would be greatly appreciated. Thank you. beemoth-diagnostics-20191024-0137.zip Edited October 24, 2019 by Rubydoo Quote Link to comment
limetech Posted October 24, 2019 Share Posted October 24, 2019 2 hours ago, Rubydoo said: The syslog is showing drive errors, which I realise is Unraid being incapable of spinning down SAS drives, and another feature of the SAS controller that Unraid cannot support. Try clicking on your SAS drives on Main and set 'Spin down delay' to Never. It's not an issue with Unraid supporting SAS controllers - they work fine with SATA devices - it's SCSI device spin down which is not supported at this time. Quote Link to comment
Rubydoo Posted November 4, 2019 Author Share Posted November 4, 2019 Hi, Did this, gave it time, still does exactly the same thing. Slowing to a complete crawl again after some time. Can anyone offer some help? Thanks. Quote Link to comment
Rubydoo Posted November 4, 2019 Author Share Posted November 4, 2019 Also here are the latest diagnostics. beemoth-diagnostics-20191104-0558.zip Quote Link to comment
Rubydoo Posted November 4, 2019 Author Share Posted November 4, 2019 And after reboot it runs like this. Quote Link to comment
glennv Posted November 4, 2019 Share Posted November 4, 2019 Just a hunch but could it be your cache ssd that is not trimmed untill you reboot ?That will slow it down to a crawl.You can run the fstrim command manualy agianst the /mnt/cache or schedule it with the usr scripts plugin. I run it nightly against my ssds. Quote Link to comment
testdasi Posted November 4, 2019 Share Posted November 4, 2019 Where are you doing the copying from and to? In other words, Where are the "FreeBSD...iso" and "highsierra.img" saved? Where are "Downloads" and "Linux" located? You screenshot looks to be MacOS-based. Are you copying over wifi / wired network / br0 bridge? Are you able to test on a Windows-based machine? Quote Link to comment
Rubydoo Posted November 4, 2019 Author Share Posted November 4, 2019 Hi, The first screenshots are from Unraid to the client (FreeBSD copied to the client machine), and the second screenshots are from the client to Unraid (highsierra.img copied to the Unraid share). Unraid is accessed via SMB in both cases. The test is the same on Windows, and the clients are wired/lan. The problem appears after the server has been on for a few days. It runs perfectly for about a day or so, then gradually slows to a crawl. Thanks. Quote Link to comment
Rubydoo Posted November 4, 2019 Author Share Posted November 4, 2019 Hmm will try that Glenn, good idea. I didn't think of that. Will let you know how it goes. Thank you for the suggestion. Quote Link to comment
hans-peter123 Posted November 4, 2019 Share Posted November 4, 2019 I have a Idea because i saw something similar. No solution, but maybe one step forward. 1. Install the stats plugin (if you don't have it yet). 2. Then reboot and directly open the stats page, switch to system stats and change the time to 10 minutes or so. 3. After that, start the copy like before. Maybe its fast as long the data goes into the cached RAM. And as soon you RAM is full (only the cached RAM is full) the transnsfer speed drops. Watch for the speed drop and make a screenshot from the stats. Only a idea... Quote Link to comment
Rubydoo Posted November 5, 2019 Author Share Posted November 5, 2019 Ok, after doing fstrim speeds appeared good on unraid running locally (duplication copy), but network drives no change. But, after smb stop and start, and stopping and starting the array, then unmounting and mounting the drives again, back to normal, so fixed somewhere between that. Will wait for it to slow down again and then test it step by step to find exactly where it is. Thanks all for the input, I will post the results in case someone else has the same issue. Quote Link to comment
Rubydoo Posted November 7, 2019 Author Share Posted November 7, 2019 Update: so the problem appears to be with smbd. I have just rebooted so I will need to let it run for a couple of days again to slow down. I will post my testing results and do some tweaking to see if I can't fix the problem. In the meantime, I'll just use nfs (media share) and afp (hackintosh workstation) for my primary needs. There is only one pc running smb and it's just for incremental backups, so bandwidth there isn't crucial. Will continue to update for the purposes of any user reading this who also encounters this same issue. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.