Vexorg Posted July 16, 2018 Share Posted July 16, 2018 Hello all, I've just build a new unraid server and after playing around with a bit before I really start using it I was finding it the drive writes seemed to be a bit slow. I first noticed the slowness while I was doing a badblock run on the new drives and after 50 hours it had just finished the first pass. For 4TB drives I understand it should take a total of 40hours or so for this size of drive. I did not see any errors and killed the process and proceeded to build the array with the drives and when it was doing the parrity build the stated write speed to the drive with 47MB/sec which seemed to me a bit slow for the drives. I checked the spec's from the manufacturer and it said it should be closer to 200MB/sec so I'm getting about 25% of the speed. Next thing I did was boot into windows and ran a few benchmarking tools from there and they all reported read/write speeds about 200MB/sec... I did find that one of the LSI cards was in a PCIe 4x slot instead of an 8X one so I moved it to a new slot. Next try was a live linux USB to see if the problem was with unix after things seem to be fine in windows. Used knoppix and ran a few tests and the drives speeds where in the 200MB/sec again?!? My conclusion is the Unraid distro is the problem and not with hardware. Now for some hardware details of my system: Supermicro x10DRi-ln4+ 2x intel e5-2630's 3x LSI 9211 HBA's connected to a supermicro BPN-SAS-846A backplane 6x HGST 4tb 3.5" HDD HUS726040ALS214 SAS drives root@Thor:~# hdparm -Tt /dev/sdh /dev/sdh: Timing cached reads: 21860 MB in 1.99 seconds = 10970.43 MB/sec Timing buffered disk reads: 570 MB in 3.00 seconds = 189.82 MB/sec root@Thor:~# dd if=/dev/zero of=/dev/sdh bs=128k count=10k 10240+0 records in 10240+0 records out 1342177280 bytes (1.3 GB, 1.2 GiB) copied, 28.5249 s, 47.1 MB/s I'm not really sure where to look at next. thor-diagnostics-20180716-1149.zip 1 Quote Link to comment
JorgeB Posted July 17, 2018 Share Posted July 17, 2018 50 to 60MB/s are the normal unRAID writing speed due to how parity works with the default settings, you can enable turbo write for faster writes. 1 Quote Link to comment
Solution Vexorg Posted July 17, 2018 Author Solution Share Posted July 17, 2018 After a ton of Google-fu I was able to resolve my problem. TL;DR Write cache on drive was disabled found an page called How to Disable or Enable Write Caching in Linux. The artical covers both ata and scsi drives which i needed as SAS drive are scsi and are a total different beast. root@Thor:/etc# sdparm -g WCE /dev/sdd /dev/sdd: HGST HUS726040ALS214 MS00 WCE 0 [cha: y, def: 0, sav: 0] This shows that the write cache disabled root@Thor:/etc# sdparm --set=WCE /dev/sdd /dev/sdd: HGST HUS726040ALS214 MS00 This enables it and my writes returned to the expected speeds root@Thor:/etc# sdparm -g WCE /dev/sdd /dev/sdd: HGST HUS726040ALS214 MS00 WCE 1 [cha: y, def: 0, sav: 0] confirms the write cache has been set Now I'm not total sure why the write cache was disabled under unraid, bug or feature? While doing my googling there was a mention of a kernel bug a few years ago that if system ram was more then 8G it disables the write cache. My current system has a little more then 8G so maybe? 5 1 6 Quote Link to comment
JorgeB Posted July 17, 2018 Share Posted July 17, 2018 3 minutes ago, Vexorg said: Now I'm not total sure why the write cache was disabled under unraid, bug or feature? First time I hear that, maybe if it's limited to SAS drives since they are not used by many, still would think other people would have complained by now it it's a general problem. Quote Link to comment
rinseaid Posted July 17, 2018 Share Posted July 17, 2018 Thanks for this Vexorg. I was searching for a different issue (SAS spindown issues) and saw your post. I had been building a new server and had poor performance but hadn't had time to research. What I found was interesting - all my HP branded SAS drives had write cache disabled, while all my Seagate drives had it enabled. These drives were split across two servers so my assumption is that this was set on the drives themselves at some point and that this has nothing to do with unRAID. Enabled write caching on the HP drives and performance is back to where I'd expect, so you saved me a ton of time. Thanks again. Quote Link to comment
AnnabellaRenee87 Posted July 17, 2018 Share Posted July 17, 2018 (edited) Thanks for the info, does this have to be done each boot? Also anyone that finds this latter, spparm can be installed from the Nerd Pack Plugin. @limetech Can this been fixed/added in the Options Menu under Disk Settings? @Squid Maybe add this in to the Turbowrite Plugin? Edited July 17, 2018 by AnnabellaRenee87 1 Quote Link to comment
Vexorg Posted July 18, 2018 Author Share Posted July 18, 2018 5 hours ago, AnnabellaRenee87 said: Thanks for the info, does this have to be done each boot? I didn't have to do it again. I just had to do a reboot and it seems the setting has suck so far. 1 Quote Link to comment
slimshizn Posted December 24, 2018 Share Posted December 24, 2018 Glad I saw this post, ran the commands in the 3rd post with my disk name. Worked instantly, thanks a ton. Quote Link to comment
mbc0 Posted January 25, 2019 Share Posted January 25, 2019 Just wanted to add a thanks for this as it has brought my write speed up from 45mb/s to 240mb/s! Many Many Thanks! 🙂 Hopefully this will be built into unRAID in the future! Quote Link to comment
hasselltech Posted April 3, 2019 Share Posted April 3, 2019 Awesome post. I saw that my write cache was disabled and wasn't sure about enabling it. Thanks for posting your experiences and the commands ran to change the states. I was also experiencing the sub 50MB/s speeds prior to enabling write cache and now get over 200MB/s. GIGABYTE GA-7PESH2 board with onboard sas controller flashed to IT mode with Seagate ST6000NM0034 drives. Quote Link to comment
kinaley Posted April 5, 2019 Share Posted April 5, 2019 johnnie.black redirected me to this post As the writing speed of Parity Clearing time was showing 12 hrs. sdparm command not found was the message at first but as per AnnabellaRenee87 post, installed nerdpack GUI plugin. after that everything was magic. 12 hrs parity work reduced to 3 hrs. This is very helpful post. Thanks 1 Quote Link to comment
bladpart Posted April 24, 2019 Share Posted April 24, 2019 On 7/17/2018 at 11:25 AM, Vexorg said: After a ton of Google-fu I was able to resolve my problem. TL;DR Write cache on drive was disabled found an page called How to Disable or Enable Write Caching in Linux. The artical covers both ata and scsi drives which i needed as SAS drive are scsi and are a total different beast. Thanks a lot, I spent a ton of time spinning my wheels on this problem! I added a Dell PERC H310 SAS card to my setup along with 6x 3TB SAS drives. The parity sync was crawling along around 40 MB/s. Turns out all the write cache's were disabled for the SAS drives (enabled for my SATA drives on the separate on-board SATA controller). Enabled the write cache on the SAS drives and I'm at 140 MB/s now which is perfectly acceptable. Some other stuff if anyone else bought an H310 and stumbles onto this. The H310 requires some work to get going (apparently well known). Had to tape pins B5 and B6 on the card itself so my PC would boot. Initially after doing that unRaid noticed all the SAS drives but parity sync was around 10 MB/s. That's when I stumbled onto flashing the H310 firmware with the IT firmware (I had the RAID firmware). That's what brought me up to 40 MB/s. The write caching was the last piece of the puzzle here. Link to the firmware bit. 1 Quote Link to comment
dbinott Posted April 28, 2019 Share Posted April 28, 2019 Wholly hell! Thank you! and my 7 SAS drives thank you! from ~35 > ~175! Quote Link to comment
Runear Posted May 13, 2019 Share Posted May 13, 2019 This just fixed the same problem. LEGEND! Wonder if this could be added to the "Fix Common Problems" plugin or something! Quote Link to comment
XtianApi Posted June 21, 2019 Share Posted June 21, 2019 Thank you so much for this. Sas drives don't have write cache on, at least not with my expansion card. Thank you for this, truly. Quote Link to comment
Mbur371189 Posted July 27, 2019 Share Posted July 27, 2019 (edited) This works! many thanks Edited July 27, 2019 by Mbur371189 Quote Link to comment
BRiT Posted July 27, 2019 Share Posted July 27, 2019 You should likely get your own topic thread since many users are less likely to post in threads already marked as SOLVED. Quote Link to comment
DrGrinch Posted August 31, 2019 Share Posted August 31, 2019 Just experienced this exact same phenomenon with a brand new 10TB WD Drive I shucked from an EasyStore. Ran the command and went from a locked rate of 40MB/Sec to 127MB/Sec. Quote Link to comment
Endda Posted September 1, 2019 Share Posted September 1, 2019 @Squid @limetech any word on this fix being added to unRAID Disk Settings or a popular plugin like Turbowrite? Quote Link to comment
Squid Posted September 1, 2019 Share Posted September 1, 2019 43 minutes ago, Endda said: @Squid @limetech any word on this fix being added to unRAID Disk Settings or a popular plugin like Turbowrite? It's not a problem with the OS, but rather how the manufacturers choose to configure their hard drives. Which is why Fix Common Problems is telling you about it. Not sure if I'd want unRaid (or any OS) automatically doing this for me. On the other hand though, a setting in Disk Settings would make life easier. 2 Quote Link to comment
rocky_mtn Posted October 13, 2019 Share Posted October 13, 2019 (edited) Man, this restored write speeds to 130 MB/s consistently, thank you SO much! Saved my Unraid install...I was about to abort! Thanks @Vexorg Edited October 13, 2019 by rocky_mtn Quote Link to comment
sota Posted October 18, 2019 Share Posted October 18, 2019 I'm apparently having the same problem. My write speeds are in the sub-30MB/s range. I checked the WCE flag and it's enabled on all my drives. This has been consistent across two different builds (other was an HP 8000 Elite SFF C2Q Q9650, 16GB ram, native SATA connectors.) I'm still looking for a solution. Quote Link to comment
trurl Posted October 18, 2019 Share Posted October 18, 2019 2 hours ago, sota said: I'm apparently having the same problem. My write speeds are in the sub-30MB/s range. I checked the WCE flag and it's enabled on all my drives. This has been consistent across two different builds (other was an HP 8000 Elite SFF C2Q Q9650, 16GB ram, native SATA connectors.) I'm still looking for a solution. Tools - Diagnostics, attach complete diagnostics zip file to your NEXT post. Quote Link to comment
wonderbread24 Posted October 18, 2019 Share Posted October 18, 2019 I found that setting the WCE flag was not persistent after a reboot. I had to set the -S flag as well. Quote Link to comment
sota Posted October 19, 2019 Share Posted October 19, 2019 2 hours ago, trurl said: Tools - Diagnostics, attach complete diagnostics zip file to your NEXT post. Here you go. I have no idea what changed, but all of the sudden i'm now getting 100MB/s transfer speeds into unRAID. I'm trying to think of what it is I messed with during this time: Hot added a 4th drive to the machine, which showed up as /dev/sde. Mounted sde as /mnt/ntfs1 (it's an NTFS formatted drive.) Did some file copy tests from it to one of the array disks (/mnt/disk2.) Got a computed 112MB/s data transfer rate, which to me is appropriate. Deleted the copied files and unmounted /mnt/ntfs1. After all that, I went back and looked at the network copy activity, and it was running near 100MB/s. i could see on the graph (Windows Server 2012R2 source) the big hump and jump up. I guess whatever Fat Electron cleared out of the wire? cube-diagnostics-20191019-0015.zip Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.