Hello all,

 

I've just build a new unraid server and after playing around with a bit before I really start using it I was finding it the drive writes seemed to be a bit slow.

 

I first noticed the slowness while I was doing a badblock run on the new drives and after 50 hours it had just finished the first pass. For 4TB drives I understand it should take a total of 40hours or so for this size of drive. I did not see any errors and killed the process and proceeded to build the array with the drives and when it was doing the parrity build the stated write speed to the drive with 47MB/sec which seemed to me a bit slow for the drives. I checked the spec's from the manufacturer and it said it should be closer to 200MB/sec so I'm getting about 25% of the speed.

 

Next thing I did was boot into windows and ran a few benchmarking tools from there and they all reported read/write speeds about 200MB/sec... I did find that one of the LSI cards was in a PCIe 4x slot instead of an 8X one so I moved it to a new slot. 

 

Next try was a live linux USB to see if the problem was with unix after things seem to be fine in windows. Used knoppix and ran a few tests and the drives speeds where in the 200MB/sec again?!? 

 

My conclusion is the Unraid distro is the problem and not with hardware.

 

Now for some hardware details of my system:

Supermicro x10DRi-ln4+

2x intel e5-2630's

3x LSI 9211 HBA's connected to a supermicro BPN-SAS-846A backplane

6x HGST 4tb 3.5" HDD HUS726040ALS214 SAS drives

 

root@Thor:~# hdparm -Tt /dev/sdh

/dev/sdh:
 Timing cached reads:   21860 MB in  1.99 seconds = 10970.43 MB/sec
 Timing buffered disk reads: 570 MB in  3.00 seconds = 189.82 MB/sec

root@Thor:~# dd if=/dev/zero of=/dev/sdh bs=128k count=10k
10240+0 records in
10240+0 records out
1342177280 bytes (1.3 GB, 1.2 GiB) copied, 28.5249 s, 47.1 MB/s

I'm not really sure where to look at next.

thor-diagnostics-20180716-1149.zip

50 to 60MB/s are the normal unRAID writing speed due to how parity works with the default settings, you can enable turbo write for faster writes.

After a ton of Google-fu I was able to resolve my problem.

 

TL;DR Write cache on drive was disabled

 

found an page called How to Disable or Enable Write Caching in Linux. The artical covers both ata and scsi drives which i needed as SAS drive are scsi and are a total different beast.

 

root@Thor:/etc# sdparm -g WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00
WCE           0  [cha: y, def:  0, sav:  0]

This shows that the write cache disabled

 

root@Thor:/etc# sdparm --set=WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00

This enables it and my writes returned to the expected speeds

 

root@Thor:/etc# sdparm -g WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00
WCE           1  [cha: y, def:  0, sav:  0]

confirms the write cache has been set

 

Now I'm not total sure why the write cache was disabled under unraid, bug or feature? While doing my googling there was a mention of a kernel bug a few years ago that if system ram was more then 8G it disables the write cache. My current system has a little more then 8G so maybe?

3 minutes ago, Vexorg said:

Now I'm not total sure why the write cache was disabled under unraid, bug or feature?

First time I hear that, maybe if it's limited to SAS drives since they are not used by many, still would think other people would have complained by now it it's a general problem.

Thanks for this Vexorg. I was searching for a different issue (SAS spindown issues) and saw your post. I had been building a new server and had poor performance but hadn't had time to research. What I found was interesting - all my HP branded SAS drives had write cache disabled, while all my Seagate drives had it enabled. These drives were split across two servers so my assumption is that this was set on the drives themselves at some point and that this has nothing to do with unRAID. 

 

Enabled write caching on the HP drives and performance is back to where I'd expect, so you saved me a ton of time. Thanks again.

Thanks for the info, does this have to be done each boot?

 

Also anyone that finds this latter, spparm can be installed from the Nerd Pack Plugin.

 

@limetech Can this been fixed/added in the Options Menu under Disk Settings?

 

@Squid Maybe add this in to the Turbowrite Plugin?

Edited July 17, 20187 yr by AnnabellaRenee87

5 hours ago, AnnabellaRenee87 said:

Thanks for the info, does this have to be done each boot?

 

I didn't have to do it again. I just had to do a reboot and it seems the setting has suck so far.

Glad I saw this post, ran the commands in the 3rd post with my disk name. Worked instantly, thanks a ton.

Just wanted to add a thanks for this as it has brought my write speed up from 45mb/s to 240mb/s! Many Many Thanks! 🙂  

 

Hopefully this will be built into unRAID in the future!

Awesome post. I saw that my write cache was disabled and wasn't sure about enabling it. Thanks for posting your experiences and the commands ran to change the states. I was also experiencing the sub 50MB/s speeds prior to enabling write cache and now get over 200MB/s.

GIGABYTE GA-7PESH2 board with onboard sas controller flashed to IT mode with Seagate ST6000NM0034 drives. 

johnnie.black redirected me to this post

As the writing speed of Parity Clearing time was showing 12 hrs.

sdparm command not found was the message at first but as per AnnabellaRenee87 post, installed nerdpack GUI plugin.

after that everything was magic.

12 hrs parity work reduced to 3 hrs.

This is very helpful post.

Thanks

On 7/17/2018 at 11:25 AM, Vexorg said:

After a ton of Google-fu I was able to resolve my problem.

 

TL;DR Write cache on drive was disabled

 

found an page called How to Disable or Enable Write Caching in Linux. The artical covers both ata and scsi drives which i needed as SAS drive are scsi and are a total different beast.

 

Thanks a lot, I spent a ton of time spinning my wheels on this problem!  I added a Dell PERC H310 SAS card to my setup along with 6x 3TB SAS drives.  The parity sync was crawling along around 40 MB/s.  Turns out all the write cache's were disabled for the SAS drives (enabled for my SATA drives on the separate on-board SATA controller).  Enabled the write cache on the SAS drives and I'm at 140 MB/s now which is perfectly acceptable.

 

Some other stuff if anyone else bought an H310 and stumbles onto this.  The H310 requires some work to get going (apparently well known).  Had to tape pins B5 and B6 on the card itself so my PC would boot.  Initially after doing that unRaid noticed all the SAS drives but parity sync was around 10 MB/s.  That's when I stumbled onto flashing the H310 firmware with the IT firmware (I had the RAID firmware).  That's what brought me up to 40 MB/s.  The write caching was the last piece of the puzzle here.  Link to the firmware bit.

 

 

Wholly hell! Thank you! and my 7 SAS drives thank you! from ~35 > ~175!

This just fixed the same problem. LEGEND!

 

Wonder if this could be added to the "Fix Common Problems" plugin or something!

Thank you so much for this. Sas drives don't have write cache on, at least not with my expansion card. Thank you for this, truly. 

This works! many thanks

Edited July 27, 20196 yr by Mbur371189

You should likely get your own topic thread since many users are less likely to post in threads already marked as SOLVED.

Just experienced this exact same phenomenon with a brand new 10TB WD Drive I shucked from an EasyStore. Ran the command and went from a locked rate of 40MB/Sec to 127MB/Sec.

@Squid @limetech any word on this fix being added to unRAID Disk Settings or a popular plugin like Turbowrite?

43 minutes ago, Endda said:

@Squid @limetech any word on this fix being added to unRAID Disk Settings or a popular plugin like Turbowrite?

It's not a problem with the OS, but rather how the manufacturers choose to configure their hard drives.  Which is why Fix Common Problems is telling you about it.  Not sure if I'd want unRaid (or any OS) automatically doing this for me.  On the other hand though, a setting in Disk Settings would make life easier.

Man, this restored write speeds to 130 MB/s consistently, thank you SO much! Saved my Unraid install...I was about to abort! Thanks @Vexorg

Edited October 13, 20196 yr by rocky_mtn

I'm apparently having the same problem.  My write speeds are in the sub-30MB/s range.  I checked the WCE flag and it's enabled on all my drives.  This has been consistent across two different builds (other was an HP 8000 Elite SFF C2Q Q9650, 16GB ram, native SATA connectors.)

 

I'm still looking for a solution.

2 hours ago, sota said:

I'm apparently having the same problem.  My write speeds are in the sub-30MB/s range.  I checked the WCE flag and it's enabled on all my drives.  This has been consistent across two different builds (other was an HP 8000 Elite SFF C2Q Q9650, 16GB ram, native SATA connectors.)

 

I'm still looking for a solution.

Tools - Diagnostics, attach complete diagnostics zip file to your NEXT post.

I found that setting the WCE flag was not persistent after a reboot. I had to set the -S flag as well.

2 hours ago, trurl said:

Tools - Diagnostics, attach complete diagnostics zip file to your NEXT post.

Here you go.

 

I have no idea what changed, but all of the sudden i'm now getting 100MB/s transfer speeds into unRAID.

I'm trying to think of what it is I messed with during this time:

Hot added a 4th drive to the machine, which showed up as /dev/sde.

Mounted sde as /mnt/ntfs1 (it's an NTFS formatted drive.)

Did some file copy tests from it to one of the array disks (/mnt/disk2.)

Got a computed 112MB/s data transfer rate, which to me is appropriate.

Deleted the copied files and unmounted /mnt/ntfs1.

After all that, I went back and looked at the network copy activity, and it was running near 100MB/s.  i could see on the graph (Windows Server 2012R2 source) the big hump and jump up.  I guess whatever Fat Electron cleared out of the wire?

 

cube-diagnostics-20191019-0015.zip