Hello, just signed up to say thanks and share my config.

 

I was having similar issues with slow write speeds, around 25 MB/s but on benchmarks it peaks around 200 MB/s

 

 

Now with disk cache enabled peaks around 190 MB/s both read and write, without using the cache drive and with turbo write enabled.

 

sdparm --set=WCE /dev/sdb

From NetData running in a docker container, before the read and write was at 100% utilization.

 

Im using a DELL PERC H330 controller (even updated the firmware from unraid), tested both RAID (Physical disk mode) and HBA mode without any difference.

 

With 4x Dell / Seagate model: ST2000NM0018

 

Edited October 23, 20196 yr by Robin AK
Added HDD model.

Back to it being slow.
Had the machine off for a couple days, waiting on some parts to come in.

Moving more data on to it, and write speeds are down again to 30MB/s.

Did the WCE thing on all drives, no effect.

Will try mounting an NTFS disk in a bit, do a test transfer, and see if things change.

 

Local disk-to-disk transfer of a 4GB file resulted in a 168MB/s data rate.

 

cube-diagnostics-20191026-2353.zip

Edited October 26, 20196 yr by sota

Looks like you're having ACPI Errors too.

Oct 26 16:12:33 Cube kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20180810/exfield-393)
Oct 26 16:12:33 Cube kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20180810/psparse-516)
Oct 26 16:12:33 Cube kernel: ACPI Error: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20180810/power_meter-338)
Oct 26 16:18:02 Cube kernel: ACPI Error: SMBus/IPMI/GenericSerialBus write requires Buffer of length 66, found length 32 (20180810/exfield-393)
Oct 26 16:18:02 Cube kernel: ACPI Error: Method parse/execution failed \_SB.PMI0._PMM, AE_AML_BUFFER_LIMIT (20180810/psparse-516)
Oct 26 16:18:02 Cube kernel: ACPI Error: AE_AML_BUFFER_LIMIT, Evaluating _PMM (20180810/power_meter-338)

 

 

Looks like you don't have write cache enabled on some of the drives.

Oct 26 16:02:19 Cube kernel: sd 1:0:6:0: [sdg] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:02:29 Cube kernel: sdg: sdg1
Oct 26 16:02:35 Cube kernel: sd 1:0:5:0: [sdf] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:02:45 Cube kernel: sdf: sdf1
Oct 26 16:02:48 Cube kernel: sd 1:0:4:0: [sde] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:02:48 Cube kernel: sde: sde1
Oct 26 16:02:53 Cube kernel: sd 1:0:4:0: [sde] Write cache: enabled, read cache: enabled, supports DPO and FUA
Oct 26 16:02:53 Cube kernel: sde: sde1
Oct 26 16:04:12 Cube kernel: sd 1:0:3:0: [sdd] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:04:12 Cube kernel: sdd: sdd1
Oct 26 16:04:16 Cube kernel: sd 1:0:3:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
Oct 26 16:04:16 Cube kernel: sdd: sdd1
Oct 26 16:04:17 Cube kernel: sd 1:0:2:0: [sdc] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:04:27 Cube kernel: sdc: sdc1
Oct 26 16:04:31 Cube kernel: sd 1:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:04:41 Cube kernel: sdb: sdb1
Oct 26 16:08:20 Cube kernel: sd 1:0:1:0: [sdb] Write cache: enabled, read cache: enabled, supports DPO and FUA
Oct 26 16:08:20 Cube kernel: sdb: sdb1
Oct 26 16:08:46 Cube kernel: sd 1:0:1:0: [sdb] Write cache: disabled, read cache: enabled, supports DPO and FUA
Oct 26 16:08:46 Cube kernel: sdb: sdb1

 

I'm not sure how to interpret the ACPI errors; keep in mind there's 3 controllers on this machine: CD-ROM SATA, HP B210i (not used), HP P420 (used).  And of course the USB slot for the unRAID stick (inside the case.)

 

I re-checked the cache settings and --get says they're all set to enabled.  However when I did force it for sdf and sdg and refreshed the log I see an entry each for sdf and sdg saying it's now enabled.  Weird.

 

eta: dug more on the ACPI errors... they're nothing to worry about.  They're related to HP being HP; see https://www.serveradminblog.com/2015/05/kernel-acpi-error-smbusipmigenericserialbus/

I'll worry about dealing with that later.

Edited October 27, 20196 yr by sota

On 7/18/2018 at 12:25 AM, Vexorg said:

root@Thor:/etc# sdparm --set=WCE /dev/sdd     /dev/sdd: HGST      HUS726040ALS214   MS00

this really helps! followed with this command and worked out finally.  many many thanks for that! 

Could someone help me diagnose slow write speeds on my server? 

I'm getting about 15-20MB/S, and when copying over multiple small files it takes an extremely long time. 

I've checked the write cache as per post two, and all seem to be enabled with the exception of the boot flash drive. 

 

Diagnostics attached! 

TIA!

 

 

 

edit: UPDATE

So I did some more testing, and it appears that the slow write speeds (and read speeds) were due to MacOS - I have two Macs as my main production machines, and a windows box as a fail-over. 

I tested the windows box and was consistently getting 105ish MB/s read over the network and ~70-80MB write, compared to ~50MB/s read on both Macs and 15-20MB/s write. 

After some hunting and troubleshooting, I found that MACos SMB implementation is apparently pretty crap, but turning off delayed ACK seemed to fix the issue. 

 

To test it (non permanent) in terminal: 

sudo sysctl -w net.inet.tcp.delayed_ack=0

 

All of a sudden, I was getting read and write speeds the same as the windows box. This is a MASSIVE improvement!!

 

For anyone that is on MACos and needs a permanent fix, this is what I did: 

 

 

To make the change permanent (requires reboot):

 

Create/edit the file /etc/sysctl.conf

Terminal:

sudo vim /etc/sysctl.conf

 

Add this line to the configuration file:

net.inet.tcp.delayed_ack=0

 

Save and close file (type “:wq” without quotes, then hit enter)

reboot computer

 

 

Edited November 5, 20205 yr by BenW

This helped all these years later even in Windows 10.

 

🍻

On 7/17/2018 at 6:25 PM, Vexorg said:

After a ton of Google-fu I was able to resolve my problem.

 

TL;DR Write cache on drive was disabled

 

found an page called How to Disable or Enable Write Caching in Linux. The artical covers both ata and scsi drives which i needed as SAS drive are scsi and are a total different beast.

 

root@Thor:/etc# sdparm -g WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00
WCE           0  [cha: y, def:  0, sav:  0]

This shows that the write cache disabled

 

root@Thor:/etc# sdparm --set=WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00

This enables it and my writes returned to the expected speeds

 

root@Thor:/etc# sdparm -g WCE /dev/sdd
    /dev/sdd: HGST      HUS726040ALS214   MS00
WCE           1  [cha: y, def:  0, sav:  0]

confirms the write cache has been set

 

Now I'm not total sure why the write cache was disabled under unraid, bug or feature? While doing my googling there was a mention of a kernel bug a few years ago that if system ram was more then 8G it disables the write cache. My current system has a little more then 8G so maybe?

 

this post show how everyone is really excited about this feature.

Currently im getting around 40MB/S write speed and i would like to enable this feature either.

could you write down all steps?

I do not understand where and how i can write this commands down, and how i handle my own HDD/SSD to this command.

4 hours ago, TechnoWerk said:

Currently im getting around 40MB/S write speed

If you are writing to the parity array this isn't unusual and has nothing to do with drive write cache.

 

https://wiki.unraid.net/Manual/Storage_Management#Array_Write_Modes

Grand! 

As i dont have space for a cache drive in my DL60 and use SAS disks this really helped out! 

Was getting the 40-50mb write over network smb and now its 105mb ish. 

What i had tho was to add the save prefix as well for it to kick in

sdparm --set=WCE --save /dev/sda

 

 

from https://linux.die.net/man/8/sdparm

To set the "Writeback Cache Enable" bit in the current values page:

sdparm --set=WCE /dev/sda

To set the "Writeback Cache Enable" bit in the current and saved values page:

sdparm --set=WCE --save /dev/sda

 

 

 

 

 

  

Sorry to necro bump but I seem to be having the same issue, I've double checked in my bios for my pcm setting for my card that sas drives are all enabled for write cache. I've gone through the three drives with

sdparm -g WCE

saw that all 3 were disabled, set the cache to enable, and verified with sdparm again, all three drives now show

WCE           1  [cha: y, def:  0, sav:  1]

 

Rebooted, and still getting the same slow ~15MB/s. Verified primary storage for new files is set to cache, secondary storage as array. I can see the cache drive filling up, and then things move over to the array when i use the mover as a manual task. My cache is a 2tb NVME. Something has to be setup wrong, any info or guidance here?

 

 

burnside-diagnostics-20250104-1720.zip

9 hours ago, 3th0s said:

Rebooted, and still getting the same slow ~15MB/s.

Is this writing to the array or to the pool?

I believe it's writing to the pool, I'm just getting this setup and started, tested copying over a few folders and files, and I'm getting 15MB/s. I'm writing to a share folder that has cache drive as the primary storage, and the array as secondary

Enable turbo write and repeat the test when writing to the array.

On 1/5/2025 at 9:09 AM, 3th0s said:

I believe it's writing to the pool, I'm just getting this setup and started, tested copying over a few folders and files, and I'm getting 15MB/s. I'm writing to a share folder that has cache drive as the primary storage, and the array as secondary

Did you check to see if WCE was enabled on the cache drive(s) also?

Edited January 6, 20251 yr by wonderbread24
words

18 hours ago, wonderbread24 said:

Did you check to see if WCE was enabled on the cache drive(s) also?

  /dev/nvme0n1: NVMe      TEAM xxxxxxxxxxx   SN13
WCE           1  [cha: y, def:  1]

Didn't think to try the cache drive but looks like it was already enabled, good idea though.

One thing to note, that if your using file compression like with ZFS, I notice a considerable decrease in performance when changing my power profile from "Performance" to "balanced" profiles, I was getting under 120MB/sec between a lone M.2 to a RAIDZ1 SSD but the second I swapped back to performance profile it went back to the expected 400mb/sec transfer speed.  Looking at the clock frequency of my cores, "Performance" parks the cores at their base clock of 2600mhz, but balanced parks them at 1200mhz. 

 

 

SOLVED my problem, I'm an idiot lol. Built, setup and tested my server in my back office that uses a wifi bridge/repeater, so I was just connected to that very slow network. Once I moved the server to my actual router, speeds above 100MBps. lol

Glad you figured it out!