Parity calculation speed

[woocash]

Hi there!

 

Let me start by saying that i am just trying unRaid for the last few days and like it a lot!

I do have a question about parity calculation though. My HW is HP Microserver gen8 with G1610T CPU, two WD REDs 3 TB, two shucked HGST HE 8TB and 128GB SSD for cache.

 In my PC al HDDs bench slightly below 200MBps, in unRaid while doing parity calculation with only 2x8TB connected (2x3TB are unmounted, and not associated) the speed I am seeing is 40-50MBps. Is this to be expected? CPU load is never greater than 20%, so what is the bottleneck here?

Edited April 17, 2019 by woocash

[trurl]

That is normal for "normal" writes to parity array. Here is an explanation of the 2 methods parity can use:

 

https://forums.unraid.net/topic/50397-turbo-write/

 

[woocash]

Thanks for the answer. In disk settings, the Tunable(md_write_method), is since 5 minutes after the installation set to "reconstruct write". Is it possible that the disks do not use the setting, or have it reset somehow? Can i verify the method that is actually in use?

Edited April 17, 2019 by woocash

[trurl]

You should be able to see which disks are being read and written in the Reads and Writes columns on Main - Array Devices.

[woocash]

I am, the data drive has read speed matching the write speed of parity drive. Currently 47,9 MBps. If it helps diagnostically, I'm running 6.7.0-rc7

Edited April 17, 2019 by woocash

[trurl]

What I was getting at is that if you have reconstruct write on, then when you write to a disk in the array, all the other disks should be read.

 

Are you saying reading from a data disk is that slow? Reading a data drive doesn't involve parity, just the drive being read.

[JorgeB]

11 hours ago, woocash said:

in unRaid while doing parity calculation

If you mean parity sync, check in the bios that write cache is enable, if it is and still slow then please post the diagnostics: Tools -> Diagnostics

[woocash]

6 hours ago, trurl said:

What I was getting at is that if you have reconstruct write on, then when you write to a disk in the array, all the other disks should be read.

 

Are you saying reading from a data disk is that slow? Reading a data drive doesn't involve parity, just the drive being read.

Yes, read speed from data disk is the same as write speed to parity disk - around 45-49 MBps.

 

1 hour ago, johnnie.black said:

If you mean parity sync, check in the bios that write cache is enable, if it is and still slow then please post the diagnostics: Tools -> Diagnostics

Not sure if I named it correctly - basically in my array there is one 8TB data drive. I added another identical 8TB new and empty drive to the array as parity disk. I clicked "Parity-sync/data rebuild" to fill parity disk with parity data. Since then the speed of this process is very slow. The caching is enabled in BIOS.

 

Looking through diagnostics I found lsscsi.txt which showed queue_depth of both my 8TB drives as 1, which if i understand correctly means NCQ is disabled. On the other hand in unRaid settings, NCQ is set to be enabled. Could this be the problem?

 

I do attach diagnostics below. 

 

 

Edited April 18, 2019 by woocash

[JorgeB]

Write cache for parity disk is disabled, you can turn it on with:

 

hdparm -W1 /dev/sdX

 

Might go back to default after a reboot.

[JorgeB]

In fact it's disabled for all your disks, very unusual. something is/was disabling it.

[woocash]

46 minutes ago, johnnie.black said:

Write cache for parity disk is disabled, you can turn it on with:

 

hdparm -W1 /dev/sdX

  

Might go back to default after a reboot.

 

42 minutes ago, johnnie.black said:

In fact it's disabled for all your disks, very unusual. something is/was disabling it.

 

I guess that was the fastest and finest diagnosis of any problem I had ever. 160MBps now Worked like a charm, appreciate it!

 

Curious about that cache being disabled everywhere, even though in BIOS it's enabled. Will investigate further and update this post if i find the reason. Wonder if maybe DiskSpeed or other benchmarking tool i used, disabled it to get more "pure" results.

Edited April 18, 2019 by woocash