Parity check times...Is this ok for my array size?

[ClawSS]

 

Last checked on Thu 01 Feb 2018 11:04:04 AM EST (two days ago), finding 0 errors.  Duration: 11 hours, 4 minutes, 3 seconds. Average speed: 75.3 MB/s  ...for an array of 1 parity and 10 data disks and 1 cache   1x 3TB WD Red Parity 2x 3TB WD Red Data 1x 2TB WD Red Data 1x 2TB WD Cavier Black Data 6x 2TB WD Cavier Green Data 1x 1TB WD Cavier Black Cache   I get some slow write speeds to the array (5MB/s direct write to parity then share, so I am not using the cache to it's fullest extent and mainly using it as a staging drive for docker programs).   I want to get a 2nd parity (probably move up to a 6TB Red).  If I use 2 parity drives, should they be = in size and how bad will this slow things down?

Edited February 3, 2018 by ClawSS

[S80_UK]

Parity check times are largely a function of the largest disk in the sytem and the slowest controller path to the disks.  The total size generally has little bearing on the timing.  You'd need to tell us a bit about your hardware and which drives are conencted to which SATA ports.  I assume you have at least one controller card added to the system.  What that controller is and how many drives are on it may make a difference.  11 hours is not as fast as you could get to for a system with one or more 3TB drives.  I generally see around 7 hours 25 minutes with a mixture or 2 and 3TB drives. 

 

A write speed of only 5MB/s for some transactions is quite slow.  I assume that is only when writing to specific disks?  

 

If using dual parity, the two parity drives don't have to be the same size, but both must be at least as large as your biggest data drive.  Whether there is a speed penalty will again depend on your hardware.

[ClawSS]

Sorry for the delay...weekends, kid activities, etc, etc.

 

The server is set up with:

M/B: ASRock - B75 Pro3
CPU: Intel® Core™ i3-2100 CPU @ 3.10GHz  (5x SATA 3Gb/s & 3x SATA 6Gb/s connections on board)
Memory: 4 GB

also installed

2x Adaptec 1430SA cards on PCI bus
Adapter 1 has 3x 2TB WD Green Data disks
Adapter 2 has 2x 3TB WD Red Data disks & 1x 1TB WD Black Cache disk + 1x 3TB WD Red disk (precleared & ready as backup)

OnBoard

1x 3TB WD Red Parity

1x 2TB WD Red Data

1x 2TB WD Black Data

3x 2TB WD Green Data

1x 2TB WD Green (precleared and ready as backup)

 

I use 3 SUPERMICRO CSE-M35T-1B 3 x 5.25" to 5 x 3.5" Hot-swap SATA HDD Trays in a tower case configuration.

 

Can I move disk connections around on the MB and cards without having to rebuild them in the array?
 

 

 

[JorgeB]

The average speed is not bad for the hardware, there might be some room for improvement, if you're using the defaults tunables try these:

 

Settings -> Disk Settings

 

Tunable (md_num_stripes): 4096
Tunable (md_sync_window): 2048
Tunable (md_sync_thresh): 2000

 

Also recommend you run the diskspeed test to see if there are older slower disks keeping your speed down, parity check speed can't be faster than your slowest disk at any point.

 

https://lime-technology.com/forums/topic/29435-drive-performance-testing-version-264-for-unraid-5-thru-63/?do=findComment&comment=630190

 

[ClawSS]

Can I move disk connections around on the MB and cards without having to rebuild them in the array?

 

I guess I have a few slowpokes.

 

Drive Identification

Parity:	WDC WD30EFRX-68AX9N0 WD-WMC1T2811947	3 TB	116 MB/sec avg
Disk 1:	WDC WD20EFRX-68AX9N0 WD-WMC301312514	2 TB	118 MB/sec avg
Disk 2:	WDC WD20EARS-00MVWB0 WD-WMAZA0765543	2 TB	85 MB/sec avg
Disk 3:	WDC WD2002FAEX-007BA0 WD-WMAY01280042	2 TB	119 MB/sec avg
Disk 4:	WDC WD20EZRX-00DC0B0 WD-WMC300404285	2 TB	109 MB/sec avg
Disk 5:	WDC WD20EZRX-00DC0B0 WD-WMC300443717	2 TB	116 MB/sec avg
Disk 6:	WDC WD20EZRX-00DC0B0 WD-WMC300364081	2 TB	115 MB/sec avg
Disk 7:	WDC WD30EFRX-68EUZN0 WD-WCC4NKKP1LR4	3 TB	115 MB/sec avg
Disk 8:	WDC WD20EZRX-00DC0B0 WD-WMC300473838	2 TB	113 MB/sec avg
Disk 9:	WDC WD20EZRX-00DC0B0 WD-WMC300180557	2 TB	83 MB/sec avg
Disk 10:	WDC WD30EFRX-68EUZN0 WD-WMC4N0E0HX8X	3 TB	119 MB/sec avg
Cache:	WDC WD1001FALS-00E3A0 WD-WCATR0086107	1 TB	106 MB/sec avg
sdd:	WDC WD20EZRX-00DC0B0 WD-WMC300365010	2 TB	120 MB/sec avg
sdf:	WDC WD30EFRX-68EUZN0 WD-WCC4N4ZT3A2U	3 TB	119 MB/sec avg

 

 

diskspeed_stopped.html

[JorgeB]

17 minutes ago, ClawSS said:

Can I move disk connections around on the MB and cards without having to rebuild them in the array?

Yes, disks are tracked by s/n

 

Those 2 slower disks are bring your average speed down, they should be the first ones replaced when you need to upgrade to larger size.

[ClawSS]

I find it interesting that my 3TB Parity being a 6 GB/s Red running OB is just as "fast" as one of my 3TB data 6GB/s Reds running on the bus throttled at 3 GB/s.  Not that I planned it this way, but it is interesting to see it proven that a spinning disk has a hard time getting 6Gbps speeds.  I guess the only drives that can saturate 6Gbps SATA connections are solid-state disks.

 

Parity: @ 116 MB/sec avg

Model family:    Western Digital Red
User capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector sizes:    512 bytes logical, 4096 bytes physical
SATA version:    SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)

 

Disk 7: @ 115 MB/sec avg

Model family:    Western Digital Red
User capacity:    3,000,592,982,016 bytes [3.00 TB]
Sector sizes:    512 bytes logical, 4096 bytes physical
Rotation rate:    5400 rpm
SATA version:    SATA 3.0, 6.0 Gb/s (current: 3.0 Gb/s)

 

Edited February 4, 2018 by ClawSS
clarity

[JorgeB]

1 hour ago, ClawSS said:

I guess the only drives that can saturate 6Gbps SATA connections are solid-state disks.

Correct, though some disks are finally getting close:

 

https://lime-technology.com/forums/topic/29435-drive-performance-testing-version-264-for-unraid-5-thru-63/?do=findComment&comment=630319

 

 

[S80_UK]

2 hours ago, ClawSS said:

I find it interesting that my 3TB Parity being a 6 GB/s Red running OB is just as "fast" as one of my 3TB data 6GB/s Reds running on the bus throttled at 3 GB/s.  Not that I planned it this way, but it is interesting to see it proven that a spinning disk has a hard time getting 6Gbps speeds.  I guess the only drives that can saturate 6Gbps SATA connections are solid-state disks.

 

If the drive already has needed sectors in its local cache as the consequence of an earlier read operation from the same cylinder and head on the disk, then the faster speed of SATA3 can help.  But that assumes you are not reading large amounts of data sequentially.  If you are, then the cache on the drive is of little benefit.  The disk speed test is intended to show the raw disk speed without the effect of the cache by reading large numbers of sequential sectors, so that it is a measure of the disk speed rather than the cache speed.  

Edited February 4, 2018 by S80_UK

[pwm]

The following would indicate that the controller ports do matters in your system - two identical disks with significant transfer speed differences.

Disk 8: 	WDC WD20EZRX-00DC0B0 WD-WMC300473838  	2 TB	  113 MB/sec avg
Disk 9: 	WDC WD20EZRX-00DC0B0 WD-WMC300180557  	2 TB	  83 MB/sec avg

The alternative would be that the disk with higher serial number has some firmware improvement making it perform better.

[JorgeB]

Or there are problems with that disk.