[CPU_IOWAIT]

I've been searching for how to solve this problem for weeks to no solution, so hopefully this post will help.

 

I have a new server, about a month old now,

 

ASUS Z12PE-D16 WS, XEON E5-2697 v4 @ 2.3Ghz, 128Gb RAM @ 2133Mhz....  not exactly a budget build...  16X6TB Drives in Array so far.,...

 

During copies from my old server over the network everything will be going find 60+ MB/s (in and out limitation of PC doing the transfer). Then suddenly everything will drop to 1 MB/s and access all files on the array and the entire web interface comes to a hault. This occours and can go on for hours and hours at full speed before this happens and sometimes minutes...

 

I've just finished a party check at 130+ MB/s and I can pull off each drive at atleast 150+ MB/s ...   

 

The only error I'm getting as such is in the glances docker (screenshot attached) that shows CUP_IOWAIT going through the roof.

 

With Cache Drive disabled for partition, 60+ MB/s copy normal, 1+MB/s copy during CPU_IOWAIT issues..

 

With Cache Drive Enabled for partition, 60+ MB/s Copy normal, but also 60+ MB/s copy during IO-WAIT but server is still very sluggish..... that makes me think this is happening all the time and I only notice it when copies are happening...

 

Most drives are WD RED 6TB connected directly to motherboard, including parity, with 4 drives being enterprise seagate SAS through a LSI 9311-8I.

 

Any thoughts? this is driving me crazy!!

 

Thanks

tower-diagnostics-20170207-1917.zip

[Can't Add Drives]

Thanks Very much, this seems to have worked...

 

Unraid was not a fan of my adding a second party and two data drives at the same time... so I had to create a new config in setting and then assign the new drives and it was good to go...

 

I've got to rebuild the party now so this is going to take a few hours... but all seems to be working!! Thanks Again

[Can't Add Drives]

Hi,

 

So I've got a relatively new setup, about a week old, with 11 x 6TB WD Red drives...  1 Partity, 10 Data.. There is Also a 1TB SSD cache Drive...

 

Currently I'm trying to add a number of Seagate 6TB Drives, but when I stop the array and try to assign the drive, the screen flashes and then the drop down box stays unassigned....

 

Here is an example log below, where I try to add the following disk... to slot 11, it's picked up fine in the list but never gets imported, I've tried all the Seagate disks, which have been precleared, and also tryed restarting the system. Thoughts?  Thanks

 

ST6000NM0034_Z4D1562E0000R525K1X1_35000c500634e8c8b

 

 

 

Jan 12 18:32:10 Tower emhttp: shcmd (198): rmmod md-mod |& logger

Jan 12 18:32:10 Tower kernel: md: unRAID driver removed

Jan 12 18:32:10 Tower emhttp: shcmd (199): modprobe md-mod super=/boot/config/super.dat |& logger

Jan 12 18:32:10 Tower kernel: md: unRAID driver 2.6.8 installed

Jan 12 18:32:10 Tower emhttp: Pro key detected, GUID: 0781-5583-1001-480824121415 FILE: /boot/config/Pro.key

Jan 12 18:32:10 Tower emhttp: Device inventory:

Jan 12 18:32:10 Tower emhttp: shcmd (200): udevadm settle

Jan 12 18:32:10 Tower emhttp: SanDisk_Ultra_Fit_4C531001480824121415-0:0 (sda) 30031216

Jan 12 18:32:10 Tower emhttp: ST6000NM0034_Z4D1562E0000R525K1X1_35000c500634e8c8b (sdb) 5860522532

Jan 12 18:32:10 Tower emhttp: ST6000NM0034_Z4D0WJE80000R525HFLP_35000c500634dd8e3 (sdc) 5860522532

Jan 12 18:32:10 Tower emhttp: ST6000NM0034_Z4D0VN4A0000R525J3YS_35000c500634e0b9f (sdd) 5860522532

Jan 12 18:32:10 Tower emhttp: Samsung_SSD_850_EVO_1TB_S2RFNX0HB09491N (sde) 976762552

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9XK (sdf) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX31DA43K9XS (sdg) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9FX (sdh) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA427YNH (sdi) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX31DA43K8DP (sdj) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA40LFUK (sdk) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA40L7KE (sdl) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WXL1H8443X1P (sdm) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX51DA476T9N (sdn) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX51DA47621D (sdo) 5860522532

Jan 12 18:32:10 Tower emhttp: WDC_WD60EFRX-68MYMN1_WD-WX41DA42734P (sdp) 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (1): import 0 sdf 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9XK

Jan 12 18:32:10 Tower kernel: md: import disk0: (sdf) WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9XK size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (2): import 1 sdi 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA427YNH

Jan 12 18:32:10 Tower kernel: md: import disk1: (sdi) WDC_WD60EFRX-68MYMN1_WD-WX41DA427YNH size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (3): import 2 sdl 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA40L7KE

Jan 12 18:32:10 Tower kernel: md: import disk2: (sdl) WDC_WD60EFRX-68MYMN1_WD-WX41DA40L7KE size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (4): import 3 sdn 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX51DA476T9N

Jan 12 18:32:10 Tower kernel: md: import disk3: (sdn) WDC_WD60EFRX-68MYMN1_WD-WX51DA476T9N size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (5): import 4 sdm 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WXL1H8443X1P

Jan 12 18:32:10 Tower kernel: md: import disk4: (sdm) WDC_WD60EFRX-68MYMN1_WD-WXL1H8443X1P size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (6): import 5 sdo 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX51DA47621D

Jan 12 18:32:10 Tower kernel: md: import disk5: (sdo) WDC_WD60EFRX-68MYMN1_WD-WX51DA47621D size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (7): import 6 sdp 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA42734P

Jan 12 18:32:10 Tower kernel: md: import disk6: (sdp) WDC_WD60EFRX-68MYMN1_WD-WX41DA42734P size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (: import 7 sdk 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA40LFUK

Jan 12 18:32:10 Tower kernel: md: import disk7: (sdk) WDC_WD60EFRX-68MYMN1_WD-WX41DA40LFUK size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (9): import 8 sdg 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX31DA43K9XS

Jan 12 18:32:10 Tower kernel: md: import disk8: (sdg) WDC_WD60EFRX-68MYMN1_WD-WX31DA43K9XS size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (10): import 9 sdh 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9FX

Jan 12 18:32:10 Tower kernel: md: import disk9: (sdh) WDC_WD60EFRX-68MYMN1_WD-WX41DA40L9FX size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (11): import 10 sdj 5860522532 0 WDC_WD60EFRX-68MYMN1_WD-WX31DA43K8DP

Jan 12 18:32:10 Tower kernel: md: import disk10: (sdj) WDC_WD60EFRX-68MYMN1_WD-WX31DA43K8DP size: 5860522532

Jan 12 18:32:10 Tower kernel: mdcmd (12): import 11

Jan 12 18:32:10 Tower kernel: mdcmd (13): import 12

Jan 12 18:32:10 Tower kernel: mdcmd (14): import 13

Jan 12 18:32:10 Tower kernel: mdcmd (15): import 14

Jan 12 18:32:10 Tower kernel: mdcmd (16): import 15

Jan 12 18:32:10 Tower kernel: mdcmd (17): import 16

Jan 12 18:32:10 Tower kernel: mdcmd (18): import 17

Jan 12 18:32:10 Tower kernel: mdcmd (19): import 18

Jan 12 18:32:10 Tower kernel: mdcmd (20): import 19

Jan 12 18:32:10 Tower kernel: mdcmd (21): import 20

Jan 12 18:32:10 Tower kernel: mdcmd (22): import 21

Jan 12 18:32:10 Tower kernel: mdcmd (23): import 22

Jan 12 18:32:10 Tower kernel: mdcmd (24): import 23

Jan 12 18:32:10 Tower kernel: mdcmd (25): import 24

Jan 12 18:32:10 Tower kernel: mdcmd (26): import 25

Jan 12 18:32:10 Tower kernel: mdcmd (27): import 26

Jan 12 18:32:10 Tower kernel: mdcmd (28): import 27

Jan 12 18:32:10 Tower kernel: mdcmd (29): import 28

Jan 12 18:32:10 Tower kernel: mdcmd (30): import 29

Jan 12 18:32:10 Tower kernel: md: import_slot: 29 empty

Jan 12 18:32:10 Tower emhttp: import 30 cache device: sde

Jan 12 18:32:10 Tower emhttp: import 31 cache device: no device

Jan 12 18:32:10 Tower emhttp: check_pool: /sbin/btrfs filesystem show 29b20289-7fff-49ee-a267-75de4d197212 2>&1

Jan 12 18:32:10 Tower emhttp: cacheUUID: 29b20289-7fff-49ee-a267-75de4d197212

Jan 12 18:32:10 Tower emhttp: cacheNumDevices: 1

Jan 12 18:32:10 Tower emhttp: cacheTotDevices: 1

Jan 12 18:32:10 Tower emhttp: cacheNumMissing: 0

Jan 12 18:32:10 Tower emhttp: cacheNumMisplaced: 0

Jan 12 18:32:10 Tower emhttp: cacheNumExtra: 0

Jan 12 18:32:10 Tower emhttp: import flash device: sda