December 16, 201312 yr Hi, I have had an Unraid server on 4.7 for almost 2 years. It consisted of 1 parity and 7 data disks of various size and manufacture. Write speed was not spectacular, but acceptable at 25 MB/s. I recently upgraded to 5.0 and replaced my old 2 TB parity disk with a 3 TB WD Green drive, in preparation for adding more disks. I also reused the old parity disk as a new data disk. I recently noticed that the write performance has dropped dramatically. I now get around 8 MB/s write speed! I have tried writing to each separate drive (ie. \tower\disk1, etc) and the slowdown is the same for all disks. This is serious enough that I really have to solve it. I have spent some time reading the docs on slow speed but so far, nothing jumps out at me. Here is what I have done so far: Checked the ethernet connection. It is running fine at 1000 MB/s. Read performance is outstanding at 100 MB/s from the server to the my PC. I ran a bunch of local write tests (dd from /dev/zero) and all the disk are pretty close to 30 MB/s except for one which is 15 MB/s (more on that). Still, I don't see how this would result in such slow write performance. The way I understand unRAID is this: Read: as fast as the target disk/network channel will run. Write: server has to read all the disks to compute parity and then write data to data disk and parity to parity disk, so it would run as slow as the slowest disk read (assuming parallel SATA commands). Given this, I don't see anything in my results that points to the culprit unless it is the write speed of the parity disk itself and I don't know of any way to test that (aside from destroying parity, am I right?) If anyone has any suggestions as to what to do next, I would be very grateful. Here are the results of dd copies and hdparms: ========================================= Parity drive is 3TB WDC-WD30EZRX-00DC0B0 ========================================= disk1 2TB WD20EARS-00MVB0 root@Tower:~# dd if=/dev/zero of=/mnt/disk1/Videos/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 135.72 s, 30.9 MB/s root@Tower:~# hdparm -tT /dev/sdc /dev/sdc: Timing cached reads: 3718 MB in 2.00 seconds = 1859.06 MB/sec Timing buffered disk reads: 376 MB in 3.01 seconds = 124.76 MB/sec ========================================= disk2 3 2TB ST2000DL003-9VT166 root@Tower:~# dd if=/dev/zero of=/mnt/disk2/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 108.486 s, 38.7 MB/s root@Tower:~# hdparm -tT /dev/sdd /dev/sdd: Timing cached reads: 3702 MB in 2.00 seconds = 1850.87 MB/sec Timing buffered disk reads: 422 MB in 3.00 seconds = 140.54 MB/sec root@Tower:~# ========================================= disk3 2TB ST32000542AS root@Tower:~# dd if=/dev/zero of=/mnt/disk3/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 131.233 s, 32.0 MB/s root@Tower:~# hdparm -tT /dev/sde /dev/sde: Timing cached reads: 3684 MB in 2.00 seconds = 1842.34 MB/sec Timing buffered disk reads: 344 MB in 3.01 seconds = 114.35 MB/sec root@Tower:~# ====================================== disk4 2TB WD20EARX-00PASB0 root@Tower:~# dd if=/dev/zero of=/mnt/disk4/Backup1/test.dd count=819200 819200+0 records in 819200+0 records out 419430400 bytes (419 MB) copied, 20.5765 s, 20.4 MB/s root@Tower:~# hdparm -tT /dev/sdg /dev/sdg: Timing cached reads: 3920 MB in 2.00 seconds = 1960.33 MB/sec Timing buffered disk reads: 338 MB in 3.01 seconds = 112.23 MB/sec ========================================= disk5 2TB WD20EARS-00MVWB0 root@Tower:~# dd if=/dev/zero of=/mnt/disk5/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 274.682 s, 15.3 MB/s root@Tower:~# hdparm -tT /dev/sdi /dev/sdi: Timing cached reads: 3640 MB in 2.00 seconds = 1820.02 MB/sec Timing buffered disk reads: 332 MB in 3.01 seconds = 110.28 MB/sec ========================================= disk6 1TB WDC_WD10EADS-00 root@Tower:~# dd if=/dev/zero of=/mnt/disk6/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 140.744 s, 29.8 MB/s root@Tower:~# hdparm -tT /dev/sdj /dev/sdj: Timing cached reads: 3718 MB in 2.00 seconds = 1858.90 MB/sec Timing buffered disk reads: 272 MB in 3.01 seconds = 90.43 MB/sec ========================================= disk7 1TB ST31000528AS root@Tower:~# dd if=/dev/zero of=/mnt/disk7/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 135.284 s, 31.0 MB/s root@Tower:~# hdparm -tT /dev/sdh /dev/sdh: Timing cached reads: 3684 MB in 2.00 seconds = 1842.46 MB/sec Timing buffered disk reads: 352 MB in 3.01 seconds = 117.06 MB/sec ========================================== disk8 1TB SAMSUNG_HD103UI root@Tower:~# dd if=/dev/zero of=//mnt/disk8/Videos/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 175.949 s, 23.8 MB/s root@Tower:~# hdparm -tT /dev/sdf /dev/sdf: Timing cached reads: 3844 MB in 2.00 seconds = 1922.02 MB/sec Timing buffered disk reads: 300 MB in 3.02 seconds = 99.47 MB/sec
December 16, 201312 yr is your sata controller set to AHCI mode? even 30mb/s is horrible for the dd tests. also, post a syslog and systems specs.
December 16, 201312 yr Write: server has to read all the disks to compute parity and then write data to data disk and parity to parity disk, so it would run as slow as the slowest disk read (assuming parallel SATA commands). Just to clarify, on writes it does not read all disks. It reads existing parity from the parity drive, calculates new parity from existing parity and new data, then writes new parity to the parity drive and data to the data drive. The only drives involved in a write are the given data drive and the parity drive.
December 17, 201312 yr Author is your sata controller set to AHCI mode? even 30mb/s is horrible for the dd tests. also, post a syslog and systems specs. I just checked and,yes, the sata controller on the motherboard is set to AHCI. Is this bad? The other options are IDE and RAID. Can I simply set it RAID or will this cause problems. Also, I have a Supermicro AOC-SASLP-MV8 SATA card, too. How can I tell how it is configured? Attached is my syslog. Thanks for your help. Hunter [EDIT] forgot my specs: Unraid: 5.0 Motherboard: ECS A885GM-A2 SATA card: SUPERMICRO AOC-SASLP-MV8 Disks, MB=MB sata port device URAID port number size type/brand assign /dev/sdb parity MB 1 3TB WDC-WD30EZRX-00D /dev/sdc disk1 MB 2 2TB WD20EARS-00MVB0 /dev/sdd disk2 MB3 2TB ST2000DL003-9VT1 /dev/sde disk3 MB4 2TB ST32000542AS /dev/sdf disk8 MB5 1T SAMSUNG_HD103UI /dev/sdi disk5 Supermicro1 2T WD20EARS-00MVWB0 /dev/sdj disk6 Supermicro2 1T WDC_WD10EADS-00 /dev/sdh disk7 Supermicro3 1T ST31000528AS /dev/sdg disk4 Supermicro4 2T WD20EARX-00PASB0 20131217-_Tower_syslog.zip
December 17, 201312 yr Author Write: server has to read all the disks to compute parity and then write data to data disk and parity to parity disk, so it would run as slow as the slowest disk read (assuming parallel SATA commands). Just to clarify, on writes it does not read all disks. It reads existing parity from the parity drive, calculates new parity from existing parity and new data, then writes new parity to the parity drive and data to the data drive. The only drives involved in a write are the given data drive and the parity drive. Of course. Parity is cumulative (actually additive). Should have known that. Thanks.
December 17, 201312 yr is your sata controller set to AHCI mode? even 30mb/s is horrible for the dd tests. also, post a syslog and systems specs. I just checked and,yes, the sata controller on the motherboard is set to AHCI. Is this bad? The other options are IDE and RAID. Can I simply set it RAID or will this cause problems. Also, I have a Supermicro AOC-SASLP-MV8 SATA card, too. How can I tell how it is configured? Attached is my syslog. Thanks for your help. Hunter AHCI is what you want. Do not set it to IDE or RAID. As for the SASLP card, I don't own one, but you should be able to configure it by pressing a certain key combo (CTRL+M I believe) when the card is initializing.
December 19, 201312 yr Author is your sata controller set to AHCI mode? even 30mb/s is horrible for the dd tests. also, post a syslog and systems specs. So my SATA ports on the MB are set to AHCI. Any ideas why raw write speed would be so slow? What is a good write speed for modern disks? Given that I can get over 100 MB/s reads, should I expect that for writes, too?
December 29, 201312 yr I too am testing read/write speeds to and from the array. Internally between disks and external via network. I get 25 MB/s to perhaps 35 MB/s when writing to the array. Reading - I max out my gigabit connection, easily. Even if I have the parity drive disabled. Here are two different inputs of hdparm and dd like OP did: One is from running the ever popular TAMs machine with 3x AOC-SAS cards. root@Tower:~# hdparm -tT /dev/sdd /dev/sdd: Timing cached reads: 1812 MB in 2.00 seconds = 906.62 MB/sec Timing buffered disk reads: 562 MB in 3.01 seconds = 186.72 MB/sec root@Tower:~# dd if=/dev/zero of=/mnt/disk1/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 61.6544 s, 68.0 MB/s root@Tower:~# that's my test array. My production array is actually pitiful. This is a VM, with a Dell H310 HBA; the drives are connected to a Norco ss-400. root@Rigel:~# hdparm -tT /dev/sdg /dev/sdg: Timing cached reads: 17096 MB in 1.99 seconds = 8570.08 MB/sec Timing buffered disk reads: 370 MB in 3.00 seconds = 123.22 MB/sec root@Rigel:~# dd if=/dev/zero of=/mnt/disk1/Videos/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 149.959 s, 28.0 MB/s the above is to a wd green drive. here's to a 7200 spinner on the same machine: root@Rigel:~# hdparm -tT /dev/sdb /dev/sdb: Timing cached reads: 16526 MB in 2.00 seconds = 8278.69 MB/sec Timing buffered disk reads: 318 MB in 3.01 seconds = 105.79 MB/sec root@Rigel:~# dd if=/dev/zero of=/mnt/disk7/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 119.682 s, 35.0 MB/s For what it's worth - the Test is running latest 5.0.4; production is running 5.0rc16c.
December 29, 201312 yr Upgrade to 5.0.4. thanks - i'm planning on doing that later today or this week. i hadn't seen anything about performance being addressed in between - but always good to run latest. is SimpleFeatures the only plug in that does regularly scheduled array status emails? i'm really going to miss that (and the status screen)
December 30, 201312 yr Upgrade to 5.0.4. not much of a difference for me. root@Rigel:/mnt/disk7# dd if=/dev/zero of=/mnt/disk7/test.dd count=8192000 8192000+0 records in 8192000+0 records out 4194304000 bytes (4.2 GB) copied, 106.194 s, 39.5 MB/s soo at this point - the only thing i can blame is either the fact that it's a VM, my parity drive is too slow (shouldn't be the case) - Or the norco ss-400 is somehow interfering. at least now i'm on the latest and greatest.
December 30, 201312 yr is SimpleFeatures the only plug in that does regularly scheduled array status emails? i'm really going to miss that (and the status screen) SimpleFeatures is no longer compatible. Dynamix is the "new" SimpleFeatures.
December 30, 201312 yr is SimpleFeatures the only plug in that does regularly scheduled array status emails? i'm really going to miss that (and the status screen) SimpleFeatures is no longer compatible. Dynamix is the "new" SimpleFeatures. thanks - i saw that - but didn't want to try it due to that potential slow down issue on the polling. though i think that was fixed. anyhooo - i've tried: [*]different HBA PERC H310 and BR10i [*]increased VM ram from 4GB to 8GB with nothing else running [*]running unraid on baremetal (instead of inside VM). [*]changing BIOS settings to ensure AHCI (for built-in ports which I am not using None of these yielded any discernible difference outside of a +/- 3MB/s so I didn't really record the results. So we are left with: Motherboard/PCI-e issue (ports are set to run at x16, x8 and x8; hba being in x16 slot), Parity Drive issue (2TB, 7200 with 64MB cache Link ) or the Norco SS-400 Anything else I can/should check ? Hopefully i'm not jacking the thread from OP - I was thinking of starting this same thread. I'm tempted to do a survey to see what people are getting and what's acceptable, etc.
December 30, 201312 yr axeman start a new thread. If you're wondering if you're jacking a thread then you are.
December 30, 201312 yr CaptainTivo, zip and attach the syslog. Show the output of "ifconfig" and "ethtool eth0"
February 5, 201412 yr You ever figure this out? is SimpleFeatures the only plug in that does regularly scheduled array status emails? i'm really going to miss that (and the status screen) SimpleFeatures is no longer compatible. Dynamix is the "new" SimpleFeatures. thanks - i saw that - but didn't want to try it due to that potential slow down issue on the polling. though i think that was fixed. anyhooo - i've tried: [*]different HBA PERC H310 and BR10i [*]increased VM ram from 4GB to 8GB with nothing else running [*]running unraid on baremetal (instead of inside VM). [*]changing BIOS settings to ensure AHCI (for built-in ports which I am not using None of these yielded any discernible difference outside of a +/- 3MB/s so I didn't really record the results. So we are left with: Motherboard/PCI-e issue (ports are set to run at x16, x8 and x8; hba being in x16 slot), Parity Drive issue (2TB, 7200 with 64MB cache Link ) or the Norco SS-400 Anything else I can/should check ? Hopefully i'm not jacking the thread from OP - I was thinking of starting this same thread. I'm tempted to do a survey to see what people are getting and what's acceptable, etc.
Archived
This topic is now archived and is closed to further replies.