August 25, 201510 yr Great tool. Loved it. Curious how my cache is showing 90MB avg vs the graph and the console showing 381 average
August 26, 201510 yr Author Great tool. Loved it. Curious how my cache is showing 90MB avg vs the graph and the console showing 381 average Can you please run it with the following options, then zip up the file and email it to [email protected]? This will create the file diskspeed.log and only test your cache drive. NOTE: Please verify that your cache drive is sdk and alter the command below to accommodate (add sdk, remove current id) diskspeed.sh -l -x sdb,sdc,sdd,sde,sdf,sdg,sdh,sdi,sdj,sdl,sdm
August 27, 201510 yr Great tool. Loved it. Curious how my cache is showing 90MB avg vs the graph and the console showing 381 average Can you please run it with the following options, then zip up the file and email it to [email protected]? This will create the file diskspeed.log and only test your cache drive. NOTE: Please verify that your cache drive is sdk and alter the command below to accommodate (add sdk, remove current id) diskspeed.sh -l -x sdb,sdc,sdd,sde,sdf,sdg,sdh,sdi,sdj,sdl,sdm Sent!
September 10, 201510 yr Great tool. Loved it. Curious how my cache is showing 90MB avg vs the graph and the console showing 381 average Can you please run it with the following options, then zip up the file and email it to [email protected]? This will create the file diskspeed.log and only test your cache drive. NOTE: Please verify that your cache drive is sdk and alter the command below to accommodate (add sdk, remove current id) diskspeed.sh -l -x sdb,sdc,sdd,sde,sdf,sdg,sdh,sdi,sdj,sdl,sdm Sent! This would be very cool if it were part of the UI and you can just select the drive for testing. Anyway, how would you get the most accurate results with one drive? Like /dev/sdb ?
September 10, 201510 yr Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Yep, something is holding something back and this prior hardware came from my office home computer which was rock solid and performed very well. What to do.
September 10, 201510 yr Author Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Yep, something is holding something back and this prior hardware came from my office home computer which was rock solid and performed very well. What to do. Try running the tunables script at http://lime-technology.com/forum/index.php?topic=29009.0
September 10, 201510 yr Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Just wanted to provide a little perspective, as those numbers are far from 'bleek' for some of us! I would love to see numbers like those! Some of us have less money, more bills, but we do like seeing what we *could* get if only ... (I suspect that johnnie.black is a little spoiled by his all SSD array!) By the way, 6gbps is not likely to help you, I don't think. I don't remember the numbers (too lazy to figure them out right now), but I believe that 3gbps is more than fast enough for all but the fastest drives (and combined RAID'd drive setups).
September 10, 201510 yr Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Yep, something is holding something back and this prior hardware came from my office home computer which was rock solid and performed very well. What to do. I would have imagined better performance. Maybe its the H310? I have a smaller array with a couple of older drives on a sas2lp I have attached my diskspeed for comparision.
September 10, 201510 yr Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Just wanted to provide a little perspective, as those numbers are far from 'bleek' for some of us! I would love to see numbers like those! Some of us have less money, more bills, but we do like seeing what we *could* get if only ... (I suspect that johnnie.black is a little spoiled by his all SSD array!) By the way, 6gbps is not likely to help you, I don't think. I don't remember the numbers (too lazy to figure them out right now), but I believe that 3gbps is more than fast enough for all but the fastest drives (and combined RAID'd drive setups). I guess there is a performance cap on some systems, regardless of how much memory or powerful components you have. I guess I was just expecting a little more from a higher powered system. I wanted to rebuild my office home computer so the leftover parts became my new unraid build which is more or less way over powered in every aspect. I'll never expect a full 6Gbps at all, but read tests with all 6Gbps drives would expect 150MB/sec sustained. I'm all 6Gbps compliant so have support for the latest NCQ. If I take one of my array drives out of my unraid server and do a read test on an Windows external dock I can easily get a sustained 150MB/sec, so I guess I'm expecting that also on read tests on my unraid box. Bottleneck somewhere. Wrong to think that?
September 10, 201510 yr Ran the speedtest and results were pretty bleek. I think I do remember better speeds on V5. sdh and sdi are SSD drives. These are all 6Gbps drives, attached to 6Gbps controllers running a highly capable system to keep up with the speed. Yep, something is holding something back and this prior hardware came from my office home computer which was rock solid and performed very well. What to do. I would have imagined better performance. Maybe its the H310? I have a smaller array with a couple of older drives on a sas2lp I have attached my diskspeed for comparision. Very well could be the H310 cards. I flashed both of them pretty easily with the same firmware and checked my syslog line by line, nothing out of the ordinary. I'm also using a Norco 4224 base, so maybe the backplanes aren't very good? Who knows. I could try throwing some of the drives on my mainboards built in SATA controller to see if there is any difference but not sure it is worth the time or work.
September 12, 201510 yr Author For shitz-n-grins for maxing out a bus, I have four SDD's in a RAID0 on my new X99 PC build and the throughput caps out at 1.5 GByte/sec, which is just marginally better than three SDD's in a RAID0.
September 12, 201510 yr Yeah, that’s also my experience, Intel’s DMI 2.0 has 2GB/s max in theory but about 1.5 to 1.6GB/s in “real world” use. New Skylake chipsets now use DMI 3.0 that doubles the bandwidth.
September 12, 201510 yr Latest run after setting Tunables to most aggressive settings. New 8TB keeps up well. Check out that Parity speed (RAID0 of 2x4TB HGST). What does the "Moo" prefix on the drive refer too? Parity: Moo ARC-1231-RAID00_0000002273521567 8.0 TB 335 MB/sec avg Disk 1: TOSHIBA MD04ACA500 24R6K00RFS9A 5.0 TB 178 MB/sec avg Disk 2: Moo ST6000DX000-1H21_Z4D04L2A 6.0 TB 110 MB/sec avg Disk 3: WDC WD60EFRX-68MYMN1 WD-WX51D6402971 6.0 TB 140 MB/sec avg Disk 4: WDC WD20EZRX-00DC0B0 WD-WMC1T3951491 2.0 TB 123 MB/sec avg Disk 5: WDC WD20EZRX-00DC0B0 WD-WMC1T0154634 2.0 TB 130 MB/sec avg Disk 6: WDC WD20EZRX-00D8PB0 WD-WMC4M2038014 2.0 TB 128 MB/sec avg Disk 7: WDC WD20EZRX-00DC0B0 WD-WCC300673196 2.0 TB 131 MB/sec avg Disk 8: ST8000AS0002-1NA17Z Z8406M0L 8.0 TB 159 MB/sec avg Disk 9: Moo ST4000DM000-1F21_Z3024WY8 4.0 TB 161 MB/sec avg Disk 10: Moo ST4000DM000-1F21_Z3024WMZ 4.0 TB 157 MB/sec avg Disk 11: Moo ST2000DM001-1CH1_Z1F5DKBF 2.0 TB 167 MB/sec avg Cache: Samsung SSD 840 EVO 250GB S1DBNEAD812507A 250 GB 394 MB/sec avg
September 13, 201510 yr Author Pro tip: You can click on a drive in the legend to hide ones that push the other lines down.
September 13, 201510 yr Latest run after setting Tunables to most aggressive settings. New 8TB keeps up well. Check out that Parity speed (RAID0 of 2x4TB HGST). What does the "Moo" prefix on the drive refer too? Parity: Moo ARC-1231-RAID00_0000002273521567 8.0 TB 335 MB/sec avg Disk 1: TOSHIBA MD04ACA500 24R6K00RFS9A 5.0 TB 178 MB/sec avg Disk 2: Moo ST6000DX000-1H21_Z4D04L2A 6.0 TB 110 MB/sec avg Disk 3: WDC WD60EFRX-68MYMN1 WD-WX51D6402971 6.0 TB 140 MB/sec avg Disk 4: WDC WD20EZRX-00DC0B0 WD-WMC1T3951491 2.0 TB 123 MB/sec avg Disk 5: WDC WD20EZRX-00DC0B0 WD-WMC1T0154634 2.0 TB 130 MB/sec avg Disk 6: WDC WD20EZRX-00D8PB0 WD-WMC4M2038014 2.0 TB 128 MB/sec avg Disk 7: WDC WD20EZRX-00DC0B0 WD-WCC300673196 2.0 TB 131 MB/sec avg Disk 8: ST8000AS0002-1NA17Z Z8406M0L 8.0 TB 159 MB/sec avg Disk 9: Moo ST4000DM000-1F21_Z3024WY8 4.0 TB 161 MB/sec avg Disk 10: Moo ST4000DM000-1F21_Z3024WMZ 4.0 TB 157 MB/sec avg Disk 11: Moo ST2000DM001-1CH1_Z1F5DKBF 2.0 TB 167 MB/sec avg Cache: Samsung SSD 840 EVO 250GB S1DBNEAD812507A 250 GB 394 MB/sec avg Isn't the tunable script only to help with parity speeds? What were your aggressive tunable numbers at? Decent speeds. Also are you doubling up on your parity just for fault tolerance or speed?
September 13, 201510 yr Latest run after setting Tunables to most aggressive settings. New 8TB keeps up well. Check out that Parity speed (RAID0 of 2x4TB HGST). What does the "Moo" prefix on the drive refer too? Parity: Moo ARC-1231-RAID00_0000002273521567 8.0 TB 335 MB/sec avg Disk 1: TOSHIBA MD04ACA500 24R6K00RFS9A 5.0 TB 178 MB/sec avg Disk 2: Moo ST6000DX000-1H21_Z4D04L2A 6.0 TB 110 MB/sec avg Disk 3: WDC WD60EFRX-68MYMN1 WD-WX51D6402971 6.0 TB 140 MB/sec avg Disk 4: WDC WD20EZRX-00DC0B0 WD-WMC1T3951491 2.0 TB 123 MB/sec avg Disk 5: WDC WD20EZRX-00DC0B0 WD-WMC1T0154634 2.0 TB 130 MB/sec avg Disk 6: WDC WD20EZRX-00D8PB0 WD-WMC4M2038014 2.0 TB 128 MB/sec avg Disk 7: WDC WD20EZRX-00DC0B0 WD-WCC300673196 2.0 TB 131 MB/sec avg Disk 8: ST8000AS0002-1NA17Z Z8406M0L 8.0 TB 159 MB/sec avg Disk 9: Moo ST4000DM000-1F21_Z3024WY8 4.0 TB 161 MB/sec avg Disk 10: Moo ST4000DM000-1F21_Z3024WMZ 4.0 TB 157 MB/sec avg Disk 11: Moo ST2000DM001-1CH1_Z1F5DKBF 2.0 TB 167 MB/sec avg Cache: Samsung SSD 840 EVO 250GB S1DBNEAD812507A 250 GB 394 MB/sec avg Isn't the tunable script only to help with parity speeds? What were your aggressive tunable numbers at? Decent speeds. Also are you doubling up on your parity just for fault tolerance or speed? tunables: Unthrottled values for your server came from Test 29 with a speed of 105.5 MB/s Tunable (md_num_stripes): 5848 Tunable (md_write_limit): 2632 Tunable (md_sync_window): 2632 These settings will consume 251MB of RAM on your hardware. This is 185MB more than your current utilization of 66MB. RAID0 (parity volume) is definitely not fault tolerant, in fact it doubles my chance of a failure . It is for speed.
September 13, 201510 yr That's right. One drive fails your parity goes bubbye. Are you doing that for testing or something?
September 13, 201510 yr The tunables script has no effect on individual drive performance. It's merely for parity check speed.
September 14, 201510 yr That's right. One drive fails your parity goes bubbye. Are you doing that for testing or something? Nope. Its is a permanent performance enhancement
September 14, 201510 yr That's right. One drive fails your parity goes bubbye. Are you doing that for testing or something? Parity is the least important drive in the set of disks. Lose parity = no data loss, just rebuild parity. While rebuilding, risk of losing data is still just limited to any other drive that fails. 2 simultaneous drive failures, if one is parity, you only lose 1 drive worth of data, if both are data drives, you lost both data drives, even if parity is still fine. In his case, he can lose both parity drives and 1 data drive and still only lose 1 drive worth of data. I see it as an acceptable increase in risk considering the gains that are possible with multiple concurrent write streams.
September 14, 201510 yr That's right. One drive fails your parity goes bubbye. Are you doing that for testing or something? Nope. Its is a permanent performance enhancement Do you do parity checks that often where it becomes an issue? Just trying to make sense of it that's all. I ask questions to dig around for reasons so I can justify in my mind to force me to do it myself.
September 14, 201510 yr That's right. One drive fails your parity goes bubbye. Are you doing that for testing or something? Nope. Its is a permanent performance enhancement Do you do parity checks that often where it becomes an issue? Just trying to make sense of it that's all. I ask questions to dig around for reasons so I can justify in my mind to force me to do it myself. Sorry for the long post. Its a general performance boost when writing to the array. More truthfully, I have a RAID card I was using as an HBA for JBOD for my array drives. I read someone else had done the RAID0 thing and was intrigued by the concept, if not just the write performance boost. Give some background... my early plans were to fill it full of hard drives and periodically replace drives and/or parity as drive capacities grew. So I ended up with a case full of low priced (slow) green drives some which exhibited slow write performance (yeah EARS, all gone now w00t!). Then I switched to the Areca RAID card which gave me 12 ports over the 8 on the outgoing Supermicro. So following my previous planning, I started looking for a new case to add more drives... Then the 8TB SMR drives came to market. Full Stop! Why increase the number of drives? I should be increasing the capacity of each drive instead. Only way to use them (I thought) was to use one as parity. But they are even slower than the greens (actually find them slightly faster but that was the thought then) until I recalled others who had used RAID0 volumes as parity and the idea of two 7200 RPM HGST drives in RAID0 was born. So the new plan is to replace the 2TBs as I need more capacity with 8TB. So I am trending now toward fewer LARGER drives. I currently have 12.9 TB free so won't need anything very soon (famous last words).
September 20, 201510 yr Seems all my 2TB drives are slow as heck. Except for disk5, the Hitachi 2TB. I guess that has a different platter configuration then the older WD 2TB drives? I guess one slow drive in the array would slow down a parity check overall right? I think most of my 2TB drives are out of warranty and have a huge power on hours. Don't have issues with them, just a little slow I guess. Drive Identification Parity: WDC WD4001FAEX-00MJRA0 WD-WCC1F0127613 4.0 TB 133 MB/sec avg Disk 1: WDC WD2002FAEX-007BA0 WD-WMAWP0099064 2.0 TB 114 MB/sec avg Disk 2: HGST HDN724040ALE640 PK2338P4HE525C 4.0 TB 139 MB/sec avg Disk 3: Hitachi HDS723020BLA642 MN5220F32HHMJK 2.0 TB 118 MB/sec avg Disk 4: HGST HDN724040ALE640 PK2338P4HEPH8C 4.0 TB 138 MB/sec avg Disk 5: Hitachi HDS723020BLA642 MN1220F30803DD 2.0 TB 134 MB/sec avg Disk 6: WDC WD2002FAEX-007BA0 WD-WMAWP0284322 2.0 TB 114 MB/sec avg Disk 7: HGST HDN724040ALE640 PK1334PCJY9BRS 4.0 TB 139 MB/sec avg Disk 8: WDC WD4001FAEX-00MJRA0 WD-WCC130263021 4.0 TB 147 MB/sec avg Disk 9: WDC WD4003FZEX-00Z4SA0 WD-WCC130966733 4.0 TB 149 MB/sec avg Disk 10: HGST HDN724040ALE640 PK2334PCGYD32B 4.0 TB 141 MB/sec avg Disk 11: WDC WD2001FASS-00U0B0 WD-WMAUR0142521 2.0 TB 117 MB/sec avg Disk 12: WDC WD2002FAEX-007BA0 WD-WCAY01300087 2.0 TB 119 MB/sec avg sdg: WDC WD2002FAEX-007BA0 WD-WMAY01552774 2.0 TB 118 MB/sec avg sdh: Samsung SSD 850 EVO 120GB S21TNXAG505255N 120 GB 118 MB/sec avg sdi: Samsung SSD 840 EVO 120GB S1D5NSBFB10859N 120 GB 118 MB/sec avg
Archived
This topic is now archived and is closed to further replies.