November 13, 200916 yr Hi Tom Can we get suggested tuning for various configurations, for those who have quit powerful CPU and 4GB RAM? -- Michael The whole point of him putting these new tunables in was so that you could mess with it. I doubt he has the time to test this sort of thing ad nauseum. He gave you a good description of what each of these settings do, so my suggestion is to start messing with the settings and find what works best for you. There are going to be many many factors that go into something like this that only you will be able to control. Take his descriptions, set some new values for the tunables, and start copying files that are larger then 4GB (I suggest files like a full DVD rip) and see what you get.
November 13, 200916 yr Author Hi Tom Can we get suggested tuning for various configurations, for those who have quit powerful CPU and 4GB RAM? -- Michael The whole point of him putting these new tunables in was so that you could mess with it. I doubt he has the time to test this sort of thing ad nauseum. He gave you a good description of what each of these settings do, so my suggestion is to start messing with the settings and find what works best for you. There are going to be many many factors that go into something like this that only you will be able to control. Take his descriptions, set some new values for the tunables, and start copying files that are larger then 4GB (I suggest files like a full DVD rip) and see what you get. Yes, these 'tunables' were included to let people experiment with their particular h/w confguration; though admittedly direct editing of a config file is is not the most 'user friendly' method. Also, michael123 has an MD-1510 which is going to require different 'optimal' values. In fact it has become apparent that different server h/w is going to require different 'optimal' values. Therefore in next release (-beta9) I have made these configurable via 'Settings' page & implemented a method to specify different default values for different h/w configs.
November 13, 200916 yr thanks, Tom so I will wait until next beta :-) So far, anyway the performance improvement is 100% on writes!
November 13, 200916 yr Hi Tom Can we get suggested tuning for various configurations, for those who have quit powerful CPU and 4GB RAM? -- Michael The whole point of him putting these new tunables in was so that you could mess with it. I doubt he has the time to test this sort of thing ad nauseum. He gave you a good description of what each of these settings do, so my suggestion is to start messing with the settings and find what works best for you. There are going to be many many factors that go into something like this that only you will be able to control. Take his descriptions, set some new values for the tunables, and start copying files that are larger then 4GB (I suggest files like a full DVD rip) and see what you get. Yes, these 'tunables' were included to let people experiment with their particular h/w confguration; though admittedly direct editing of a config file is is not the most 'user friendly' method. Also, michael123 has an MD-1510 which is going to require different 'optimal' values. In fact it has become apparent that different server h/w is going to require different 'optimal' values. Therefore in next release (-beta9) I have made these configurable via 'Settings' page & implemented a method to specify different default values for different h/w configs. Good to know. I had forgotten that these settings were not visible in the web config quite yet. I remember reading that you were going to add those into the settings page, but I am kinda use to editing files myself that i take that for granted sometimes.
November 13, 200916 yr good job !!!!! write performance is x2 minimum !!!!!! 24 mo minimum 48 mo max in write !!!!!!
November 13, 200916 yr Yes, these 'tunables' were included to let people experiment with their particular h/w confguration; though admittedly direct editing of a config file is is not the most 'user friendly' method. Also, michael123 has an MD-1510 which is going to require different 'optimal' values. In fact it has become apparent that different server h/w is going to require different 'optimal' values. Therefore in next release (-beta9) I have made these configurable via 'Settings' page & implemented a method to specify different default values for different h/w configs. It would be nice if the community were able to determine the optimal values for various amounts of memory available in a system - i.e. 512 MB, 1 GB, 2 GB, etc. and have a simple drop down box selection for the user to specify how much RAM they want to dedicate to the unRAID services. I have 6 GB of memory in my system, but I would only like to allocate 2 GB to the unRAID services since I run other processes on the system; therefore an auto-detect feature would not work for me. This would be a whole lot more user friendly and less intimidating to the novice user. It just depends on what the goal is for unRAID long-term - to cater to the expert user and expose the settings (easy coding) or aim for a larger user base with less experienced users (more coding and research required). By the way, write performance went from 13 MB/s up to 28-30 MB/s for my system! I have also had some odd issues with a NIC that may or may not be related. My onboard NIC stopped working (Marvel 8053 - Gigabyte 965P-DS3 Motherboard) after I upgraded to 4.5beta8. I tried backing off back to 4.5beta7, but it still didn't work, so the NIC may have just failed. I only mention this as I have seen other NIC related posts in this thread.
November 13, 200916 yr good job !!!!! write performance is x2 minimum !!!!!! 24 mo minimum 48 mo max in write !!!!!! Are you using cache drive? How do you get 48MB/sec??
November 13, 200916 yr I can say that write performance has improved for me also, however I'm not getting quite the same performance boost as others. I get about a 33-50% improvement. I don't have Vista therefore I may not have the same benchmark to work with. When doing rsyncs from another host. I was getting bursty 20MB/s with long pauses depending on how full the cache was. Overall it was around 10-12Mb/s in the end. If I adjusted bwlimit I would get a "steady" 12MB/s without any pauses. Now I get a steady 18MB/s without any pauses and without any bwlimit options (4GB abit ab9 pro). I have not adjusted scheduler, buffering or any NCQ parameters. DD test of writing Segate 1.5tb 7200rpm 32MB cache parity write1gb: 1024000000 bytes (1.0 GB) copied, 35.4304 s, 28.9 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 140.515 s, 29.1 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 285.156 s, 14.4 MB/s - Seagate 1.0tb 7200rpm 32MB data 89% full. write4gb: 4096000000 bytes (4.1 GB) copied, 137.584 s, 29.8 MB/s - Seagate 1.5tb 5900rpm 32MB data 75% full. write4gb: 4096000000 bytes (4.1 GB) copied, 154.002 s, 26.6 MB/s - WD 1.0tb 5400rpm 16MB data 84% full. Here is my script in case anyone wants to review or comapre. Mare sure you cd to the directory/drive you want to test or run the script with the full path to the drive you want to test. #!/bin/bash if [ ! -z "${1}" ] then TMPFILE="${1}" else TMPFILE="test.$$" fi trap "rm -f ${TMPFILE}" EXIT HUP INT QUIT TERM echo "`date` writing to: ${TMPFILE}" dd if=/dev/zero of="${TMPFILE}" count=4000000 bs=1024 echo "`date` Done." ls -l --si ${TMPFILE} rm -f "${TMPFILE}"
November 14, 200916 yr i not use disk cache i use teracopy to transfer the data i have hitachi 2to hdds very speed disk !!!!!!
November 14, 200916 yr i not use disk cache i use teracopy to transfer the data i have hitachi 2to hdds very speed disk !!!!!! This one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822145298&cm_re=HITACHI_2TB-_-22-145-298-_-Product ?
November 14, 200916 yr In the next releases of unRAID is there going to be a option if you like to replace the cache disk with a data disc, reason that I ask is that I have one more disc I like to add to the server //Peter
November 14, 200916 yr DD test of writing Segate 1.5tb 7200rpm 32MB cache parity write1gb: 1024000000 bytes (1.0 GB) copied, 35.4304 s, 28.9 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 140.515 s, 29.1 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 285.156 s, 14.4 MB/s - Seagate 1.0tb 7200rpm 32MB data 89% full. write4gb: 4096000000 bytes (4.1 GB) copied, 137.584 s, 29.8 MB/s - Seagate 1.5tb 5900rpm 32MB data 75% full. write4gb: 4096000000 bytes (4.1 GB) copied, 154.002 s, 26.6 MB/s - WD 1.0tb 5400rpm 16MB data 84% full. CacheDirs running, no-op scheduler, WD 2TB 5400rpm parity drive. write 4gb: 4096000000 bytes (4.1 GB) copied, 49.7164 s, 82.4 MB/s - CACHE WD 1TB 7200rpm data 18% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 114.979 s, 35.6 MB/s - WD 2TB 5400rpm data 1% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 118.985 s, 34.4 MB/s - WD 2TB 5400rpm data 1% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 127.094 s, 32.2 MB/s - WD 2TB 5400rpm data 76% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 128.071 s, 32.0 MB/s - WD 2TB 5400rpm data 76% full. /boot/config/extra.cfg, not really tweaked, mostly just doubled the defaults. md_num_stripes=2784 md_write_limit=1536 md_sync_window=576
November 14, 200916 yr DD test of writing Segate 1.5tb 7200rpm 32MB cache parity write1gb: 1024000000 bytes (1.0 GB) copied, 35.4304 s, 28.9 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 140.515 s, 29.1 MB/s - Seagate 1.5tb 7200rpm 32MB data 71% full. write4gb: 4096000000 bytes (4.1 GB) copied, 285.156 s, 14.4 MB/s - Seagate 1.0tb 7200rpm 32MB data 89% full. write4gb: 4096000000 bytes (4.1 GB) copied, 137.584 s, 29.8 MB/s - Seagate 1.5tb 5900rpm 32MB data 75% full. write4gb: 4096000000 bytes (4.1 GB) copied, 154.002 s, 26.6 MB/s - WD 1.0tb 5400rpm 16MB data 84% full. What is "no-op scheduler"? CacheDirs running, no-op scheduler, WD 2TB 5400rpm parity drive. write 4gb: 4096000000 bytes (4.1 GB) copied, 49.7164 s, 82.4 MB/s - CACHE WD 1TB 7200rpm data 18% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 114.979 s, 35.6 MB/s - WD 2TB 5400rpm data 1% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 118.985 s, 34.4 MB/s - WD 2TB 5400rpm data 1% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 127.094 s, 32.2 MB/s - WD 2TB 5400rpm data 76% full. write 4gb: 4096000000 bytes (4.1 GB) copied, 128.071 s, 32.0 MB/s - WD 2TB 5400rpm data 76% full. /boot/config/extra.cfg, not really tweaked, mostly just doubled the defaults. md_num_stripes=2784 md_write_limit=1536 md_sync_window=576
November 14, 200916 yr What is "no-op scheduler"? http://en.wikipedia.org/wiki/Noop_scheduler OK, thanks. But practically, how do I configure it? What is the default method (if I need to go back)? What is the "official" position of Lime? When is it recommended, if at all?
November 14, 200916 yr To see what you have on your SATA disks: for i in /sys/block/sd*/queue/scheduler; do cat $i; done To set it to noop, or cfq, or the others available: echo noop > /sys/block/sd#/queue/scheduler
November 14, 200916 yr Apologies in advance if this is the wrong place to post this. md_num_stripes=2560 md_write_limit=1536 md_sync_window=572 And New parity 1.5TB Seagate Barracuda 5900rpm ST315005N4A1AS-RK server RAM 2GB Parity Sync is at: Total size: 1,465,138,552 KB Current position: 1,159,859,200 (79.1%) Estimated speed: 65,320 KB/sec Estimated finish: 77.8 minutes Started at about 2am currently at 11am I tried some larger values (4X) but server responded to ping but not to the web interface. I am adding another 1.5TB Seagate Barracuda 5900rpm ST315005N4A1AS-RK I will post some numbers for the disk rebuild and copy stats later today. Also largest drive in array was 1TB until this point. so I think I am at the point where parity is at it's fastest since there is no read operations at this point? Then again the bottleneck is probably write regardless and since these are the slowest spinning drives at 5900rpm this was probably never the issue anyway?
November 14, 200916 yr To see what you have on your SATA disks: for i in /sys/block/sd*/queue/scheduler; do cat $i; done To set it to noop, or cfq, or the others available: echo noop > /sys/block/sd#/queue/scheduler root@VAULT:~# for i in /sys/block/sd*/queue/scheduler; do cat $i; done [noop] anticipatory deadline cfq [noop] anticipatory deadline cfq [noop] anticipatory deadline cfq [noop] anticipatory deadline cfq [noop] anticipatory deadline cfq [noop] anticipatory deadline cfq root@VAULT:~# Does it mean that Tom already set no-op for me?
November 14, 200916 yr /boot/config/extra.cfg, not really tweaked, mostly just doubled the defaults. md_num_stripes=2784 md_write_limit=1536 md_sync_window=576 After I adjusted my numbers to the same here, I saw some tests change from 29MB/s to 31.5MB/s. Thanks.
November 15, 200916 yr i not use disk cache i use teracopy to transfer the data i have hitachi 2to hdds very speed disk !!!!!! This one: http://www.newegg.com/Product/Product.aspx?Item=N82E16822145298&cm_re=HITACHI_2TB-_-22-145-298-_-Product ? yes this one
Archived
This topic is now archived and is closed to further replies.