jbartlett

Ran the script twice. First time was after a couple days of use, 2nd time was fresh after a reboot. ETA: There's up to 5mb improvement after a reboot and I did get a different end result Tunables Report from unRAID Tunables Tester v2.2 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --- FULLY AUTOMATIC TEST PASS 1 (Rough - 20 Sample Points @ 3min Duration)--- 1 | 1408 | 768 | 512 | 75.8 MB/s 2 | 1536 | 768 | 640 | 58.1 MB/s 3 | 1664 | 768 | 768 | 62.3 MB/s 4 | 1920 | 896 | 896 | 58.0 MB/s 5 | 2176 | 1024 | 1024 | 59.6 MB/s 6 | 2560 | 1152 | 1152 | 54.9 MB/s 7 | 2816 | 1280 | 1280 | 55.6 MB/s 8 | 3072 | 1408 | 1408 | 57.6 MB/s 9 | 3328 | 1536 | 1536 | 53.1 MB/s 10 | 3584 | 1664 | 1664 | 48.1 MB/s 11 | 3968 | 1792 | 1792 | 51.3 MB/s 12 | 4224 | 1920 | 1920 | 55.9 MB/s 13 | 4480 | 2048 | 2048 | 48.0 MB/s 14 | 4736 | 2176 | 2176 | 53.2 MB/s 15 | 5120 | 2304 | 2304 | 51.8 MB/s 16 | 5376 | 2432 | 2432 | 55.1 MB/s 17 | 5632 | 2560 | 2560 | 52.0 MB/s 18 | 5888 | 2688 | 2688 | 50.6 MB/s 19 | 6144 | 2816 | 2816 | 42.9 MB/s 20 | 6528 | 2944 | 2944 | 58.6 MB/s --- Targeting Fastest Result of md_sync_window 512 bytes for Special Pass --- --- FULLY AUTOMATIC TEST PASS 1b (Rough - 4 Sample Points @ 3min Duration)--- 21 | 896 | 768 | 128 | 83.2 MB/s 22 | 1024 | 768 | 256 | 78.6 MB/s 23 | 1280 | 768 | 384 | 78.1 MB/s 24 | 1408 | 768 | 512 | 71.8 MB/s --- Targeting Fastest Result of md_sync_window 128 bytes for Final Pass --- --- FULLY AUTOMATIC TEST PASS 2 (Final - 16 Sample Points @ 4min Duration)--- 25 | 856 | 768 | 8 | 72.8 MB/s 26 | 864 | 768 | 16 | 68.1 MB/s 27 | 880 | 768 | 24 | 68.7 MB/s 28 | 888 | 768 | 32 | 74.7 MB/s 29 | 896 | 768 | 40 | 74.9 MB/s 30 | 904 | 768 | 48 | 78.4 MB/s 31 | 912 | 768 | 56 | 78.9 MB/s 32 | 920 | 768 | 64 | 82.4 MB/s 33 | 928 | 768 | 72 | 80.4 MB/s 34 | 936 | 768 | 80 | 79.8 MB/s 35 | 944 | 768 | 88 | 82.0 MB/s 36 | 960 | 768 | 96 | 81.8 MB/s 37 | 968 | 768 | 104 | 81.3 MB/s 38 | 976 | 768 | 112 | 81.7 MB/s 39 | 984 | 768 | 120 | 80.5 MB/s 40 | 992 | 768 | 128 | 82.0 MB/s Completed: 2 Hrs 21 Min 56 Sec. Best Bang for the Buck: Test 1 with a speed of 75.8 MB/s Tunable (md_num_stripes): 1408 Tunable (md_write_limit): 768 Tunable (md_sync_window): 512 These settings will consume 55MB of RAM on your hardware. Unthrottled values for your server came from Test 32 with a speed of 82.4 MB/s Tunable (md_num_stripes): 920 Tunable (md_write_limit): 768 Tunable (md_sync_window): 64 These settings will consume 35MB of RAM on your hardware. This is -85MB less than your current utilization of 120MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values. Tunables Report from unRAID Tunables Tester v2.2 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --- FULLY AUTOMATIC TEST PASS 1 (Rough - 20 Sample Points @ 3min Duration)--- 1 | 1408 | 768 | 512 | 75.6 MB/s 2 | 1536 | 768 | 640 | 66.2 MB/s 3 | 1664 | 768 | 768 | 60.4 MB/s 4 | 1920 | 896 | 896 | 59.0 MB/s 5 | 2176 | 1024 | 1024 | 56.7 MB/s 6 | 2560 | 1152 | 1152 | 53.6 MB/s 7 | 2816 | 1280 | 1280 | 56.1 MB/s 8 | 3072 | 1408 | 1408 | 59.4 MB/s 9 | 3328 | 1536 | 1536 | 54.4 MB/s 10 | 3584 | 1664 | 1664 | 54.3 MB/s 11 | 3968 | 1792 | 1792 | 51.5 MB/s 12 | 4224 | 1920 | 1920 | 58.8 MB/s 13 | 4480 | 2048 | 2048 | 55.8 MB/s 14 | 4736 | 2176 | 2176 | 53.2 MB/s 15 | 5120 | 2304 | 2304 | 60.7 MB/s 16 | 5376 | 2432 | 2432 | 46.2 MB/s 17 | 5632 | 2560 | 2560 | 54.1 MB/s 18 | 5888 | 2688 | 2688 | 53.9 MB/s 19 | 6144 | 2816 | 2816 | 52.9 MB/s 20 | 6528 | 2944 | 2944 | 67.3 MB/s --- Targeting Fastest Result of md_sync_window 512 bytes for Special Pass --- --- FULLY AUTOMATIC TEST PASS 1b (Rough - 4 Sample Points @ 3min Duration)--- 21 | 896 | 768 | 128 | 81.2 MB/s 22 | 1024 | 768 | 256 | 83.3 MB/s 23 | 1280 | 768 | 384 | 83.3 MB/s 24 | 1408 | 768 | 512 | 78.4 MB/s --- Targeting Fastest Result of md_sync_window 256 bytes for Final Pass --- --- FULLY AUTOMATIC TEST PASS 2 (Final - 16 Sample Points @ 4min Duration)--- 25 | 1000 | 768 | 136 | 85.4 MB/s 26 | 1008 | 768 | 144 | 83.1 MB/s 27 | 1016 | 768 | 152 | 82.1 MB/s 28 | 1024 | 768 | 160 | 80.4 MB/s 29 | 1040 | 768 | 168 | 79.2 MB/s 30 | 1048 | 768 | 176 | 80.8 MB/s 31 | 1056 | 768 | 184 | 80.2 MB/s 32 | 1064 | 768 | 192 | 80.7 MB/s 33 | 1072 | 768 | 200 | 80.3 MB/s 34 | 1080 | 768 | 208 | 83.6 MB/s 35 | 1088 | 768 | 216 | 81.0 MB/s 36 | 1096 | 768 | 224 | 82.1 MB/s 37 | 1104 | 768 | 232 | 83.3 MB/s 38 | 1120 | 768 | 240 | 82.9 MB/s 39 | 1128 | 768 | 248 | 81.7 MB/s 40 | 1136 | 768 | 256 | 85.3 MB/s Completed: 2 Hrs 21 Min 42 Sec. Best Bang for the Buck: Test 1 with a speed of 75.6 MB/s Tunable (md_num_stripes): 1408 Tunable (md_write_limit): 768 Tunable (md_sync_window): 512 These settings will consume 55MB of RAM on your hardware. Unthrottled values for your server came from Test 25 with a speed of 85.4 MB/s Tunable (md_num_stripes): 1000 Tunable (md_write_limit): 768 Tunable (md_sync_window): 136 These settings will consume 39MB of RAM on your hardware. This is -81MB less than your current utilization of 120MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Still have a couple pages to read but was your test writes to the share or to the disk directly? I find that writing to the share is both slower and the speed more variable. -John

Here is the results of the rebuild test. Tunables Report from unRAID Tunables Tester v2.2 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --------------------------------------------------------------------------- 1 | 1544 | 768 | 8 | 45.2 MB/s 2 | 1544 | 768 | 72 | 59.8 MB/s 3 | 1544 | 768 | 136 | 59.9 MB/s 4 | 1544 | 768 | 200 | 73.9 MB/s 5 | 1544 | 768 | 264 | 88.4 MB/s 6 | 1544 | 768 | 328 | 101.6 MB/s 7 | 1544 | 768 | 392 | 106.9 MB/s 8 | 1544 | 768 | 456 | 112.4 MB/s Completed: 0 Hrs 33 Min 52 Sec. Best Bang for the Buck: Test 2 with a speed of 59.8 MB/s Tunable (md_num_stripes): 1544 Tunable (md_write_limit): 768 Tunable (md_sync_window): 72 These settings will consume 60MB of RAM on your hardware. Unthrottled values for your server came from Test 8 with a speed of 112.4 MB/s Tunable (md_num_stripes): 1544 Tunable (md_write_limit): 768 Tunable (md_sync_window): 456 These settings will consume 60MB of RAM on your hardware. This is -60MB less than your current utilization of 120MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Restarted the test after watching a tail on the syslog - there's two places the NumStripes variable is calculated - in the Calc function and inside the RunTest loop at the end.

Kicked off, with a 64 byte increment. I can also repeat the test with the SSD to see how much of an impact it had. -John

Very unlikely, since it's a Gb network. I can do writes over 100MB/s between my Windows machines ... so the ~50MB/s I'm getting on UnRAID isn't even close to what the desktop and network can provide. Just to be sure, I just wrote the same test file I've been using to my wife's machine across the network, and it completed in 28 seconds. Try copying a file from the cache drive.

Here are the results of running the script with parity rebuild and the following settings: Thorough (4 min), 128 byte increment, start at 8, end at 1920. Is this suitable or would you like me to update the following /root/mdcmd set md_write_limit $WriteLimit to /root/mdcmd set md_write_limit 768 ETA: This is with the new 4TB drive replacing the wonky 32GB SSD Tunables Report from unRAID Tunables Tester v2.2 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --------------------------------------------------------------------------- 1 | 776 | 768 | 8 | 43.2 MB/s 2 | 904 | 768 | 136 | 59.0 MB/s 3 | 1032 | 768 | 264 | 86.4 MB/s 4 | 1288 | 768 | 392 | 106.5 MB/s 5 | 1416 | 768 | 520 | 113.1 MB/s 6 | 1544 | 768 | 648 | 113.7 MB/s 7 | 1680 | 776 | 776 | 113.8 MB/s 8 | 1936 | 904 | 904 | 113.3 MB/s 9 | 2192 | 1032 | 1032 | 113.8 MB/s 10 | 2576 | 1160 | 1160 | 114.0 MB/s 11 | 2832 | 1288 | 1288 | 113.9 MB/s 12 | 3088 | 1416 | 1416 | 114.1 MB/s 13 | 3344 | 1544 | 1544 | 114.1 MB/s 14 | 3600 | 1672 | 1672 | 114.0 MB/s 15 | 3984 | 1800 | 1800 | 114.2 MB/s Completed: 1 Hrs 2 Min 59 Sec. Best Bang for the Buck: Test 6 with a speed of 113.7 MB/s Tunable (md_num_stripes): 1544 Tunable (md_write_limit): 768 Tunable (md_sync_window): 648 These settings will consume 60MB of RAM on your hardware. Unthrottled values for your server came from Test 15 with a speed of 114.2 MB/s Tunable (md_num_stripes): 3984 Tunable (md_write_limit): 1800 Tunable (md_sync_window): 1800 These settings will consume 155MB of RAM on your hardware. This is 112MB more than your current utilization of 43MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Buggar it. About to crash (edit: sleep), will run a spread-spectrum scan with the script as-is except for parity rebuilds. Thorough (4 min), 128 byte increment, start at 8, end at 1920

The difference is seeing what the effect of the different values on a parity rebuild vs parity checks using the tunables script which plugs in different numbers for each iteration. It's about finding out the optimized value for a rebuild to see if it's on par for the optimized value for sync. I just need to know the correct way to do it.

I was trying to stop the array, invoked by the GUI. It is repeatable for me - reseted the box twice before I realized what was going on.

The script tests parity checks which uses md_sync_window but parity builds uses md_write_limit. Would it compare apples to apples if I do the parity build checks by updating the script to reverse the two values or set them both to md_write_limit?

After discovering that you can't stop an array with the preclear running, I said buggars and dropped the 4 TB in. I'll let a full parity check after the rebuild tell me if the drive gets any pending sectors. I'm not worried about the data, it's backed up elsewhere. Running the script with full-auto and modified to perform a parity rebuild. After it finishes, I'll let the parity fully rebuild and then run the script again unmodified. ETA: Correction, decided to do a 4 minute test run with 128 byte spread from 8 to 2944 bytes. ETA: Scratch that. New post below.

Discovered that I am unable to stop the array while the preclear is running with UNRAID version 5.0 (final). It hangs on the sync command. Cancel the pre-clear, UNRAID sync command completes.

I'd love to get that new 4TB drive in there but I've started the preclear process 3 times now. 1st time seemed to hang, 2nd time the putty window simply vanished.

Man! I was hoping I wouldn't have to open up the tower since I decided to upgrade the SSD (external drive bays).... *snerk* I'm pretty sure they are, I wouldn't short-change a card in a slot. -John ETA: Specs report the slots as PCIe 2.0 x16

Here's the two SATA PCI-E controllers I'm using in my Bizarro box. http://www.newegg.com/Product/Product.aspx?Item=N82E16816132018 -John

I think you're right about my server being from the Bizarro world. It's nearing the end of a 3TB drive and the speeds are estimated at 105mb/sec

Oh, my recomendation if you decide to do away with the "unthrottled" label is to use "balls to the wall"

Come to think of it, I can use your script with a minor modification to do all the parity rebuild tests. You have the following which kicks off a parity check: /root/mdcmd check NOCORRECT What is the command to kick off a parity rebuild?

With upgrading the 32GB SSD, I would only need to let the parity rebuild run for less than 10 minutes to see how long it takes to rebuild the 32GB range. If you really want to test the rebuild timings, create a script that'll kick off the parity rebuild, let it run to 32 GB, then abort the rebuild and repeat looping over the different values. I'm game with running it on my server - as long as one of my normal drives doesn't fail

Would replacing the 32GB SSD with the new 4TB and doing the rebuild suit your test needs or would it be better with upgrading one of the 1.5TB drives? I've decided to remove the SSD as it's primary purpose is no longer being utilized. I'm currently running a parity check on the last tests recommended values. It's at 46%, 1.7TB mark. Currently getting 84.7MB/sec It'll be tomorrow at the earliest before I can do the rebuild but I can do the other request once the parity check completes in 7-8 hours.

You did. I have it in there as an xbmc cache drive for my xbmc clients so there's no spinup delays in accessing thumbnails & the like. Though I'm starting to lean towards not having a central cache and letting the icons exist on each server. Here's the results of version 2.2: Tunables Report from unRAID Tunables Tester v2.2 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --- FULLY AUTOMATIC TEST PASS 1 (Rough - 20 Sample Points @ 3min Duration)--- 1 | 1408 | 768 | 512 | 90.5 MB/s 2 | 1536 | 768 | 640 | 87.7 MB/s 3 | 1664 | 768 | 768 | 87.2 MB/s 4 | 1920 | 896 | 896 | 86.9 MB/s 5 | 2176 | 1024 | 1024 | 86.0 MB/s 6 | 2560 | 1152 | 1152 | 86.5 MB/s 7 | 2816 | 1280 | 1280 | 86.4 MB/s 8 | 3072 | 1408 | 1408 | 86.7 MB/s 9 | 3328 | 1536 | 1536 | 86.6 MB/s 10 | 3584 | 1664 | 1664 | 87.4 MB/s 11 | 3968 | 1792 | 1792 | 85.9 MB/s 12 | 4224 | 1920 | 1920 | 84.6 MB/s 13 | 4480 | 2048 | 2048 | 87.4 MB/s 14 | 4736 | 2176 | 2176 | 86.6 MB/s 15 | 5120 | 2304 | 2304 | 86.4 MB/s 16 | 5376 | 2432 | 2432 | 85.2 MB/s 17 | 5632 | 2560 | 2560 | 83.7 MB/s 18 | 5888 | 2688 | 2688 | 85.6 MB/s 19 | 6144 | 2816 | 2816 | 86.3 MB/s 20 | 6528 | 2944 | 2944 | 85.2 MB/s --- Targeting Fastest Result of md_sync_window 512 bytes for Special Pass --- --- FULLY AUTOMATIC TEST PASS 1b (Rough - 4 Sample Points @ 3min Duration)--- 21 | 896 | 768 | 128 | 93.3 MB/s 22 | 1024 | 768 | 256 | 95.7 MB/s 23 | 1280 | 768 | 384 | 93.6 MB/s 24 | 1408 | 768 | 512 | 89.5 MB/s --- Targeting Fastest Result of md_sync_window 256 bytes for Final Pass --- --- FULLY AUTOMATIC TEST PASS 2 (Final - 16 Sample Points @ 4min Duration)--- 25 | 1000 | 768 | 136 | 92.3 MB/s 26 | 1008 | 768 | 144 | 92.0 MB/s 27 | 1016 | 768 | 152 | 93.3 MB/s 28 | 1024 | 768 | 160 | 94.0 MB/s 29 | 1040 | 768 | 168 | 94.0 MB/s 30 | 1048 | 768 | 176 | 90.7 MB/s 31 | 1056 | 768 | 184 | 94.2 MB/s 32 | 1064 | 768 | 192 | 95.2 MB/s 33 | 1072 | 768 | 200 | 97.3 MB/s 34 | 1080 | 768 | 208 | 96.2 MB/s 35 | 1088 | 768 | 216 | 96.1 MB/s 36 | 1096 | 768 | 224 | 95.6 MB/s 37 | 1104 | 768 | 232 | 98.0 MB/s 38 | 1120 | 768 | 240 | 96.8 MB/s 39 | 1128 | 768 | 248 | 96.5 MB/s 40 | 1136 | 768 | 256 | 95.9 MB/s Completed: 2 Hrs 21 Min 31 Sec. Best Bang for the Buck: Test 1 with a speed of 90.5 MB/s Tunable (md_num_stripes): 1408 Tunable (md_write_limit): 768 Tunable (md_sync_window): 512 These settings will consume 55MB of RAM on your hardware. Unthrottled values for your server came from Test 37 with a speed of 98.0 MB/s Tunable (md_num_stripes): 1104 Tunable (md_write_limit): 768 Tunable (md_sync_window): 232 These settings will consume 43MB of RAM on your hardware. This is 8MB more than your current utilization of 35MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Seeing differing results on 5.0 final, lot higher and more steady. Going to restart with the script unmodified. Test 1 - md_sync_window=512 - Completed in 604.779 seconds = 94.0 MB/s Test 2 - md_sync_window=640 - Completed in 605.112 seconds = 89.1 MB/s Test 3 - md_sync_window=768 - Completed in 605.048 seconds = 89.5 MB/s Test 4 - md_sync_window=896 - Completed in 605.122 seconds = 88.6 MB/s Test 5 - md_sync_window=1024 - Completed in 604.995 seconds = 89.2 MB/s

I just kicked off a fullauto with version 2.2 with the following changes: Setting TestLen to 599 instead of 239 (in 3 places in the full-auto logic) to run for 10 minutes in order to get past my 32GB SSD drive which seems to skew the results. System specs MB: ASUSTeK Computer INC. - M4A88TD-V EVO/USB3 CPU: AMD Phenom II X4 965 - 3.4 GHz RAM: 4GB HD: 1.5TB WD EARS (3, jumpered), 2TB WD (1), 3TB Seagate (3), 4TB Seagate (2+parity), 32GB SSD (1), 500GB WD (Cache) Hard drives are connected to MB and to two PCI-E SATA extenders. Been replacing the EARS drives, one a month, with the 4TB Seagates. Preclearing another 4TB now. -John

Interesting. Seems the lowest possible setting is the best setting for me. Tunables Report from unRAID Tunables Tester v2.1 by Pauven NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Test | num_stripes | write_limit | sync_window | Speed --------------------------------------------------------------------------- 1 | 896 | 768 | 128 | 83.5 MB/s 2 | 1024 | 768 | 256 | 82.9 MB/s 3 | 1280 | 768 | 384 | 80.1 MB/s 4 | 1408 | 768 | 512 | 74.6 MB/s 5 | 1536 | 768 | 640 | 66.9 MB/s Completed: 0 Hrs 52 Min 20 Sec. Best Bang for the Buck: Test 1 with a speed of 83.5 MB/s Tunable (md_num_stripes): 896 Tunable (md_write_limit): 768 Tunable (md_sync_window): 128 These settings will consume 35MB of RAM on your hardware. Unthrottled values for your server came from Test 1 with a speed of 83.5 MB/s Tunable (md_num_stripes): 896 Tunable (md_write_limit): 768 Tunable (md_sync_window): 128 These settings will consume 35MB of RAM on your hardware. This is -15MB less than your current utilization of 50MB. NOTE: Adding additional drives will increase memory consumption. In unRAID, go to Settings > Disk Settings to set your chosen parameter values.