Pauven

PITA. That's the truth. I guess I was hoping for a miracle one-liner, like how: eval $(egrep 'md_num_stripes|md_sync_window|md_sync_thresh|nr_requests' /boot/config/disk.cfg | tr -d '\r') creates 4 vars with values assigned. Freaking awesome. I was also hopeful to find the data in the "mdcmd status" output, as at least that identifies the drive # on each line. But I don't see spin status or name there. If it is simply a matter of looping through, finding one value, then trusting the next value is the one I'm looking for, I can do that. PITA, but doable. Thanks, Paul

Exactly what I was looking for, thanks! The official documentation should be updated to include this solution. I'm not to the plug-in stage yet. Someday.

You're Hired! I would appreciate it if, during the beta as you post your results, you clarify which server the results apply to and re-list the config for that server. -Paul

There's a lot of interesting data in the disks.ini file, and I would like to pull some of it out, and would appreciate some help with the logic for doing so. I primarily would like the name value (i.e. "disk18" or "cache" or "parity2" ) along with the device (i.e. "sdd"). I would want to filter on the status (don't want "DISK_NP"). It would also be great if I could get the spin status for each drive, either from here or somewhere else. I see a rotational value, but this seems to be a way to distinguish HD's from SSD's. Thanks, Paul

Okay, got a more [minor] challenge. I'm now checking the unRAID version in the script, for compatibility reasons. Based upon Lime-Tech documentation, I'm pulling this value from the SysLog. But the tunables test script creates enough log entries that eventually this value rolls off. Is there a better way of getting the unRAID version? Thanks, Paul

Sorry, didn't notice this before: Dual Parity - Pentium G2030/8GB/22HDD(2TB)/SAS2LP+Adaptec 1430SA Dual Parity - Celeron G560/8GB/14HDD(3&4TB)/LSI9211-8i Single Parity - Celeron G560/8GB/14HDD(2,3&6TB)/SASLP+Adaptec 1430SA Dual Parity - Celeron G570/8GB/8HDD(4&8TB)/Adaptec 1430SA Dual Parity - Pentium G4400/8GB/30SSD(64,80,120&256)/LSI9207-8i(with expander RES2SV240) + LSI9211-8i Dual Parity - Pentium G2030/8GB/22HDD(2&3TB)/SAS2LP+Adaptec 1430SA Single Parity - Xeon E3-1220/32GB/5HDD(3TB)/Asmedia 1601 What the #*%$!!!! Do you really have 7 unRAID servers!!! 115 various HD's and SSD's!!! And what looks like in the neighborhood of 225TB of storage!!!!!! Are you affiliated with Backblaze? I'll take a leap of faith and assume those dual parity servers are 6.2. What about the two single parity? Since you listed them all, are you planning on testing on all of them? I've very curious about that SSD server you got. And the one with the expander. And the... oh hell all of em! Wow. Paul

For all those who are interested, may I be so bold as to request that you not only provide system specs, but also put them in your signature. It's difficult to keep everyone's configuration(s) straight in my head, and inconvenient for me to hunt through pages of a thread trying to find that one post where you listed it. Also, now is a good time to make sure your signature is up to date. Please make sure your signature includes: UnRAID version and license type If you're visualized, the vm platform Motherboard, cpu, ram Controllers in use, including mb controllers and HBA's and multipliers Single /Dual Parity and size of those drives All hard drives in array, and if connected to more than one controller, grouped by their connections Any cache drives Any additional drives outside the array ljm42, StevenD, Scott, squid and myself are examples of what I'm looking for. Thanks Paul

unRAID version? Your signature says 6.0.

Requirements: unRAID v6.2, that's it. Crazy servers welcome. Would be interested in a VM, and various different drive controllers. Single and dual parity. Single controllers and multiple controllers. Pro and non-pro licenses. I see most of you have system specs in your signatures. If you don't, please detail them. Thanks, Paul

Okay, time for a BETA sign-up. I would like 3-5 users willing to beta test. Just respond here. Thanks. Plan to have a unRAID Tunables Tester v4.0b ready for test tomorrow (Tuesday). -Paul

What values were used for each test? The wrong values were used for the two 8.5 hour tests. A combination of stock unRAID values and stripes/window values that were optimized for 5.0. The 7:40 test used md_num_stripes=4848, md_sync_window=2424, md_sync_thresh=2423, nr_requests=8. These are the values my tool discovered for my server. I'm thinking 7:20 was my previous best on v5.0, with only a single parity. My CPU is so low-end it is probably slowing down the dual parity check. Or there may still be another combination of values that wrings some more performance, but I doubt it. I've been making some nice coding enhancements to the tool today. I added new routines to better exclude testing both extra low values, high values, and really high values, unless the tool first observes good numbers in the low/high ranges. I think that's one of the last wish list items I picked up from this thread. By excluding the low/high ranges, the default test will run 9 hours, up to a maximum of about 16 hours. My server would run 15 hours since it likes the really high values, but not the low values. A server that likes the low values would run in about 10 hours. Yes, I know that sounds long, but these same improvements on the old 2.2 code would have brought the run time down to about 95 minutes. The new test runs so much longer because it's running two test at every sample point to test md_sync_thresh, I've doubled the sample points, almost doubled each test length, and added in nr_requests tests. So basically you can thank unRAID v6.2 for a much longer test. Getting closer to a beta release. -Paul

Parity check just completed. 2016-08-21, 17:16:08 7 hr, 40 min, 37 sec 108.6 MB/s OK 2016-08-19, 07:06:26 8 hr, 20 min, 36 sec 99.9 MB/s OK 2016-08-13, 21:57:52 8 hr, 25 min, 47 sec 98.9 MB/s OK

I'm a long way away from making a GUI version, everything is still command line. I don't plan to even consider starting a GUI conversion until I have this script working right on 6.2. But thanks for thinking optimistically!!! -Paul

Okay, thanks. I have some logic in the script to look at it if it exists, wasn't sure if it was still needed. I'll keep it.

Thanks guys. Does the "/boot/config/extra.cfg" configuration file still work with 6.2, or is that obsolete or replaced/changed?

Reaching out for help from some users running 6.2 with Dual Parity and NOT a PRO license. It might also be interesting to see those with a PRO license, but the Array drive count configured to less than the default 24. I need the output of this command: mdcmd status | grep "rdevStatus." Here is what I see on my 6.2 server with Dual Parity and a PRO license: rdevStatus.0=DISK_OK rdevStatus.1=DISK_OK rdevStatus.2=DISK_OK rdevStatus.3=DISK_OK rdevStatus.4=DISK_OK rdevStatus.5=DISK_OK rdevStatus.6=DISK_OK rdevStatus.7=DISK_OK rdevStatus.8=DISK_OK rdevStatus.9=DISK_OK rdevStatus.10=DISK_OK rdevStatus.11=DISK_OK rdevStatus.12=DISK_OK rdevStatus.13=DISK_NP rdevStatus.14=DISK_NP rdevStatus.15=DISK_NP rdevStatus.16=DISK_NP rdevStatus.17=DISK_OK rdevStatus.18=DISK_OK rdevStatus.19=DISK_NP rdevStatus.20=DISK_NP rdevStatus.21=DISK_NP rdevStatus.22=DISK_NP rdevStatus.23=DISK_NP rdevStatus.24=DISK_NP rdevStatus.25=DISK_NP rdevStatus.26=DISK_NP rdevStatus.27=DISK_NP rdevStatus.28=DISK_NP rdevStatus.29=DISK_OK I'm trying to determine if the second parity drive always shows up as drive 29, or if this moves around based upon license or configuration. Thanks, Paul

Thanks jack0w! Just watch this thread during this week, I expect to have it available soon. -Paul

Hey Frank, Welcome to the thead, and thanks for sharing your output. You mentioned v6 RC4 twice, do you mean v6.2 RC4? You said there's no place to set md_sync_window in the GUI - yes there is, right above the setting for md_sync_thresh. You can ignore the md_write_limit values in the results, they no longer apply to 6.2. You said you left md_sync_window at stock (assuming 384) and md_sync_thresh at 50% (assuming 192). From reading your test results, I think this has a big speed penalty. In your test results, at md_sync_window=512 you're at 51.2 MB/s, increasing to 57+ at 640. Assuming this is a curve, I would expect 384 to be significantly less than 51.2, probably closer to 40 MB/s. You could verify this by running a (M)anual test using the tool, with a start and end point at 384, any interval, for a duration of 3 minutes or more. I think you're running this in your testbed, right, the one with these drives?: 1TB Hitachi DeskStar HD(Parity 2), 3 X 1TB Hitachi DeskStar HD If so, those 1TB drives are holding back ultimate speeds, but perhaps you already know that. My new script should do a much better job of evaluating settings on a 6.2 server, but it isn't quite ready for public consumption. Maybe in a week or so you can be among my first testers of the new script. Your results look similar to mine, peaking low and then significantly dropping off, before I added in the new special tests for nr_requests and md_sync_thresh. With my new tests, the drop off disappeared and the highest speeds came from a much higher number of stripes. v2.2 of the tool, that you're running, isn't capable of finding the best values in unRAID v6.2. -Paul

The test took 15 hours to complete, but I'm pretty happy with the result. On unRAID v5, my server was able to hit 139 MB/s in this test. Upgrading to v6.2 plus adding a second parity dropped the speeds to about 110 MB/s. I figured dual parity would never run as fast as single parity, and while that may be true, the results below show that I've gotten really close at 131.5 MB/s. Before tweaking the md_sync_thresh and nr_requests values, I found the data to be somewhat chaotic, so I added a lot of additional test points. The test data below is the result of 164 parity check start/stops. The 8 tests for md_sync_thresh/nr_requests ran for 10 minutes each, while the other 156 md_sync_window tests ran for 5 minutes each. That's 14 hours and 20 minutes of pure test data. The actual test took an extra 40 minutes, primarily because I don't start each test at the beginning of each parity check, but rather let it run a little bit to let it get up to speed before I start measuring, on average about 14 seconds for each one. Tunables Report from unRAID Tunables Tester v4.0 by Pauven (for unRAID v6.2) Automatic Parity Sync Test run on Sat Aug 20 16:01:24 EDT 2016 --- FULLY AUTOMATIC TEST PASS 1 (Rough - 45 Sample Points @ 5min Duration)--- NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Current Values: md_num_stripes=6256, md_sync_window=2816, md_sync_thresh=192 Test | num_stripes | sync_window | nr_requests | sync_thresh | Speed 1 | 1536 | 768 | 128 | 767 | 91.7 MB/s 2 | 1536 | 768 | 128 | 384 | 122.2 MB/s 3 | 1536 | 768 | 8 | 767 | 126.4 MB/s 4 | 1536 | 768 | 8 | 384 | 121.8 MB/s Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 126.4 MB/s These values will be used for the remainder of the test Beginning md_sync_window testing: Test | num_stripes | sync_window | Speed 1a | 6400 | 384 | 117.7 MB/s 1b | 6400 | 384 | 115.2 MB/s 2a | 6400 | 448 | 120.9 MB/s 2b | 6400 | 448 | 118.0 MB/s 3a | 6400 | 512 | 121.2 MB/s 3b | 6400 | 512 | 126.9 MB/s 4a | 6400 | 576 | 119.9 MB/s 4b | 6400 | 576 | 123.4 MB/s 5a | 6400 | 640 | 120.6 MB/s 5b | 6400 | 640 | 124.3 MB/s 6a | 6400 | 704 | 124.1 MB/s 6b | 6400 | 704 | 123.0 MB/s 7a | 6400 | 768 | 124.6 MB/s 7b | 6400 | 768 | 122.9 MB/s 8a | 6400 | 832 | 129.7 MB/s 8b | 6400 | 832 | 124.6 MB/s 9a | 6400 | 896 | 129.6 MB/s 9b | 6400 | 896 | 124.9 MB/s 10a | 6400 | 960 | 127.6 MB/s 10b | 6400 | 960 | 125.9 MB/s 11a | 6400 | 1024 | 128.7 MB/s 11b | 6400 | 1024 | 126.8 MB/s 12a | 6400 | 1088 | 129.2 MB/s 12b | 6400 | 1088 | 126.9 MB/s 13a | 6400 | 1152 | 129.4 MB/s 13b | 6400 | 1152 | 126.9 MB/s 14a | 6400 | 1216 | 129.9 MB/s 14b | 6400 | 1216 | 128.0 MB/s 15a | 6400 | 1280 | 130.2 MB/s 15b | 6400 | 1280 | 128.3 MB/s 16a | 6400 | 1344 | 130.2 MB/s 16b | 6400 | 1344 | 129.0 MB/s 17a | 6400 | 1408 | 130.5 MB/s 17b | 6400 | 1408 | 128.9 MB/s 18a | 6400 | 1472 | 130.6 MB/s 18b | 6400 | 1472 | 129.1 MB/s 19a | 6400 | 1536 | 130.7 MB/s 19b | 6400 | 1536 | 129.5 MB/s 20a | 6400 | 1600 | 130.5 MB/s 20b | 6400 | 1600 | 129.8 MB/s 21a | 6400 | 1664 | 130.8 MB/s 21b | 6400 | 1664 | 130.2 MB/s 22a | 6400 | 1728 | 130.8 MB/s 22b | 6400 | 1728 | 130.4 MB/s 23a | 6400 | 1792 | 130.8 MB/s 23b | 6400 | 1792 | 130.4 MB/s 24a | 6400 | 1856 | 130.8 MB/s 24b | 6400 | 1856 | 130.6 MB/s 25a | 6400 | 1920 | 130.7 MB/s 25b | 6400 | 1920 | 130.6 MB/s 26a | 6400 | 1984 | 131.0 MB/s 26b | 6400 | 1984 | 130.6 MB/s 27a | 6400 | 2048 | 131.1 MB/s 27b | 6400 | 2048 | 130.4 MB/s 28a | 6400 | 2112 | 130.9 MB/s 28b | 6400 | 2112 | 130.6 MB/s 29a | 6400 | 2176 | 131.0 MB/s 29b | 6400 | 2176 | 130.6 MB/s 30a | 6400 | 2240 | 130.8 MB/s 30b | 6400 | 2240 | 130.8 MB/s 31a | 6400 | 2304 | 131.0 MB/s 31b | 6400 | 2304 | 130.8 MB/s 32a | 6400 | 2368 | 131.3 MB/s 32b | 6400 | 2368 | 130.8 MB/s 33a | 6400 | 2432 | 130.7 MB/s 33b | 6400 | 2432 | 130.7 MB/s 34a | 6400 | 2496 | 131.0 MB/s 34b | 6400 | 2496 | 130.8 MB/s 35a | 6400 | 2560 | 131.1 MB/s 35b | 6400 | 2560 | 130.8 MB/s 36a | 6400 | 2624 | 130.8 MB/s 36b | 6400 | 2624 | 130.8 MB/s 37a | 6400 | 2688 | 130.9 MB/s 37b | 6400 | 2688 | 130.6 MB/s 38a | 6400 | 2752 | 131.1 MB/s 38b | 6400 | 2752 | 130.9 MB/s 39a | 6400 | 2816 | 130.7 MB/s 39b | 6400 | 2816 | 131.0 MB/s 40a | 6400 | 2880 | 130.7 MB/s 40b | 6400 | 2880 | 131.0 MB/s 41a | 6400 | 2944 | 131.1 MB/s 41b | 6400 | 2944 | 131.0 MB/s 42a | 6400 | 3008 | 131.1 MB/s 42b | 6400 | 3008 | 131.2 MB/s 43a | 6400 | 3072 | 131.1 MB/s 43b | 6400 | 3072 | 130.9 MB/s 44a | 6400 | 3136 | 130.9 MB/s 44b | 6400 | 3136 | 130.8 MB/s 45a | 6400 | 3200 | 131.0 MB/s 45b | 6400 | 3200 | 131.0 MB/s The Fastest Sync Speed tested was md_sync_window=2368 at 131.3 MB/s This will consume 200 MB just for the Syncs (additional for Writes and Reads) The Thriftiest Sync Speed tested was md_sync_window=832 at 129.7 MB/s This will consume 70 MB just for the Syncs (additional for Writes and Reads) --- Targeting Fastest Result of md_sync_window 2368 bytes for Final Pass --- --- FULLY AUTOMATIC TEST PASS 2 (Fine - 33 Sample Points @ 5min Duration)--- NOTE: Use the smallest set of values that produce good results. Larger values increase server memory use, and may cause stability issues with unRAID, especially if you have any add-ons or plug-ins installed. Current Values: md_num_stripes=6256, md_sync_window=2816, md_sync_thresh=192 Test | num_stripes | sync_window | nr_requests | sync_thresh | Speed 1 | 1536 | 768 | 128 | 767 | 91.5 MB/s 2 | 1536 | 768 | 128 | 384 | 123.3 MB/s 3 | 1536 | 768 | 8 | 767 | 124.8 MB/s 4 | 1536 | 768 | 8 | 384 | 123.0 MB/s Fastest vals were nr_reqs=8 and sync_thresh=99% of sync_window at 124.8 MB/s These values will be used for the remainder of the test Beginning md_sync_window testing: Test | num_stripes | sync_window | Speed 1a | 4992 | 2240 | 130.9 MB/s 1b | 4992 | 2240 | 130.8 MB/s 2a | 4992 | 2248 | 130.9 MB/s 2b | 4992 | 2248 | 130.7 MB/s 3a | 4992 | 2256 | 131.2 MB/s 3b | 4992 | 2256 | 131.2 MB/s 4a | 4992 | 2264 | 131.1 MB/s 4b | 4992 | 2264 | 130.9 MB/s 5a | 4992 | 2272 | 131.0 MB/s 5b | 4992 | 2272 | 130.7 MB/s 6a | 4992 | 2280 | 131.0 MB/s 6b | 4992 | 2280 | 130.9 MB/s 7a | 4992 | 2288 | 130.8 MB/s 7b | 4992 | 2288 | 131.0 MB/s 8a | 4992 | 2296 | 130.8 MB/s 8b | 4992 | 2296 | 130.9 MB/s 9a | 4992 | 2304 | 130.8 MB/s 9b | 4992 | 2304 | 130.7 MB/s 10a | 4992 | 2312 | 130.9 MB/s 10b | 4992 | 2312 | 131.1 MB/s 11a | 4992 | 2320 | 131.1 MB/s 11b | 4992 | 2320 | 131.0 MB/s 12a | 4992 | 2328 | 131.0 MB/s 12b | 4992 | 2328 | 131.1 MB/s 13a | 4992 | 2336 | 131.0 MB/s 13b | 4992 | 2336 | 130.6 MB/s 14a | 4992 | 2344 | 131.0 MB/s 14b | 4992 | 2344 | 130.9 MB/s 15a | 4992 | 2352 | 131.0 MB/s 15b | 4992 | 2352 | 130.9 MB/s 16a | 4992 | 2360 | 131.0 MB/s 16b | 4992 | 2360 | 130.8 MB/s 17a | 4992 | 2368 | 131.1 MB/s 17b | 4992 | 2368 | 130.9 MB/s 18a | 4992 | 2376 | 131.0 MB/s 18b | 4992 | 2376 | 130.8 MB/s 19a | 4992 | 2384 | 131.0 MB/s 19b | 4992 | 2384 | 130.8 MB/s 20a | 4992 | 2392 | 130.9 MB/s 20b | 4992 | 2392 | 130.8 MB/s 21a | 4992 | 2400 | 131.0 MB/s 21b | 4992 | 2400 | 130.9 MB/s 22a | 4992 | 2408 | 130.9 MB/s 22b | 4992 | 2408 | 130.6 MB/s 23a | 4992 | 2416 | 130.9 MB/s 23b | 4992 | 2416 | 130.8 MB/s 24a | 4992 | 2424 | 131.5 MB/s 24b | 4992 | 2424 | 130.7 MB/s 25a | 4992 | 2432 | 131.3 MB/s 25b | 4992 | 2432 | 131.0 MB/s 26a | 4992 | 2440 | 131.0 MB/s 26b | 4992 | 2440 | 130.7 MB/s 27a | 4992 | 2448 | 130.9 MB/s 27b | 4992 | 2448 | 130.9 MB/s 28a | 4992 | 2456 | 131.1 MB/s 28b | 4992 | 2456 | 130.9 MB/s 29a | 4992 | 2464 | 131.0 MB/s 29b | 4992 | 2464 | 131.1 MB/s 30a | 4992 | 2472 | 131.0 MB/s 30b | 4992 | 2472 | 130.8 MB/s 31a | 4992 | 2480 | 130.7 MB/s 31b | 4992 | 2480 | 131.0 MB/s 32a | 4992 | 2488 | 131.3 MB/s 32b | 4992 | 2488 | 130.9 MB/s 33a | 4992 | 2496 | 131.0 MB/s 33b | 4992 | 2496 | 131.0 MB/s The Fastest Sync Speed tested was md_sync_window=2424 at 131.5 MB/s This will consume 204 MB just for the Syncs (additional for Writes and Reads) The Thriftiest Sync Speed tested was md_sync_window=2240 at 130.9 MB/s This will consume 189 MB just for the Syncs (additional for Writes and Reads) Completed: 15 Hrs 01 Min 36 Sec. I've still got plenty of tweaking to do. Now that I've got a full set of quality data, I'll evaluate if I can reduce the # of test points to cut down on running time. My first thought is no, because if I go back to sampling every 128 stripes, instead of the 64 stripes of this test, I would have missed the peak at 2368, and instead found the second highest peak at 3008, centering Pass 2 there. While the final speed would likely have been comparable, it would consume about 25% more memory for the additional stripes. I think I prefer the additional sample points, even if the test takes significantly longer. It's not like this is a daily or even monthly job, after all. I'm also running the md_sync_thresh/nr_requests test twice, once before Pass 1, and again before Pass 2, but both times it runs with the same parameters. I'm thinking the second run should be using the fastest md_sync_window from Pass 1. Going to the charts (made in Excel, not from the utility), you can see a lot of volatility in the results at low md_sync_window values. Above 1000, the server starts to settle down and you can see some nice curves forming. The md_sync_thresh at 50% Max line gradually catches up with the Max-1 line, and finally passes it at 2816 - the exact same value that produced the fastest results on my server on unRAID v5. Coincidence? I think not. The fastest result was 131.3 MB/s at md_sync_window=2368 with md_sync_thresh=2367 (Max-1). Also noteworthy is a 131.2 MB/s at md_sync_window=3008 with md_sync_thresh=1504 (50% Max). Basically an identical speed, but takes more stripes/memory to get there. While I talked about getting rid of the Best Bang for the Buck routine altogether on unRAID v6.2, I already had my new routine coded 3 years ago, so I just left it in. It's now called Thriftiest, and it correctly identifies the lowest value that gives 95% of peak speed achieved during the test. The new routine no longer gets confused like the old routine, as I'm using an array to store all the values, so I can sort and analyze them in the right order. The Thriftiest value it found was at md_sync_window=832, and it achieved 98.8% of peak speed at significantly lower stripes/memory. There's always a possibility that this spike is just a non-repeatable fluke, so if I was going to use it, I would run a manual test targeting that range with more detail and longer run times to ensure the result is real (or, hey, just run a parity check with that value and see if it's good). The next chart is Pass 2. It uses the fastest result from Pass 1, and does a more thorough scan of the range around it. That's right, I now scan both below and above the fastest value from Pass 1. In v2.2 of this utility I only scanned below, something that got questioned. And rightfully so, as the fastest result of 131.5 MB/s came from above the fastest md_sync_window of Pass 1. The chart makes the 131.5 MB/s look a lot faster than it really is. From slowest point (130.6 MB/s) to fastest point is only 0.9 MB/s, within the margin of error, and all are good speeds. I just kicked off a Correcting Parity Check using the fastest values from above. Since upgrading to v6.2 and adding a 2nd parity drive, my checks have been taking 8.5 hours. I think the new settings will bring me down below 8 hours, but probably not to the 7.5 hours I got on v5.0. -Paul

One of the reasons I'd like to get this converted to a GUI plug-in is so we can add graphical elements like line charts, showing the results. It looks like my server settles down nicely md_sync_window values over 1000, and using Max-1 for md_sync_thresh offers a consistent increase of 50% Max. Test isn't even at the halfway point yet...

I'm not convinced. I increased the fullauto test time from 4 minutes up to 5 minutes, and started up a new run (expecting it to take about 14 hours). Checked in on it 30 minutes later to see the results of the four nr_requests & md_sync_thresh tests, and was surprised to see that Test 2 had won (nr_request=128) by a smidge over Test 3 (nr_requests=. Granted, that's only happened once in 5+ tests, but it happened all the same. Based upon this, I think it best to leave in the test. For all I know, there's some more bizaro servers out there like JBartlett's that behave the exact opposite of a typical server. Anyway, frustrated with these results, I cancelled the test. I really wanted to see a fullauto test result with nr_request=8 on my server. I again tweaked the routine, doubling the amount of time spent on the four nr_requests & md_sync_thresh tests, from 5 to 10 minutes each, thinking an extra 20 minutes is well worth the increase in accuracy to determine the right nr_requests value for the rest of the test. I also doubled the md_num_stripes and md_sync_window to twice stock unRAID values while running these four tests (I figured that put it more in the middle of the overall test range, giving more balanced results). Keep in mind, I'm just playing around here, trying different strategies to find a way to make this tool even work, so don't be concerned that I'm hard coding values just yet. I started the fullauto test again, and this time while Test 3 won out by 4MB/s, Test 2 beat Test 4. Test 1 - nr_req=128 sync_thresh=767 - 53.604 GB in 598.443 secs = 91.7 MB/s Test 2 - nr_req=128 sync_thresh=384 - 71.401 GB in 598.327 secs = 122.2 MB/s Test 3 - nr_req= 8 sync_thresh=767 - 73.899 GB in 598.487 secs = 126.4 MB/s Test 4 - nr_req= 8 sync_thresh=384 - 71.205 GB in 598.409 secs = 121.8 MB/s There's some really weird interplay between these two values. With nr_requests=128, then md_sync_thresh=Max-1 becomes the slowest method, but with nr_requests=8, Max-1 becomes the fastest method. With md_sync_thresh=50% of Max, the nr_requests value has little impact on my server, as both results were neck and neck, easily within the margin of error. Both results were very good, but still 3%-4% slower the fastest result. The fullauto test is continuing nicely, this time with nr_requests=8, and the highest result I have so far is 130.2 MB/s at md_sync_window=1280 and md_sync_thresh=1279, almost 6 MB/s faster than the value I got yesterday at md_sync_window=1280 but with unRAID stock nr_requests and md_sync_thresh values. Even better, in general most of the results at every test point are faster than the test I ran last night, and most are close to maxing out the server, so even if I can't find the absolute best value, I sure can find some really good ones... a Tests are md_sync_thresh=md_sync_window-1, b Tests =50% md_sync_window Test 1a - md_sync_window= 384 - 34.399 GB in 299.196 seconds = 117.7 MB/s Test 1b - md_sync_window= 384 - 33.674 GB in 299.224 seconds = 115.2 MB/s Test 2a - md_sync_window= 448 - 35.336 GB in 299.335 seconds = 120.9 MB/s Test 2b - md_sync_window= 448 - 34.476 GB in 299.180 seconds = 118.0 MB/s Test 3a - md_sync_window= 512 - 35.415 GB in 299.260 seconds = 121.2 MB/s Test 3b - md_sync_window= 512 - 37.088 GB in 299.155 seconds = 126.9 MB/s Test 4a - md_sync_window= 576 - 35.046 GB in 299.219 seconds = 119.9 MB/s Test 4b - md_sync_window= 576 - 36.070 GB in 299.209 seconds = 123.4 MB/s Test 5a - md_sync_window= 640 - 35.232 GB in 299.214 seconds = 120.6 MB/s Test 5b - md_sync_window= 640 - 36.328 GB in 299.193 seconds = 124.3 MB/s Test 6a - md_sync_window= 704 - 36.257 GB in 299.219 seconds = 124.1 MB/s Test 6b - md_sync_window= 704 - 35.934 GB in 299.182 seconds = 123.0 MB/s Test 7a - md_sync_window= 768 - 36.430 GB in 299.277 seconds = 124.6 MB/s Test 7b - md_sync_window= 768 - 35.909 GB in 299.183 seconds = 122.9 MB/s Test 8a - md_sync_window= 832 - 37.932 GB in 299.366 seconds = 129.7 MB/s Test 8b - md_sync_window= 832 - 36.401 GB in 299.242 seconds = 124.6 MB/s Test 9a - md_sync_window= 896 - 37.873 GB in 299.292 seconds = 129.6 MB/s Test 9b - md_sync_window= 896 - 36.511 GB in 299.247 seconds = 124.9 MB/s Test 10a - md_sync_window= 960 - 37.297 GB in 299.335 seconds = 127.6 MB/s Test 10b - md_sync_window= 960 - 36.797 GB in 299.216 seconds = 125.9 MB/s Test 11a - md_sync_window=1024 - 37.634 GB in 299.349 seconds = 128.7 MB/s Test 11b - md_sync_window=1024 - 37.048 GB in 299.287 seconds = 126.8 MB/s Test 12a - md_sync_window=1088 - 37.747 GB in 299.280 seconds = 129.2 MB/s Test 12b - md_sync_window=1088 - 37.078 GB in 299.239 seconds = 126.9 MB/s Test 13a - md_sync_window=1152 - 37.818 GB in 299.275 seconds = 129.4 MB/s Test 13b - md_sync_window=1152 - 37.084 GB in 299.269 seconds = 126.9 MB/s Test 14a - md_sync_window=1216 - 37.968 GB in 299.337 seconds = 129.9 MB/s Test 14b - md_sync_window=1216 - 37.400 GB in 299.277 seconds = 128.0 MB/s Test 15a - md_sync_window=1280 - 38.054 GB in 299.330 seconds = 130.2 MB/s Test 15b - md_sync_window=1280 - Test Range Entered - Time Remaining: 242s

True. It's probably me that minds the most. This is a very frustrating project to work on, because each test takes so much time. I've gone from just over 2 hours for a FullAuto test on v2.2 of this utility, to now 8-16 hours for the new version, with new parameters being tested for v6.2, more test points, and longer individual tests to improve accuracy. I make a 5-second code change, and it takes 30+ minutes to test, if I'm lucky. Makes progress very slow. I just ran a test again on my server, this time using an md_sync_window of 760 (which my full test picked out as the fastest when I ran it last night). Test 3 edged back into the lead again. md_sync_window=760: Test 1 - nr_req=128 sync_thresh=759 - 20.356 GB in 241.028 secs = 86.5 MB/s Test 2 - nr_req=128 sync_thresh=380 - 27.237 GB in 240.932 secs = 115.8 MB/s Test 3 - nr_req= 8 sync_thresh=759 - 28.881 GB in 241.016 secs = 122.7 MB/s Test 4 - nr_req= 8 sync_thresh=380 - 28.777 GB in 240.943 secs = 122.3 MB/s Beginning md_sync_window testing: Test 1 - md_sync_window= 760 - 29.109 GB in 241.045 seconds = 123.7 MB/s There's also the run-to-run variance. Test 3 above used the exact same settings as that md_sync_window Test 1 in the same results, run 4 minutes apart from each other. Over a 4 minute test cycle there was a 1MB/s variance. Yes, I suppose that's less than 1% variance, but it is also bigger than the variance between Test 3 vs 4. Not really enough accuracy to pick a winning combination. Kinda consistent, in a really inconsistent way. I just reran this test at md_sync_window=2816 (the first one I posted several posts back)... md_sync_window=2816 (first test): Test 1 - nr_req=128 sync_thresh=2815 - 18.109 GB in 239.968 secs = 77.3 MB/s Test 2 - nr_req=128 sync_thresh=1408 - 18.571 GB in 239.959 secs = 79.3 MB/s Test 3 - nr_req= 8 sync_thresh=2815 - 29.960 GB in 240.030 secs = 127.8 MB/s Test 4 - nr_req= 8 sync_thresh=1408 - 28.717 GB in 240.006 secs = 122.5 MB/s And this time I got dramatically different results: md_sync_window=2816 (second test): Test 1 - nr_req=128 sync_thresh=2815 - 28.737 GB in 241.066 secs = 122.1 MB/s Test 2 - nr_req=128 sync_thresh=1408 - 29.903 GB in 241.083 secs = 127.0 MB/s Test 3 - nr_req= 8 sync_thresh=2815 - 30.556 GB in 241.038 secs = 129.8 MB/s Test 4 - nr_req= 8 sync_thresh=1408 - 30.484 GB in 241.025 secs = 129.5 MB/s This time Test 2 was high enough to beat Test 4 from the previous run. What changed between runs? Nothing, except the results. Technically, Test 3 still won, and nr_requests still worked better at 8 than 128, but on this last run it's almost too close to call. This is getting back to the point where I got some flak years ago because the Fastest and Best Bang results didn't always come out right. Trying to automate this testing process is hard enough on my own server, but to come up with a methodology that reliably works on other servers.... sheez. Can't believe I'm actually doing this again. On the flip side, the new tests are helping. Previously, I was getting < 110MB/s with md_sync_thresh and nr_requests both at stock values and md_sync_window at 2816 - and this was turning in a decent parity check time. Now by tweaking those new settings, I'm squeaking out an additional 20 MB/s... nice. I'm also finding that the same high md_sync_window #'s that worked so well on 5.0 are again working well on 6.2, but only in combination with tweaks to md_sync_thresh and nr_requests. Oh, and BRiT, I agree with you, I think I can just use this preliminary test to get the nr_requests value, and just test md_sync_thresh twice with each md_sync_window. -Paul

Hey BRiT, I'm not sure what I mean... yet. I was thinking that, in order to consistently determine if md_sync_thresh should be at -1 or 50%, and nr_requests at 8 or 128, that I should set the server to default unRAID values for the purpose of testing those two parameters. This would be temporary. But I just ran a test this way, and I got very different results than my previous test. Now, keep in mind that md_sync_window at 384 is not optimized for my server, so the numbers should be lower than my post above. That said, the results of this test didn't match my previous test, with md_sync_window at 2816. Test 1 - nr_req=128 sync_thresh=383 - 12.255 GB in 240.802 secs = 52.1 MB/s Test 2 - nr_req=128 sync_thresh=192 - 26.474 GB in 241.087 secs = 112.4 MB/s Test 3 - nr_req= 8 sync_thresh=383 - 27.300 GB in 241.066 secs = 116.0 MB/s Test 4 - nr_req= 8 sync_thresh=192 - 27.665 GB in 241.016 secs = 117.5 MB/s Previously, at md_sync_window=2816, Test 3 was my fastest. Now, at 384, Test 4 is my fastest. I'm now thinking I need to test each md_sync_window value in combination with both md_sync_thresh values - doubling the # of tests!!! -Paul

Hey Squid, thanks for the info. That makes sense. It also makes it seem unwise for me to attempt testing nr_requests, especially on multi-drive setups. However, I had already coded and tested it before you posted your reply, so take a gander at these results: Test 1 - nr_req=128 sync_thresh=2815 - 18.109 GB in 239.968 secs = 77.3 MB/s Test 2 - nr_req=128 sync_thresh=1408 - 18.571 GB in 239.959 secs = 79.3 MB/s Test 3 - nr_req= 8 sync_thresh=2815 - 29.960 GB in 240.030 secs = 127.8 MB/s Test 4 - nr_req= 8 sync_thresh=1408 - 28.717 GB in 240.006 secs = 122.5 MB/s Here's what is interesting about those results: With all WD Red 3TB drives, on my HighPoint 2760A controller, I was anticipating that nr_request at a stock value of 128, and md_sync_thresh at a 50% value (1408 in this case) was going to be the fastest. Not only was the opposite true, 127.8 is easily the fastest time I have seen in 2 days of testing. This is doubly (is that a word?) interesting since the md_sync_window was set to 2816 during this test, which is my old optimized setting under 5.0, not the new 760 value that gives me 127.2 on 6.2 (with nr_requests=128 and md_sync_thresh=192 stock values). Now the flaw in my earlier tests was that I was leaving md_sync_thresh at the stock value, 192, regardless of what I set md_sync_window to for each test. I'll have to modify my code and test again, setting md_sync_window to a stock 384 before evaluating the md_sync_thresh at 383 and 192, to figure out if 50% or 99% is the right solution for md_sync_thresh, then use that ratio for all of the md_sync_window tests. -Paul

Okay, I have my first challenge, and can use a little help. I can set the md_sync_thresh value using mdcmd, but this doesn't work for nr_requests. In a thread I found that you can set nr_requests individually per drive using the following: echo 8 > /sys/block/sdX/queue/nr_requests Question 1, is there a way to set nr_requests globally for the array, like Tunable(nr_requests) setting on the "Disk Settings" panel, but from the command line? Question 2, if I have to set each drive individually using the above format, what is an efficient routine? I basically need a list of sdX array drives to cycle through, but at the same time, skipping any cache or other drives not in the array. Strike that question. I already solved the problem 3 years ago, I have a routine that finds all the disks in the array, and I can easily cycle through them. Still would prefer a global method, though. Thanks, Paul