JorgeB

Thanks for the script, very useful. One suggestion, since vdisks by default are sparse use --sparse on rsync (or cp instead) to make the backups, say you have a 120GB vdisk but only 20GB are allocated, the way it is now the backup will use 120GB, with --sparse or cp it would only use 20GB.

Looks good, I like it.

It doesn't, though you can achieve better performance with a managed switch using VLANs, see here: http://louwrentius.com/achieving-450-mbs-network-file-transfers-using-linux-bonding.html The main issue with this mode, like you mentioned, is that it only works with other linux computers.

Agree, this would be nice.

Only mode 4 requires a smart or managed switch, all others work with any switch.

unRAID FAQ - Can I change my cache pool to RAID0 or other modes?

I've ran 5+ preclears at once without issues.

Nice! One suggestion, if disable, enable turbo write before starting, clearing can be up to 3 times faster. As for the progress info, there's an easy way for v6.2, if the script can check current unRAID version, you can add status=progress to the dd command.

Thanks bonienl, looks cool, and certainly more useful than the reads/writes numbers that don't mean much. I also see some heavy fluctuating speeds, from ~170MB/s to over 400MB/s on my test server, maybe a higher sample time, if possible, will provide more accurate readings. Any way would could add another toggle to show the total bytes read/written (reset when pressing clear statistics)? This could also be useful. P.S.: it doesn't work on my cache disk, is this expected?

I see your point, I think it's better for the script to take a few hours more but make sure it finds the optimal values.

16 hours is not that long for a test that most people only need to run once, but I wonder if running the nr_requests test before or after test 2 wouldn't give similar results in less time, i.e., after test 1 test nr_requests 8/16/128 with the best result from test 1, use the better result from then on, or as an alternative, after test 2, this way the last test would be done with a single nr_requests.

How about using the first test to find only sync_window and sync_thresh? Looks to me like with nr_requests set at default there's a better chance of finding the optimal sync_thresh, also looks like the best sync_thresh is the same (or in the case of your last test, practically the same) with the various nr_request values, so after finding the optimal window and thresh values you could do a test on those changing only nr_requests, I believe this would be faster and provide better results than trying to find optimal values for all 3 settings at the same time.

Agree, these settings may be optimal for some servers, and while I didn't have much time this week and intend to do more testing with different controllers later, all results point to a optimal setting that is usually a little below sync_window, just not sure if there is a set value, like -60 that is optimal for everything, ideally the script would run a short test in the beginning to try and find it. I retested a single LSI9211 with larger (and faster) SSDs in the hopes of seeing better defined results, and while they are, the optimal thresh value changed from the previous tests (previous tests where done with 2 controllers at the same time, so maybe why the difference, but I don't have 16 of largest SSDs to test with both again, and using only one controller with the smallest SSDs won't help either because results will be limited by their max speed in almost all tests) Sync_window=2048 stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 128 | 2047 | 289.7MB/s 4096 | 2048 | 128 | 2040 | 321.7MB/s 4096 | 2048 | 128 | 2036 | 335.2MB/s 4096 | 2048 | 128 | 2032 | 337.0MB/s 4096 | 2048 | 128 | 2028 | 340.5MB/s 4096 | 2048 | 128 | 2024 | 333.5MB/s 4096 | 2048 | 128 | 2016 | 330.0MB/s 4096 | 2048 | 128 | 1984 | 330.0MB/s 4096 | 2048 | 128 | 1960 | 330.0MB/s 4096 | 2048 | 128 | 1952 | 330.0MB/s 4096 | 2048 | 128 | 1920 | 330.0MB/s 4096 | 2048 | 128 | 1856 | 325.0MB/s 4096 | 2048 | 128 | 1792 | 326.6MB/s 4096 | 2048 | 128 | 1536 | 323.3MB/s 4096 | 2048 | 128 | 1280 | 320.1MB/s 4096 | 2048 | 128 | 1024 | 314.4MB/s Same sync_window but nr_requests=8 for the 4 fastest results (like before, looks like it doesn't make a big difference with LSI controllers) stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 8 | 2036 | 337.0MB/s 4096 | 2048 | 8 | 2032 | 340.5MB/s 4096 | 2048 | 8 | 2028 | 340.5MB/s 4096 | 2048 | 8 | 2024 | 335.2MB/s Sync_window=1024 and nr_request back to default: stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 2048 | 1024 | 128 | 1023 | 293.7MB/s 2048 | 1024 | 128 | 1016 | 328.3MB/s 2048 | 1024 | 128 | 1012 | 331.7MB/s 2048 | 1024 | 128 | 1008 | 333.5MB/s 2048 | 1024 | 128 | 1004 | 337.0MB/s 2048 | 1024 | 128 | 1000 | 325.0MB/s 2048 | 1024 | 128 | 996 | 316.9MB/s Sync_window=3072 stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 6144 | 3072 | 128 | 3071 | 295.0MB/s 6144 | 3072 | 128 | 3064 | 321.7MB/s 6144 | 3072 | 128 | 3056 | 335.2MB/s 6144 | 3072 | 128 | 3052 | 337.0MB/s 6144 | 3072 | 128 | 3048 | 333.5MB/s 6144 | 3072 | 128 | 3040 | 333.5MB/s 6144 | 3072 | 128 | 3032 | 331.7MB/s 6144 | 3072 | 128 | 3024 | 331.7MB/s 6144 | 3072 | 128 | 3016 | 326.6MB/s Best results were always with a thresh=sync_window-20, previous tests with 2 controllers best setting for thresh was sync_window-60.

I don't know what the upper limit is, but tried up to 131072 and it works, didn't went any higher, doubt a higher number will help unRAID though, but only testing can confirm.

Interesting results, look forward to the normal test results. PS: are you sure nr_requests=1 works? You can check the current value after setting it to 1, for me it never goes lower than 4. cat /sys/block/sdX/queue/nr_requests

Yep, mistype.

Requested values tests: stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 128 | 2047 | 72.4MB/s 4096 | 2048 | 128 | 2040 | 76.8MB/s 4096 | 2048 | 128 | 2032 | 78.3MB/s 4096 | 2048 | 128 | 2024 | 78.9MB/s 4096 | 2048 | 128 | 2016 | 80.0MB/s 4096 | 2048 | 128 | 1984 | 80.0MB/s 4096 | 2048 | 128 | 1960 | 79.8MB/s 4096 | 2048 | 128 | 1952 | 80.0MB/s 4096 | 2048 | 128 | 1920 | 79.8MB/s 4096 | 2048 | 128 | 1856 | 79.8MB/s 4096 | 2048 | 128 | 1792 | 79.8MB/s 4096 | 2048 | 128 | 1728 | 79.8MB/s 4096 | 2048 | 128 | 1664 | 78.5MB/s 4096 | 2048 | 128 | 1536 | 77.7MB/s 4096 | 2048 | 128 | 1280 | 77.5MB/s 4096 | 2048 | 128 | 1024 | 77.1MB/s 78.8 to 80.0MB/s is a single second difference in total time. stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 128 | 1024 | 77.1MB/s 4096 | 2048 | 64 | 1024 | 77.5MB/s 4096 | 2048 | 32 | 1024 | 77.3MB/s 4096 | 2048 | 16 | 1024 | 79.6MB/s 4096 | 2048 | 8 | 1024 | 79.8MB/s 4096 | 2048 | 4 | 1024 | 80.0MB/s 4096 | 2048 | 1 | 1024 | ? MB/s Although it can be set to 1 in unRAID, it will remain at 4, I believe that is the minimum possible setting.

Script is not patched, don't forget you need to patch the one located in "/boot/config/plugins/preclear.disk".

All my previous tests were done using 2 LSI 9211 (flashed H310), I now did some tests using a SASLP, since it's bandwidth challenged and a parity check will take more time the differences should be more noticeable, also it responds differently to the tunable changes. Only thresh was changed to find the optimal values stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 128 | 2047 | 72.4MB/s 4096 | 2048 | 128 | 2016 | 80.0MB/s 4096 | 2048 | 128 | 1984 | 80.0MB/s 4096 | 2048 | 128 | 1952 | 80.0MB/s 4096 | 2048 | 128 | 1920 | 79.8MB/s 4096 | 2048 | 128 | 1856 | 79.8MB/s 4096 | 2048 | 128 | 1792 | 79.8MB/s 4096 | 2048 | 128 | 1728 | 79.8MB/s 4096 | 2048 | 128 | 1664 | 78.5MB/s 4096 | 2048 | 128 | 1536 | 77.7MB/s 4096 | 2048 | 128 | 1280 | 77.5MB/s 4096 | 2048 | 128 | 1024 | 77.1MB/s With a sync_window of 2048 there's a big range where it works very well, from ~1728 to ~2016, with an apparent sweet spot from ~1950 to ~2000 ,and like the LSI neither sync_window-1 nor /2 provide the best results. Note also that with nr_requests=8 this controller performs always at optimal speed, making the tresh setting practically irrelevant. Of course if this controller is used with one that responds differently the trick is to find the best values with both together. Using nr_requests=8 with the 2 slowest thresh values: stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 8 | 2047 | 79.8MB/s 4096 | 2048 | 8 | 1024 | 79.8MB/s Next I'm going to test the SAS2LP, same chipset as your controller, but since I don't have a spare I'll have to use one from a server, so I'll do it as soon as I can, IIRC the results were similar to the SASLP but with much bigger differences.

Exactly, although consistent this test server is good for a rough idea, a longer parity check would be better for fine tuning.

stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2240 | 8 | 1960 | 206.6MB/s stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 4096 | 2048 | 8 | 1984 | 207.9MB/s 4096 | 2048 | 8 | 1968 | 207.9MB/s 4096 | 2048 | 8 | 1952 | 209.3MB/s 4096 | 2048 | 8 | 1920 | 207.9MB/s Note that a check with the same settings sometimes is a second shorter or longer, because it's a very small array it makes a difference of a few MB/s, so when the results are very close they could be practically considered the same, e.g.: Duration: 2 minutes, 32 seconds. Average speed: 210.6 MB/s Duration: 2 minutes, 33 seconds. Average speed: 209.3 MB/s Duration: 2 minutes, 34 seconds. Average speed: 207.9 MB/s So with a sync_window=2048 a sync_thresh from ~1900 to ~1990 gives very similar results.

Here you go: Test | stripes | window | nr_reqs | thresh | Speed ----------------------------------------------------------- 1 | 6400 | 1960 | 8 | 1960 | 177.9MB/s 2 | 6400 | 1968 | 8 | 1960 | 205.2MB/s 3 | 6400 | 2240 | 8 | 1960 | 206.6MB/s 4 | 6400 | 2614 | 8 | 1960 | 203.9MB/s 5 | 6400 | 3920 | 8 | 1960 | 203.9MB/s 6 | 6400 | 5880 | 8 | 1960 | 202.6MB/s

num_stripes=4096: 210.6 num_stripes=2096: 209.3 You're right, difference represents only one second for the total time, so we can consider the same speed.

After some more testing looks like the important setting is md_sync_thresh, if set to the optimal value md_num_stripes can be left at 2 x md_sync_window. Is it possible for the script to test a few values for md_sync_thresh between md_sync_window-1 and md_sync_window/2?

Script was not finding the best values for some of my servers, so I've doing some testing on my test server since I don't remember for sure how I arrived at the settings I've been using, some interesting findings, not sure if this helps or makes it more difficult, but after all there was a good reason why I chose my go to values: So, because the script only tests 2 sync_thresh settings and md_num_stripes is always 2 x md_sync_windows, it can't find the best settings. PS: test server has only LSI controllers, I think that's the reason why nr_request doesn't make much difference. Can't run the normal test on this server since the parity check finishes in 2.5minutes