unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables


Recommended Posts

Are you referring to speeding up every time you past a disc threshold? That's to be expected.

 

No but I think the whole thing was a red herring anyway. This is new kit and the first 1.5% of checking was worryingly different to the number we expected. Once I passed that 1.5% threshold the numbers became what I would typically expect but since I had already created the thread it seemed sensible to complete the data entry.

Link to comment
  • 3 weeks later...

I just wanted to say thanks for all the work that has been done here.  I ran this util a few months back and my supermicro X7SPA build went from an average read speed of 15megs a second I have experienced for the last few years up to 50!  I just came looking for this thread to check for any updates and found it on page 3.  It really should be stickied!

Link to comment

I just wanted to say thanks for all the work that has been done here.  I ran this util a few months back and my supermicro X7SPA build went from an average read speed of 15megs a second I have experienced for the last few years up to 50!  I just came looking for this thread to check for any updates and found it on page 3.  It really should be stickied!

 

Agreed. When I setup unRAID, I used a 'rule of thumb' I found in the Wiki...I didn't try the tunables script until this  month, but I'm seeing a similar 3x speedup on throughput.

 

VOTE on STICKY = YES. and add it to the unofficial documentation!

Link to comment
  • 3 weeks later...

My results;

Tunables Report from  unRAID Tunables Tester v2.2 by Pauven

NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.

Test | num_stripes | write_limit | sync_window |   Speed 
--- FULLY AUTOMATIC TEST PASS 1 (Rough - 20 Sample Points @ 3min Duration)---
   1  |    1408     |     768     |     512     |  76.3 MB/s 
   2  |    1536     |     768     |     640     |  76.4 MB/s 
   3  |    1664     |     768     |     768     |  76.4 MB/s 
   4  |    1920     |     896     |     896     |  76.3 MB/s 
   5  |    2176     |    1024     |    1024     |  76.3 MB/s 
   6  |    2560     |    1152     |    1152     |  76.4 MB/s 
   7  |    2816     |    1280     |    1280     |  76.3 MB/s 
   8  |    3072     |    1408     |    1408     |  76.3 MB/s 
   9  |    3328     |    1536     |    1536     |  76.4 MB/s 
  10  |    3584     |    1664     |    1664     |  76.3 MB/s 
  11  |    3968     |    1792     |    1792     |  76.3 MB/s 
  12  |    4224     |    1920     |    1920     |  76.4 MB/s 
  13  |    4480     |    2048     |    2048     |  76.4 MB/s 
  14  |    4736     |    2176     |    2176     |  76.3 MB/s 
  15  |    5120     |    2304     |    2304     |  76.3 MB/s 
  16  |    5376     |    2432     |    2432     |  76.3 MB/s 
  17  |    5632     |    2560     |    2560     |  76.4 MB/s 
  18  |    5888     |    2688     |    2688     |  76.3 MB/s 
  19  |    6144     |    2816     |    2816     |  76.4 MB/s 
  20  |    6528     |    2944     |    2944     |  76.3 MB/s 
--- Targeting Fastest Result of md_sync_window 640 bytes for Final Pass ---
--- FULLY AUTOMATIC TEST PASS 2 (Final - 16 Sample Points @ 4min Duration)---
  21  |    1424     |     768     |     520     |  76.4 MB/s 
  22  |    1440     |     768     |     528     |  76.3 MB/s 
  23  |    1448     |     768     |     536     |  76.3 MB/s 
  24  |    1456     |     768     |     544     |  76.3 MB/s 
  25  |    1464     |     768     |     552     |  76.4 MB/s 
  26  |    1472     |     768     |     560     |  76.3 MB/s 
  27  |    1480     |     768     |     568     |  76.3 MB/s 
  28  |    1488     |     768     |     576     |  76.3 MB/s 
  29  |    1496     |     768     |     584     |  76.4 MB/s 
  30  |    1504     |     768     |     592     |  76.3 MB/s 
  31  |    1520     |     768     |     600     |  76.3 MB/s 
  32  |    1528     |     768     |     608     |  76.4 MB/s 
  33  |    1536     |     768     |     616     |  76.3 MB/s 
  34  |    1544     |     768     |     624     |  76.3 MB/s 
  35  |    1552     |     768     |     632     |  76.4 MB/s 
  36  |    1560     |     768     |     640     |  76.4 MB/s 

Completed: 2 Hrs 8 Min 24 Sec.

Best Bang for the Buck: Test 1 with a speed of 76.3 MB/s

     Tunable (md_num_stripes): 1408
     Tunable (md_write_limit): 768
     Tunable (md_sync_window): 512

These settings will consume 38MB of RAM on your hardware.


Unthrottled values for your server came from Test 21 with a speed of 76.4 MB/s

     Tunable (md_num_stripes): 1424
     Tunable (md_write_limit): 768
     Tunable (md_sync_window): 520

These settings will consume 38MB of RAM on your hardware.
This is 3MB more than your current utilization of 35MB.
NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

Link to comment

Tunables Report from  unRAID Tunables Tester v2.2 by Pauven

NOTE: Use the smallest set of values that produce good results. Larger values
      increase server memory use, and may cause stability issues with unRAID,
      especially if you have any add-ons or plug-ins installed.

Test | num_stripes | write_limit | sync_window |   Speed 
--- FULLY AUTOMATIC TEST PASS 1 (Rough - 20 Sample Points @ 3min Duration)---
   1  |    1408     |     768     |     512     |  86.1 MB/s 
   2  |    1536     |     768     |     640     |  84.5 MB/s 
   3  |    1664     |     768     |     768     |  82.9 MB/s 
   4  |    1920     |     896     |     896     |  85.4 MB/s 
   5  |    2176     |    1024     |    1024     |  84.4 MB/s 
   6  |    2560     |    1152     |    1152     |  90.0 MB/s 
   7  |    2816     |    1280     |    1280     |  92.7 MB/s 
   8  |    3072     |    1408     |    1408     |  91.1 MB/s 
   9  |    3328     |    1536     |    1536     |  93.2 MB/s 
  10  |    3584     |    1664     |    1664     |  94.0 MB/s 
  11  |    3968     |    1792     |    1792     |  84.8 MB/s 
  12  |    4224     |    1920     |    1920     |  79.6 MB/s 
  13  |    4480     |    2048     |    2048     |  85.5 MB/s 
  14  |    4736     |    2176     |    2176     |  82.9 MB/s 
  15  |    5120     |    2304     |    2304     |  82.1 MB/s 
  16  |    5376     |    2432     |    2432     |  87.3 MB/s 
  17  |    5632     |    2560     |    2560     |  86.5 MB/s 
  18  |    5888     |    2688     |    2688     |  86.9 MB/s 
  19  |    6144     |    2816     |    2816     |  88.2 MB/s 
  20  |    6528     |    2944     |    2944     |  87.2 MB/s 
--- Targeting Fastest Result of md_sync_window 1664 bytes for Final Pass ---
--- FULLY AUTOMATIC TEST PASS 2 (Final - 16 Sample Points @ 4min Duration)---
  21  |    3424     |    1544     |    1544     |  93.9 MB/s 
  22  |    3448     |    1552     |    1552     |  92.9 MB/s 
  23  |    3464     |    1560     |    1560     |  92.1 MB/s 
  24  |    3480     |    1568     |    1568     |  88.0 MB/s 
  25  |    3496     |    1576     |    1576     |  86.5 MB/s 
  26  |    3520     |    1584     |    1584     |  94.8 MB/s 
  27  |    3536     |    1592     |    1592     |  91.7 MB/s 
  28  |    3552     |    1600     |    1600     |  83.6 MB/s 
  29  |    3568     |    1608     |    1608     |  83.1 MB/s 
  30  |    3584     |    1616     |    1616     |  93.8 MB/s 
  31  |    3608     |    1624     |    1624     |  92.2 MB/s 
  32  |    3624     |    1632     |    1632     |  89.2 MB/s 
  33  |    3640     |    1640     |    1640     |  83.2 MB/s 
  34  |    3656     |    1648     |    1648     |  84.1 MB/s 
  35  |    3680     |    1656     |    1656     |  84.8 MB/s 
  36  |    3696     |    1664     |    1664     |  87.9 MB/s 

Completed: 2 Hrs 4 Min 56 Sec.

Best Bang for the Buck: Test 1 with a speed of 86.1 MB/s

     Tunable (md_num_stripes): 1408
     Tunable (md_write_limit): 768
     Tunable (md_sync_window): 512

These settings will consume 126MB of RAM on your hardware.


Unthrottled values for your server came from Test 26 with a speed of 94.8 MB/s

     Tunable (md_num_stripes): 3520
     Tunable (md_write_limit): 1584
     Tunable (md_sync_window): 1584

These settings will consume 316MB of RAM on your hardware.
This is 132MB more than your current utilization of 184MB.
NOTE: Adding additional drives will increase memory consumption.

In unRAID, go to Settings > Disk Settings to set your chosen parameter values.

i chose 94.8 over 86.1

thanks for this great tool. appreciate every little bit of speed improvement

Link to comment

That are very decent speeds you are getting there ...  8)

 

It's amazing the improvements you get when you take out older & slower drives  ;D

 

I'm putting the finishing touches on a new script that'll benchmark all drives in the UNRAID system (with graph!) so you can identify any bottlenecks.

Link to comment

That are very decent speeds you are getting there ...  8)

 

It's amazing the improvements you get when you take out older & slower drives  ;D

 

I'm putting the finishing touches on a new script that'll benchmark all drives in the UNRAID system (with graph!) so you can identify any bottlenecks.

 

I like your graphing tool, is it available to the public ?

 

Link to comment

I like your graphing tool, is it available to the public ?

 

http://www.highcharts.com/

 

I actually meant the script which you created to do the measurements and turn it into a graph. I see you have made available "diskspeed.sh v1.0" this is what I was looking for. Thanks.

 

Ps. Been using "highcharts" quite extensively for the "Stats" page (both under SimpleFeatures and Dynamix), it is a very nice tool to generate graphs.

Link to comment
  • 4 weeks later...

I'm currently running a parity check on 5.0.4 and tracking the speed every 15 seconds, should be done in about an hour. Then I'll upgrade to v6, run the turnables script again, and then run another parity check - again tracking the speed every 15 seconds.

 

I'll be able to present a spiffy graph comparing the two when it's done.

Link to comment

15 seconds is pretty frequent and may very well negatively impact your results.  I'd say every minute at the most.  I'd suggest taking a best guess, given your typical parity-check time, at how often you should check to get a data point every 5%.  So for me with a 6 or 7 hour check time I'd set it at about every 20 minutes.  Go crazy at 10 minutes for about every 2.5%

Link to comment

15 seconds is pretty frequent and may very well negatively impact your results.  I'd say every minute at the most.  I'd suggest taking a best guess, given your typical parity-check time, at how often you should check to get a data point every 5%.  So for me with a 6 or 7 hour check time I'd set it at about every 20 minutes.  Go crazy at 10 minutes for about every 2.5%

The process makes a copy of /proc/mdcmd which from every indication of my current parity speeds of 161.9 MB/sec @ 150GB isn't being affected in the slightest.

 

Though I could run another parity check after the current run but without the monitor and compare it to see.

Link to comment

15 seconds is pretty frequent and may very well negatively impact your results.  I'd say every minute at the most.  I'd suggest taking a best guess, given your typical parity-check time, at how often you should check to get a data point every 5%.  So for me with a 6 or 7 hour check time I'd set it at about every 20 minutes.  Go crazy at 10 minutes for about every 2.5%

The process makes a copy of /proc/mdcmd which from every indication of my current parity speeds of 161.9 MB/sec @ 150GB isn't being affected in the slightest.

 

Though I could run another parity check after the current run but without the monitor and compare it to see.

 

hmmm can't argue with that  :-X  Looking forward to your results.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.