2 minutes ago, BRiT said:

Was your system performance changed much through changes in these tunables on the older versions? 

Not significantly.  About the only tunable I've got changed in nr_requests at 128 and num_stripes at 4000

8 hours ago, BRiT said:

So this is something to report to Limetech and they will have to make the changes required in order to get solid performance on wider variety of hardware?

Tom is already aware of my results, now we need feedback form other users to see how if this is really an issue or not, but based on my tests only users with very wide arrays and/or very fast devices should notice any difference, still those who do should report it together with the array size/hardware used, you just post the diags.

I have just tried 6.8.0-rc1 to compare its parity checks to both 6.7.2 and the Tunables script since my array is very straightforward and the result is interesting and shows there is still room for improvement.

 

The first thing I did before upgrading from 6.7.2 was reset all the Disk Settings tunables back to default (including nr-requests which defaults to auto now).

 

My array is 2x WD Red 8Tb dual parity and 6x WD Red 8Tb data with 2x 512Gb SSD for cache.

 

6.7.2 default - 17h15 to 17h30

6.7.2 tunables - 15h30

6.8.0-rc1 default - 16h45

 

I have added a screenshot of the 6.7.2 Tunables and also 6.8.0-rc1 default values for comparison and I think there is still scope for improvement. I suspect that on my server we could find up to an hour in there that could be tuned out.

On 10/11/2019 at 7:15 PM, BRiT said:

Looks like the 6.8 series will present a new set of challenges on tuning...

Drats.

 

On 10/12/2019 at 3:04 AM, johnnie.black said:

v.6.8 should autotune and use the best settings and there's not much to manually tune for parity check speed

So wait... I was actually too quick to create the new version of UTT?  I could have sat on my tookus and let Limetech fix the issue for me?

 

13 hours ago, DanielCoffey said:

I have just tried 6.8.0-rc1 to compare its parity checks to both 6.7.2 and the Tunables script since my array is very straightforward and the result is interesting and shows there is still room for improvement.

...

6.7.2 default - 17h15 to 17h30

6.7.2 tunables - 15h30

6.8.0-rc1 default - 16h45

That's disappointing.  But your tuned values are sky-high, probably among the highest I have ever seen shared here.  I would say that you have a special needs controller.  Definitely share this with Limetech.

 

13 hours ago, DanielCoffey said:

I have added a screenshot of the 6.7.2 Tunables and also 6.8.0-rc1 default values for comparison

Very interesting, thanks for sharing.  It took me a long time and a lot of effort to come up with a testing strategy for the v6.0-v6.7 tunables, and these changes with 6.8 pretty much throw all that out the window.

 

If anyone sees any info regarding the new tunables, please repost here.

 

And fingers crossed that Limetech makes UTT unnecessary, as I really really really don't want to do it all over yet again...

What else would you be doing over the holidays? 🍬 🦃 🕯️ 🕎 🎅 🎆

10 hours ago, Pauven said:

So wait... I was actually too quick to create the new version of UTT?  I could have sat on my tookus and let Limetech fix the issue for me?

Yeah, bad timing, though like mentioned new code needs work as it can be considerably slower than before, about 40% slower on my test server, but since the only remaining tunable (md_num_stripes) doesn't make any difference for me, other than slowing things more if set very low, I believe we'll need to wait for Tom to optimize it.

so is this now dead?

It worked well for 6.7.3.

 

However 6.8 flipped the tables on tunables.

 

Things may yet change again once 6.9 final hits, it all depends on what Limetech implements to address the performance decreases in parity build and check times some users on 6.8 have reported.

Good evening,

 

I am attempting to use this on my UnRAID 6.8.0 system.  Copied the file from my Windows 10 machine to the flash drive (/boot/utt) as indicated in the instructions.  I am having an issue with permissions.  I can't seem to chmod 777 the file successfully.  Also, attempting to change permissions using WinSCP doesn't seem to work.  Obviously, I am getting a permission denied when attempting to run it.  Can someone advise where I went wrong?

 

Thanks,

Erik

You can't directly run scripts on the flash with 6.8+ (security measure)  You could just

 

bash /boot/utt/scriptname

Beyond that, there's a multitude of changes in 6.8 that largely negates this script, and in theory 6.8+ is self-optimizing on read speeds

OK, thanks... I'll leave it alone then.

UTT is not compatible with Unraid v6.8 or later.

 

I developed the latest version using Unraid 6.6.6 (which is what I'm still running).  I've avoided the 6.7.x series due to some known performance issues, and 6.8 for even bigger issues. So I don't have more recent versions available for testing and development.   6.6.6 works perfectly for me, and I have zero reason to chase version upgrades just to be on a #, so I might be here for a while.

 

Which is all really pointless anyway, since Lime-Tech took away the tunables that UTT tunes in v6.8.  In theory, UTT is dead and no longer even needed with v6.8, since Lime-Tech took back control of these tunables and have their own internal logic for tuning them.

 

So long story short, UTT is dead for Unraid 6.8 or later.  Though it still works for 6.7.3 and earlier.

Thank you!


—Sent from my iPhone using Tapatalk Pro

That is sad and I understand your reasoning. And nothing I say below should be read as a call to change your current plan.

 

That said, I disagree that LT has made it self tuning. Default settings for even the one main tunable, md_num_stripes, makes a difference for my 7 disk array. Setting it to 5120 improved my parity by 10% from default. Not sure if this is the best speed:memory option, but i also guess I don't need a full script to test out a few options. But for sure default is slower than tweaked. FWIW, I'm getting same speed in 6.8 tweaked as 6.7 tweaked which was the same speed as 6.6 tweaked

 

nr_requests I'm not so sure but I did increase mine from default to 256. This is where maybe a script might help to see if there is an interaction. Though my research of what nr_requests does leads me to believe there probably isn't any covariance.

 

In any case, thank you for all your time and effort. Even if it ends up being the stuff of legend, your contribution to the community really is greatly appreciated.

 

Thank You Pauven!

24 minutes ago, jumperalex said:

That said, I disagree that LT has made it self tuning. Default settings for even the one main tunable, md_num_stripes, makes a difference for my 7 disk array. Setting it to 5120 improved my parity by 10% from default. Not sure if this is the best speed:memory option, but i also guess I don't need a full script to test out a few options. But for sure default is slower than tweaked. FWIW, I'm getting same speed in 6.8 tweaked as 6.7 tweaked which was the same speed as 6.6 tweaked

 

Interesting, I did not know this.  I knew some users had performance issues with 6.8 and that Lime-Tech was still refining their logic, but I hadn't heard that some of the tunables can still help.  I guess once I get off my legacy version I can revisit this again.  Most likely I'm waiting for 6.9.1, fingers crossed.

 

Though to be honest, I do hope Lime-Tech can figure out the logic to truly make UTT unnecessary.

 

27 minutes ago, jumperalex said:

In any case, thank you for all your time and effort. Even if it ends up being the stuff of legend, your contribution to the community really is greatly appreciated.

Thanks, it feels good to help and even better to be appreciated!

Hi

 

im a new Unraid user, still in trial mode, running 6.9.1, on a HP DL380 G8, with the inbuilt P420i in HBA mode, with 20 x 600GB 10K sas drives (got them stupidly cheap)

 

i tried to run this script this morning and got a few issues appear during the run cycle.

 

./unraid6x-tunables-tester.sh: line 1546: [: : integer expression expected
./unraid6x-tunables-tester.sh: line 1548: [: : integer expression expected
./unraid6x-tunables-tester.sh: line 1550: [: : integer expression expected

 

and

Setting all drives to nr_requests=128 for the following tests/usr/local/sbin/mdcmd: line 11: echo: write error: Invalid argument

/usr/local/sbin/mdcmd: line 11: echo: write error: Invalid argument

 

are these expected? or is this a unraid/script versioning related issue?

 

Thanks

Edited March 16, 20215 yr by Kill3r
more info

5 minutes ago, Kill3r said:

i tried to run this script this morning and got a few issues appear during the run cycle.

This script is only for v6.7.x or older, it's not needed for newer releases.

Hi JorgeB,

 

ok, does 6.9.1 auto tune these values?

 

Thanks

2 minutes ago, Kill3r said:

ok, does 6.9.1 auto tune these values?

Since v6.8 some tunables changed and are no longer used, in practice Unraid should auto tune for best performance and the remaining tunables don't do a lot (if any) difference, but it's known to be slower in some cases, especially for larger arrays.

Everything these fine gents wrote is correct.  I stopped development of UTT after Unraid v6.8 came out. 

 

There was some chatter that even v6.8 had some tunables that affected performance, and that what LimeTech was doing didn't work perfectly on all hardware, though as you can see it has been quiet here for well over a year, so I'm guessing the issues weren't enough for users to chase solutions.  And perhaps LT did resolve some of those earlier v6.8 performance issues a few users experienced.

 

Ultimately, my perspective is that beginning with v6.8, LT was actively working on internalizing performance tuning, and the need for UTT is no more.

 

Additionally, the original major performance issue that I experienced on my hardware, that led me to create this tool, is gone since v6.8.  So even if there were performance issues affecting some hardware configs, I'm lacking the motivation or time to troubleshoot them by revamping this code.  I willingly pass the mantle on to anyone else that has a need to refine the code for newer Unraid versions.  My shift has ended.

So what should we set md_num_stripes to these days? on 32gb of RAM system.

Its the only option left in settings so it must be doing something

 

If 6.12.4 is autotuning, it's doing a horrible job. Upping the strpes value made a HUGE diff in my rebuild speed, like 5x at least.

 

24gb RAM, Intel® Xeon® CPU E5-2697 v2 @ 2.70GHz Dell R720XD

 

Is the script still relevant / still works / applies for Unraid 7.x?

I stumbled upon here while browsing the meaning of the various disk settings.

Edited February 9Feb 9 by rug

1 hour ago, rug said:

Is the script still relevant / still works / applies for Unraid 7.x?

Not really, there were signifcant changes.