unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables

Pauven · August 23, 2016

Okay, got a more [minor] challenge.

I'm now checking the unRAID version in the script, for compatibility reasons. Based upon Lime-Tech documentation, I'm pulling this value from the SysLog.

But the tunables test script creates enough log entries that eventually this value rolls off.

Is there a better way of getting the unRAID version?

Thanks,

Paul

JorgeB · August 23, 2016

I'll take a leap of faith and assume those dual parity servers are 6.2. What about the two single parity?

Since you listed them all, are you planning on testing on all of them? I've very curious about that SSD server you got. And the one with the expander. And the... oh hell all of em!

Actually have 8 unRAID servers :-[ , the eight is being used as test server only at the moment.

All are running 6.2, and I do plan to test them all, but it will take some time.

JorgeB · August 23, 2016

This is probably why he doesn't have them in his sig. johnnie seems to throw them together just for the fun of testing.

Main reason I don't have a sig config is because I'm constantly tinkering with them, upgrading disks, controllers, etc, I would spend too much time keeping it up to date.

Pauven · August 23, 2016

You should be able to parse /var/local/emhttp/disks.ini

In the [parity2] section it has an IDX label (which on my system spec in sig) comes up with 29

There's a lot of interesting data in the disks.ini file, and I would like to pull some of it out, and would appreciate some help with the logic for doing so.

I primarily would like the name value (i.e. "disk18" or "cache" or "parity2" ) along with the device (i.e. "sdd"). I would want to filter on the status (don't want "DISK_NP").

It would also be great if I could get the spin status for each drive, either from here or somewhere else. I see a rotational value, but this seems to be a way to distinguish HD's from SSD's.

Thanks,

Paul

Pauven · August 23, 2016

Actually have 8 unRAID servers , the eight is being used as test server only at the moment.

All are running 6.2, and I do plan to test them all, but it will take some time.

You're Hired!

I would appreciate it if, during the beta as you post your results, you clarify which server the results apply to and re-list the config for that server.

-Paul

bonienl · August 23, 2016

Okay, got a more [minor] challenge.

I'm now checking the unRAID version in the script, for compatibility reasons. Based upon Lime-Tech documentation, I'm pulling this value from the SysLog.

But the tunables test script creates enough log entries that eventually this value rolls off.

Is there a better way of getting the unRAID version?

Thanks,

Paul

Are you developing this as a plugin?

If yes you can specify a "min" and/or "max" version identifier to ensure the plugin installs only on the intended unRAID version(s).

E.g.

<PLUGIN name="&name;" author="&author;" version="&version;" pluginURL="&pluginURL;" min="6.1.0">

The same approach with Min/Max (note the capital letter) is used to do conditional loading of files inside the PLG file.

If you simply want to read the unRAID version directly then open the file /etc/unraid-version

bonienl · August 23, 2016

It would also be great if I could get the spin status for each drive, either from here or somewhere else. I see a rotational value, but this seems to be a way to distinguish HD's from SSD's.

rotational is indeed used to distinguish a HDD or SSD disk.

Use the "color" field to determine the spin status:

- green-on = disk active

- green-blink = disk inactive

Pauven · August 23, 2016

If you simply want to read the unRAID version directly then open the file /etc/unraid-version

Exactly what I was looking for, thanks! The official documentation should be updated to include this solution.

I'm not to the plug-in stage yet. Someday.

bonienl · August 23, 2016

I'm not to the plug-in stage yet. Someday.

I encourage you to go that direction. It will make installation and updating so much easier (and larger exposure of your work when included in CA)

Squid · August 23, 2016

I'm not to the plug-in stage yet. Someday.

I encourage you to go that direction. It will make installation and updating so much easier (and larger exposure of your work when included in CA)

I had originally offered to help on that one but now with you piping in here you're the better choice as you are the html css God around here

Sent from my SM-T560NU using Tapatalk

bonienl · August 23, 2016

What you say? Needed to google that ... How To Make Life Cozy, Shiny & Sensible

Anyway it is a small step to extend a script to a plugin, and if help is required, not a problem.

Squid · August 23, 2016

What you say? Needed to google that ... How To Make Life Cozy, Shiny & Sensible

Anyway it is a small step to extend a script to a plugin, and if help is required, not a problem.

Its because something like this ideally should have the graphs displayed, which is more work (and learning) for me. My original suggestion was actually just going to be a .page file wrapper for the script.

Squid · August 23, 2016

You should be able to parse /var/local/emhttp/disks.ini

In the [parity2] section it has an IDX label (which on my system spec in sig) comes up with 29

There's a lot of interesting data in the disks.ini file, and I would like to pull some of it out, and would appreciate some help with the logic for doing so.

I primarily would like the name value (i.e. "disk18" or "cache" or "parity2" ) along with the device (i.e. "sdd"). I would want to filter on the status (don't want "DISK_NP").

It would also be great if I could get the spin status for each drive, either from here or somewhere else. I see a rotational value, but this seems to be a way to distinguish HD's from SSD's.

Thanks,

Paul

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them

), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

saarg · August 23, 2016

You should be able to parse /var/local/emhttp/disks.ini

In the [parity2] section it has an IDX label (which on my system spec in sig) comes up with 29

There's a lot of interesting data in the disks.ini file, and I would like to pull some of it out, and would appreciate some help with the logic for doing so.

I primarily would like the name value (i.e. "disk18" or "cache" or "parity2" ) along with the device (i.e. "sdd"). I would want to filter on the status (don't want "DISK_NP").

It would also be great if I could get the spin status for each drive, either from here or somewhere else. I see a rotational value, but this seems to be a way to distinguish HD's from SSD's.

Thanks,

Paul

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

Have you checked they are not on eBay, craigslist or something similar?

Pauven · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

PITA. That's the truth.

I guess I was hoping for a miracle one-liner, like how:

eval $(egrep 'md_num_stripes|md_sync_window|md_sync_thresh|nr_requests' /boot/config/disk.cfg | tr -d '\r')

creates 4 vars with values assigned. Freaking awesome.

I was also hopeful to find the data in the "mdcmd status" output, as at least that identifies the drive # on each line. But I don't see spin status or name there.

If it is simply a matter of looping through, finding one value, then trusting the next value is the one I'm looking for, I can do that. PITA, but doable.

Thanks,

Paul

Pauven · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

Have you checked they are not on eBay, craigslist or something similar?

Good call, I just checked and found a new ebay listing for a bunch of used drives: 2x Seagate 4TB, 9x Seagate 3TB, 4x Seagate 2TB, 3x Seagate 1TB, 2x WD 3TB, 1x WD 2TB, 1x Hitachi 2TB, and 1x Hitachi 1TB.

Score!

saarg · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

Have you checked they are not on eBay, craigslist or something similar?

Good call, I just checked and found a new ebay listing for a bunch of used drives: 2x Seagate 4TB, 9x Seagate 3TB, 4x Seagate 2TB, 3x Seagate 1TB, 2x WD 3TB, 1x WD 2TB, 1x Hitachi 2TB, and 1x Hitachi 1TB.

Score!

;D

Squid · August 23, 2016

Nah. I'm thinking you just call my script with the disk# you want the info on and I return the values. And then you can grep that (or we can get fancier and you supply my script with the disk and the attribute you want and I'll just return that) we're talking a 3 line script in PHP that you'll include with your package

Sent from my LG-D852 using Tapatalk

Squid · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

Have you checked they are not on eBay, craigslist or something similar?

Good call, I just checked and found a new ebay listing for a bunch of used drives: 2x Seagate 4TB, 9x Seagate 3TB, 4x Seagate 2TB, 3x Seagate 1TB, 2x WD 3TB, 1x WD 2TB, 1x Hitachi 2TB, and 1x Hitachi 1TB.

Score!

;D

Bastards.

Sent from my LG-D852 using Tapatalk

bonienl · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

PITA. That's the truth.

I guess I was hoping for a miracle one-liner, like how:
eval $(egrep 'md_num_stripes|md_sync_window|md_sync_thresh|nr_requests' /boot/config/disk.cfg | tr -d '\r')
creates 4 vars with values assigned. Freaking awesome.

I was also hopeful to find the data in the "mdcmd status" output, as at least that identifies the drive # on each line. But I don't see spin status or name there.

If it is simply a matter of looping through, finding one value, then trusting the next value is the one I'm looking for, I can do that. PITA, but doable.

Thanks,

Paul

Going the PHP way is much easier ... it has native support.

But if you want to go with bash, then associative arrays may help.

Next is not really a one-liner but may do what you want.

while read -r line; do
  if [[ ${line:0:1} == '[' ]]; then
    disk=${line:2:-2}
    declare -A $disk
  else
    ifs=IFS; IFS='='
    key=($line)
    eval $disk[${key[0]}]=${key[1]}
    IFS=$ifs
  fi
done </var/local/emhttp/disks.ini

The above reads the values of all the disks in the array and create the corresponding associative arrays.

You can now do something like this

if [[ ${disk1[status]} == DISK_OK && ${disk1[color]} == green-on ]]; then
  echo "disk1 is well and active"
fi

edit: fixed typo

Squid · August 23, 2016

Parsing a file like that is a PITA with bash. I'm on the road for a couple of days (and somehow my wife has managed to completely take down my entire network and servers and she can't seem to find the reset button on them ), but shoot me a PM with exactly what you want (and output format), and I'll give you a php script to call when I get back (Friday)

PITA. That's the truth.

I guess I was hoping for a miracle one-liner, like how:
eval $(egrep 'md_num_stripes|md_sync_window|md_sync_thresh|nr_requests' /boot/config/disk.cfg | tr -d '\r')
creates 4 vars with values assigned. Freaking awesome.

I was also hopeful to find the data in the "mdcmd status" output, as at least that identifies the drive # on each line. But I don't see spin status or name there.

If it is simply a matter of looping through, finding one value, then trusting the next value is the one I'm looking for, I can do that. PITA, but doable.

Thanks,

Paul
Going the PHP way is much easier ... it has native support.

But if you want to go with bash, then associative arrays may help.

Next is not really a one-liner but may do what you want.
while read -r line; do
  if [[ ${line:0:1} == '[' ]]; then
    disk=${line:2:-2}
    declare -A $disk
  else
    ifs=IFS; IFS='='
    key=($line)
    eval $disk[${key[0]}]=${key[1]}
    IFS=$ifs
  fi
done </var/local/emhttp/disks.ini
The above reads the values of all the disks in the array and create the corresponding associative arrays.

You can now do something like this
if [[ ${disk1[status]} == DISK_OK && ${disk1[color]} == green-on ]]; then
  echo "disk1 is well and active"
fi
edit: fixed typo

And that is why I don't touch bash anymore. So much simpler to interface to a PHP wrapper from bash than to deal with ^^^

Sent from my LG-D852 using Tapatalk

bonienl · August 23, 2016

And that is why I don't touch bash anymore. So much simpler to interface to a PHP wrapper from bash than to deal with ^^^

Totally agree, writing the above in PHP is basically a one-liner. Just wanted to point out that things are possible in bash. It's up to Pauven to decide how to move forward!

Pauven · August 23, 2016

Nah. I'm thinking you just call my script with the disk# you want the info on and I return the values. And then you can feel that (or we can get fancier and you supply my script with the disk and the attribute you want and I'll just return that) we're talking a 3 line script in PHP that you'll include with your package

I think I got what I need.

Not sure how you could do it in 3 lines (unless you mean I can call your script with just 3 lines in my script), as I think 21 is the best I could do if I really tried. Here's what I came up with, which is meeting my needs:

	count=-1
while read line
do
	if [[ $line == "["*"]" ]]; then 
		((count++)) #Increase the counter
		if [ $count -gt 0 ]; then
			DiskNamePretty[$idx]=$name
			DiskType[$idx]=$type
			DiskName[$idx]=$device
			if [ $color == "green-on" ]; then
				DiskSpin[$idx]="UP"
			else
				DiskSpin[$idx]="down"
			fi
		fi
	else 
		eval $line  #convert the line into a var
	fi
done < /var/local/emhttp/disks.ini

#process the final disk from the file
DiskNamePretty[$idx]=$name
DiskType[$idx]=$type
DiskName[$idx]=$device
if [ $color == "green-on" ]; then
	DiskSpin[$idx]="UP"
else
	DiskSpin[$idx]="down"
fi

Oh, look I see bonienl showed me up!!! Very nice. I see it was a similar solution, but you grabbed all values and threw then into an array, and silly me I used separate arrays to hold just the values I wanted.

Eh, for now it is working. But I really do like what you did there.

Thanks,

Paul

Pauven · August 23, 2016

And that is why I don't touch bash anymore. So much simpler to interface to a PHP wrapper from bash than to deal with ^^^

Totally agree, writing the above in PHP is basically a one-liner. Just wanted to point out that things are possible in bash. It's up to Pauven to decide how to move forward!

Okay, I'm totally curious... what's the one line?

I do a little amateur PHP coding on web pages, but it never even crossed my mind that I could do it in a unRAID script.

Squid · August 23, 2016

parse_ini_file(filename,true)

Followed by an echo of whatever variable you want so that bash reads it.

Sent from my LG-D852 using Tapatalk

unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Pauven

Pauven

Pauven

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation