Spin up script not working since update 6.9

[coblck]

How do you apply your method and can this be done on a schedule? So the drives do finally spin down. 

[SimonF]

59 minutes ago, MightyT said:

For now I just 'cat' a small txt file for each disk, that wakes the disks up and refreshes the status,  I'm hoping this keeps them spinning since there is no  read caching that I'm aware of.

 

Use case is, like coblck said, keeping the disks spun during a certain time of day while having relatively strict spindown settings at other times. For me it is too noisy while I'm working (and rarely accessing anything), and then in the evening and on the weekends I just want to keep them spinning since I'm not in the room aynway, to reduce spinup/spindown cycles.

I was thinking along the similar lines of a file per disk that gets read. Only thing we may need to consider if the read gets cached in ram so not physically reading from the spinning rust.

[MightyT]

1 hour ago, coblck said:

How do you apply your method and can this be done on a schedule? So the drives do finally spin down. 

 

My script now looks like this:

#!/bin/bash
cat /mnt/user/<share>/<file>;

With 1 line per disk, that just outputs the file to the console, or in that case the log. So same as before, script as cronjob for the time that you want them to keep spinning, spindown settings for the rest of the time.

 

1 hour ago, SimonF said:

I was thinking along the similar lines of a file per disk that gets read. Only thing we may need to consider if the read gets cached in ram so not physically reading from the spinning rust.

 

Yeah, that remains to be seen, my script will start soon and I'll report back if it kept working through the evening.

 

Still this is a clunky workaround, there has got to be something lke the old version with 'sdspin'

 

[doron]

Here is what I came up with (for the SAS Spindown plugin diagnostic utility), avoiding caching:

 

echo 1 > /proc/sys/vm/drop_caches     # Make sure Linux not reading from cache

dd if=/dev/sdX of=/dev/null bs=4K count=1 skip=$(( $RANDOM * 1024 + 1 )) &> /dev/null

 

[MightyT]

So just outputting the same file is not working because of caching, also it should be echo not cat. But, like doron suggested I just ended up with this:

#!/bin/bash

#disk1 = sdc
#disk2 = sde
for ((N=1; N<20; N++))
do
  dd if=/dev/sdc of=/dev/null skip=$(($RANDOM % 1024*1024)) bs=$((1024 + $RANDOM % 2048)) count=$((N*10)) &> /dev/null
  dd if=/dev/sde of=/dev/null skip=$(($RANDOM % 1024*1024)) bs=$((1024 + $RANDOM % 2048)) count=$((N*10)) &> /dev/null
done

 

So a few relatively random reads, a first try with smaller/fewer block didn't work because of caching and I hadn't thought of dropping the cache. Unsure if I want to with the Folder Caching plugin, let's see how that goes

[coblck]

Just stick with 6.9 for the moment with the other script as working for me. 

[MightyT]

Alright, yesterday's version was a bit overkill, I turned it down a little and ended up with this:

 

#!/bin/bash
for ((N=0; N<5; N++))
do
  dd if=/dev/sdc of=/dev/null skip=$(($RANDOM % 4*1024*1024*1024)) bs=$((1024 + $RANDOM % 1024)) count=10 &> /dev/null
  dd if=/dev/sde of=/dev/null skip=$(($RANDOM % 4*1024*1024*1024)) bs=$((1024 + $RANDOM % 1024)) count=10 &> /dev/null
done

 

So 5 rounds (per disk) of reading 10 random blocks of a size between 1 and 2 kb, skipping up to 4*2^9 blocks, so that should if I'm not mistaken cover the first 8tb of my 10tb disks, way too much to be cached, with relatively little read activity. Works perfectly fine so far.

Edited March 11, 2021 by MightyT

[coblck]

@MightyT is script working fine ? also what would i need to chnage for 5tb disks

[MightyT]

Has been working just fine all week. I'd change the size of the skip parameter, to only skip half the amount of blocks, at the most, and then just add a line per disk  inside the loop, like this:

 

#!/bin/bash
for ((N=0; N<5; N++))
do
  dd if=/dev/sdc of=/dev/null skip=$(($RANDOM % 2*1024*1024*1024)) bs=$((1024 + $RANDOM % 1024)) count=10 &> /dev/null
  dd if=/dev/sde of=/dev/null skip=$(($RANDOM % 2*1024*1024*1024)) bs=$((1024 + $RANDOM % 1024)) count=10 &> /dev/null
done

 

[coblck]

@MightyTFantastic thanks, does this script just read from the discs every so often keeping them awake and will have to do a cron schedule for the times i require it too work 

Edited March 13, 2021 by coblck

[doron]

Note the first line in my suggested code above:

echo 1 > /proc/sys/vm/drop_caches     # Make sure Linux not reading from cache

This causes the kernel to drop all page caches so the chances of reading a cached buffer decrease dramatically. 

You'd then not need to read 50 (potentially massive) blocks each time...

 

Note all kernel page caches are dropped (cleared), not just one device's, so you don't want to do this too often.

 

(Purists will echo "3" and not "1" but that would be going over the top IMHO).

[MightyT]

1 hour ago, coblck said:

@MightyTFantastic thanks, does this script just read from the discs every so often keeping them awake and will have to do a cron schedule for the times i require it too work 

exactly

 

1 hour ago, doron said:

Note all kernel page caches are dropped (cleared), not just one device's, so you don't want to do this too often.

 

Yeah, I figured I wouldn't want to do that every 15 minutes, but just spreading out the potential blocks so I got a chance of reading something that isn't cached seems to work consistently. And now I'm only reading 5 blocks of max. 2kb per disk, I think that is negligible.

[doron]

3 minutes ago, MightyT said:

And now I'm only reading 5 blocks of max. 2kb per disk, I think that is negligible.

You did specify "count=10" which makes that 10 such blocks times 5 iterations...

[MightyT]

2 minutes ago, doron said:

You did specify "count=10" which makes that 10 such blocks times 5 iterations...

Yeah, I forgot about that. Still that would bring the max only up to 100kb, still ok, I might just turn it down more.

[Squid]

This script will spin up every drive and keep the GUI in sync with everything (as it does it the exact same way the GUI does). 

 

It doesn't appear the emhttp supports spin up or spin down.  Rather it only supports toggling between the two, hence the script checking for spundown

 

If someone want to convert to bash feel free

#!/usr/bin/php
<?

function emhttpd($cmd) {
  global $state, $csrf;
  $ch = curl_init("http://127.0.0.1/update");
  $options = array(CURLOPT_UNIX_SOCKET_PATH => '/var/run/emhttpd.socket',
                   CURLOPT_POST => 1,
                   CURLOPT_POSTFIELDS => "$cmd&startState=$state&csrf_token=$csrf");
  curl_setopt_array($ch, $options);
  curl_exec($ch);
  curl_close($ch);
}

$disks = (array)parse_ini_file('/var/local/emhttp/disks.ini',true);
$vars = parse_ini_file('/var/local/emhttp/var.ini',true);
$state = $vars['mdState'];
$csrf = $vars['csrf_token'];

foreach ($disks as $disk) {
  if ($disk['status'] != 'DISK_OK') continue;
  if ( ! $disk['spundown'] ) continue;
  echo "Spinning up {$disk['name']}\n";
  emhttpd("cmdSpinup={$disk['name']}");
}
?>

 

[Squid]

Simplification.  Tom let me know you can spin up all drives via one call

 

#!/usr/bin/php
<?

function emhttpd($cmd) {
  global $state, $csrf;
  $ch = curl_init("http://127.0.0.1/update");
  $options = array(CURLOPT_UNIX_SOCKET_PATH => '/var/run/emhttpd.socket',
                   CURLOPT_POST => 1,
                   CURLOPT_POSTFIELDS => "$cmd&startState=$state&csrf_token=$csrf");
  curl_setopt_array($ch, $options);
  curl_exec($ch);
  curl_close($ch);
}

$vars = parse_ini_file('/var/local/emhttp/var.ini',true);
$state = $vars['mdState'];
$csrf = $vars['csrf_token'];

echo "Spinning up disks\n";
emhttpd("cmdSpinupAll=apply");
?>

 

[JorgeB]

16 hours ago, Squid said:

Tom let me know you can spin up all drives via one call

Since I'm now using a pool instead of an array for the the drives I want to spin up, any chance you know how to change the script to spin up all devices in a pool instead?

[Squid]

33 minutes ago, JorgeB said:

Since I'm now using a pool instead of an array for the the drives I want to spin up, any chance you know how to change the script to spin up all devices in a pool instead?

Try changing this line:

emhttpd("cmdSpinupAll=apply");

to instead be

emhttpd("cmdSpinup=nameOfCachePool");

 

But, the name of the cache pool should be all in lower case (ie: your pool is named Virtual_machines you use virtual_machines)

 

You should also be able to spindown by just changing cmdSpinup to be cmdSpindown

[coblck]

17 hours ago, Squid said:

Simplification.  Tom let me know you can spin up all drives via one call

 

#!/usr/bin/php
<?

function emhttpd($cmd) {
  global $state, $csrf;
  $ch = curl_init("http://127.0.0.1/update");
  $options = array(CURLOPT_UNIX_SOCKET_PATH => '/var/run/emhttpd.socket',
                   CURLOPT_POST => 1,
                   CURLOPT_POSTFIELDS => "$cmd&startState=$state&csrf_token=$csrf");
  curl_setopt_array($ch, $options);
  curl_exec($ch);
  curl_close($ch);
}

$vars = parse_ini_file('/var/local/emhttp/var.ini',true);
$state = $vars['mdState'];
$csrf = $vars['csrf_token'];

echo "Spinning up disks\n";
emhttpd("cmdSpinupAll=apply");
?>

 

@SquidSo this as is simple as creating new script and copying this over no need to chnage anything , will the drives spin down after default time has passed.

Edited March 18, 2021 by coblck

[JorgeB]

32 minutes ago, Squid said:

Try changing this line:

emhttpd("cmdSpinupAll=apply");

to instead be

emhttpd("cmdSpinup=nameOfCachePool");

 

Thanks, It works, but it only spins up the first pool device, though if I don't find a better way I already tested and if I repeat with nameofpool2, nameofpool3, etc it will spin up the others.

[Squid]

I don't have any multi-device pools.  

 

If you replace /usr/local/emhttp/plugins/dynamix/include/ToggleState.php with the attached, manually spin up your pool and then give me back the contents of /boot/blah you can adjust to suit.  (It'll list the commands it's sending to emhttpd)

ToggleState.php

[JorgeB]

51 minutes ago, Squid said:

If you replace /usr/local/emhttp/plugins/dynamix/include/ToggleState.php with the attached, manually spin up your pool

It's outputting this:

 

cmdSpinup=lts
cmdSpinup=lts2
cmdSpinup=lts3
cmdSpinup=lts4

 

Before I did try to add the multiple spin up command to the script above like so:

 

#!/usr/bin/php
<?

function emhttpd($cmd) {
  global $state, $csrf;
  $ch = curl_init("http://127.0.0.1/update");
  $options = array(CURLOPT_UNIX_SOCKET_PATH => '/var/run/emhttpd.socket',
                   CURLOPT_POST => 1,
                   CURLOPT_POSTFIELDS => "$cmd&startState=$state&csrf_token=$csrf");
  curl_setopt_array($ch, $options);
  curl_exec($ch);
  curl_close($ch);
}

$vars = parse_ini_file('/var/local/emhttp/var.ini',true);
$state = $vars['mdState'];
$csrf = $vars['csrf_token'];

echo "Spinning up disks\n";
emhttpd("cmdSpinup=lts");
emhttpd("cmdSpinup=lts2");
emhttpd("cmdSpinup=lts3");
emhttpd("cmdSpinup=lts4");
?>

 

but it failed to spin up the rest, not really a surprise since I know nothing about scripts and just just hoping it would be that easy.

[Squid]

IDK.  What you've done should work....

 

 

[JorgeB]

16 minutes ago, Squid said:

IDK.  What you've done should work....

OK, strange, so running it manually using user scripts it didn't work, it only spun up the first device (had to manually close the script window, no OK, so possibly the script wasn't being executed correctly till the end), but running the same script on schedule worked, so problem solved, since that's what I want, to it run on a schedule.

[ken-ji]

On 3/14/2021 at 10:04 PM, Squid said:

This script will spin up every drive and keep the GUI in sync with everything (as it does it the exact same way the GUI does). 

 

It doesn't appear the emhttp supports spin up or spin down.  Rather it only supports toggling between the two, hence the script checking for spundown

 

If someone want to convert to bash feel free

#!/usr/bin/php
<?

function emhttpd($cmd) {
  global $state, $csrf;
  $ch = curl_init("http://127.0.0.1/update");
  $options = array(CURLOPT_UNIX_SOCKET_PATH => '/var/run/emhttpd.socket',
                   CURLOPT_POST => 1,
                   CURLOPT_POSTFIELDS => "$cmd&startState=$state&csrf_token=$csrf");
  curl_setopt_array($ch, $options);
  curl_exec($ch);
  curl_close($ch);
}

$disks = (array)parse_ini_file('/var/local/emhttp/disks.ini',true);
$vars = parse_ini_file('/var/local/emhttp/var.ini',true);
$state = $vars['mdState'];
$csrf = $vars['csrf_token'];

foreach ($disks as $disk) {
  if ($disk['status'] != 'DISK_OK') continue;
  if ( ! $disk['spundown'] ) continue;
  echo "Spinning up {$disk['name']}\n";
  emhttpd("cmdSpinup={$disk['name']}");
}
?>

 

 

Because of a weird speed issue where I now need to spin up my array before running parity checks etc. and my personal opinions about php,

I had to take up the bash challenge (this might even be sh compatible)

Only did the spin them all up one though.

#!/bin/bash
. /usr/local/emhttp/state/var.ini

curl --unix-socket /var/run/emhttpd.socket \
  --data-urlencode cmdSpinupAll=apply \
  --data-urlencode startState=$mdState \
  --data-urlencode csrf_token=$csrf_token \
  http://127.0.0.1/update