SPINCONTROL: Local and Remote Management of Drive Standby State.


Recommended Posts

Ah, so when you say the web page could be out of sync, could the drives be spun up and it just doesn't show?  It does a lot of clicking like it is spinning them up, so unless it is spinning them back down immediately (which I guess could be the case), maybe they are spun up.  Is there another way besides the web page that I could check their status? I have unmenu installed, but it seems to also indicate they are spun down (there is no temperature shown for the drives).

 

Regardless, thanks for the tool and the help.

Link to comment

Ah, so when you say the web page could be out of sync, could the drives be spun up and it just doesn't show?  It does a lot of clicking like it is spinning them up, so unless it is spinning them back down immediately (which I guess could be the case), maybe they are spun up.  Is there another way besides the web page that I could check their status? I have unmenu installed, but it seems to also indicate they are spun down (there is no temperature shown for the drives).

 

Regardless, thanks for the tool and the help.

The new spin-up logic will immediately spin down a drive if its timer has expired and you do not either use one of the "spinup" commands or access the disk through the /dev/md device.

 

You cannot just read a block from the raw /dev/sdX disk  to spin up a drive as was the case in the past. 

Now, for those disks not in the array you must still use the old methods.

Link to comment

NOTES

I should add, I spin UP the drives by issuing the hdparm auto spinddown command.

That is exactly how I used to do it too before the 4.5.beta series.

But ever since one of the betas (I forgot which one) I started getting nasty device not ready reset errors in syslog whenever I did that.

It is strange that hdparm should cause such a problem when setting the spindown timer on a disk that's spun-down at the moment.

Maybe it has something to do with these particular drives, I don't know. (mine are WD-EADS)

Anyway, now I would spin up a drive by a random read from it, and everything is well.

Or, if I want to spin up all drives, I just issue a `sync` command.

 

Link to comment

The new spin-up logic will immediately spin down a drive if its timer has expired and you do not either use one of the "spinup" commands or access the disk through the /dev/md device.

 

Joe, I saw one of your other posts in the 4.5 announcement thread where you talk about the "spinup" commands that Tom provided, but I can't find any reference to those commands in the release notes or elsewhere. Can you give me a pointer?  Can the commands be executed remotely, maybe via HTTP?  I just need a way to send a spin-up command (preferably with granularity, but I'd settle for spin-up all) remotely from a windows computer. WeeboTech's tool will work if he wants to update it to not use hdparm, but I'm open to other ideas as well. Thanks.

Link to comment

The new spin-up logic will immediately spin down a drive if its timer has expired and you do not either use one of the "spinup" commands or access the disk through the /dev/md device.

 

Joe, I saw one of your other posts in the 4.5 announcement thread where you talk about the "spinup" commands that Tom provided, but I can't find any reference to those commands in the release notes or elsewhere. Can you give me a pointer?  Can the commands be executed remotely, maybe via HTTP?  I just need a way to send a spin-up command (preferably with granularity, but I'd settle for spin-up all) remotely from a windows computer. WeeboTech's tool will work if he wants to update it to not use hdparm, but I'm open to other ideas as well. Thanks.

The commands are in the mdcmd interface to the "md" driver

 

They are

 

/root/mdcmd spinup X

where X = 0 through 19  0=parity drive, 1-19 are disk1 through disk19 data drives.

or

/root/mdcmd spindown X

 

The description from limetech was here: http://lime-technology.com/forum/index.php?topic=4782.msg44093#msg44093

 

If you do a

/root/mdcmd status | grep rdevLastIO

you will get the fields in the "md" interface that indicate the disk spin-down status.  If a value for a disk is 0, it is spun down

If non-zero, it is the time-stamp-in-seconds since Jan 1,1970 when the disk was last accessed.

 

If you get the current time in the same format and subtract, you can tell how long the disk has been idle.

 

Here is a quick little script showing how to interpret the rdevLastIO values.

#!/bin/bash

 

function hms() {

    seconds=$1

    hours=$(($seconds / 3600))

    seconds=$(($seconds - ($hours * 3600)))

    minutes=$(($seconds / 60))

    seconds=$(($seconds - ($minutes * 60)))

    [ "$hours" = "0" ] && hours="" || hours="$hours hours, "

    [ "$minutes" = "0" ] && minutes="" || minutes="$minutes minutes, "

    echo $hours$minutes$seconds

}

 

#Get the disk spinup status

disk_status=`/root/mdcmd status | strings | grep rdevLastIO`

# Get the current time.

now=`date +%s`

 

for i in 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

do

    last_access=`echo "$disk_status" | grep "rdevLastIO.$i=" | cut -d"=" -f2`

    if [ "$last_access" = "" ]

    then

      continue

    fi

 

    elapsed_time=$(expr $now - $last_access)

    i=`printf "%2d" $i`

    if [ "$last_access" = "0" ]

    then

      status="disk $i not currently spinning"

    else

      status="disk $i = `hms $elapsed_time` seconds since last I/O"

    fi

    echo $status

done

 

 

Joe L.

Link to comment

The commands are in the mdcmd interface to the "md" driver

 

They are

...

 

Thanks for all the info, a lot to digest for a windows programmer.  I'll try to plow through it this weekend. WeeboTech, if you think you will be updating SPINCONTROL, please let me know and I may just wait for you to do your magic before jumping in myself.

Link to comment

The commands are in the mdcmd interface to the "md" driver

 

They are

...

 

Thanks for all the info, a lot to digest for a windows programmer.  I'll try to plow through it this weekend. WeeboTech, if you think you will be updating SPINCONTROL, please let me know and I may just wait for you to do your magic before jumping in myself.

 

Actually taking a second, closer look at it, it's a lot more straightforward than I thought at first glance. Just need to fashion a TCP/IP interface to those commands.

Link to comment

The commands are in the mdcmd interface to the "md" driver

 

They are

...

 

Thanks for all the info, a lot to digest for a windows programmer.  I'll try to plow through it this weekend. WeeboTech, if you think you will be updating SPINCONTROL, please let me know and I may just wait for you to do your magic before jumping in myself.

 

Actually taking a second, closer look at it, it's a lot more straightforward than I thought at first glance. Just need to fashion a TCP/IP interface to those commands.

 

I'll take a look at my script and update it with the new interface.

 

It might be quicker to expose the mdcmd via inetd or an unmenu plugin and let the client program have the intelligence to decide what to do.

Link to comment

I'll take a look at my script and update it with the new interface.

 

It might be quicker to expose the mdcmd via inetd or an unmenu plugin and let the client program have the intelligence to decide what to do.

 

Thanks.  Also thanks for pointing out inetd, that's a handy tool I'll take a closer look at.

Link to comment

I'll take a look at my script and update it with the new interface.

 

It might be quicker to expose the mdcmd via inetd or an unmenu plugin and let the client program have the intelligence to decide what to do.

 

Thanks.  Also thanks for pointing out inetd, that's a handy tool I'll take a closer look at.

 

Quick -n- dirty way of exposing the mdcmd via tcpip.

Note this does very little in the way of error checking the string entered.

It takes no regard to permissions, it's just a way of playing for now.

This really would be better done with a c program that inspected the input and write directly to the /proc/mdcmd file.

In any case it's an interesting experiment.

 

 

#!/bin/bash

trap "rm -f /tmp/mdcmd.$$" EXIT HUP INT QUIT TERM PIPE

# Define to 1 if you want to exit after each command sent
# EXITAFTERREPLY=0
# EXITAFTERREPLY=1

while read -t30
do
    REPLY=`echo $REPLY| tr -d '\000-\010\013-\037\041-\056\072-\100\133-\140\173-\377'| tr A-Z a-z`
    # echo "==>'$REPLY'"
    # echo "$REPLY" | od -x

    [ -z "${REPLY}" ] && exit

    case "${REPLY}" in 
    exit|EXIT|quit|QUIT|"" ) exit;;
    esac

    echo "${REPLY}" > /proc/mdcmd
    cat /proc/mdcmd > /tmp/mdcmd.$$
    cat < /tmp/mdcmd.$$
    rm -f /tmp/mdcmd.$$
    [ "${EXITAFTERREPLY:=0}" -gt 0 ] && exit
done

 

inetd segment.

9090 stream tcp nowait nagios /usr/sbin/tcpd /tmp/mdcmd.sh

 

After changing the inetd.conf file

do the following

root@Atlas /tmp #ps -ef | grep inetd

root      1361    1  0 Feb16 ?        00:00:00 /usr/sbin/inetd

 

Then do a kill -1 to reload the inetd.conf file

root@Atlas /tmp #kill -1 1361

 

Link to comment
  • 4 months later...

I haven't had time to play with this until last night. But I took Weebotech's advice and wrote a C program to receive remote network commands that can be plugged into inetd. I've attached the code below for anyone interested. Basically, it allows you to open a UDP connection and send commands like "disk3\n" to the port configured in inetd.conf and it will spin up that disk by calling mdcmd. The only valid commands are "disk1\n" ... to ... "disk19\n" and "exit\n", it will just ignore anything else it receives.

 

I've had to implement a slight pause (250ms, although shorter may work as well) between commands in my client or else it only responds to the last one, I think maybe because I am using UDP (per Wikipedia to use a single server instance). It may also work if you use TCP (perhaps without a pause), but I haven't tested it.

 

Anyways, for posterity, here is mdcmd_inetd.c:

 

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <syslog.h>
#include <string.h>

int main(int argc, char **argv)
{
  char p[4096];
  openlog("mdcmd.inetd", LOG_PID|LOG_CONS, LOG_USER);
  //inetd passes its information to us in stdin.
  while(fgets(p, sizeof(p), stdin))
  {
    // Remove the newline from the string
    p[strlen(p)-1] = 0;
    if (strcmp(p, "exit") == 0)
    {
      syslog(LOG_INFO, "Received exit request");
      break;
    }
    if ((strlen(p) < 5) || (strlen(p) > 6))
    {
      continue;
    }

    if (strncmp(p, "disk", 4) != 0)
    {
      continue;
    }

    int disk = atoi(p+4);
    if ((disk < 0) || (disk > 19))
    {
      continue;
    }

    // Fork and execute the command to spin up the disk.
    if (!fork())
    {
      // Child
      char buf[256];
      sprintf(buf, "Spinning up %s", p);
      syslog(LOG_INFO, buf);
      closelog();
      execl("/root/mdcmd", "/root/mdcmd", "spinup", p+4, (char *)0);
      _exit(0); // Shouldn't be needed.
    }
  }

  closelog();
  exit(0);
}

Link to comment

nice and simple, your using UDP or TCP?

What's the line you put in inetd.conf?

 

also the

_exit(0); // Shouldn't be needed.

customarily is _exit(127);

 

>> or posterity, here is mdcmd_inetd.c:

I would call it mdspinup_inetd.c since you're only exposing the spinup interface.

This interface is safer, as it does not expose the rest of the mdcmd interface.

 

Good job.

Link to comment

Thanks for the tips, it's been 10+ years since I've done any C programming or Unix programming. A lot of it came back easily but I'm not surprised some details (like _exit(127)) have left the building.

 

Line for /etc/services:

mdspinup           15490/udp # mdspinup server, portnumber/udp

 

Line for /etc/inetd.conf:

mdspinup dgram udp wait root /boot/custom/mdspinup_inetd/mdspinup.inetd mdspinup.inetd

Link to comment

My client is just a few lines of test code in a C# application I am working on, it can do TCP just by replacing UdpClient with TcpClient and then changing c.Send to c.Client.Send.

 

            UdpClient c = new UdpClient("watchtower", 15490); // host, port
            Byte[] b = Encoding.ASCII.GetBytes("disk3\n");
            c.Send(b, b.Length);
            Thread.Sleep(250);
            b = Encoding.ASCII.GetBytes("disk7\n");
            c.Send(b, b.Length);

 

I thought Wikipedia said that inetd would spawn a new server for every TCP request though which didn't seem as efficient using a single server for UDP. Maybe I misunderstood.

Link to comment

I thought Wikipedia said that inetd would spawn a new server for every TCP request though which didn't seem as efficient using a single server for UDP. Maybe I misunderstood.

 

Thanks, now you're teaching me.

 

Yes inetd would spawn a new tcp client per request. As far as it being inefficient, with the amount of use this is going to be getting, It's nothing to worry about.

At the most it would be 20 or so requests. 

Although fork is an expensive system call, it's small and for a short period of time with an upper limit (unless someone is trying to cause a denial of service).

A reason to use TCP vs the UDP is that you can telnet to the port or use something like netcat, socat and others.

 

Since you have inetd running an external program -> /boot/custom/mdspinup_inetd/mdspinup.inetd

It's still forking and running an external program. With UDP It will queue up other requests until the internal buffer fills up then throw them away.

 

With UDP on the client side, if the server is not up, the client will not hang. The message will go out on then network to a recipient that does not exist and the message will die on the network.

 

For this purpose, I think you're fine. If you had multiple machines sending these UDP control messages, then there may be a problem if one of the connections hangs for a while.

Link to comment
  • 10 years later...

It is now 2020 (almost 2021) and I am running on Unraid 6.8.3.  Many years ago (unraid 4.x)  I would send a TCP command to port 80 that looked like (without the clauses):

 

GET /update.htm?cmdSpinupAll=Spin%20Up HTTP/1.1  ... (plus clauses)

 

It returned a lot of web stuff but a status of 200 and the first thing in the body was "e81" which I assume is some sort of an error message.
That command no longer works, the drives do not spin up. 

 

This thread describes alternate ways of doing this.  Do the methods described in this thread still work?

Would anyone be willing to write a plugin that would simply install the necessary scripts and set up such that I could send reasonable commands over TCP to the unraid IP at some port.

 

I am a decent programmer but have no experience with C, Linux, Bash. I am 81 years old with little patience, but do enjoy coding . I rely on the unraid system for all my important storage and all my movies (MKV files) for my theater.  The theater runs on a PC using a DuneHD movie files (.mkv) for rendering with a movie librarian I wrote and modeled after the Kelaidescape system.   I have had Unraid system up times measured in years and do not wish to screw up the Unraid system.  Security is not a major issue as the house is well protected and has almost nil access from the Internet.

 

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.