How to add stuff to the powerdown script

[MortenSchmidt]

Success!!! Gotta love it when stuff finally works 

[WeeboTech]

If you are in the mood to add things... we need to check for files (on the protected array) mounted as loop mounted file systems.

 

They will show up in the "mount" command, but do not show up as processes, or as open files, but if mounted on an unRAID disk that is in the array, they will keep the array from stopping.

 

Something like this might work:

mount | grep "loop=" | sed -n "s/.* on \(.*\) type .*/\1/p" | xargs umount

It will find the entries mounted as loop types and umount them

 

My initial tests show this does not quite always work and it may provide errors if there are no loop mounted filesystems.

I have this snippet to check for mounted loop filesystems

 

#!/bin/bash 

TMPFILE=/tmp/tmpfile.$$
trap "rm -f $TMPFILE" EXIT HUP INT QUIT

mount | grep "loop=" | sed -n "s/.* on \(.*\) type .*/\1/p" > $TMPFILE 
while read FS
do export LOOPFS="${LOOPFS} $FS"
done < $TMPFILE

echo $LOOPFS
# umount -v $FS

 

what i'll do is use this to build a LOOPFS list.

 

    logger "Killing active pids on the array drives"
    for fs in ${LOOPFS} /mnt/user /mnt/disk* 
    do  [ ! -d ${fs} ] && continue
        for signal in TERM TERM KILL
        do  for pid in $(fuser -cu $fs 2>/dev/null)
            do  ps --no-headers -fp ${pid}
                if kill -0 ${pid} 2>/dev/null
                   then kill -${signal} ${pid}
                fi
            done
            [ ! -z "${pid}" ] && sleep 1
        done
    done 2>&1 | logger

   
    logger "Umounting the drives"
    for disk in ${LOOPFS} /mnt/disk*
    do  /bin/umount -v ${disk}
    done 2>&1 | logger

 

 

I've added the mv for the /usr/local/sbin/powerdown command in the powercontol slackware package.

Although, I really feel people should just call it by the full command name so there is no ambiguity.

 

 

I just need to work on the code to run every script with parameters in the /boot/custom/etc/rc.d/unraid.d directory.

 

I was thinking to take out the diagnostic dump and syslog save segments. then add them into the /boot/custom/etc/rc.d/unraid.d  directory.

 

This keeps rc.unRAID smaller.

 

Thoughts? comments?

 

 

[purko]

I just need to work on the code to run every script with parameters in the /boot/custom/etc/rc.d/unraid.d directory.

Please don't think about adding any complexity into this one.

Make my day, and make it as simple as...

/boot/custom/etc/rc.d/unraid.d/rc.* stop &
sleep 5

Put that early in your powerdown script, then go about your other business.

 

[BRiT]

Why have the 'sleep 5' in the stop scripts at all?

 

[WeeboTech]

I just need to work on the code to run every script with parameters in the /boot/custom/etc/rc.d/unraid.d directory.

Please don't think about adding any complexity into this one.

Make my day, and make it as simple as...

/boot/custom/etc/rc.d/unraid.d/rc.* stop &
sleep 5

Put that early in your powerdown script, then go about your other business.

 

 

Think I'm a little too late for this response.

Besides, that command will not be executed properly.

 

Here is what I have so far for the new RCDIR script directory. (Which is configurable at install time or via a static conf file)

 

 

runparts()
{

   for script in $1/*[^~,] ; do
       [ -d "${script}" ]     && continue
       [ ! -x "${script}" ]   && continue
       [ "${script%.swp}" != "${script}" ] && continue
       [ "${script%.bak}" != "${script}" ] && continue
       [ "${script%,v}"   != "${script}" ] && continue

       logger  -t"$P[$$]" "Running: $script, arg: $2"

       dos2unix "${script}" > "${TMPFILE}.sh"
       chmod u+rwx "${TMPFILE}.sh"
       exec 3>"${TMPFILE}.log"
       "${TMPFILE}.sh" ${2} >&3 2>&3
       RC=$?
       logger  -t"$P[$$]" "Ended: RC: ${RC}"
       exec 3>-
       rm -f "${TMPFILE}.sh" "${TMPFILE}.log"
   done

}

[Joe L.]

I just need to work on the code to run every script with parameters in the /boot/custom/etc/rc.d/unraid.d directory.

Please don't think about adding any complexity into this one.

Make my day, and make it as simple as...

/boot/custom/etc/rc.d/unraid.d/rc.* stop &
sleep 5

Put that early in your powerdown script, then go about your other business.

 

 

I don't think that syntax will work..

If you had rc.script1 and rc.script2 and rc.script3 and invoked the command as you illustrated it would expand in the shell to:

rc.script1 rc.script2 rcscript3 stop &

 

The rc.script1 script would get all the arguments, probably not stop since it is looking for $1 as the "stop" command

None of the others would even be invoked.

 

You really need something like

for i in /boot/custom/etc/rc.d/unraid.c/rc.*

do

  $i stop &

done

wait

 

That way, each will be called in turn.

You'll also need a "wait" at the end to wait for all the processes you started to complete their tasks, otherwise, the shutdown would continue while they were trying to cleanly terminat their affiliated process, but before they were finished.

 

Joe L.

 

[WeeboTech]

Do we really want to fork all those commands in the background?

That could be a heavy load all at once. If memory is short for any reason, the system will crash.

In addition I would like the ability to capture the start/stop to syslog for review later.

see my code snippet below. It just does not have the script output captured yet.

 

[purko]

Why have the 'sleep 5' in the stop scripts at all?

 

Some scripts may need a few seconds to stop their stuff properly.

RTorrent comes to mind: the spec says it will be fully gone within 5 seconds of receiving a kill -INT signal.

So the powerdown would fire up all stop scripts simultaneously, then wait a few seconds before umounting disks and stuff from underneath them.

 

Another way would be to simply do...

/boot/custom/etc/rc.d/unraid.d/rc.* stop

...and let all of them finish in sequence. But I don't see much benefit in that, and the time penalty will be heavy if you have too many rc. scripts there, each of them with some wait of their own. Besides, some crappy script may get stuck and never finish, which will get Weebo's powerdown script stuck. I don't want the powerdown script to be worrying about anything like that. So running them all in backgraund, and just giving them a few seconds before we proceed is the perfect way to do that.  If my package is so stupid to fail to kill itself nicely, it will be killed later anyway.

 

When I requested such directory I did not intend to dwell on whether the stuff in in will work properly.

My intention is to give packages a way and a chance to do their own 'nice' stopping, if they care to take advantage of such a chance.

 

[WeeboTech]

...and let all of them finish in sequence. But I don't see much benefit in that, and the speed penalty will be heavy if you have too many rc. scripts there. Besides, some crappy script may get stuck and never finish, which will get Weebo's powerdown script stuck. I don't want the powerdown script to be worrying about anything like that. So running them all in backgraund, and just giving them a few seconds before we proceed is the perfect way to do that.  If my package is so stupid to fail to kill itself nicely, it will be killed later anyway.

 

When I requested such directory it I did not intend to dwell on whether the stuff in in will work properly.

My intention is to give packages a way and a chance to do their own 'nice' stopping, if they care to take advantage of such a chance.

 

This point has merit. Let me think about it s'more.

Purko do you have a google account and do you have svn installed anywhere?

[Joe L.]

Do we really want to fork all those commands in the background?

That could be a heavy load all at once. If memory is short for any reason, the system will crash.

In addition I would like the ability to capture the start/stop to syslog for review later.

see my code snippet below. It just does not have the script output captured yet.

 

Very true... I was just using syntax similar to what he had initially posted.  I prefer shutting down one at a time.  I like your code, but I'd restrict it to files starting with "rc."  

Is there a reason you are processing other files?

 

I like the elimination of the backup copies, etc.  Take a look at the script used by cron.hourly in

/usr/bin/runparts

It uses a similar concept for the scripts dumped in /etc/cron.hourly but also excludes files you did not.

 

Joe L.

[Joe L.]

...and let all of them finish in sequence. But I don't see much benefit in that, and the speed penalty will be heavy if you have too many rc. scripts there. Besides, some crappy script may get stuck and never finish, which will get Weebo's powerdown script stuck. I don't want the powerdown script to be worrying about anything like that. So running them all in backgraund, and just giving them a few seconds before we proceed is the perfect way to do that.  If my package is so stupid to fail to kill itself nicely, it will be killed later anyway.

 

When I requested such directory it I did not intend to dwell on whether the stuff in in will work properly.

My intention is to give packages a way and a chance to do their own 'nice' stopping, if they care to take advantage of such a chance.

Good point about one script possible preventing the others from running.

In that case, a fixed sleep at the end (to give them some time to complete) and everything running/stopping in the background might be the answer.

 

Oh yes... good to hash out the ideas like this.  Helpful for all involved.

[WeeboTech]

I prefer shutting down one at a time.  I like your code, but I'd restrict it to files starting with "rc." 

Is there a reason you are processing other files?

 

I like the elimination of the backup copies, etc.  Take a look at the script used by cron.hourly in

/usr/bin/runparts

It uses a similar concept for the scripts dumped in /etc/cron.hourly but also excludes files you did not.

 

Joe L.

 

I can change my code to be configurable like run-parts.

I can also change the stop segments to do the runparts block in the background and wait a configurable time period.

[purko]

runparts()
{

   for script in $1/*[^~,] ; do
       [ -d "${script}" ]     && continue
       [ ! -x "${script}" ]   && continue
       [ "${script%.swp}" != "${script}" ] && continue
       [ "${script%.bak}" != "${script}" ] && continue
       [ "${script%,v}"   != "${script}" ] && continue

       logger  -t"$P[$$]" "Running: $script, arg: $2"

       dos2unix "${script}" > "${TMPFILE}.sh"
       chmod u+rwx "${TMPFILE}.sh"
       exec 3>"${TMPFILE}.log"
       "${TMPFILE}.sh" ${2} >&3 2>&3
       RC=$?
       logger  -t"$P[$$]" "Ended: RC: ${RC}"
       exec 3>-
       rm -f "${TMPFILE}.sh" "${TMPFILE}.log"
   done

}

 

None of this should be any of the powerdown's script business.

If my script there is 'rc.mypackage', you should execute it with a 'stop' parameter.

Why should you make me name it .sh or .awk or something?

If I want to sometimes chmod it to non-executble, that's my right. Why would you be chmoding stuff?

And, regular users don't edit scripts in that directory, my installer puts them there. So why the 'dos2unix'?

 

See what I mean that you shouldn't try to add any complexity into this?  

Just sweep in one move whatever cares to exist there, then move on, without further thought.

 

[WeeboTech]

None of this should be any of the powerdown's script business.

If my script there is 'rc.mypackage', you should execute it with a 'stop' parameter.

It does.

 

Why should you make me name it .sh or .awk or something?

I don't look closer. I dos2unix the file into /tmp, chmod it then execute it.

 

If I want to sometimes chmod it to non-executble, that's my right. Why would you be chmoding stuff?

I only chmod stuff that was already executable.

 

And, regular users don't edit scripts in that directory, my installer puts them there. So why the 'fromdos'??

 

Just in case someone does.

 

See what I mean that you shouldn't try to add any complexity into this?

Just sweep in one move whatever cares to exist there, then move on, without further thought.

 

There has to be some thought.

the only protections(restrictions) are.

1. the file must be executable already.

2. the file cannot end with extensions that should not be run.

3. the file is run in dos2unix as a safeguard as does unraid during the normal startup.

 

The only point I may be spending too much time on is capturing the log.

Perhaps I should simplify that.

The other parts work just like runparts does in /usr/bin.

 

Joe voted for only allowing execution of files named rc.*

Other opinions are welcome.

[Joe L.]

runparts()
{

   for script in $1/*[^~,] ; do
       [ -d "${script}" ]     && continue
       [ ! -x "${script}" ]   && continue
       [ "${script%.swp}" != "${script}" ] && continue
       [ "${script%.bak}" != "${script}" ] && continue
       [ "${script%,v}"   != "${script}" ] && continue

       logger  -t"$P[$$]" "Running: $script, arg: $2"

       dos2unix "${script}" > "${TMPFILE}.sh"
       chmod u+rwx "${TMPFILE}.sh"
       exec 3>"${TMPFILE}.log"
       "${TMPFILE}.sh" ${2} >&3 2>&3
       RC=$?
       logger  -t"$P[$$]" "Ended: RC: ${RC}"
       exec 3>-
       rm -f "${TMPFILE}.sh" "${TMPFILE}.log"
   done

}

 

None of this should be any of the powerdown's script business.

If my script there is 'rc.mypackage', you should execute it with a 'stop' parameter.

Why should you make me name it .sh or .awk or something?

Look closer. He is not.

If I want to sometimes chmod it to non-executble, that's my right. Why would you be chmoding stuff?

Look closer.  If a script in the directory is not executable it will be skipped.

And, regular users don't edit scripts in that directory, my installer puts them there. So why the 'fromdos'??

I know it, and you know it, but who know who will put something there.  It is just being proactive.

See what I mean that you shouldn't try to add any complexity into this?  

Just sweep in one move whatever cares to exist there, then move on, without further thought.

Yes, the "fromdos" stuff can probably be left off, but once you pass the script through it you must either make it executable or invoke a shell with it as the parameter.  That's what those commands are doing.  The appending of a .sh to the temp file in /tmp is just a frill.  It does not hurt anything.

 

The complexity is to keep a single badly edited/written script from stopping the whole process.  We have a lot of unix newbies out there.  

 

Joe L.

[Joe L.]

Joe voted for only allowing execution of files named rc.*

Other opinions are welcome.

Reason is... if it does not have a name like rc.* it will probably also not understand the "stop" parameter you will be giving it as an argument.  It then would run, perhaps "starting" instead.

 

I'd even go so far as to suggest if you cannot grep the word "stop" in the file, skip it too in the shutdown sequence.

[purko]

I know it, and you know it, but who know who will put something there.  It is just being proactive.

 

That's probably how Limetech thinks, as they still haven't given us any post_array_start and pre_array_stop hooks!

 

The complexity is to keep a single badly edited/written script from stopping the whole process. 

 

That is why I suggested that you don't wait for them, just give them a few seconds.

 

We have a lot of unix newbies out there.

 

Those 'unix newbies' are free to crash their server as they please.  You can't stop them from doing that.

 

[purko]

How about

/boot/custom/etc/rc.d/unraid.d

Will that work for you as the directory for the add-on rc.* scripts that can handle "start" and "stop" arguments (Y/N)?  (and anyone else who wants to add their ideas)

 

This works well.

 

Guys, something just dawned on me. I hope it's not to late to open this.

 

It just occured to me that from the very begining, while I was talking about directory like "/boot/custom/etc/rc.d/...", my mind was working in the ramfs, "/custom/etc/rc.d/...", and not on the flash key!  I can't believe my mind and my typing hands miscommunicated so bad!   

 

Do we want this thing on the flash key?  I think not!  Any scripts that get put in that place, they get put there by package installers upon boot.

 

If the user removes a certain package from their 'go' script, that package's script won't be in the "/custom/etc/rc.d/..." directory next time they boot.  That's the most logical way to do this.  We really don't want to write all that stuff to the flash key upon every boot.  And, normally there's no reason for users to access those scripts.

 

So think about this please, and let's pick a place on the ramfs for our custom .../rc.d/... directory.

 

[WeeboTech]

If the user removes a certain package from their 'go' script, that package's script won't be in the "/custom/etc/rc.d/..." directory next time they boot.  That's the most logical way to do this.   We really don't want to write all that stuff to the flash key upon every boot.  And, normally there's no reason for users to access those scripts.

 

So think about this please, and let's pick a place on the ramfs for our custom .../rc.d/... directory.

 

 

The modified rc.unRAID script has a configurable RCDIR.

I can change it to /etc/rc.d/unraid.d rather then /boot/custom/etc/rc.d/unraid.d

 

Is this what is being asked for?

[Joe L.]

How about

/boot/custom/etc/rc.d/unraid.d

Will that work for you as the directory for the add-on rc.* scripts that can handle "start" and "stop" arguments (Y/N)?  (and anyone else who wants to add their ideas)

 

This works well.

 

Guys, something just dawned on me. I hope it's not to late to open this.

 

It just occured to me that from the very begining, while I was talking about directory like "/boot/custom/etc/rc.d/...", my mind was working in the ramfs, "/custom/etc/rc.d/...", and not on the flash key!  I can't believe my mind and my typing hands miscommunicated so bad!   

 

Do we want this thing on the flash key?  I think not!  Any scripts that get put in that place, they get put there by package installers upon boot.

 

If the user removes a certain package from their 'go' script, that package's script won't be in the "/custom/etc/rc.d/..." directory next time they boot.  That's the most logical way to do this.   We really don't want to write all that stuff to the flash key upon every boot.  And, normally there's no reason for users to access those scripts.

Good point.  No need dealing with a rc.* script on the flash drive when it has not even been installed. (or the package is no longer being installed)

 

I did not consider that... so I think you are right, it really should be in the ram file system, not the flash drive itself.

So think about this please, and let's pick a place on the ramfs for our custom .../rc.d/... directory.

I think that /etc/rc.d/unraid.d is as good a place as any.  You've convinced me the original idea I had was not the best place for these rc.* scripts we add.

 

There are some install packages I know will install their own rc.* script in the /etc/rc.d directory.  When I installed the slackware mysql packace it put a rc.mysql script in /etc/rc.d.  We'll need to have a supplemental rc.mysql script in /etc/rc.d/unraid.d to call the "stop" event in /etc/rc.d/rc.msql.  Otherwise, the array won't stop and the one in /etc/rc.d won't ever be reached.  By the time it (in /etc/rc.d) is invoked, the database will already be shut down.

[purko]

If the user removes a certain package from their 'go' script, that package's script won't be in the "/custom/etc/rc.d/..." directory next time they boot.  That's the most logical way to do this.   We really don't want to write all that stuff to the flash key upon every boot.  And, normally there's no reason for users to access those scripts.

 

So think about this please, and let's pick a place on the ramfs for our custom .../rc.d/... directory.

 

 

I can change it to /etc/rc.d/unraid.d rather then /boot/custom/etc/rc.d/unraid.d

 

Is this what is being asked for?

 

Yes. Thank you.

 

The modified rc.unRAID script has a configurable RCDIR.

This I don't understand.  Why would you want this?

You simply decide, and say: There will be a directory here, and any script that gets put there will be run this way by the powerdown script. That's all.

 

My package will run on a user's computer which may or may not have the powerdown script installed at that time.  

That's why the powerdown hook location should be preagreed  upon, and not moved around or configured.  

 

When my package installs, with or without a powerdown script present, I'll put my stop script in that preagreed location.

Even if they have a powerdown package in their 'go' script, it may happen to get installed after my package.

 

[WeeboTech]

I will change the script to use the initramfs, in that case, I will not need to do the fromdos conversion.

It is expected scripts on root in /etc/rc.d will be in proper format.

 

As for the supplemental rc.mysql, it need only be linked.

This is how sysv systems do it, files are put in /etc/init.d or /etc/rc.d/init.d then linked to the proper runlevels.

 

on sysv they are named according to the order as in

S##-filename for start scripts and K##-filename for stop scripts.

This way an order can be maintained. I'm not going to move in this direction at the current time.

a simple sweep and execute is all that will be done.

 

The reason I did not balk at using the flash key was for users who create static and specific start/stop scripts. This is something that I do.

 

I do not create a package for everything, but I do create a number of start stop customizations.

I'll leave that for another discussion.

 

As for configuration, I always do that. no one has to know or use it.

I could be useful for testing.

[purko]

I will change the script to use the initramfs, in that case, I will not need to do the fromdos conversion.

It is expected scripts on root in /etc/rc.d will be in proper format.

 

As for the supplemental rc.mysql, it need only be linked.

This is how sysv systems do it, files are put in /etc/init.d or /etc/rc.d/init.d then linked to the proper runlevels.

 

on sysv they are named according to the order as in

S##-filename for start scripts and K##-filename for stop scripts.

This way an order can be maintained. I'm not going to move in this direction at the current time.

a simple sweep and execute is all that will be done.

 

The reason I did not balk at using the flash key was for users who create static and specific start/stop scripts. This is something that I do.

 

I do not create a package for everything, but I do create a number of start stop customizations.

I'll leave that for another discussion.

 

As for configuration, I always do that. no one has to know or use it.

I could be useful for testing.

 

That's all I needed.

 

Thank you.

 

[Joe L.]

I will change the script to use the initramfs, in that case, I will not need to do the fromdos conversion.

It is expected scripts on root in /etc/rc.d will be in proper format.

That will make it simpler.

As for the supplemental rc.mysql, it need only be linked.

This is how sysv systems do it, files are put in /etc/init.d or /etc/rc.d/init.d then linked to the proper runlevels.

That works for me.

The reason I did not balk at using the flash key was for users who create static and specific start/stop scripts. This is something that I do.

 

I do not create a package for everything, but I do create a number of start stop customizations.

I'll leave that for another discussion.

 

As for configuration, I always do that. no one has to know or use it.

I could be useful for testing.

We are working two different, but related issues.

 

1. How to cleanly stop/start process when the array being stopped/started.

 

2. How to initially install and start add-on processes (either on demand, or upon reboot)

 

Both potentially involve rc.* scripts.  The directory /etc/rc.d/unraid.d can hold the rc.* scripts for installed custom packages, or links to those in the /etc/rc.d directory is we install a pre-compiled package snagged from the web.

 

Those scripts in /etc/rc.d/unraid.d can be processed in turn as described already, eventually with triggers from unRAID 5.0.  In the interim, I'm fairly confident we can write a script that provides the triggers. (I already sent WeeboTech a proof-of-concept of such a script)

 

unRAID is different than systems installed to a hard-disk in that we do not un-install a package, we simply elect to not install it the next time we reboot.  In that way, the /etc/rc.d (and /etc/rc.d/unraid.d) directories will only have installed packages.

This would not be the case, as purco pointed out, if we put the rc.* scripts on the flash drive.  Then we would end up with scripts with potentially nothing for them to control (since we did not re-install it on re-boot)

 

The /boot/packages directory is being scanned for *.auto_install scripts now.  Something like it is needed regardless of where we put it.  To me, having the *.auto_install files in the same folder as the packages they were installing made it much easier on the linux newbies.    I think they should stay there, unless I hear a good reason why they should not.  These are not rc.* scripts, but scripts of commands to perform the actual initial installation, which might put into place an rc.* script in either /etc/rc.d, or in /etc/rc.d/unraid.d after doing the appropriate installation.  It also makes it easier to manage, regardless of the package-manager screens we use. (I strongly suspect the unMENU package-manager will be replaced with one written in ruby/sinatra, once we get that far.  The package.conf files can still be used as they are not language specific.  All a package-manager need do is impliment the same logic as the unMENU one to parse and read their contents.

 

So... two different, but related issues.  One, the initial installation to a initfs file-system upon reboot, the other the routing start/stop when the array is started/stopped.

 

Joe L.

[purko]

Joe, let's stay focused here. In this discussion we are talking about one very narrow topic: a powerdown hook provided by the powerdown package. That's all. We need not involve unmenu stuff here.

 

I am very interested in the larger discussion, about a triggering program that will execute all scripts in a certain directory upon certain interesting events. For that we should start a new thread.