unMENU plug-in: Go Script Manager

[JarDo]

Would you like me to remove the backup go file feature in future versions?

 

Interesting question.  My thoughts:

 

I'm only a user/tester/commentator and I'm not actively involved with helping develop the Go Script Manager on a technical level.  Joe, I think comes closest to being that type of contributor to your project.  Having said that I don't think I have a place to say Yes or No to that question.

 

However, (Uh, Ohh...  Here comes the opinion )it would seem to me that since the same functionality does already exist within unMenu if you did remove it from the Go Script Manager nothing would be lost for unMenu users and it would remove some complexity from your code and make it that much easier for you to develop/maintain/improve the unique functionality that the Go Script Manager brings to unMenu.

 

It also might remove potential confusion for unMenu users.  Reading how you and Joe have been working together with ideas and such, I was thinking that your go script backup had to be different from the functionality that Joe's plugin provided.  I thought maybe your backup included information about the go scripts that I had enabled.  But, I wasn't sure and couldn't verify for myself because the backup wasn't working correctly for me.  That's why I was asking the question.  I was trying to understand.

 

As to order of execution for the go scripts.  Would there be a way to append a sequence number based on the order in which the go scripts were enabled???  I like how your conf files don't have numbers pre-appended to them.  That makes them more generic.  This way the user could decide the best order for their setup via your plugin.  All a user would have to do to change the order of execution would be to disable all of the availble scripts and then re-enable in the preferred order.

 

Heck, if you could figure out how to pull off that bit of magic with the Go Script Manager, maybe Joe could do the same thing with the package manager.

[Joe L.]

You are correct, but what I was envisioning is that people may make additions/modifications to the go script manually because a package did not exist yet for them to use and did not want to make one themselves (think of a new tweak/user mod that may not have a .conf package for the package manager or the go script manager yet). 

 

Isn't this what Joe's "Config View/Edit" unMenu plugin is for?

Yes, but in fairness, I had the code written and working(v0.4 released Dec 26th) BEFORE that plug-in was available (Dec 28th or 29th), it made the feature obsolete, I guess.  I left it in because I still felt the option should be there for people who might have edited the go file manaully with a text editor and want to back it up.

 

As of right now, I can't easily link to the config view/edit plugin without causing formatting problems on the go script manager.  I was hoping that when Joe L. wrote the libraries that I might be able to do away with the backup button by giving them a place to go to edit the file and leave that stuff up to the config view/edit plugin libraries but can't do that easily yet.  At the very least I can't do it without taking Joe L's code and re-writting it (either like I did in the "unMENU Plugin: Config View/Edit" thread OR by adding it directly to my own).

 

Would you like me to remove the backup go file feature in future versions?

 

Cheers,

Matt

Yes, the work that Biggy2872 did, to create a way to back up the "go" script, did pre-date the Config View/Edit plug in I was withing concurrently.  He was not aware of my effort, nor was I aware of his.  He did publish his before I did mine.

 

There is some overlap in capability.

 

I think the backup of the "go" script is now better covered by my Config View/Edit plug-in.  I don't say this because I wrote it, but because of my background in human factors engineering of web-sites and computer systems.  All edited scripts, and their backups, are handled in exactly the same way, and through the same screen.

 

This is not the first time a screen has changed to incorporate suggestions, it happened when my original syslog viewer was changed by another contributor to color code lines, and when I completely re-wrote my original screen to incorporate plug-ins.

 

I expect things to evolve, I know the myMain plug-in is better than my own "Main" link... it does not yet have all the same capabilities with drives outside of the array, but as soon as it does I'll get rid of one of the links at the top of my own unmenu page.  It is impossible for any of us to envision everything, or to think of all the potential features, but that is exactly why it is a great rapid prototyping environment, and together we form a great design team.    At least one person in the past has said "Don't be afraid to throw the first version away once it is complete.  It is at that point you will better know how you should have done it differently."

 

I  agree that the removal of a section of Biggy2872's plug-in dealing with backup of the "go" script could reduce his plug-in complexity. As a bonus it will make Biggy2872's plug-in almost identical in how it is used as the package manager.  This makes it much easier on users of both.  (A very old unix methodology, each program should do one thing well. Now, you could argue that emacs does not follow that methodology, most programs do.)

 

I do like the idea you presented about prepending a number to the front of the "auto_install" scripts to force the order of execution.  If you re-name them yourself it will do exactly as you said... it will execuite them in the order they show in an "ls" command.  There might be an easier method to re-order them though... Your thought about running them in the order in which they are "enabled" will work, but might be tricky over time when you just want to add one take in the middle.

 

I'm thinking of a series of fields on the screen with "^" and "v" buttons to change the order.  This will make it lots easier for all.

It might just be a future enhancement...

 

Now, I've been too tied up with things around here to make the routine Biggy2872 described into a library...  (lately been re-machining an old Server case from the 8086 days to fit a new unRAID server.  It was so old it had a "turbo" switch to change speed down to "33 Mhz" for the programs that would not run at higher speeds...  The turbo switch is now gone... Now I'm building a drive rack.  It has 4 120mm fans covering the most of the entire front panel of the server.) Hardware/software/firmware... it is all fun.

 

Joe L.

[Biggy2872]

I'm wondering if I could get some feedback...

 

What about standardizing the scripts that get placed in /boot/custom/etc/rc.d?  We could create a new topic to track all the scripts people use, develop standard names and by use the order of these files to determine a boot roder for enabled scripts.

 

For instance, the package manager uses the S10-install_custom_files, I saw someone using the S08-update_hosts file for the /etc/hosts netweork tweak.

 

By standardizing these scripts and their names we are already defining the boot sequence and the go script manager would not need that information from a user.  I could then provide options for moving undefined (in terms of boot order) user added scripts up and down the list of pre-determined scripts(commonly used) boot order.

 

thoughts?

 

Matt

 

[JarDo]

If the Go Script Manager already had this capability:

 

I could then provide options for moving undefined (in terms of boot order) user added scripts up and down the list of pre-determined scripts(commonly used) boot order.

 

Then I don't see why we'd need to go through this:

 

What about standardizing the scripts that get placed in /boot/custom/etc/rc.d?  We could create a new topic to track all the scripts people use, develop standard names and by use the order of these files to determine a boot roder for enabled scripts.

 

Am I understanding your suggestion correctly?

[Biggy2872]

If the Go Script Manager already had this capability:

 

I could then provide options for moving undefined (in terms of boot order) user added scripts up and down the list of pre-determined scripts(commonly used) boot order.

 

Then I don't see why we'd need to go through this:

 

What about standardizing the scripts that get placed in /boot/custom/etc/rc.d?  We could create a new topic to track all the scripts people use, develop standard names and by use the order of these files to determine a boot roder for enabled scripts.

 

Am I understanding your suggestion correctly?

No I don't think you are...

 

What if John Doe had a tweak that was added a year ago named /boot/custom/etc/rc.d/S02-johndoe_script and I create a package for the go script manager to utilize the same script, i would want to know it is already on the system (by standardizing the file name for said script) and using that filename as the PACKAGE_INSTALLED line from the .conf for instance (not exactly what I have planned but close).  The future version of the go script manager would be able to know that it is there and therefore already enabled... and can even disable, allowing the go script manager to actually manage what you want it to... your booting scripts

 

The people that use the S## files have already developed a boot sequence by the naming of the files... I believe it's inhernet to the filenaming convention.. if the system to organize booting ALREADY exists, why try and program another solution? (same arguments against me having the backup go file button) The sequence can then be controlled at a community level, to avoid someone making the wrong choices for boot sequence and causing unnecessary troubleshooting for experts on the forum.  When new tweaks are developed that need to be added to the booting process then we could determine at that time an appropriate  "S" number to assign it and voila... done, I could make a package that would then use that filename. 

 

Cheers,

Matt

 

[prostuff1]

I like your suggestions Biggy2872.

 

I think there needs to be a standard setup, somehow.  There is a rough one in place but i think something needs to be set in stone and then forced, if someone wants to do it differently then it will be there problem to mess with it.

 

I say we need to start forcing the Third Party Boot Flash Architecture ( http://lime-technology.com/wiki/index.php?title=Third_Party_Boot_Flash_Plugin_Architecture )

 

I'm not sure if it can be done (probably can) but we need a script that can be run that will create this layout for the newbie.  I envision something to the effect of they start up there server, mount the flash share and place the script in /boot, telnet into the tower, navigate to /boot and find the script, run the script that will create the the /boot/packages and so on folders.

 

Personally, I think that /boot/packages should be moved into the /boot/custom/ folder along with the /boot/unmenu folder.  I think we need to leave the root of the flash as clean and as untouched as possible.

 

 

I currently have this setup on my flash:

/boot/packages

/boot/custom

/boot/unmenu

 

I think that it could all be condensed down into /boot/custom

 

I have also adopted using the rc.local_startup file.  This should be the default i think.  In my go scirpt i have the line:

fromdos < /boot/custom/etc/rc.d/rc.local_startup | sh

so that any script i create and place in /boot/custom/etc/rc.d is run though fromdos and any screwups i might have caused with editing files on my mac are fixed.

My rc.local_startup file looks like this:

#!/bin/bash
logger -trc.local_startup -plocal7.info -is "Initiating Local Custom Startup."
scripts=`ls /boot/custom/etc/rc.d/ | egrep -v "rc.local_startup|rc.local_shutdown"`
cd /boot/custom/etc/rc.d
for script in $scripts
do scriptbase=${script##*/}      # Strip pathname
   ( echo "Processing $script" 
fromdos < $script | sh -xv
   ) 2>&1 | logger -t$scriptbase -plocal7.info -is
done

 

In the /boot/custom/etc/rc.d folder i also have a file labeled S10-install_custom_packages with the following code:

cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c 

this gets all of my stuff going that is in /boot/packages.

I realize that i could add the above line directly to my go script but i prefer the fromdos line.

I also have a script in this /boot/custom/etc/rc.d that is my weekly_parity_check.sh (obviously i have this run a parity check every week).  With the way i have this set up that script is run every time the server starts.

 

 

Well, thats about all i can think of right now (probably forgot something).

[Biggy2872]

I also have a script in this /boot/custom/etc/rc.d that is my weekly_parity_check.sh (obviously i have this run a parity check every week).  With the way i have this set up that script is run every time the server starts.

 

In honour of your support for this system, could you please post the contents of your weekly_parity_check.sh.. there are some options in the go script manager which are just variations of others (this one being a variation of the monthly one), I plan on making it so that such variations share the same script name.. for instance selecting the package for a read ahead of 256 will create the same script file (say S02-read_ahead for example) as selecting the option of a read ahead of 2048 and so forth.  I might start a message in here to start collecting them so that maybe we could start the process.

 

Cheers,

Matt

[JarDo]

Hmmm.

 

I guess I just don't have much faith in the success of this:

 

create a new topic to track all the scripts people use, develop standard names and by use the order of these files to determine a boot roder for enabled scripts.

 

I imagine all the same issues that could arise out of pre-pending numbers to the Package Manager conf files.  I was so glad to see those go away.

 

I can only imagine the arguments over 'what is dependent upon what' happening over time.  Specially as different user's needs diverge over time.  Having a standard install sequence based on 'standard' file names might actually create more complexity in the user community as it would remove control from the individual user.

 

Our discussion is a good one.  There is a balance that needs to be reached between assumptions (static values) and variables in an application such as this.  I guess my vote is for variables, and to keep the user in the user interface as much as possible.  If one of the user requirements is an understanding of the sequence number on the conf files, I feel that would distract from the simplicity the Go Script Manager is attempting to offer the user.  At that, I might as well go back to manually editing my go script.

 

I don't want to do that!! 

[Biggy2872]

Hmmm.

 

I guess I just don't have much faith in the success of this:

 

create a new topic to track all the scripts people use, develop standard names and by use the order of these files to determine a boot roder for enabled scripts.

 

I imagine all the same issues that could arise out of pre-pending numbers to the Package Manager conf files.  I was so glad to see those go away.

 

I can only imagine the arguments over 'what is dependent upon what' happening over time.  Specially as different user's needs diverge over time.  Having a standard install sequence based on 'standard' file names might actually create more complexity in the user community as it would remove control from the individual user.

 

Our discussion is a good one.  There is a balance that needs to be reached between assumptions (static values) and variables in an application such as this.  I guess my vote is for variables, and to keep the user in the user interface as much as possible.  If one of the user requirements is an understanding of the sequence number on the conf files, I feel that would distract from the simplicity the Go Script Manager is attempting to offer the user.  At that, I might as well go back to manually editing my go script.

 

I don't want to do that!! 

 

My sytem would not change anything to the .conf filenames.. instead i would propose adding a value to the .confs instead... something like "PACKAGE_RCD_NAME".  Then if the go script manager detects the rc.d structure it knows to look for a file by then name define in the .conf.. if someone wanted to change it in the future, they could edit the .conf with the config view/edit plugin.

 

If someone wasn't using the rc.d structure.. the .auto_goscript file would be installed with the leading S## added to the .conf name.. (S##-unMENUstart-unmenu-goscript.conf.auto_goscript for example) and the boot priority defined by the rc.d structure and standardization would be applied to the go file approach.

 

A balance could be that my idea is the default behaviour but that if you wanted, you could enable a feature to add a value of say "PACKAGE_FORCE_BOOT" or something like that, it would overwrite the "PACKAGE_RCD_NAME" variable by changing the leading S## value.  I could even add interfaces to change the value without the need to directly edit the .conf.

 

thoughts?

Matt

[prostuff1]

 

In honour of your support for this system, could you please post the contents of your weekly_parity_check.sh.. there are some options in the go script manager which are just variations of others (this one being a variation of the monthly one), I plan on making it so that such variations share the same script name.. for instance selecting the package for a read ahead of 256 will create the same script file (say S02-read_ahead for example) as selecting the option of a read ahead of 2048 and so forth.  I might start a message in here to start collecting them so that maybe we could start the process.

 

Cheers,

Matt

 

Sure can

 

Here be my weekly_parity_check.sh script.  It runs every Sunday morning at 1am

 

#!/bin/sh
crontab -l >/tmp/crontab
grep -q "/root/mdcmd check" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
   echo "# check parity every Sun at 1 am:" >>/tmp/crontab
   echo "0 1 * * 0 /root/mdcmd check 1>/dev/null 2>&1" >>/tmp/crontab
   crontab /tmp/crontab
fi

 

It is essentially the same one that Joe put up in another thread modified to run every week, instead of every month.  Given, this may be a little to much, but i figure better safe then sorry.

 

I support the idea of placing an Sxx number in front of the files inside the third party boot architecture layout.  I know some people are using this and I think it should become the standard.  I have not yet gone to doing this but i will be messing with this very soon.  I guess the "great minds" here need to get together and discuss what order for startup would be best and how we want to determine the numbering.

 

 

Maybe we need to create a couple of threads to collect this kind of stuff.  Maybe a:

 

"My scripts (please post yours)" thread - this could be were everyone that has created scripts like my weekly_parity_check.sh could post them.  We could limit the discussion in the thread so that it is not hard to read/get through.  This thread could simplify the wiki a little and make it much easier to update.

 

"Third Party Boot Architecture" thread - obviously this would contain a link to the wiki and would explain why this is the suggested layout.  I know we mention this in a lot of threads but it seems like this topic is always barried and hard to find.  Not to mention when someone reads it they can get confused if they read something different somewhere else. I would like to see a script to create this layout, like i suggested in my last post, as we might have some newbies try to create these folders over SMB which might/can cause problems.  A script that is run would alleviate some of that problem (and shorten the instructions for us).

[Biggy2872]

Sure can

 

Here be my weekly_parity_check.sh script.  It runs every Sunday morning at 1am

 

#!/bin/sh
crontab -l >/tmp/crontab
grep -q "/root/mdcmd check" /tmp/crontab 1>/dev/null 2>&1
if [ "$?" = "1" ]
then
   echo "# check parity every Sun at 1 am:" >>/tmp/crontab
   echo "0 1 * * 0 /root/mdcmd check 1>/dev/null 2>&1" >>/tmp/crontab
   crontab /tmp/crontab
fi

 

It is essentially the same one that Joe put up in another thread modified to run every week, instead of every month.  Given, this may be a little to much, but i figure better safe then sorry.

 

I support the idea of placing an Sxx number in front of the files inside the third party boot architecture layout.  I know some people are using this and I think it should become the standard.  I have not yet gone to doing this but i will be messing with this very soon.  I guess the "great minds" here need to get together and discuss what order for startup would be best and how we want to determine the numbering.

 

 

Maybe we need to create a couple of threads to collect this kind of stuff.  Maybe a:

 

"My scripts (please post yours)" thread - this could be were everyone that has created scripts like my weekly_parity_check.sh could post them.  We could limit the discussion in the thread so that it is not hard to read/get through.  This thread could simplify the wiki a little and make it much easier to update.

 

"Third Party Boot Architecture" thread - obviously this would contain a link to the wiki and would explain why this is the suggested layout.  I know we mention this in a lot of threads but it seems like this topic is always barried and hard to find.  Not to mention when someone reads it they can get confused if they read something different somewhere else. I would like to see a script to create this layout, like i suggested in my last post, as we might have some newbies try to create these folders over SMB which might/can cause problems.  A script that is run would alleviate some of that problem (and shorten the instructions for us).

Well there was already a post on this matter at one point and i have been eyeing it up lately... it might give us a start... see here: http://lime-technology.com/forum/index.php?topic=1953.msg14021#msg14021...

 

So using this as a rough guide we have:

 

S01-blockdev OR S01-set_read_ahead (which do you guys prefer?) *

S03-cpufreq (saved for individual user to use cpu frequency mods if they are supported)

S10-custom_install_packages (as defined by Joe L in package manager) *

S20-init.spincontrol (for drive spin up/down control)

S75-useradds (for adding additonal users)

S80-update_hosts (updating /etc/hosts file) *

S90-smb-shares (for changing values default smb behaviour, will require some work)

S99-syslog-save (for saving the syslog at the end of boot, do we need/want this?)

(*) denotes mods that already have packages...

 

where should starting unmenu fall into this?

well how about S30?

 

what about monthly parity check?

I would say S21... we could use the 20s for cron jobs?

 

Can anyone think of any that should be "locked" and not allow it to be disabled?

Any additons you would like to see added?

 

Input please,

Matt

[JarDo]

OMG

[Biggy2872]

OMG

Care to elaborate?

 

Cheers,

Matt

[dhy8386]

As a newbie to all this, i appreciate what you are trying to do but let me throw out a few points from my simple perspective:

 

1) I think Joes suggestion about having a "up" and "down" arrow to move the packages into the order you would want them installed is an absolute requirement. Right now, there are not an unwieldy amount of packages and scripts but as the project develops, gains momentum, and becomes more user friendly for the average user, you will be faced with 100's of these which will begin to necessitate you having to rename some as new ones come on board. This will require people to change the names or redownload. Rather, if there was a thread which maintained the "Recommended Boot Order", it could easily be updated and people could adjust their systems accordingly with a simple moving of the package up or down.

 

2) To your suggestion, it would also be good to have a thread (actually sticky) that was the Official Packages and Scripts thread. I think the thread needs a moderator or two that has the power to suspend or delete packages/scripts posted that do not conform to the agreed upon format (whatever that may be). Again, I am thinking ahead here and trying to prevent lots and lots of rework as this project expands.

 

3) If we do have an official thread as per 2 above, I could see an easy way to provide package boot order updates would be to have one file which controls this order. That way, users coudl simply download and install (or overwrite) it over the previous one without having to do anything else. The more that is involved which changes, the higher the likelihood of user mistakes and errors=more forum posts for help

 

4) I 100% agree with a naming convention for the boot drive for folders. However, I think there should only be 1 approved method. It does not make sense to have some people doing it one way and some another. This is easily enforceable by having a standardization for packages and scripts (per 2 above) that are written with these naming conventions in mind. That way, if someone tries something different, you can easily point out the error. I also agree that there should be a script file that creates these folders if Tom does not want to levy this as part of the standard install.

 

Maybe this is all obvious but as someone who plays with this kind of stuff all the time, i find it common that the lack of the above often leads to lots of unnecessary troubleshooting and frustration from both the users and those trying to help...

 

I wish I could contribute but have no experience with designing this stuff. May have to start as this is a great project.  

[Biggy2872]

As a newbie to all this, i appreciate what you are trying to do but let me throw out a few points from my simple perspective:

 

1) I think Joes suggestion about having a "up" and "down" arrow to move the packages into the order you would want them installed is an absolute requirement. Right now, there are not an unwieldy amount of packages and scripts but as the project develops, gains momentum, and becomes more user friendly for the average user, you will be faced with 100's of these which will begin to necessitate you having to rename some as new ones come on board. This will require people to change the names or redownload. Rather, if there was a thread which maintained the "Recommended Boot Order", it could easily be updated and people could adjust their systems accordingly with a simple moving of the package up or down.

 

2) To your suggestion, it would also be good to have a thread (actually sticky) that was the Official Packages and Scripts thread. I think the thread needs a moderator or two that has the power to suspend or delete packages/scripts posted that do not conform to the agreed upon format (whatever that may be). Again, I am thinking ahead here and trying to prevent lots and lots of rework as this project expands.

 

3) If we do have an official thread as per 2 above, I could see an easy way to provide package boot order updates would be to have one file which controls this order. That way, users coudl simply download and install (or overwrite) it over the previous one without having to do anything else. The more that is involved which changes, the higher the likelihood of user mistakes and errors=more forum posts for help

 

4) I 100% agree with a naming convention for the boot drive for folders. However, I think there should only be 1 approved method. It does not make sense to have some people doing it one way and some another. This is easily enforceable by having a standardization for packages and scripts (per 2 above) that are written with these naming conventions in mind. That way, if someone tries something different, you can easily point out the error. I also agree that there should be a script file that creates these folders if Tom does not want to levy this as part of the standard install.

 

Maybe this is all obvious but as someone who plays with this kind of stuff all the time, i find it common that the lack of the above often leads to lots of unnecessary troubleshooting and frustration from both the users and those trying to help...

 

I wish I could contribute but have no experience with designing this stuff. May have to start as this is a great project.  

1. I fully plan to add this to the functionality.  It will be a feature that can be toggled on and off for each package.  It will overwrite the boot sequence for a particular file by utilizing a "placeholder" value in the .conf.  this value will change the S## of the installed script but leave the .conf filename alone.  If you wanted to directly edit the name of the file or the boot sequence manually, you can edit the .conf with the Config View/edit plugin as it was designed (for the manual control you were looking for).

I could build an installations script in the future which may help to "update" the .confs to new standards without changing the boot order someone has already chosen independently (by automatically changing the S## in the PACKAGE_RCD_NAME variable and by leaving the placeholder variable untouched (or if it doesn't exitst moving the old sequence number to the placeholder value).

I could add a place to display the recommended S## (from the PACKAGE_RCD_NAME variable)

The placeholder value would be controllable through the up(^) and down(V) buttons we have been discussing.

 

2.  I already have an official post for the .confs utilized by the Go Script Manager in this thread and plan to keep this updated with the official .confs.  Maybe Joe L. could do the same for the package manager, but that is up to him.

 

3. See the update idea above.

 

4. I guess it will be helpful if I outlined my brainstorming process to better explain what I plan on doing...

 

One of the first things the new go script manager will check is if the rc.d structure exists.  if it does not, it will operate closely to the way it does now (minus the backup features... they're gone) in terms of package management + the new way to overwrite boot sequence.

 

If you do have an rc.d structure then it will try to account for all the script in the directory by comparing them to the defined packages in terms of installed .confs.  If it runs into a package it can't associate with one of the .confs, it will create a new .conf for the package that will have a PACKAGE_NAME based on the file name, a stock PACKAGE_DESCR line which will ask the user to edit the package details in the Config View/Edit.  This way at the very least, the script will show up in the go script manager and will be managable by the interface.  The user can then edit the .conf to their liking.

 

I also have some ideas for handling some packages which differ only in values in the script.  This can be seen with the read-ahead scripts.  Instead of having 4 different .confs to cover the one script, I have an idea to combine them and provide radio buttons for selected the value each user wants to utilize.  This will be done by adding one or more PACKAGE_OPTIONS lines to the .confs with the correct format.

 

One thought I had was the need to lock certain .confs into enabled mode, like for instance the S10-custom_install_packages.  You don't want to turn that off by mistake and not have your packages install on next boot.  Who knows, maybe that is a bad example of a particular file (i could see the need to disable the script for troubleshooting), but i think you get the idea.

 

Cheers,

Matt

 

Cheers,

Matt

[Joe L.]

As a newbie to all this, i appreciate what you are trying to do but let me throw out a few points from my simple perspective:

 

1) I think Joes suggestion about having a "up" and "down" arrow to move the packages into the order you would want them installed is an absolute requirement. Right now, there are not an unwieldy amount of packages and scripts but as the project develops, gains momentum, and becomes more user friendly for the average user, you will be faced with 100's of these which will begin to necessitate you having to rename some as new ones come on board. This will require people to change the names or re-download. Rather, if there was a thread which maintained the "Recommended Boot Order", it could easily be updated and people could adjust their systems accordingly with a simple moving of the package up or down.

Of all the scripts mentioned so far, about the only "dependency" I am aware of is for the /mnt/md* drives to be available before attempting to set their read-ahead.  The remaining can be performed in almost any order.  A true dependency would be where a library must be installed before a script using it is run in a second "go" initiated script. 

 

Honestly, I'm not sure I have any like that on my server.  The closest would be the missing c++ library installation, as "smartctl" is used in unmenu.awk.  However, it would not matter if it was installed before or after unmenu.awk starts, as it is only used when reading the drive temperature, or when specifically requesting a SMART report.  There really is no need to specific an installation order for those two tasks.

 

With that in mind, the need to resolve dependencies is far less that just starting up (and shutting down) processes in a controlled manner.

 

2) To your suggestion, it would also be good to have a thread (actually sticky) that was the Official Packages and Scripts thread. I think the thread needs a moderator or two that has the power to suspend or delete packages/scripts posted that do not conform to the agreed upon format (whatever that may be). Again, I am thinking ahead here and trying to prevent lots and lots of rework as this project expands.

You do mean the Unofficial-Official Packages and Scripts thread.  It is a great idea, and would work well if moderated, and in fact exists minimally (for a few packages and scripts) in the wiki. 

 

I am in favor of putting the list in the Wiki, and a locked thread pointing to it, otherwise it is impossible to edit over time a post made by someone else.  I think it will always be "UnOfficial" as Tom @ Lime-Tech has enough to do without trying to figure out if it is "Officially" sanctioned and compatible with everything he is doing or plans do do in the future.

3) If we do have an official thread as per 2 above, I could see an easy way to provide package boot order updates would be to have one file which controls this order. That way, users coudl simply download and install (or overwrite) it over the previous one without having to do anything else. The more that is involved which changes, the higher the likelihood of user mistakes and errors=more forum posts for help

I too like a single file with the names of the scripts to be executed in the order desired.  It is easy to edit via any number of tools/editors.  In its simplest form it would be used as a shell script. One line per command.  If it exists, it is used instead of the ".auto_goscript" and ".auto_install" commands.

4) I 100% agree with a naming convention for the boot drive for folders. However, I think there should only be 1 approved method. It does not make sense to have some people doing it one way and some another. This is easily enforceable by having a standardization for packages and scripts (per 2 above) that are written with these naming conventions in mind. That way, if someone tries something different, you can easily point out the error. I also agree that there should be a script file that creates these folders if Tom does not want to levy this as part of the standard install.

I too like standards... We've been evolving as we go.  Originally /boot/packages was used to make it easier for beginners to download and move packages to the flash drive. (in their browsers, and in windows it is \\tower\flash\packages)  The idea of a "custom" folder was not thought of at that time.  I do agree, the "packages" folder probably should be under /boot/custom.

 

We will need a better set of rc.install scripts to deal with BOTH startup and shut-down.  There has been great work done in the "powerdown" script to create an rc.xxxxxx set of scripts accept a "start" and "stop" argument.  We might be able to adopt something similar.  If nothing else, the scripts in /boot/custom/etc/rc.d must be able with shutdown tasks as well as startup.  According to one reference to /etc/rc.d, the capital "S" preface on some tasks is supposed to indicate statrup tasks, and those with a capital "K" the stop tasks.  Our current loop does not make a distinction.  It probably needs some fixes to handle different array states.

Maybe this is all obvious but as someone who plays with this kind of stuff all the time, i find it common that the lack of the above often leads to lots of unnecessary troubleshooting and frustration from both the users and those trying to help...

 

I wish I could contribute but have no experience with designing this stuff. May have to start as this is a great project.  

Often ideas as more important than the actual code.  Keep them coming.

 

The desire to have an "official" set of scripts run in a specific order is great, but for 99.99% or unRAID users not helpful at all. 

The curious/unexperienced will try to install and run everything, even if not needed, the very-experienced can manage on their own.  Hopefully most everybody in-between will think about what they install before doing so.

 

Joe L.

[Biggy2872]

We will need a better set of rc.install scripts to deal with BOTH startup and shut-down.  There has been great work done in the "powerdown" script to create an rc.xxxxxx set of scripts accept a "start" and "stop" argument.  We might be able to adopt something similar.   If nothing else, the scripts in /boot/custom/etc/rc.d must be able with shutdown tasks as well as startup.   According to one reference to /etc/rc.d, the capital "S" preface on some tasks is supposed to indicate statrup tasks, and those with a capital "K" the stop tasks.  Our current loop does not make a distinction.  It probably needs some fixes to handle different array states

This is why I would like to continue to use the S## designation, if in the future our loop was to make the distinction, we would already have the appropriate filenames and standards for the startup (maybe need some modification) and we could add the K files at a later date (and have the go script manger manage those as well).  from my idea of tracking the boot sequence of the individual user in the .confs, I could be able to easily add the appropriate value to the "K" files to the code with little modification to the go script manager.

 

I too like a single file with the names of the scripts to be executed in the order desired.  It is easy to edit via any number of tools/editors.  In its simplest form it would be used as a shell script. One line per command.  If it exists, it is used instead of the ".auto_goscript" and ".auto_install" commands.

 

The spirit of our ideas are not too different.. if your single file exists, I could simply use the single file to ovewrite the boot placeholders, I could allow for the go script manager then to reorder it again according to the user input inside the plugin via (^) and (V) links as suggested, once completed, the packagemanger would update the palceholders and recreate the single file if it is changed.  My idea of the placeholder was at the root the same as this idea, although the information is not stored in one location (single file) but across all the .confs.  The file could be edited outside of the plugin as well, once the go script manager is run, it will update the boot sequence to the order defined in the file.

 

thoughts?

 

Matt

[jimwhite]

We are really in desparate need of a unified approach to posting and distributing all of the plugins, addons and scripts that all our resident geniuses come up with.  I think that the model of the first post is always updated with the latest version with NO links within the thread, or a wiki-repository again with NO links in the threads are the best way.  Currently, we have some with updated posts in the middle of the thread, and most with various updates sprinkled throughout the thread and often in multiple threads.  I hosed my system yesterday because I ended up with a mix of things from two older versions of GoScript manager... what a mess !! 

 

Moreover, I may not be alone in that I seem to be unable to grasp what GoScript manager is supposed to do!

 

[NAS]

I agree.

 

I would go one further and say the forum shouldn't be used for distribution at all and only for discussion.

 

I would also say that the amount of reading a user needs to do to keep up with scripts or even get a good feel for the whole thing in the first place is nothing short of unfeasible.

 

I dont want to fork this thread any more but it seems sensible to get back to basics and let more of the community join in with using these excellent addons and to do that we need simplicity.

 

 

[Biggy2872]

We are really in desparate need of a unified approach to posting and distributing all of the plugins, addons and scripts that all our resident geniuses come up with.  I think that the model of the first post is always updated with the latest version with NO links within the thread, or a wiki-repository again with NO links in the threads are the best way.  Currently, we have some with updated posts in the middle of the thread, and most with various updates sprinkled throughout the thread and often in multiple threads.  I hosed my system yesterday because I ended up with a mix of things from two older versions of GoScript manager... what a mess !! 

 

Moreover, I may not be alone in that I seem to be unable to grasp what GoScript manager is supposed to do!

 

The goal of the go file manager is very simple, provide a way for inexperienced users to add/remove scripts from the boot up procedures(go file OR rc.d structure which is system dependent), without the hassle of forgetting to run them through fromdos commands and all the errors inexperienced users run into on an near constant basis with scripts in linux.  It will also help experts to diagnose system problems by being able to diasable all booting scripts in one interface.  This will essentially revert them to a "stock" go file to help with diagnostics.  People seem to want more control of their system from the unmenu interface.  I'm trying to provide a way to do that with the booting scripts.  

 

In the future when run levels and shutdown procedures are supported (K files), the go script manager will install with appropriate startup and shutdown scripts for each package in the rc.d sctructure and not just the startup scripts.

 

I think rebranding the name to Boot Manager (or boot script manager) in the next release might be a better idea to help people understand the purpose of the plug-in.

 

Cheers,

Matt  

[JarDo]

OMG

Care to elaborate?

 

Cheers,

Matt

 

Sorry, it was late.  I was tired.  With what I was reading at that time, it was all seeming to get more complicated than needed.

 

This morning there were new posts with more info.  It hadn't yet occured to me until this morning that you guys had been thinking about a single file to maintain the installation order.  I think that is a great idea.  If that were all that were needed then the solution could be implemented in two stages.  First release the Go Script Manager simply adds an entry to this file for every enabled script. Then the user can use Joe's Config View/Edit plug-in to control the order of the entries (thus, installation order).  A subsequent release could remove the need for users to manually edit this file. 

 

If needed, I support the idea of a variable in the conf file itself to support the installation sequence functionality.  I'm not a big fan of doing anything with the conf file name (i.e. sequence numbers in the conf file name)  I really think that's asking for trouble.  I'm a little on the fence about the "S" or "K" pre-pended to the file name.  That doesn't seem as bad as sequence numbers to me.  But, if the single file idea is one that is adopted...  How about having two files.  One for startup scripts and another for shutdown scripts??  This way you could even have a single script run at either event (startup and shutdown) without having to have two identical scripts with diferrent names.

 

[Biggy2872]

If needed, I support the idea of a variable in the conf file itself to support the installation sequence functionality.  I'm not a big fan of doing anything with the conf file name (i.e. sequence numbers in the conf file name)  I really think that's asking for trouble.  I'm a little on the fence about the "S" or "K" pre-pended to the file name.  That doesn't seem as bad as sequence numbers to me.  But, if the single file idea is one that is adopted...  How about having two files.  One for startup scripts and another for shutdown scripts??  This way you could even have a single script run at either event (startup and shutdown) without having to have two identical scripts with diferrent names.

From my reading(some of it anyways)

 

The S and K filename followed by numbers is the linux standard(or one of them) for dealing with these kinds of scripts...

see here: http://www.netbsd.org/docs/guide/en/chap-rc.html

here: http://www.comptechdoc.org/os/linux/startupman/linux_surc.html *** this has the filename pairs I was talking about (S AND K for each script)

here: http://www.linux.com/articles/114107

and here: http://www.linuxjournal.com/article/1274 *** another good one describing s and k files

 

because we don't have an rcoder utiity, you are suggesting I make a file which does the same thing, but the convention to do that in linux is the S and K prepending of the script filenames and having them placed directly in /etc/rd.c or in our case /boot/custom/etc/rc.d.

 

This is important for run levels(if we ever develop them) but I'm still learning the ins and outs of the rc.d system.  I have used linux in the past few years but never to the point where I needed to fully understand this runlevel system in great detail.  I'm trying to play catch up now...  I wouldn't want to program something that will interfere with future desires to have working run-levels.

 

Cheers,

Matt

[prostuff1]

LONG POST TO COME

 

Well there was already a post on this matter at one point and i have been eyeing it up lately... it might give us a start... see here: http://lime-technology.com/forum/index.php?topic=1953.msg14021#msg14021...

That thread needs to be placed prominently somewhere, especially if we are going to start adopting that architecture. He also has a .zip of his go script and all of the scripts that he is using.  Some of them are very nice and I have "stolen" them for use in my startup.

 

So using this as a rough guide we have:

 

S01-blockdev OR S01-set_read_ahead (which do you guys prefer?) *

S03-cpufreq (saved for individual user to use cpu frequency mods if they are supported)

S10-custom_install_packages (as defined by Joe L in package manager) *

S20-init.spincontrol (for drive spin up/down control)

S75-useradds (for adding additonal users)

S80-update_hosts (updating /etc/hosts file) *

S90-smb-shares (for changing values default smb behaviour, will require some work)

S99-syslog-save (for saving the syslog at the end of boot, do we need/want this?)

(*) denotes mods that already have packages...

I like S01-set_read_ahead as it is a little more apparent to the newbie what it might do.

 

where should starting unmenu fall into this?

well how about S30?

Sound about right to me

 

what about monthly parity check?

I would say S21... we could use the 20s for cron jobs?

That is a very good thought.  Maybe we should shot for breaking it up into sections of 10 and putting specific things in a given section.  Like you said, cron jobs would go in the 20-29 section.

 

Can anyone think of any that should be "locked" and not allow it to be disabled?

Any additons you would like to see added?

The S10 script should probably be "locked."  We could allow it to be moved but up and down and disabled, but i think a pop-up warning message should be set off when the user tries to do something to it.  If we made that one script the "set point" and made it so that it could not be moved up of down, only disabled, then it might make diagnosing and fixing problems easier.

 

 

 

1) I think Joes suggestion about having a "up" and "down" arrow to move the packages into the order you would want them installed is an absolute requirement. Right now, there are not an unwieldy amount of packages and scripts but as the project develops, gains momentum, and becomes more user friendly for the average user, you will be faced with 100's of these which will begin to necessitate you having to rename some as new ones come on board. This will require people to change the names or redownload. Rather, if there was a thread which maintained the "Recommended Boot Order", it could easily be updated and people could adjust their systems accordingly with a simple moving of the package up or down.

I agree that the up and down arrows should be included, but i also feel a standard naming convention should be set.  The Sxx-name-of-package seems like a good and logical one to use at this point.  Maybe it will change over time, but i think for right now Sxx is the way to go.

 

2) To your suggestion, it would also be good to have a thread (actually sticky) that was the Official Packages and Scripts thread. I think the thread needs a moderator or two that has the power to suspend or delete packages/scripts posted that do not conform to the agreed upon format (whatever that may be). Again, I am thinking ahead here and trying to prevent lots and lots of rework as this project expands.

Agree with this also, i feel we actually need a few threads of this sort and some mod's to police this board.

Are there any mod's on this board?  I don't think there are and with the community here growing i think it is something that needs to be started.  I don't see any problems with members getting out of control, but having someone with the power to sticky threads, move threads, delete posts, and do general house cleaning would be a good thing.

 

3) If we do have an official thread as per 2 above, I could see an easy way to provide package boot order updates would be to have one file which controls this order. That way, users coudl simply download and install (or overwrite) it over the previous one without having to do anything else. The more that is involved which changes, the higher the likelihood of user mistakes and errors=more forum posts for help

Yup, a thread or two for some of this stuff that is just for updating and the like would come in handy.

 

4) I 100% agree with a naming convention for the boot drive for folders. However, I think there should only be 1 approved method. It does not make sense to have some people doing it one way and some another. This is easily enforceable by having a standardization for packages and scripts (per 2 above) that are written with these naming conventions in mind. That way, if someone tries something different, you can easily point out the error. I also agree that there should be a script file that creates these folders if Tom does not want to levy this as part of the standard install.

Agreed, that is the reason for this discussion.  I am very much a fan of the Third Party Boot Architecture along with the fromdos line i have in my go script.  I think that should be the way it is done from here on forward.  It simplifies what is in the go script and then from there we can add stuff to the /boot/custom/etc/rc.d folder.  Like i said earlier we need to start forcing this layout so that everything becomes standardized.  Moving the /boot/packages and /boot/unmenu folders into the /boot/custom/ folder i think should become the default also.

 

Maybe this is all obvious but as someone who plays with this kind of stuff all the time, i find it common that the lack of the above often leads to lots of unnecessary troubleshooting and frustration from both the users and those trying to help...

 

I wish I could contribute but have no experience with designing this stuff. May have to start as this is a great project.  

I am in the same boat. I can sit here and conceptualize this stuff but i am not much for the programing stuff.  I can do a little bit, but i am not familiar with any of the languages that are being used here.  I could probably figure it out (and i might) but i would probably be better used on the organization end of things.

[prostuff1]

I think rebranding the name to Boot Manager (or boot script manager) in the next release might be a better idea to help people understand the purpose of the plug-in.

 

I think it needs to be renamed to Boot Script Manager.  That way we are not implying that that this plugin is a Boot Manager, it is purely something to manage boot scripts

[SSD]

Instead of trying to standardize the numbers, you might consider creating a set of reusable startup scripts.  And have the boot script manager create short little script files that call those scripts.

 

I think this is clear but the numbers are used as the ordering mechanism.  The lower the number, the earlier in the sequence it is executed.  In my system, S05 is a 30 second sleep.  A script with a number less that S05 happens before the sleep, greater thatn S05 happens after the sleep.  The 30 second sleep allows unRAID to bring the array online so that you can be reasonably sure that the drives are mounted and Samba is configured.

 

If you give two scripts the same number, subsequent characters will determine the colating sequence.  I have used the same number when I really didn't care which one executed first.  Perhaps you want to avoid this.

 

My scripts in this directory tend to be very short.  If I am doing anything complex, the script just calls another script.  For example, I get a smart report on every boot. This is in a separate script and called.