Logic flow for a .tgz or .zip file in /boot/packages:

 

If there is no .ini file of the same appname, extract it from /unraid in the package file

 

If still no .ini file, make a default stub

 

Parse the .ini

 

If disabled, exit

 

Check dependancies, install dependancies, handle failures (Report?  Abort?).

 

Extract and run the scripts as needed from /unraid in the package

 

If no “install” script, default to extract/installpkg on each .tgz file in the package.

(Note if multiple .tgz files and you want them in a particular order, write an install script!)

 

Capture output from whole process, and write results to /boot/packages/messages/<appname>txt.

 

 

The preinstall.sh could download it on demand also. I've seen a number of slackbuilds do this.

 

I'm no fan of this, could be mischief.  But that can simply be handled by the "preinstall=" line pointing to a url instead of a filename.  php can get files via url easily.

 

XML is not easy to read, and Tom has already standardized on .ini file format (which is the same as Apache config files)

 

There must be a reason... Apache uses it

 

There might be a way to encapsulate more information within the XML.

 

The php "parse_ini" function does a bang-up job, and includes ability to even populate arrays via the .ini file.

 

http://php.net/manual/en/function.parse-ini-file.php

 

I think we need 2 package managers:

 

- a simple one that will handle enable/disable in booth /boot/extra, and /boot/packages ... and this has to be part of stock unRAID, because if you have something in /boot/extra or /boot/packages that is crashing unRAID, and you boot in "safe mode", you HAVE to have some way to disable the misbehaving package.

 

- the full community package manager will have all the bells and whistles for the user, and it does the processing of the package/ini files in /boot/packages.

 

It would be a good idea to start another thread on implementing safe mode, and ask Tom to include a variable in $var indicating boot mode (safe vs standard?) and have it written to the syslog.

Just in following all of this I do have one comment about something mentioned earlier.

 

I think using .zip would be a better choice for a new dev not use to a linux environment. .zip is supported natively on Windows and OS X so it would probably make it easier.

 

I questioned this avenue of thought last night.

It seems like there is an effort to protect a windows user from using linux.

I think there is something wrong with that if that windows user is going to write any form of scripts.

 

Teach a man to fish feed them for life.

 

If a person is doing anything more advanced then dropping files off, they should be familiar with the environment they are tweaking.

 

It doesn't make sense to have a windows person, write scripts, to package them up as a zip because they might be unfamiliar with a tar.gz.

 

Tools and scripts could be written to handle this.

Some form of autopackager with a dedicated directory on the array.

I don't see many windows people doing advanced scripting.

 

With respect to the community, I've never seen anyone say RTFM, in fact this community is extremely helpful to the noobs.

 

Where I do see .zip's value is there are functions in php to manipulate them. Yet, I've just located some functions to do similar things with tar.

I questioned this avenue of thought last night.

It seems like there is an effort to protect a windows user from using linux.

I think there is something wrong with that if that windows user is going to write any form of scripts.

 

I agree in principle, but disagree practically.  Honestly, I would prefer the .zip format even though I can do a .tgz file, just because in my dev environment it would be easier for me.

 

Yet, I've just located some functions to do similar things with tar.

 

But not a gzipped tar (that I know of... I welcome being educated otherwise)

I don't see many windows people doing advanced scripting.

 

I disagree.

 

I've written packages that run on a Mac, but I've never owned or used a Mac.  I wrote them (and SOLD them) even before I had a VM that could run OS X.  Same with Linux.... wrote stuff for Linux and never even rant it or tested it on Linux.  All development was done under Windows.

 

There are a number of applications I can think of that would be at home as an unRAID plugin, that could easily come from a Windows person who has little or no Linux knowledge.

Yet, I've just located some functions to do similar things with tar.

 

But not a gzipped tar (that I know of... I welcome being educated otherwise)

Correct, not a gzipped one, but then there is gunzip and gzip.

 

I do all of my development of unmenu packages on a Mac and then transfer the needed stuff to the unraid environment.

Correct, not a gzipped one, but then there is gunzip and gzip.

 

Which is added complication, shelling out, managing temporary files, etc.

 

When 5.0 came out, I needed to make a single small change to the php.ini file in the unraidWeb package.  It took me 15 minutes since I had not been in there for a long time.  Had it been a zip file, I could have done it in 30 seconds, by just clicking on the php.ini file, editing it, and saving it and winrar or winzip would have simply updated the package with the changed file.

 

It might not be a big difference when building an original package, but for simple maintenance of an existing package, I find the .zip much more preferable.

 

 

I don't see many windows people doing advanced scripting.

 

I disagree.

 

I've written packages that run on a Mac, but I've never owned or used a Mac.  I wrote them (and SOLD them) even before I had a VM that could run OS X.  Same with Linux.... wrote stuff for Linux and never even rant it or tested it on Linux.  All development was done under Windows.

 

There are a number of applications I can think of that would be at home as an unRAID plugin, that could easily come from a Windows person who has little or no Linux knowledge.

 

We are talking about advanced system administrative scripting.

Not everyone has the same caliber of experience as you. (/me salutes you).

 

We're trying to accommodate a windows user developing root based installation procedures outside the destination platform.

 

I have 20 disks of data, and we're going to support advanced root scripting in .zip packages to assist a windows user so they do not have to learn and use native packaging tools.

The same tools that are used to build the native system.

 

I think my preference would be a front end that has fields to local browseable sources.

Fill in all the information about a package.

Click Submit and it creates the advanced installation .tgz

 

I think I'm going to take a step back from this.

This gives me the shivers. I'm going to take a break from this for a short while.

I don't know... I've seen plenty of experienced <fill in name of OS> folks pull some bonehead stuff and cause havoc in <name of same OS>.... but I understand your concern.

 

However, anyone who blindly runs any package/plugin on their production unRAID system is playing with fire.  Test and sandbox first... including all applications from Bubba.... hell, including new versions of unRAID itself.

 

I think the paradigm of the plugin/package system is that it manages and installs the packages you chose.  It is up to you to chose wisely.  Perhaps you chose to not add any packages that come in a .zip?

 

I would much prefer to prototype this pig with .zip support.  But after it is prototyped and works, we can add .tgz support to it.

 

But we can table the form of package file a while, and spend some time on the elements of the .ini/.conf file.  I was thinking of starting a new thread for the .ini elements, and update the OP with the products of the discussion.  Sound good, or should we discuss it here?

 

 

I do all of my development of unmenu packages on a Mac and then transfer the needed stuff to the unraid environment.

 

Am I correct the Mac environment is a unix environment?

 

Are the unmenu packages tweaking system files, such as /etc/inetd.conf or any of the system configuration etc files?

 

Are there required packages that are downloaded, compiled and installed during these tests?

 

Once the transfer has occurred and been tested in the unraid environment, can you package it up for redistribution there?

I don't know... I've seen plenty of experienced <fill in name of OS> folks pull some bonehead stuff and cause havoc in <name of same OS>.... but I understand your concern.

 

However, anyone who blindly runs any package/plugin on their production unRAID system is playing with fire.  Test and sandbox first... including all applications from Bubba.... hell, including new versions of unRAID itself.

 

I think I'm leaning towards the same way of installing an advanced package as a regular slackware package.  If someone needs to learn the basic skills of creating the advanced package, then we will write guides.

 

One method of supported package installation.

 

installpkg. You said it yourself.  use slackware package methodology.

 

To support the advanced methodology.

Encapsulate the included slackware package inside.

 

use doinst.sh to run the preinstall, installpkg of the included virgin slackware packages(binary or source) and postinstall.

 

We have the ability now to write a plugin which will allow browsing a local filesystem and packaging the advanced files into the correct format on the server.  This would help the more casual user who does not write a script to do the rebuild/update.

 

Are the unmenu packages tweaking system files, such as /etc/inetd.conf or any of the system configuration etc files?

 

I see many flavors of packages.... some that do Linux internals, and obviously need to be done on a Linux system.

 

But I see opportunities for plenty of others, that will be pure php code, and have no interaction with the underlying Linux system at all.... such as a more full-featured graphical array status dashboard.

 

 

use doinst.sh to run the preinstall, installpkg of the included virgin slackware packages(binary or source) and postinstall.

 

So use one doinst.sh, but use the manager to extract it alone, and run it with a parameter such as "preinstall" and it would do the preinstall stuff.... then run installpkg and then it will be run by Slackware itself with no parameter, and it would do its original thing, and then the package manager would extract doinst.sh and run it a third time, with a parameter "postinstall" ... is that the idea?

 

What about dependencies?

 

Where do you want to store the .ini file?

 

Am I correct the Mac environment is a unix environment?

You are correct

 

Are the unmenu packages tweaking system files, such as /etc/inetd.conf or any of the system configuration etc files?

The ones i have created are not as far as i can remember

 

Are there required packages that are downloaded, compiled and installed during these tests?

When i do my testing on my Mac i generally download all the needed files on my own.  The unmenu package i create does that on the unraid system, but i don't let it do that on my Mac.

 

Once the transfer has occurred and been tested in the unraid environment, can you package it up for redistribution there?

Yes, with a little bit of work.  Since unmenu dumps all files in /boot/packages it takes some hunting around to find all the stuff that was needed for the package.  I know why and understand why unmenu works this way but at the same time i wish i could put all the Airvideo related files into a folder in /boot/packages.  In essence i would have /boot/packages/airvideo which woudl contain everything needed to run airvideo.  The shared libraries makes it hard to accomplish this sort of organization.

 

 

Most of the unmenu packages I write i can test out on my Mac system before i move them to unRAID, granting some of the stuff does not work.  I test out all the sed commands i might need, and compile scripts (the ffmpeg airvideo is a good example) along with some of the other stuff.

 

The Airvideo package does require a lot of things to be downloaded and it took me a good long while (with the help of JoeL) to figure out what all was needed.  On the Mac side i usually download the pieces and parts I might need and put them in an unRAID development folder.  Once i think i have everything tweaked the way it needs to be i move the .conf file over to /boot/packages and reload unmenu. Once it picks it up I go about installing it there. Unmenu downloads the files i specify in the .conf, matches MD5 to make sure all is well, and then goes about creating the .manual_install, which is pretty much just my .conf file with some of the lines removed and a few added.

 

The printer deamon package is another one that I initially created. JoeL tweaked and enhanced it when i asked for some assistance. The main reason i wanted to get the package created was for personal use but JoeL expressed some interesting in it also when i mentioned doing it.  I was given a free laser printer and unopened cartridge that had a parallel port on it and the only thing i could hook it to was my unRAID server. I have a nice wireless inkjet printer but it goes through ink like no tomorrow so i wanted to replace it with this free laser.

Are the unmenu packages tweaking system files, such as /etc/inetd.conf or any of the system configuration etc files?

I can answer that... some are.

 

The proftpd package does edit inetd.conf with these two lines to disable the built-in vsftp ftp daemon.

PACKAGE_INSTALLATION sed -i -e "s/^ftp/##ftp/" /etc/inetd.conf

PACKAGE_INSTALLATION /etc/rc.d/rc.inetd restart

The powerdown_ctlaltdel-unmenu-package.conf unMENU package edits the /etc/acpi/acpi_handler.sh so a powerdown request by a quick of the "power" button on the server is routed to WeeboTech's powerdown instead of the one lime-technology provides that simply invokes the web-interface.

PACKAGE_INSTALLATION [ -f /usr/local/sbin/powerdown ] && mv /usr/local/sbin/powerdown /usr/local/sbin/unraid_powerdown

PACKAGE_INSTALLATION [ -f /usr/local/sbin/unraid_powerdown ] && sed -i "sX/usr/local/sbin/powerdownX/sbin/powerdownX" /etc/acpi/acpi_handler.sh

PACKAGE_INSTALLATION [ ! -f /usr/local/sbin/unraid_powerdown ] && sed -i "sX/sbin/init 0X/sbin/powerdownX" /etc/acpi/acpi_handler.sh

The apcupsd package edits /etc/rc.6 to add the request to killpower to the UPS just prior to the call to poweroff with this line:

PACKAGE_INSTALLATION [ "${vUPS_KILL-YES}" != "NO" ] && sed -i -e "s/\/sbin\/poweroff/\/etc\/apcupsd\/apccontrol killpower; \/sbin\/poweroff/" /etc/rc.d/rc.6

Joe, I recall you seemed to have wanted to extend a bit more consideration to the DOS/Windows developers and maximizing their ability to enter unRAID plugins.... what's your take on .tgz native versus using .zip for the unRAID custom packages?

 

Technically, the .zip is much easier to manage from the php side, which makes writing the package manager much easier.

Joe, I recall you seemed to have wanted to extend a bit more consideration to the DOS/Windows developers and maximizing their ability to enter unRAID plugins.... what's your take on .tgz native versus using .zip for the unRAID custom packages?

 

Technically, the .zip is much easier to manage from the php side, which makes writing the package manager much easier.

The big issue is that zip is not a supplied program on unRAID today.  (and I know it is NOT recognized by installpkg if you intend to use it.)

 

Since the package manager is only written once, by experienced developers, it should not be the reason why an easy approach for it is chosen.  It is OK if it takes it a bit more work, since that coding is only done once.

 

As already mentioned, we can create a php page that will tar/gzip a directory pointed to by a user and provide fields to assist them in creating the .tgz.

 

Joe L.

 

 

The big issue is that zip is not a supplied program on unRAID today.

 

I intended to include it as part of the basic package manager install, and there is native php support. But if no one else wants it, then so be it.  It is just an order of magnitude easier for a dev who just needs to tweak one file compared to a gzipped tar file.

 

As already mentioned, we can create a php page that will tar/gzip a directory pointed to by a user and provide fields to assist them in creating the .tgz.

 

I'm sorry, this is a kludge.  Who wants to fill out a form when all you have to do is:

 

Click on .zip, click on file in zip, edit and save file and the the zip manager will replace the file with the new contents. 

 

Or from a commandline it is just as easy.  No form will be that simple.

 

On a different note, I do not like altering the doinst.sh in another developer's package.  I would much rather have our own format for the package, with the original .tgz file unmodified and stored inside our .tgz file.  If a new version of dmidecode or bwm-ng comes out with a modified doinst.sh, then we have to rewrite the thing again.  It seems much easier and more utilitarian to just pack the .tgz unmodified inside our unRAID specific package.

 

I agree, adding I don't love the idea of encapsulating another .tgz at all.  I'd rather not have to use disk space to store what can be downloaded from somewhere else.  (Although, if you do have the space, it makes it very easy to be sure the .tgz you are wrapping is always going to be there, and your script that interacts with it will always be able to find things as it expects.)

 

Joe L.

Folks, it's taken me some time, but I've been able to get a more advanced tgz to be handled by installpkg.

 

it's kinda a kludge too.

The slackbuild script, builds the advanced "unraid" slackware pacakge in sort of a bootstrapping method.

 

It's was a pain because I found out quickly that you cannot use the /install tree or you will go into a loop.

Advanced bootstrap install packages.

http://lime-technology.com/forum/index.php?topic=7093.0

 

I'd rather not have to use disk space to store what can be downloaded from somewhere else.

 

How are you going to install it if you don't download it?  It takes no more space to have it embed it than to have it standalone, and then you have two files to take care of as a set.

On a different note, I do not like altering the doinst.sh in another developer's package.  I would much rather have our own format for the package, with the original .tgz file unmodified and stored inside our .tgz file.  If a new version of dmidecode or bwm-ng comes out with a modified doinst.sh, then we have to rewrite the thing again.  It seems much easier and more utilitarian to just pack the .tgz unmodified inside our unRAID specific package.

 

See the other thread. This is what I did a proof of concept on.

If y'all really want to use .zip archives for the unRAID advanced package, I'm not going to stop it.

I think using the slackbuild method of rebuilding the package with updated files is just as fast.

 

If we do go the .zip route, then we'll need to package up the

install. preinstall, postinstall and other subordinate packages (slackbuild script?).

You see, you still need to do something to package it up and build it all together at some point.

I'd rather not have to use disk space to store what can be downloaded from somewhere else.

 

How are you going to install it if you don't download it?  It takes no more space to have it embed it than to have it standalone, and then you have two files to take care of as a set.

 

Slackbuilds.org does not always store the source locally.

But it will have the slackbuild script and other build files.

I've seen slackbuild scripts that do a wget to download the files on demand.

I have no big preference here unless I am developing.

Then I download and store all files.

 

With the proof of concept I did, you can embed or use the install.sh to download files on demand with wget.

Heck you might even be able to write something to parse the .ini file and grab the sources.

 

The idea is, if you need really advanced tasks, use the slackware package to repackage and install.

 

Zip

Pro.

  You can attach it to the board.

  Reportedly easier to update.

  php can access .zip archives programatically.

 

Con:

  It is on the board and not in a central location which makes searching for it a pain.

  You update the .zip archive, but you still have to version it and upload it versioned.

  php cannot access .tgz archives directly.

 

 

Can someone show me a code example of where php (as installed on 5.0) can access/manipulate .zip archives?

 

 

Slackbuilds.org does not always store the source locally.

 

I was talking about it being stored on the end user's machine where intallpkg is being run.  I don't see space on a dev's machine as an issue.  If I was doing an unRAID package with an embedded .tgz, I would absolutely want to keep a local copy... the remote one could disappear or be altered, and make rebuilding and troubleshooting quite a problem.

 

Can someone show me a code example of where php (as installed on 5.0) can access/manipulate .zip archives?

 

It can't... that will be added by the package manager.  But neither can it manipulate a gzipped tar.