July 21, 201015 yr In order to try and minimize package definitions. I've found a method to allow advanced complex package installation as a slackware package itself. This has the added functionality to do a preinstall, install, postinstall for a package that requires advanced handling. The benefit is you can build a preinstall/postinstall script on it's own, then put it in place. Run a slackbuild and rebuiild the .tgz file with appropriate files. It's really only an idea right now. The goal is to devise a methodology so that a standard slackware can be used in advanced methods to bootstrap/build/setup other packages or groups of packages. The advanced package has a doinst.sh script that calls the advanced packages install.sh script. The particular doinst.sh is non rentrant and will be overwritten so it has one function, call the install script. The install script does the following. 1. if exists, run a preinstall.sh 2. if exists, installs one or more *.tgz slackware packages 3. if exists, run a postinstall.sh bash-3.1# tar -tvzf advanced-1.00-i486-unRAID.tgz drwxr-xr-x root/root 0 2010-07-21 12:26 ./ drwxr-xr-x root/root 0 2010-07-21 12:26 tmp/ drwxr-xr-x root/root 0 2010-07-21 12:26 tmp/advanced-1.00/ -r--r--r-- root/root 179590 2010-07-21 12:26 tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz drwxr-xr-x root/root 0 2010-07-21 12:26 usr/ drwxr-xr-x root/root 0 2010-07-21 12:26 usr/doc/ drwxr-xr-x root/root 0 2010-07-21 12:26 usr/doc/advanced-1.00/ -rw-r--r-- root/root 2714 2010-07-21 12:26 usr/doc/advanced-1.00/advanced.SlackBuild -rw-r--r-- root/root 230 2010-07-21 12:26 usr/doc/advanced-1.00/slack-desc drwxr-xr-x root/root 0 2010-07-21 12:26 usr/libexec/ drwxr-xr-x root/root 0 2010-07-21 12:26 usr/libexec/advanced-1.00/ -r-xr--r-- root/root 166 2010-07-21 12:26 usr/libexec/advanced-1.00/preinstall.sh -r-xr--r-- root/root 489 2010-07-21 12:26 usr/libexec/advanced-1.00/install.sh -r-xr--r-- root/root 165 2010-07-21 12:26 usr/libexec/advanced-1.00/postinstall.sh drwxr-xr-x root/root 0 2010-07-21 12:26 install/ -rw-r--r-- root/root 635 2010-07-21 12:26 install/doinst.sh -rw-r--r-- root/root 230 2010-07-21 12:26 install/slack-desc #!/bin/sh # based on the Slackware 12.0 SlackBuild # Packager Rob Cotrone ( WeeboTech at mydomain dot com) # http://lime-technology.com # Required: CWD=`pwd` TMP=${TMP:-/tmp/tgz} NAME=advanced VERSION=1.00 PKG=$TMP/${NAME}-${VERSION} ARCH=${ARCH:-i486} BUILD=unRAID SOURCE=http://unraid-weebotech.googlecode.com/files/$NAME-$VERSION.tgz if [ ! -e $NAME-$VERSION.tar.gz ] then echo wget -c $SOURCE fi if [ ! -d $TMP ] then mkdir -p $TMP fi if [ ! -d $PKG ] then mkdir -p $PKG else rm -rf ${PKG} mkdir ${PKG} fi cd $TMP # unused at this point # tar xvzf $CWD/$NAME-$VERSION.tar.gz echo -e "\E[0;32m+------------------------------------+\E[0;0m" echo -e "\E[0;32m| Start SlackBuild $NAME-$VERSION |\E[0;0m" echo -e "\E[0;32m+------------------------------------+\E[0;0m" if [ ! -d $NAME-$VERSION ] then mkdir -p $NAME-$VERSION fi cd $NAME-$VERSION find . -perm 777 -exec chmod 755 {} \; find . -perm 775 -exec chmod 755 {} \; find . -perm 555 -exec chmod 755 {} \; find . -perm 666 -exec chmod 644 {} \; find . -perm 664 -exec chmod 644 {} \; find . -perm 444 -exec chmod 644 {} \; chown -R root:root . # Here is where the manual build install commands go. LIBEXEC=${PKG}/usr/libexec/${NAME}-${VERSION} install -d ${PKG}/tmp/${NAME}-${VERSION} install -d ${PKG}/usr/libexec/${NAME}-${VERSION} install -p -m544 ${CWD}/install.sh ${LIBEXEC} install -p -m544 ${CWD}/preinstall.sh ${LIBEXEC} install -p -m544 ${CWD}/postinstall.sh ${LIBEXEC} for script in ${LIBEXEC}/*.sh do sed -i -e "s/NAME=.*/NAME=${NAME}/g" -e "s/VERSION=.*/VERSION=${VERSION}/g" ${script} done install -m444 ${CWD}/md5deep-3.3-i486-1mfb.tgz ${PKG}/tmp/${NAME}-${VERSION} # The rest should be standard on all installations mkdir -p $PKG/install mkdir -p $PKG/usr/doc/$NAME-$VERSION sed "s/#.##/${VERSION}/g" < ${CWD}/slack-desc > ${PKG}/install/slack-desc sed "s/#.##/${VERSION}/g" < ${CWD}/slack-desc > $PKG/usr/doc/$NAME-$VERSION/slack-desc cat $CWD/$NAME.SlackBuild > $PKG/usr/doc/$NAME-$VERSION/$NAME.SlackBuild cat $CWD/doinst.sh > $PKG/install/doinst.sh # Boot strap libexec scripts. cat <<-EOF >> $PKG/install/doinst.sh if [ -x /usr/libexec/${NAME}-${VERSION}/install.sh ] then rm -v /install/doinst.sh exec /usr/libexec/${NAME}-${VERSION}/install.sh # rm -v /usr/libexec/${NAME}-${VERSION}/install.sh fi EOF gzip -9 $PKG/usr/man/*/* cd $PKG if [ ${DEBUG:=1} -gt 0 ] then echo "Do Inspect: ${PKG}, type exit to continue." export PS1="$0 ${PWD} $:" sh fi requiredbuilder -y -v -s $CWD $PKG /sbin/makepkg -l y -c n $CWD/$NAME-$VERSION-$ARCH-$BUILD.tgz tar -tvzf $CWD/$NAME-$VERSION-$ARCH-$BUILD.tgz if [ "$1" = "--cleanup" ] then rm -rf $TMP fi :::::::::::::: doinst.sh :::::::::::::: config() { NEW="$1" OLD="`dirname $NEW`/`basename $NEW .new`" # If there's no config file by that name, mv it over: if [ ! -r $OLD ]; then mv $NEW $OLD elif [ "`cat $OLD | md5sum`" = "`cat $NEW | md5sum`" ]; then # toss the redundant copy rm $NEW fi # Otherwise, we leave the .new copy for the admin to consider... } if [ ${DEBUG:=0} -gt 0 ] then set -x -v fi for file in config.new do config etc/$file; done :::::::::::::: install.sh :::::::::::::: if [ ${DEBUG:=0} -gt 0 ] then set -x -v fi NAME="undefined" VERSION="undefined" SCRIPT=/usr/libexec/${NAME}-${VERSION}/preinstall.sh if [ -x ${SCRIPT} ]; then echo "Running ${SCRIPT} phase" sh -c "${SCRIPT}" rm -v "${SCRIPT}" fi for PKG in /tmp/${NAME}-${VERSION}/*.tgz do installpkg "${PKG}" rm -v "${PKG}" done SCRIPT=/usr/libexec/${NAME}-${VERSION}/postinstall.sh if [ -x ${SCRIPT} ]; then echo "Running ${SCRIPT} phase" sh -c "${SCRIPT}" rm -v "${SCRIPT}" fi :::::::::::::: postinstall.sh :::::::::::::: #!/bin/bash if [ ${DEBUG:=0} -gt 0 ] then set -x -v fi NAME="undefined" VERSION="undefined" echo "Hi I'm the $0 script. I'm going to do something advanced" sleep 3 date :::::::::::::: preinstall.sh :::::::::::::: #!/bin/bash if [ ${DEBUG:=0} -gt 0 ] then set -x -v fi NAME="undefined" VERSION="undefined" echo "Hi I'm the $0 script. I'm going to do something advanced" sleep 3 date Here is what it looks like when the pacakge is built. bash-3.1# ./advanced.SlackBuild wget -c http://unraid-weebotech.googlecode.com/files/advanced-1.00.tgz +------------------------------------+ | Start SlackBuild advanced-1.00 | +------------------------------------+ gzip: /tmp/tgz/advanced-1.00/usr/man/*/*: No such file or directory Slackware package maker, version 2.1. Searching for symbolic links: No symbolic links were found, so we won't make an installation script. You can make your own later in ./install/doinst.sh and rebuild the package if you like. This next step is optional - you can set the directories in your package to some sane permissions. If any of the directories in your package have special permissions, then DO NOT reset them here! Would you like to reset all directory permissions to 755 (drwxr-xr-x) and directory ownerships to root.root ([y]es, [n]o)? n Creating tar file advanced-1.00-i486-unRAID.tar... ./ tmp/ tmp/advanced-1.00/ tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz usr/ usr/doc/ usr/doc/advanced-1.00/ usr/doc/advanced-1.00/advanced.SlackBuild usr/doc/advanced-1.00/slack-desc usr/libexec/ usr/libexec/advanced-1.00/ usr/libexec/advanced-1.00/preinstall.sh usr/libexec/advanced-1.00/install.sh usr/libexec/advanced-1.00/postinstall.sh install/ install/doinst.sh install/slack-desc tar-1.13: advanced-1.00-i486-unRAID.tar is the archive; not dumped Gzipping advanced-1.00-i486-unRAID.tar... Renaming advanced-1.00-i486-unRAID.tar.gz to advanced-1.00-i486-unRAID.tgz... Moving advanced-1.00-i486-unRAID.tgz to /home/rcotrone/src/advanced_demo... Package creation complete. drwxr-xr-x root/root 0 2010-07-21 12:33 ./ drwxr-xr-x root/root 0 2010-07-21 12:33 tmp/ drwxr-xr-x root/root 0 2010-07-21 12:33 tmp/advanced-1.00/ -r--r--r-- root/root 179590 2010-07-21 12:33 tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz drwxr-xr-x root/root 0 2010-07-21 12:33 usr/ drwxr-xr-x root/root 0 2010-07-21 12:33 usr/doc/ drwxr-xr-x root/root 0 2010-07-21 12:33 usr/doc/advanced-1.00/ -rw-r--r-- root/root 2714 2010-07-21 12:33 usr/doc/advanced-1.00/advanced.SlackBuild -rw-r--r-- root/root 230 2010-07-21 12:33 usr/doc/advanced-1.00/slack-desc drwxr-xr-x root/root 0 2010-07-21 12:33 usr/libexec/ drwxr-xr-x root/root 0 2010-07-21 12:33 usr/libexec/advanced-1.00/ -r-xr--r-- root/root 166 2010-07-21 12:33 usr/libexec/advanced-1.00/preinstall.sh -r-xr--r-- root/root 489 2010-07-21 12:33 usr/libexec/advanced-1.00/install.sh -r-xr--r-- root/root 165 2010-07-21 12:33 usr/libexec/advanced-1.00/postinstall.sh drwxr-xr-x root/root 0 2010-07-21 12:33 install/ -rw-r--r-- root/root 635 2010-07-21 12:33 install/doinst.sh -rw-r--r-- root/root 230 2010-07-21 12:33 install/slack-desc And then when it is installed bash-3.1# installpkg advanced-1.00-i486-unRAID.tgz Installing package advanced-1.00-i486-unRAID... PACKAGE DESCRIPTION: advanced: advanced Control scripts for the unRAID server Environment. advanced: advanced: Example advanced unRAID package to assist with environment advanced: Executing install script for advanced-1.00-i486-unRAID... mv: cannot stat `etc/config.new': No such file or directory removed `/install/doinst.sh' Running /usr/libexec/advanced-1.00/preinstall.sh phase Hi I'm the /usr/libexec/advanced-1.00/preinstall.sh script. I'm going to do something advanced Wed Jul 21 12:34:21 EDT 2010 removed `/usr/libexec/advanced-1.00/preinstall.sh' Installing package md5deep-3.3-i486-1mfb... PACKAGE DESCRIPTION: md5deep: md5deep (Suite of Tools to compute Hashes or Message Digests) md5deep: md5deep: md5deep, a set of cross-platform tools to computer hashes, or message md5deep: digests, for any number of files while optionally recursively digging md5deep: through the directory structure. Supports MD5, SHA-1, SHA-256, Tiger, md5deep: and Whirlpool hashes. md5deep: md5deep: More information at: http://md5deep.sourceforge.net/ md5deep: md5deep: Packaged by Marco Antonio Frias B. ([email protected]) removed `/tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz' Running /usr/libexec/advanced-1.00/postinstall.sh phase Hi I'm the /usr/libexec/advanced-1.00/postinstall.sh script. I'm going to do something advanced Wed Jul 21 12:34:25 EDT 2010 removed `/usr/libexec/advanced-1.00/postinstall.sh' Then bash-3.1# cat /var/log/packages/advanced-1.00-i486-unRAID PACKAGE NAME: advanced-1.00-i486-unRAID COMPRESSED PACKAGE SIZE: 174 K UNCOMPRESSED PACKAGE SIZE: 200 K PACKAGE LOCATION: advanced-1.00-i486-unRAID.tgz PACKAGE DESCRIPTION: advanced: advanced Control scripts for the unRAID server Environment. advanced: advanced: Example advanced unRAID package to assist with environment advanced: advanced: advanced: advanced: advanced: advanced: advanced: advanced: FILE LIST: ./ tmp/ tmp/advanced-1.00/ tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz usr/ usr/doc/ usr/doc/advanced-1.00/ usr/doc/advanced-1.00/advanced.SlackBuild usr/doc/advanced-1.00/slack-desc usr/libexec/ usr/libexec/advanced-1.00/ usr/libexec/advanced-1.00/preinstall.sh usr/libexec/advanced-1.00/install.sh usr/libexec/advanced-1.00/postinstall.sh install/ install/doinst.sh install/slack-desc bash-3.1# removepkg advanced-1.00-i486-unRAID Removing package /var/log/packages/advanced-1.00-i486-unRAID... Removing files: --> /tmp/advanced-1.00/md5deep-3.3-i486-1mfb.tgz no longer exists. Skipping. --> Deleting /usr/doc/advanced-1.00/advanced.SlackBuild --> Deleting /usr/doc/advanced-1.00/slack-desc --> Deleting /usr/libexec/advanced-1.00/install.sh --> /usr/libexec/advanced-1.00/postinstall.sh no longer exists. Skipping. --> /usr/libexec/advanced-1.00/preinstall.sh no longer exists. Skipping. --> Deleting empty directory /usr/libexec/advanced-1.00/ --> Deleting empty directory /usr/doc/advanced-1.00/ --> Deleting empty directory /tmp/advanced-1.00/ bash-3.1# ls -l /var/log/packages/md5deep-3.3-i486-1mfb -rw-r--r-- 1 root root 1495 2010-07-21 12:34 /var/log/packages/md5deep-3.3-i486-1mfb
July 21, 201015 yr Won't work on a stock unRAID system.... you want to make all devs have a full Slackware install?
July 21, 201015 yr Author Won't work on a stock unRAID system.... you want to make all devs have a full Slackware install? What's missing and can it be added?
July 21, 201015 yr Ideally, the entire pkgtools suite should be included in base unRAID. This ensures the functionality of installpkg, removepkg, makepkg, explodepkg, and upgradepkg are available. The entire package itself is 41257 bytes. Once installed, makepkg is 11315 bytes.
July 21, 201015 yr Nicely done work on getting this going. This is pretty much what I had in mind for the advanced packages while working within the constraints of existing tools.
July 21, 201015 yr Author Ideally, the entire pkgtools suite should be included in base unRAID. This ensures the functionality of installpkg, removepkg, makepkg, explodepkg, and upgradepkg are available. The entire package itself is 41257 bytes. Once installed, makepkg is 11315 bytes. many of them are already there. I think makepkg & requiredbuilder is the only one left.
July 21, 201015 yr Author Nicely done work on getting this going. This is pretty much what I had in mind for the advanced packages while working within the constraints of existing tools. It does provide a framework for some advanced setup/rebuilding.
July 21, 201015 yr What would you want to do with the .ini files for providing info to the package manager? How do you want to handle dependencies? Lets not forget, we also plan a package repository, so: - the package manager can stay informed of updates and inform the user of them and - the package manager can present the user with a uniform view of packages in the repository.
July 21, 201015 yr What would you want to do with the .ini files for providing info to the package manager? How do you want to handle dependencies? Lets not forget, we also plan a package repository, so: - the package manager can stay informed of updates and inform the user of them and - the package manager can present the user with a uniform view of packages in the repository. And for those it will probably require network connectivity. ;)
July 21, 201015 yr Author What would you want to do with the .ini files for providing info to the package manager? How do you want to handle dependencies? Lets not forget, we also plan a package repository, so: - the package manager can stay informed of updates and inform the user of them and - the package manager can present the user with a uniform view of packages in the repository. I do not have all the answers on this. My goals were 1. provide a consolidated way of incorporating/handling advanced scripting without populating the .ini 2. use the same slackware pkgtools that are readily available. You and Joe seemed to nail all the required variables. I did a proof of concept for helping alleviate the many lines of shell commands in the .ini We can add the .ini to an advanced package or use the .ini as a stub for an remote install.
July 21, 201015 yr We can add the .ini to an advanced package or use the .ini as a stub for an remote install. I want to make sure we are on the same page here. On the client unRAID system, the entire .tgz file must, MUST, be self-contained and have within it everything needed (other than dependencies) for it to install on that client system. These packages are installed at every boot.... if Internet access is down, or a server where something needed is hosted is unreachable (cough ... Comcast sucks.... cough) when the server starts, the package won't install. That is an unacceptable failure to me. If you want to make your build environment work with remotely-stored files that are only retrieved when you build a package, then that's fine, but they have to be built into the package for distribution.
July 21, 201015 yr Author We can add the .ini to an advanced package or use the .ini as a stub for an remote install. I want to make sure we are on the same page here. On the client unRAID system, the entire .tgz file must, MUST, be self-contained and have within it everything needed (other than dependencies) for it to install on that client system. These packages are installed at every boot.... if Internet access is down, or a server where something needed is hosted is unreachable (cough ... Comcast sucks.... cough) when the server starts, the package won't install. That is an unacceptable failure to me. If you want to make your build environment work with remotely-stored files that are only retrieved when you build a package, then that's fine, but they have to be built into the package for distribution. I am in agreement.
July 21, 201015 yr OK, using WeboTech's enhanced Slackware package .tgz, the install process at boot time will be a simple installpkg.... that's simple enough. Consider for now, an enhanced .tgz is sitting there in /boot/packages. Now the system boots, and the Package Manager was loaded from /boot/extra. Package Manager now starts looking in /boot/packages ... what will it do? We need standards for: - determining if the package is disabled/enabled (I like .disabled) - determining dependencies (unRAID version, RAM, array info, and installed packages) (I like an .ini file, but could use the slackware dependencies file, but I've never relied on it and I'm not sure other devs do either)) - acting on dependencies (recursive call) - logging activities - checking for upgrades? (info to do it from the .ini? I think I would not DO an upgrade, but just log the fact or provide some notification an upgrade is out there). Then the users decides to stop the array or shutdown. The PM now has to apply the event to each package. - determine what to do in that event for each package and do it (.ini file) Comments?
July 21, 201015 yr Author Would it be better to put this intermediary package configuration/ini/definition in another thread?
July 21, 201015 yr Would it be better to put this intermediary package configuration/ini/definition in another thread? Yup. http://lime-technology.com/forum/index.php?topic=7100.0
July 21, 201015 yr Is it really the package managers job to process stop events and distribute them to installed packages?
July 21, 201015 yr Is it really the package managers job to process stop events and distribute them to installed packages? Yes. The package manager has several jobs: - assist in getting packages onto the server, and updating them when needed (can be done manually for now) - install the packages at server boot (respecting disabled flags, etc) - register the event scripts for the packages by putting scripts/symlinks in the emhttp_event directories If unRAID traversed a directory when events happen, then packages could hook themselves into the event processing.... but that requires unRAID to do processing, and we didn't like that. Instead, we expect unRAID to just call one script for the event. The "smarts" will be in that script, and the package manager will hook into it. At least that's the way I understand the discussion so far.
July 25, 201015 yr We need the preinstall script to be able to stop the installpkg process. For example, a package that supplements the stock php, needs to check the installed version of php, and abort the install if the right version of php is not found.
July 25, 201015 yr Author We need the preinstall script to be able to stop the installpkg process. For example, a package that supplements the stock php, needs to check the installed version of php, and abort the install if the right version of php is not found. It may require removing the doinst.sh Maybe touching a flag file and the doinst.sh looks for.
Archived
This topic is now archived and is closed to further replies.