The problem is that, to paraphrase the Sec. Def , we don't know what we don't know.

 

There are a LOT of excellent apps out there that you can drop into most any webserver and have instant functionality, like phpsysinfo, the web interfaces for nzbget and rtorrent, ... the list is endless.

 

Some of these are affected only by the missing elements in php.

 

Others are affected by missing elements of a traditional webserver in emhttp.

 

Not to mention difficulties in troubleshooting for a dev, when there is no access log or error log for the webserver.

 

Yes, we have found a handful of specific issues that Tom might be able do some build changes on his end to make emhttp more like a real webserver.  But I see that as a never-ending cycle.... we discover something like that in emhttp, and then depend on the revision cycle from Tom to fix it.

 

And what happens when one of those things we find is not doable at all in emhttp?

 

I also question whether it is wise to have your core RAID management application doing all this potential extra work.  Just how robust is the web-server code that is being used?  Will some particular php code or something else the emhttp web server tries to process, hang emhttp?

 

We are early in the life of unRAID 5.0, and Tom may decide to include a real webserver (lighttpd) that defaults to an agreed-upon port (say 81) and plugins can continue to use emhttp if they want, or a plugin can be a stub that gives you a link to http//<server>:81/<page>.  Before we go to the extreme of a community package for lighttpd, I'd like to hear Tom's thoughts.

 

I'll start documenting some of the specific issues identified in the OP.

I agree.

 

Some of what I'm asking for should be base emhttp.  CGI should be a regular proper CGI interface without any form of templating. logging is another story, but that can be handled.

 

xmlrpc/post ability. It's a very standard API.

if the API could work, methods could be devised by community.

 

For the more advanced tasks, a webserver should be involved.

I had thought that is the way it was going to work.

A real webserver along with some interface between it and emhttp or some other core manager program.

 

Maybe it has to go through more evolutions.

Some things that emhttp does under the hood, should be externalized in to php or other managing code.

I.E. users / updating config files / restarting daemons.

 

Just my thoughts on it for now. I think we're able to help ourselves a bit more.

With upcoming events the user community expansions will be cleaner and better managed.

 

In reality, if we had the event subsystem first, many of the "plugins" would have been done under a lighthttpd/php environment.

 

The beauty of the the plugin environment is common look and feel.

I was able to slap together a number of useful plugins without much code.

You can make a wrapper around your CGI using passthru()

 

do_cgi.php:

 

<?php

parse_str(str_replace("+","%2B",$argv[1]), $_GET);

if (isset($_GET['cgifile'])) {
  $runfile = "/usr/local/webGui/" . $_GET['cgifile'] . '.cgi';
  if (file_exists($runfile)) {
     passthru($runfile);
  }
}

?>

 

 

 

Then call it with

 

 

http://tower/do_cgi.php?cgifile=xxxxx

 

 

nagios wants to call it's programs from a cgi-bin directory.

Plus it uses some of the webservers variables for internal links.

The gui programs are all in C, so it would be a bit of effort to use the php wrappers.

I have to think about maintenance later.

I can write an install or slackbuild script, but I always want to use the core unchanged source since nagios updates regularly.

I understand.

 

The wrapper is really only useful for testing, or apps that live in one, single, unified .cgi file, like the GUI display for the APC UPS monitor.

As an outside observer the question "Do we need a web server? " seems to be a very definite yes. Short of emHTTP suddenly acting like a proper web server surely the plugin devs are doomed to a life of "oh bugger another emHTTP funny"

 

Tom obviously needs to put his thoughts here before to much work is done either way.

At this point more http server functionality should exist in emhttp unless Tom decides to package up a full webserver in unRAID.

 

There is what I would consider, basic functionality missing from unRAID and the only way to get it there is install it and make it easy enough to install for the community.

 

I don't think emhttp needs to do allot of the heavy http serving.

Yet it needs to a bit more foundation support for addons.

 

Mail notifications are critical, but how do we make it possible?

 

Right now emhttp can do this as we have enough of a front end to accept input via browser and configure a configuration file.

 

We've come a long way.

 

cgi-bin is important for some add on packages other wise it forces the need for the second web server.

 

When it comes to more advanced tasks, the full http server should fill in.

 

I think cgi-bin and http post should be a workable possibility.

 

xmlrpc is possible but with helpers and more advanced programming.

cgi-bin and http post are first.

 

I'm interested in what Tom has to say.

I have built a working package of the latest lighttpd (1.4.26) for unRAID, but there are some things to think about.

 

I will set it up this way:

 

  /boot/custom/lighttpd/                      <-- home dir for configs and scripts

  /boot/custom/lighttpd/lighttpd.conf     <-- main config file

 

The lighttpd files in /etc and so on will be symlinked back to the home on the flash... so they are persistent.

 

I will also include a small unRAID plugin for configuring lighttpd, starting/stoping, etc.

 

In order to work on unRAID as-is, including the trimmed-down php on unraid, no fast-cgi, I have to do the .conf file with a lot of things disabled.

 

If someone adds capabilities to unRAID php (or adds perl, ruby, etc) the lighttpd.conf file will have to be changed.

 

OTOH, if I include a complete reinstall of php, etc, I can go ahead and enable additional features in the lighttpd.conf.... but the problem still remains for other things (like perl) that I am obviously not going to include in the lighttpd package.

 

Since I can't include everything, I am leaning heavily towards including nothing.... just pure lighttpd that will work with pure stock unRAID.

 

Please don't say "do both" 'cause I ain't got the time.

 

So what happens to lighttpd when someone adds something like perl or XML-RPC?

 

Should we leave editing the config file to users to do?

Should devs have their packages check for lighttpd and modify the lighttpd.conf file if needed?

Should we do it with includes in the lighttpd.conf file?  (this has some limitations since I believe some sections can appear only once).

 

Consider the Config options:

 

  http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs:Configuration

OTOH, if I include a complete reinstall of php, etc, I can go ahead and enable additional features in the lighttpd.conf.... but the problem still remains for other things (like perl) that I am obviously not going to include in the lighttpd package.

 

I dunno if you should go so far as a complete reinstall of php.

It needs to match what is with unRAID or something is going to break.

Ask me how I know (LOL!!)

 

Since I can't include everything, I am leaning heavily towards including nothing.... just pure lighttpd that will work with pure stock unRAID.

 

This is the way to go for now.

 

 

So what happens to lighttpd when someone adds something like perl or XML-RPC?

 

Should we leave editing the config file to users to do?

Should devs have their packages check for lighttpd and modify the lighttpd.conf file if needed?

 

For now yes, I think the patch program could be used for some of the more complex changes.

I'm thinking of asking Tom for patch as I want to modify some of the startup scripts before everything else takes off.

 

Since the /boot/extra is done very early, I could modify some things with patches in a doinst.sh.

(stuff like rc.inetd1.conf to enable multiple ethernet adapters).

mounting tmpfs on certain directories, etc, etc.

 

 

Back on topic.

 

Should we do it with includes in the lighttpd.conf file?  (this has some limitations since I believe some sections can appear only once).

 

I like the idea of using includes. Smaller blocks are easier to work with.

 

 

From your perspective, What is needed in the unRAID libraries, php, lighthttpd to support rtorrent?

From your perspective, What is needed in the unRAID libraries, php, lighthttpd to support rtorrent?

 

That will be up to the dev of an rTorrent package for unRAID to figure out. 

OK, for you devs that need to modify the lighttpd config.  I've set it up so that lighttpd will load all the .conf files from:

 

 

  /boot/custom/lighttpd/conf.d/*.conf

 

The default settigns (that will be controlled AND OVERWRITTEN) by the lighttpd plugin, are in:

 

  /boot/custom/lighttpd/default.conf

 

 

So if you add perl, for esample, you need to create a perl.conf file and stick in ht the conf.d dir under lighttpd, something like this:

 

static-file.exclude-extensions += ( ".pl"  )
cgi.assign += ( ".pl" => "/usr/bin/perl" )

 

 

Now brush up on your lighttpd config syntax. 

 

http://redmine.lighttpd.net/projects/lighttpd/wiki/Docs:Configuration

Just to make sure it doesn't get missed, a lot of the webserver issues are due to an underlying php issue.  A full version of lighttpd on the stock unRAID php still has the php issues.  I even got Apache running on unRAID, and it had the same issues with the underlying php.

 

But making php more robust after the fact will not solve all of them.  It is going to need to be compiled differently.

 

I have been considering additional potential plugins and applications for unRAID 5.0, I fear that no amount of tweaking the installed php is going to help, regardless of whether a full webserver is available, because of the compile flags on the unRAID version of php.

 

I also see a little better how emhttp is using the cli version of php, and the cgi version may actually not work for emhttp. 

 

I believe that it would be possible to do a parallel build of php-cgi, and it could live side-by-side with the cli version unRIAD is using, and both obviously share the libs.  This would allow the CGI version to be compiles with mySQL, libxml, and other support enabled that is not in the CLI version used by emhttp.

 

Or maybe I'm just sleep deprived, and I fubard the whole thing... I'm awful frustrated.

 

 

 

 

I have been considering additional potential plugins and applications for unRAID 5.0, I fear that no amount of tweaking the installed php is going to help, regardless of whether a full webserver is available, because of the compile flags on the unRAID version of php.

 

I also see a little better how emhttp is using the cli version of php, and the cgi version may actually not work for emhttp. 

 

I believe that it would be possible to do a parallel build of php-cgi, and it could live side-by-side with the cli version unRIAD is using, and both obviously share the libs.  This would allow the CGI version to be compiles with mySQL, libxml, and other support enabled that is not in the CLI version used by emhttp.

 

Or maybe I'm just sleep deprived, and I fubard the whole thing... I'm awful frustrated.

 

You are correct, the scaled down PHP is going to limit possibilities.

However I was able to compile php with a few libs and only add a few packages.

I bet we could go with a larger version of PHP if we satisfied all the dependencies first.

 

Unless I am missing which php options you are looking to compile in.

 

What is it that you are doing on unRAID that requires this much effort and Database related hooks?

 

To be clear, emhttp does not do real cgi.

I'm not sure if it supports post(it should), but I did'nt see how to get a post working to a PHP script.  I think that is important for multipart forms and uploading files.

This is what I modified in the slackbuild that Tom provided.

 

# tmm - modifed to product only php-cli for unraid

php_configure() {

EXTENSION_DIR=/usr/lib${LIBDIRSUFFIX}/php/extensions \

CFLAGS="$SLKCFLAGS" \

./configure \

 $* \

 --prefix=/usr \

 --libdir=/usr/lib${LIBDIRSUFFIX} \

 --with-libdir=lib${LIBDIRSUFFIX} \

 --sysconfdir=/etc \

 --without-apxs2 \

 --with-apache=no \

 --disable-all \

 --enable-cli \

 --with-pcre-regex \

 --enable-libxml \

 --enable-xml \

 --enable-simplexml \

 --enable-xmlreader \

 --enable-xmlwriter \

 --with-xmlrpc \

 --enable-dom \

 --enable-inifile \

 --disable-Reflection \

 --with-bz2 \

 --with-zlib \

 --enable-zip \

 --with-pear  \

 --build=$ARCH-slackware-linux

}

 

I did have to download and install a number of packages to support the shared libs. All in all it was not that much. It was still just under 3mb.

I did something a bit different.  Instead of recompiling and replacing the CLI version of php that ships with unRAID, I built and installed just the CGI version.

 

That lets emhttp have its own version of php, while lighttp runs the real CGI version.... all sharing the same libs.

 

My build is identical to what ships with Slack 13.2:

 

'./configure' '--enable-fastcgi' '--enable-pcntl' '--enable-sigchild' '--prefix=/usr' '--libdir=/usr/lib' '--with-libdir=lib' '--sysconfdir=/etc' '--disable-safe-mode' '--disable-magic-quotes' '--enable-zend-multibyte' '--enable-mbregex' '--enable-tokenizer=shared' '--with-config-file-scan-dir=/etc/php' '--with-config-file-path=/etc/httpd' '--enable-mod_charset' '--with-layout=PHP' '--enable-sigchild' '--enable-xml' '--with-libxml-dir=/usr' '--enable-simplexml' '--enable-spl' '--enable-filter' '--disable-debug' '--with-openssl=shared' '--with-pcre-regex=/usr' '--with-zlib=shared,/usr' '--enable-bcmath=shared' '--with-bz2=shared,/usr' '--enable-calendar=shared' '--enable-ctype=shared' '--with-curl=shared' '--with-curlwrappers' '--with-mcrypt=/usr' '--enable-dba=shared' '--with-gdbm=/usr' '--with-db4=/usr' '--enable-dbase=shared' '--enable-exif=shared' '--enable-ftp=shared' '--with-gd=shared' '--with-jpeg-dir=/usr' '--with-png-dir=/usr' '--with-zlib-dir=/usr' '--with-xpm-dir=/usr' '--with-freetype-dir=/usr' '--with-t1lib=/usr' '--enable-gd-native-ttf' '--enable-gd-jis-conv' '--with-gettext=shared,/usr' '--with-gmp=shared,/usr' '--with-iconv=shared' '--with-imap-ssl=/usr' '--with-imap=/usr/local/lib/c-client' '--with-ldap=shared' '--enable-mbstring=shared' '--enable-hash' '--with-mhash=shared,/usr' '--with-mysql=shared,/usr' '--with-mysqli=shared,/usr/bin/mysql_config' '--enable-pdo=shared' '--with-pdo-mysql=shared,/usr' '--with-pdo-sqlite=shared,/usr' '--with-pspell=shared,/usr' '--with-mm=/usr' '--enable-shmop=shared' '--with-snmp=shared,/usr' '--enable-soap=shared' '--enable-sockets' '--with-sqlite=shared' '--enable-sqlite-utf8' '--with-regex=php' '--enable-sysvmsg' '--enable-sysvsem' '--enable-sysvshm' '--enable-wddx=shared' '--with-xsl=shared,/usr' '--enable-zip=shared' '--with-tsrm-pthreads' '--enable-shared=yes' '--enable-static=no' '--with-gnu-ld' '--with-pic' '--build=i486-slackware-linux' 

Publish the slackbuild, How does it handle the install to avoid colliding?

I'm not using slackbuild.  I build it by hand.

 

I have to run some more comparisons.... I have only found a handful of different files -- most are identical.

 

Also by using fast-cgi, lighttpd gives you some excellent advanced config options, like specifying a different php.ini for CGI php handled by lighttpd, and leave the stock unRAID php.ini alone and php CLI (emhttp) will use that one.

Very Kewl!