Jump to content
Joe L.

An Improved unRAID web-interface, extensible, and easy to install

553 posts in this topic Last Reply

Recommended Posts

A new and improved unRAID web-interface now exists.  It does not replace the existing web-management console, but suppliments it.  

It is accessible on an entirely different port on your server (typically port 8080)  It is named "unmenu.awk"

 

You can get updated unMENU version 1.2 files including updated package .conf files from this link at google code

There is now only one file.  It in turn downloads and updates as needed.

There are three zip files at google.code.  You will want them all.   Download and unpack them all in the same directory where you want to put all the unmenu files. I suggest a folder name something like /boot/unmenu.

 

The files in the three google.code downloadable zip files entirely replace any older versions of the files you may currently have.  If you have edited your older files to make local changes, move them first before you unzip these three new files.  None of the old files are needed, and in fact, since "package config" files have been re-named from when I originally distributed them in their first version, the old will only get in the way.  Note: the old *.unmenu-package.conf files were moved to your /boot/packages directory if you ran an old version of the unmenu package manager.  The *unmenu-package.conf files will need to be removed from there, or at the least, moved elsewhere, and not exist in either the directory with all the unmenu.awk files or the /boot/packages directory..

 

There is a "start-up" script included in the zip files.  It is named "uu"  Leave it in the same directory as all the other unmenu files.  (As written, it must reside there)

 

Before you start up this new version of unmenu, you must stop any older version you might be running.  

Easiest way to stop (kill) the old version is to type:

killall -g awk

If you added a line in your "go" script to automatically start unmenu, you may need to change it as well.

 

To start up this newest version of unmenu on the command line to test it, either "cd" to the directory with the unmenu files and then type:

uu

 

or, type the full path to the directory with the files (as an example, if your folder is /boot/unmenu)

/boot/unmenu/uu

 

The line in the config/go script to start it when you reboot should be something like this:

/boot/unmenu/uu &

or this:

echo "/boot/unmenu/uu" | at now + 1 minute

 

Use the second version if you want to make sure the unRAID array is started before the unMENU web-server is invoked. (It gives it a minute to start)

 

After you start the unmenu server (by typing "uu" in the instruction above) you can see the improved interface in your browser by using the following URL:  

http://tower:8080

 

If you attempt to start a second instance of unmenu using the same port as the currently running version it will fail with an error like this:

awk: ./unmenu.awk:256: fatal: can't open two way socket `/inet/tcp/8080/0/0' for input/output (No such file or directory)

The error message is slightly misleading, it is not that the socket cannot be opened because it does not exist, but because it is already open by another process.

(If it is already started when you booted by a line in the "go" script and you attempt to start it again on the command line, you'll get the above error)

 

There have been many changes and improvements since version 1.1 of unMENU, some added new features, a few to fix tiny bugs, other in preparation for future plug-ins  (unmenu.awk can now handle POST as well as GET)

 

The biggest addition is the new version of "myMain" a super piece of coding by Brian bjp999.  Wait until you see the "Smart" view.  If you are on version 4.4 or early 4.5beta versions of unRAID you will need to use the package manager to install the "cxxlibs" package before the "smartctl" program will function to let you see drive temperatures.  You will need to install the "bwm-ng" package to see the I/O performance on the "Performance" view in myMain.   I'll let Brian describe his work, but all I can say is that it is an example of exactly what I wanted when I developed the unmenu.awk plug-in system to explore ways to improve the unRAID user-interface.

 

On the user-scripts page is a button to restart the unmenu.awk server.   If you add a new plug-in after unmenu.awk is already started, you will need to kill it and then re-start it for it to see the new plug-in and add it to its top menu.  This button is an easy way to kill/restart unmenu.awk  You no longer need to use "ps" to figure out the process IDs to kill.

 

When I posted the version 1.2 files on google.code I went through the package configuration files and confirmed they all worked.  I've discovered the servers hosting these files are constantly updating them, so the URL links in the package conf files go out of date.  If you download a package and it does not match the md5 checksum, odds are the file was changed on the package server.

 

After installing unMENU, and starting it, to invoke you would type this URL in your browser.

//tower:8080

 

The main page of the new web-interface looks like this:

rvabkn.jpg

 

The new and improved interface has many of the same features as the stock management page, and a lot of new features.  

 

At the top of the new improved interface is the array status.  It will show if the array is on-line, stopped, calculating parity, etc.  As long as the array is in a STARTED status, the main page auto-refreshes every 60 seconds. (configurable if you prefer a different refresh rate)

On the front page you can see both the disks in the protected array, and any that are connected, but not part of the protected array.

The page shows the physical disk, where it is mounted, and its status, model number, and read/write statistics.

You can also see the space used and free on the USB flash drive.

 

An improvement over the stock main page is the color coding of disk temperature reading when they reach various warning thresholds.

As delivered, the disk temperature will show highlighted in YELLOW when over 40%C, in ORANGE when over 45%C, and in RED when over 50%C.  The specific thresholds are configurable locally on your server.

 

On those disks with file systems that are mounted, you can see the total size, space used, percentage used, and the amount of free space.

There is also a line on the page showing the total disk space, total used, total percentage used, and free space.   By default, it uses the "marketing" divisor of 1000 when displaying disk sizes.  You can edit the unmenu.conf configuration file to divide by 1024 if you like the computer science definition of Megabytes, Gigabytes, and Terrabytes.

 

For those disks that are not in the protected array you can see the individual disk partitions, their file-system types, and if mounted, their free space.   (The disk-management page on the improved interface will let you mount, un-mount, share, and un-share unprotected disks, even if they are from your PC and do not have a reiserfs file-system.)  The unprotected disks will not be part of the "user-shares" merge of disks, but you will be able to browse them, read, and (if you enable it) write to them.  This feature in the improved interface is there to allow you to easily migrate and access your old data.

 

The lower portion of the main page is also configurable, as delivered, it shows the last 6 lines of the "syslog" and color codes some of the lines that are interesting in some way.  If a disk were to start having errors, odds are you might see some message here before the array takes it offline.

 

To assist in keeping an eye on the system log, there si a page dedicated to viewing the syslog.  By default, it will show the last 3000 lines (yes, configurable), and thanks to RobJ, it is color coded. (Also configurable, in a "syslog_local.conf" file, for your own additions to the color coding scheme for software you might have installed locall)

 

The syslog page also provides a link to "download" the syslog to your PC.  It even names it with the current date.  This will make it far easier to supply a syslog if you request help in this forum.

 

The array management page allows you to spin up all disks, spin down all disks, and stop the array.  You can also stop, and re-start SAMBA shares.  (Be aware, if you stop sharing, the main unRAID management console will not show any indication you are no longer sharing files on your LAN.  It has no equivalent feature)

 

You can ask SAMBA to re-read the samba share configuration file (if you edit it by hand).

 

Stopping the array from the improved interface is safer than stopping it from the stock management console because the improved interface checks for any open files or active processes using the protected array and terminates them before attempting to un-mount the disks.   (The unRAID array cannot be stopped unless all the disks can be un-mounted.  The cannot be un-mounted if they are busy.  If the array is not stopped cleanly, you will be forced to perform a full parity check when you re-start the array.

 

You cannot re-start the array from the new-and improved interface, you must use the stock unRAID console to perform that task.  The stock set of management pages is available on the unRAId Main link on the improved interface menu, so it is pretty easy to re-start the array if you have stopped it.  

 

You must use the stock unRAID interface to configure devices and to replace or add new (clear and format) devices on your array.   This is all by design and will not change.  Tom at Lime-Technology has spent many hours in getting the logic in place to allow the unRAID interface to deal with the normal management tasks involving disks in the array.  I do not intend to ever duplicate that work.    The "unmenu" interface is specifically designed to compliment the existing features available in reporting on our array status.

 

It is VERY possible to perform an action using the unmenu.awk interface that will confuse the stock interface... For example, it is possible to stop SAMBA file sharing... the stock interface has no way to display that SAMBA has been stopped.  You can get things back in sync by stopping and re-starting the array from the stock interface.  This will re-start SAMBA, or you can press the button in the improved unmenu interface to restart SAMBA.

 

This new and improved interface was written in "GNU awk," an interpreted language that is included in the stock unRAID distribution.  the interface is an "awk" program coded to be a web-server with some built in web-pages and the ability to read additional plug-in files that become additional web-pages in the interface.  These plug-in files can extend the interface to do almost anything.  

 

The "awk" web-server is limited to serving one object at a time to a browser, therefore, a complicated web-page with embedded graphics is not possible at this time. You are limited to text, html, javascript, xml...for most web-pages,  but as you will see in the File-browser plug-in, you can serve binary files (jpg, gif, etc)  so you can use it to view pictures stored on the server, It will not work serving images as a part of an html page. (basically because the browser issues overlapping requests to the web-server, one for the main HTML and subsequent requests for each image file, and the "awk" web-server is limited to handling one request at a time.  It cannot handle multiple object requests and multiple concurrent connections from the browser)

 

The unmenu.awk web-interface was developed based on lots of ideas shared in a thread discussing ways to improve the unRAID user interface.  That thread is here: http://lime-technology.com/forum/index.php?topic=2110.0  You can easily develop and add your own plug-ins for almost anything that you would like on your unRAID server.

 

It's going to take me days to document this improved web interface, so be patient with me as I get it into place.    There are many functions and features not available on the stock unRAID web-interface and, as described, it can be extended for almost any specific needs.   The main screen on it is very similar to the stock unRAID management screen, but without any buttons.  The buttons to control the array are on their own page.  

 

It is possible to see the status of disks that are not in the protected array.  It is even possible to mount and share an unprotected disk.  I use this feature to get access to a USB drive I used to use for my PC backups.

 

Be aware that it is possible to stop the unRAID server from this interface, but not possible to start it back up again. You must use the supplied unRAID interface for that.   But to make that task easier, the stock unRAID interface is available on the "unRAID Main" link at the top of screen menu.   (I wrote a plug-in to make it easy to access the stock unRAID supplied management screens)  

 

The next few  posts in this thread show the screens in the improved unmenu.awk interface.   It is the result of a lot of input and contributions from members of the unRAID Server Community.   One thing for sure, they are not shy about sharing their wish-list for features.  Most of their requests ended up implemented in some way.

 

this post has the older version 1.1 zip files if you want to look at them for any reason. You should put these in a different folder if you download them.  

 

Joe L.

Share this post


Link to post

The home page of the unmenu.awk web interface for unRAID servers looks like the illustration below.  The set of "links" in the top left portion of the screen is automatically defined based on the plug-in files found when the unmenu.awk interface is started.  Most of the functions in the list are plug-ins.  A small handful are built into the unmenu.awk program. (Main, Array-Management, Disk-Management, About, and Help)  Also built in is a simple "syslog" viewer.   (but not used, as an external plug-in has the same "url" and overrides the built-in function.)

 

You must stop the unmenu.awk program and restart it for it to see a new plug-in and add it to its menu.   You do not need to stop and re-start it when making changes to a plug-in when initially developing it, as they are interpreted when the page is invoked. 

 

The main page of the new web-interface looks like this:

rvabkn.jpg

 

Joe L.

Share this post


Link to post

Many of the features and functions in the unmenu.awk program are configurable.  This is done using two files:

unmenu.conf

and

unmenu_local.conf

 

These two files hold the configuration values...

unmenu.conf will be distributed and will have new values added over time and subsequent releases.  Although you can edit it to change a value you are much better off making a copy of the file named

unmenu_local.conf.

Then edit unmenu_local.conf to change the variable and it will be used instead of the one in unmenu.conf.

 

The unmenu.conf file currently has definitions for:

  • the "port" used by the web-server
  • the "name" used to reference the web-server
  • color coded warning indications for disk temperatures
  • The refresh-interval for the main page.  (It Automatically refreshes every 60 seconds when the array is STARTED)
  • mount options for disks not in the protected array.  (defined as read-only, but you could mount a drive as read-write if desired)
  • links for display on the "links" plug-in.
  • A definition of what to put in the bottom portion of the main screen (currently, a color-coded syslog viewer showing the last 6 lines)
  • A method to restrict the file-browser plug-in to specific folders

 

 

Joe L.

Share this post


Link to post

The unmenu.awk interface automatically includes menu links for plug-in files.  These files can be written in either "awk" or as a "shell script"

 

A plug-in is a file named

ZZ-unmenu-whatever_you_like.awk  or ZZ-unmenu-whatever_you_like.cgi where ZZ is a one, two, or three digit number. 

 

The number indicates where in sequence it will be added to the top menu. (the menu choices are in lexical order, as shown by an "ls" command on the folder holding the plug-in files.  With that in mind, files starting with 1, 10, and 100 will be in the menu before 2, 3, 4... because of the leading "1")  If you wish a specific order, use leading zeros.

001, 002, 003 ... etc.

 

The first few entries in the top menu in the improved interface are always "Main", "Array Mgmt", "Disk Mgmt", "Syslog"

These are followed by the plug-in links,

These are followed by two links that are always "About", and "Help"

 

the simplest "awk" plug-in would have the following 6 lines as contents:

BEGIN {

#define ADD_ON_MENU=Hello World

#define ADD_ON_URL=hello_1

#define ADD_ON_TYPE=awk

print "Hello World"

}

 

the simplest "shell" plug-in would have the following 3 lines as contents:

#define ADD_ON_MENU=Hello World

#define ADD_ON_URL=hello_2

echo "Hello World"

 

The valid "defines" you can put at the top of a plug-in are:

The label to be shown on the top menu.  If no label is given, the URL can still be accessed, but not by the top of screen links.

#define ADD_ON_MENU = xxxxxxxx

The "URL" affiliated with a "plug-in"

#define ADD_ON_URL = xxxxxxxxx

If set to "NO", no top of screen status box is shown

#define ADD_ON_STATUS = YES

If set to anything other than "awk" the script is invoked as an execuitable command.

#define ADD_ON_TYPE = awk

expect YES or NO... YES = include <HTML><HEAD><TITLE></TITLE></HEAD><BODY> ... </BODY></HTML>

NO = let plug in supply, default if not specified = YES

#define ADD_ON_HTML_TAGS = NO

expect YES or NO... YES = include top of page heading with menu links, NO = do not show heading, default if not specified = YES

#define ADD_ON_PAGE_HEADING = NO

A string describing the "version" of the plug-in to be shown on the "About" page, if not defined, version 0.1 will be assigned in the "About" page.

#define ADD_ON_VERSION = 1.0

 

The following define is used when the plug-in will supply the entire HTTP header... It is used when the document type returned is not the normal html/text.

Specifically, the File-Browser plug-in sets this to NO, as it has to be able to serve various object types.

When set to NO, the plug-in must supply the entire content fed to the browser, including any and all HTTP header lines.

#define ADD_ON_HTTP_HEADER = NO

 

To illustrate how easy this can be... I added these three lines to a file named 40-unmenu-uptime.cgi and put that file in the same folder as unmenu.awk.

 

#ADD_ON_MENU=Uptime

#ADD_ON_URL=uptime

uptime

 

I stopped and re-started my unmenu.awk process, and here is the result.  A new "Uptime" menu choice, and a new piece of information available via the web interface.

 

vqqxck.jpg

 

If multi-line output was present, I would have had to either add HTML formatting, or simply echo "<pre>" before the entire output.    <pre>, in HTML, indicates the following output is preformatted and should be shown as is.

 

Here is an example of a multi-line Process List plug-in (all 4 lines of it)  I named it 41-unmenu-process_list.cgi :

#ADD_ON_MENU=Process List

#ADD_ON_URL=process_list

echo "<pre>"

ps -ef

 

It looks like this now when invoked:

2q8xago.jpg

 

In the process list (near the bottom, process 2580) you can see that when the shell invokes the "cgi" plug-in, $1 is the current array state, $2 is always "GET", $3 is the URL.  This ($3) will have the querystring parameters from form fields as well.  You would need to parse them from $3 as needed.   I'm currently using $4 to pass a concatenated set of URL/Menu Label pairs, so it would be easy to add a full top menu to any plug-in, even those that do the full HTTP headers themselves. (that was not yet being passed as a parameter when I did this screen-shot)  The "File-Browser" plug-in uses this to create its top menu to look consistent with all the other screens.

 

Any command that can be output to the screen and that is not interactive (wanting input from the user) can be made into a plug-in with ease, either as an "awk" script, or as a shell script.

 

Hope this gives you a few ideas.   Most of the add-on packages we are putting on our servers have a command line interface, many have the ability to invoke a command to get status.  Now it is easy to web-enable those same commands.     

Try this one...  you get both uptime AND a process list...   Not too shabby for 4 lines of plug-in.  You could name it 25-unmenu-top.cgi

 

#ADD_ON_MENU=Top Processes

#ADD_ON_URL=top

echo "<pre>"

top -b -n1

 

or this one for SAMBA status... name it as 25-unmenu-smb_status.cgi

#ADD_ON_MENU=SAMBA Status

#ADD_ON_URL=smb_status

echo "<pre>"

smbstatus

 

Have fun...

 

Joe L.

Share this post


Link to post

We've got a great "User scripts" plug-in that is based on a suggestion from bubbaQ.  It allows you to group all the quick little admin scripts on buttons on a single page and get rid of some of the clutter in the main menu.  The button names are grabbed from the script file.   The scripts themselves are in a configurable directory, but by default it is the same directory as the the other unmenu awk and cgi scripts.

 

Give the "User-Scripts" plug-in a try.  It might work well, for example, for the "commands" you created to start and stop vmware.  Now, you can create buttons.

 

The scripts themselves are matched and made into buttons if they have a name matching "[0-9]*unmenu_user_script*" and are executable.

 

They are executed when the button is pressed, so you now can have "Start VmWare" and "Stop VmWare" buttons.

 

Here are two sample 'User Scripts'

 

root@Tower:/boot# cat 10-unmenu_user_script_hello

#define USER_SCRIPT_LABEL Hello World

#define USER_SCRIPT_DESCR Print a famous greeting.

echo "Hello World"

root@Tower:/boot# cat 20-unmenu_user_script_goodbye

#define USER_SCRIPT_LABEL Goodbye World

#define USER_SCRIPT_DESCR Print a variation of a famous greeting

#define USER_SCRIPT_DESCR written by one of the authors of "C" language.

echo "Goodbye World"

 

Note: you can have multiple USER_SCRIPT_DESCR lines.  The lines are concatenated.  They can contain HTML, but do not need to have any.

 

It looks like this when you select the "User Scripts" menu choice... presenting whatever "buttons" you have defined along with their affiliated descriptions.

xkzt6t.jpg

 

This is the first "cgi" shell script that has buttons and processes the querrystring parameter passed from the browser as $3.

 

A user defined button for the "User-Scripts" plug-in can be as simple as this 4 line example used to run disk speed tests on all the disks on your server:

#define USER_SCRIPT_LABEL Disk Speed Test

#define USER_SCRIPT_DESCR Run disk transfer speed tests (using hdparm -tT)

echo "<pre>"

hdparm -tT /dev/[hs]d?

 

 

Have fun... and thanks to bubbaQ for the suggestion for this great plug-in

 

Joe L.

Share this post


Link to post

A description of the File-Browser plug-in an extensible web interface for unRAID servers will go here:

 

Joe L.

Share this post


Link to post

You can use the "Package-Manager" plug-in to download and install "package extensions"   A sample set of package configuration files is included.

The package configuration file will automatically be MOVED to the /boot/packages folder when the package-manager plug-in is invoked.

 

Once a package is downloaded through the plug-in, it may be installed manually by simply pressing a single button on the web-interface.   After installation (and you test it works) you are presented a button to configure an "auto-re-install-on-reboot" for that package.   If you press that button your "go" script will have a line appended to it automatically ... or, if you already have the /boot/custom/etc/rc.d structure in place, a file will be put there instead of in the go script.     Only one line is ever appended to the "go" script.  It will auto-install all of the packages designated by you.

 

I'll go into a lot more detail at a later time, but for now, just be aware.  If you only want to manually install, don't enable the "auto-install-on-reboot" feature.    If you do enable it for a given package, the web-interface will then present a button to disable the auto-install-on-reboot for that package.  Pressing it will remove that package from being auto-installed in the future by removing the "packagename.auto_install" file.  (no harm done)

 

The auto-install feature creates a "packagename.auto_install" script in the /boot/packages folder.  Those scripts contain the commands needed for auto-installation and are run in turn on re-boot.  In the same way, the "manual-install" button creates a "packagename.manual_install" script.  It can be invoked to reinstall a package, or viewed to see what occurs when a package is installed.  If the /boot/packages folder does not exist, it will be created.   Most everything can be locally configured in unmenu_local.conf by copying it from the equivalent lines from unmenu.conf.

 

downloadable packages are defined in NN-unmenu*package*.conf files.  A given file can contain multiple package definitions (but it is usually best if it contains only one as it simplifies distribution if only one of the package configuration files needs updating at a later time).

A sample package configuration file (included in the zip file) looks like this:

PACKAGE_NAME List Open Files
PACKAGE_DESCR The <b>lsof</b> command is used to list open files on a file system. This is important to know if you are
PACKAGE_DESCR attempting to shut down your server.  If a file-system has open files, it may not be un-mounted.  If it
PACKAGE_DESCR cannot be un-mounted, the unRAID array cannot be shut down cleanly.
PACKAGE_URL http://slackware.mirrors.pair.com/slackware-current/slackware/ap/lsof-4.78-i486-1.tgz
PACKAGE_FILE lsof-4.78-i486-1.tgz
PACKAGE_MD5 e5f72c903462cf170a3fdaa9e01af1ff
PACKAGE_INSTALLED /usr/bin/lsof
PACKAGE_DEPENDENCIES none
PACKAGE_INSTALLATION installpkg lsof-4.78-i486-1.tgz
PACKAGE_VERSION_TEST lsof -v 2>&1 | grep revision | awk '{print $2}'
PACKAGE_VERSION_STRING 4.78
PACKAGE_MEMORY_USAGE Light (10K to 500K)

PACKAGE_NAME ntfs-3g - NTFS read/write filesystem driver
PACKAGE_DESCR The <b>ntfs-3g</b> driver adds to unRAID the ability to read and write NTFS file systems.
PACKAGE_DESCR The built-in NTFS driver is read-only.<br>
PACKAGE_DESCR The NTFS-3G driver is an open source, freely available NTFS driver
PACKAGE_DESCR for Linux with read and write support.  It provides safe and fast
PACKAGE_DESCR handling of the Windows XP, Windows Server 2003, Windows 2000 and
PACKAGE_DESCR Windows Vista file systems.  Most POSIX file system operations are
PACKAGE_DESCR supported, with the exception of full file ownership and access
PACKAGE_DESCR rights support.
PACKAGE_DESCR <br>Homepage for the ntfs-3g project: http://www.ntfs-3g.org
PACKAGE_URL http://slackware.mirrors.pair.com/slackware-current/slackware/a/ntfs-3g-1.2812-i486-1.tgz
PACKAGE_FILE ntfs-3g-1.2216-i486-1McD.tgz
PACKAGE_MD5 7d6b633fe359fc2503a7e3c0c027a9ce
PACKAGE_INSTALLED /bin/ntfs-3g
PACKAGE_DEPENDENCIES fuse file-system  (already exists in unRAID)
PACKAGE_INSTALLATION installpkg ntfs-3g-1.2216-i486-1McD.tgz
PACKAGE_VERSION_TEST /bin/ntfs-3g --help 2>&1 | grep ntfs-3g | awk '{print $2}'
PACKAGE_VERSION_STRING 1.2216
PACKAGE_MEMORY_USAGE Light (10K to 500K)

PACKAGE_NAME file - A utility to determine file type
PACKAGE_DESCR This is Ian F. Darwin's 'file' utility, used to identify files.
PACKAGE_URL http://slackware.cs.utah.edu/pub/slackware/slackware-current/slackware/a/file-4.21-i486-1.tgz
PACKAGE_FILE file-4.21-i486-1.tgz
PACKAGE_MD5 b97a3ae071ae458ad6c0500b287a292f
PACKAGE_INSTALLED /usr/bin/file
PACKAGE_DEPENDENCIES none
PACKAGE_INSTALLATION installpkg file-4.21-i486-1.tgz
PACKAGE_VERSION_TEST /usr/bin/file -v 2>&1
PACKAGE_VERSION_STRING file-4.21
PACKAGE_MEMORY_USAGE Light (10K to 500K)

 

The package manager page in the unmenu.awk interface looks like this:

15335w1.jpg

 

 

Joe L.

Share this post


Link to post

A description of the syslog browser in an extensible web interface for unRAID servers will go here:

 

Joe L.

Share this post


Link to post

The most current version of unmenu.awk and an initial set of plug-ins and config files is included in the .zip files at google.code pointed to by the link in the first post in this thread.

 

If you have customized your unmenu.conf configuration file, rename it to unmenu_local.conf before un-zipping this latest version.

Varibles defined in the unmenu_local.conf will automatically be used where appropriate.

In fact, you need never modify the unmenu.conf file... Any and all changes should be put in the equivalent attributes in the unmenu_local.conf file.

It's going to take me days to document this utility, so bear with me as I get it in place.   

 

Biggest addition in this version is a "Package-Manager" to download and install "package extensions"   A sample set of package configuration files is included.

The package configuration file will automatically be MOVED to the /boot/packages folder when the package-manager plug-in is invoked.

 

Once a package is downloaded through the plug-in, it may be installed manually by simply pressing a single button on the web-interface.   After installation (and you test it works) you are presented a button to configure an "auto-re-install-on-reboot" for that package.   If you press that button your "go" script will have a line appended to it automatically ... or, if you already have the /boot/custom/etc/rc.d structure in place, a file will be put there instead of in the go script.     Only one line is ever appended to the "go" script.  It will auto-install all of the packages designated by you.

 

I'll go into a lot more detail at a later time, but for now, just be aware.  If you only want to manually install, don't enable the "auto-install-on-reboot" feature.    If you do enable it for a given package, the web-interface will then present a button to disable the auto-install-on-reboot for that package.  Pressing it will remove that package from being auto-installed in the future by removing the "packagename.auto_install" file.  (no harm done)

 

The auto-install feature creates a "packagename.auto_install" script in the /boot/packages folder.  Those scripts contain the commands needed for auto-installation and are run in turn on re-boot.  In the same way, the "manual-install" button creates a "packagename.manual_install" script.  It can be invoked to reinstall a package, or viewed to see what occurs when a package is installed.  If the /boot/packages folder does not exist, it will be created.   Most everything can be locally configured in unmenu_local.conf by copying it from the equivalent lines from unmenu.conf.

 

downloadable packages are defined in NN-unmenu*package*.conf files.  A given file can contain multiple package definitions.

 

A second huge change in this version is the way lines in the syslog viewer are matched and color coded.  There is now a syslog_match.conf file.  It contains a set of patterns for the syslog.   If you have additional lines you would like locally highlighted, for software you have loaded locally, create a file named "syslog_user_match.conf"   Those additional patterns will be added to the supplied list in syslog_match.conf.  In this way, a subsequent update of the supplied file will not overwrite your additions.

 

Thanks to everybody who contributed their time, ideas, and plug-in code to this effort.   If you develop a plug-in, please share it with us.  If you find a bug, let us know.   

 

If you are hesitant on making changes to your system, don't download and install this.

 

Oh yes, for those who have not been following this development.

To run it:

 

download and unzip in a folder on the flash drive. (I have mine in  the top level folder, mounted in linux at /boot.  In windows, this would be \\tower\flash )

log in via telnet and cd to that folder and type the following commands (assuming you unzipped in \\tower\flash\).

/boot/uu

 

If you unzipped in a sub-folder "/boot/unmenu" you would type:

/boot/unmenu/uu

 

 

Joe L.

Share this post


Link to post

I am installed new version of unraid menu. List of syslog in web browser now working and added other nice features.

 

Joe L. very hard and nice work. Thanks, very thanks.

Share this post


Link to post

Just an FYI... If you install this on unRAID 4.4beta2, the "smartctl" command as supplied by Tom does not work and therefore, the SMART tests, and temperature readings will not work.

 

You can download the missing cxxlibs-6.0.8-i486-4.tgz library "package" if you need the temperature readings and use of "SMART" features between now and when Tom releases the 4.4beta3 version of unRAID.

 

Since unmenu.awk now has a package manager you can download a "package configuration file" usable in the unmenu.awk package-manager.  It is an attachment to this post.

 

Download it, put it in the same folder with the other unmenu.awk distribution files, click on the "Package-Manager" link, and then, at the bottom of the page a button will exist to "Download" the missing library. (since it has the highest file number, it will be listed last) 

 

Press the "Download" button and the download will occur.  After the file is downloaded, a button will then change to allow you to manually install the package. 

 

After you press it to manually install the "package" the button will again change to permit you to enable an automatic re-install on reboot. 

 

It is up to you if you want to have it automatically install itself the next time you reboot.  Hopefully, that will be when Tom releases the next beta, and the library will then exist in his release.

 

Joe L.

 

 

Share this post


Link to post

Fantastic!  I installed unMenu and it works great...

 

Just wanted to say thanks, and let you know what a great job you've done here.

 

 

Share this post


Link to post

I just installed the new version of unMenu looks fantastic (drives on unprotected array showed spun down on your screenshot - I'm very happy  ;D ;D ;D).

 

Just curious is there a reason you're using version 1.2216 of NTFS-3G? I saw the new version is 1.5012 and I am actually using 1.2412.

 

Thank you for your hard work. More power!

Share this post


Link to post

I just installed the new version of unMenu looks fantastic (drives on unprotected array showed spun down on your screenshot - I'm very happy  ;D ;D ;D).

 

Just curious is there a reason you're using version 1.2216 of NTFS-3G? I saw the new version is 1.5012 and I am actually using 1.2412.

 

Thank you for your hard work. More power!

The 1.5012 release of ntfs-3g was about four days ago on Oct 11th.  I was not aware of it.  At the time I did my development 1.2216 was the most current stable release.

 

I've not done any testing of the newer releases, odds are they would work perfectly too.

I'll download and give the 1.5012 release a try. 

 

Thanks for the feedback.   The ability to read temperature of some drives, or spin them down just does not exist (the hdparm and smartctl commands are not usable on "usb" and "scsi" devices)  If that changes, I can take that limitation out of unmenu.awk.  (I'll probably just make it configurable in unmenu.conf)  On my array the unprotected drive shown in the screenshot is connected via a "usb" cable and is in an external "usb" case.

 

Update: ntfs-3g 1.5012 is not available as a Slackware compiled package anywhere I could find.  I'ts biggest claim to fame is the ability to create over 54 million files and directories in a single NTFS file-system.  Not a feature I really need, especially since I'm currently mounting my drive as read-only.   I did find the prior version, so I'll give it a try if I get a chance in the next few weeks.

 

Edit: Nov 17, 2008 ... an updated ntfs-3g 1.5012 package configuration file now is available here: http://lime-technology.com/forum/index.php?topic=2595.msg22439#msg22439

Joe L.

Share this post


Link to post

Hi Joe L.

 

I am trying to install "List open File package" but I am getting this error message.

 

lsof-4.78-i486-1.tgz installation:

 

Cannot install lsof-4.78-i486-1.tgz: package is corrupt (failed 'gzip -l lsof-4.78-i486-1.tgz')

 

lsof-4.78-i486-1.tgz apparently did not install properly, /usr/bin/lsof does not exist.

 

It tells me that MD5 doesn't macth and file may be corrupted. I went to /boot/package and delete both files manual_install and .tgz and redownload but got the same result.

 

Did I miss anything?

Share this post


Link to post

Hi Joe L.

 

I am trying to install "List open File package" but I am getting this error message.

 

lsof-4.78-i486-1.tgz installation:

 

Cannot install lsof-4.78-i486-1.tgz: package is corrupt (failed 'gzip -l lsof-4.78-i486-1.tgz')

 

lsof-4.78-i486-1.tgz apparently did not install properly, /usr/bin/lsof does not exist.

 

It tells me that MD5 doesn't macth and file may be corrupted. I went to /boot/package and delete both files manual_install and .tgz and redownload but got the same result.

 

Did I miss anything?

Try once more.  (It will probably also fail, but once it does, this time, do an ls -l to see its size)

ls -l /boot/packages/lsof-4.78-i486-1.tgz

-rwxr-xr-x 1 root root  313616 Sep 20 16:33 lsof-4.78-i486-1.tgz

If it is only a few hundred bytes or so, it is probably an error message... to see it type:

cat /boot/packages/lsof-4.78-i486-1.tgz

 

What does it say?

 

I just tried it myself and see a message like this:

root@Tower:/boot# cat packages/lsof*

f5

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<HTML><HEAD>

<TITLE>404 Not Found</TITLE>

</HEAD><BODY>

<H1>Not Found</H1>

The requested URL /slackware-current/slackware/ap/lsof-4.78-i486-1.tgz was not found on this server.<P>

</BODY></HTML>

 

0

 

Apparently, the link I included in the package.conf file is  no longer valid for the download.  (you were doing nothing wrong, but the package-manager is not currently coded to tell you of this type of error without looking at the downloaded contents)

 

The solution is a replacement download URL

Go to this URL:

http://packages.slackware.it/

 

Type the package file name in the search box:

lsof-4.78-i486-1.tgz

 

Choose the first package listed...

Or,

Click on this URL

http://packages.slackware.it/package.php?q=current/lsof-4.78-i486-1

 

Scroll to a server in your country that has an "http" download available. 

download from there to your packages folder, then you should be OK to use the package manager to continue.

 

Or...

Use the attached package conf file, with an updated URL.

 

I'll need to figure out a way to better handle this class of error message where the link in a conf file is no longer valid. 

 

Joe L.

 

Share this post


Link to post

@ Joe L.:

The attached package conf file works perfect. Thanks!

 

@ NAS:

Thank you for the gesture. That will help a lot of people want this to work.

Share this post


Link to post

Joe L.

I installed the latest version and it's really great! Thank for your efforts!

 

Is it possible to include options to enable/disable 30% read performance tweak and directory entry caching?

Share this post


Link to post

The read performance is in the new beta already ( I believe).

 

Dentry caching is not ready for prime time IMO and shouldn't be in it yet.

Share this post


Link to post

Is it possible to include options to enable/disable 30% read performance tweak and directory entry caching?

It is easier than you think to add this type of functionality to your own unmenu.

 

I just created three "user-script" files.  They are:

 

30-unmenu_user_script_setra-0256

#define USER_SCRIPT_LABEL Set Disk Read-Ahead  256

#define USER_SCRIPT_DESCR Set the "md" device read-ahead buffer size using <b>blockdev --setra 256 /dev/md1</b>

echo "<pre>"

for i in /dev/md*

do

echo Setting $i

  blockdev --setra 256 $i

  blockdev --getra $i

done

30-unmenu_user_script_setra-1024

#define USER_SCRIPT_LABEL Set Disk Read-Ahead 1024

#define USER_SCRIPT_DESCR Set the "md" device read-ahead buffer size using <b>blockdev --setra 1024 /dev/md1</b>

echo "<pre>"

for i in /dev/md*

do

echo Setting $i

  blockdev --setra 1024 $i

  blockdev --getra $i

done

30-unmenu_user_script_setra-2048

#define USER_SCRIPT_LABEL Set Disk Read-Ahead 2048

#define USER_SCRIPT_DESCR Set the "md" device read-ahead buffer size using <b>blockdev --setra 2048 /dev/md1</b>

echo "<pre>"

for i in /dev/md*

do

echo Setting $i

  blockdev --setra 2048 $i

  blockdev --getra $i

done

 

The 4.4-beta2 version of unRAID seems to set the read-ahead to 1024.  It used to be 256 in older versions of unRAID.  In my script I was setting it to 2048.   You can now experiment to see how the different values work by pressing the appropriate button on the "User-Scripts" page in unmenu.

 

To help you, I've attached a zip file with four user-script files.  ( I've also included a button to get the current read-ahead value.)

 

Unzip in the folder with your other unmenu.awk plug-in files, then use the "User-Scripts" page to access the new buttons.

 

I'll let you create the equivalent "user-script" button file for the directory cache setting... just remember to do it in an editor that does not add trailing carriage returns, or run it through "fromdos" when you are done editing it. To be used as a button, its file name must start with a number and have unmenu_user_script as part of its name.  (Obviously, it must also have the two fields inside of it to define the label and description)

 

Oh yes,at one point it was discovered it was only necessary to set the read-ahead on one of the "md" devices...

At that time, there was no need to loop through them all, they shared the same buffer setting (and perhaps the same buffer). 

This has apparently changed in the newest unRAID.   Setting /dev/md1 did not affect /dev/md2 or any of the others.   

Therefore, we DO need to set each device in a loop..

 

Joe L.

 

Share this post


Link to post

Thank you Joe L., that was quick again!

I'll see where can I arrive with my own skills to experiment with the dir entry caching.

Share this post


Link to post
or, if you already have the /boot/custom/etc/rc.d structure in place, a file will be put there instead of in the go script

 

Could someone give me a clue as to how to implement the "rc.d structure".  I added the /boot/custom/etc/rc.d directory and unmenu added the necessarry file to that directory, but the packages don't auto install.  I'm missing a step, aren't I?  I searched the forum as I figured this was something discussed prior but I couldn't find anything.

Share this post


Link to post

or, if you already have the /boot/custom/etc/rc.d structure in place, a file will be put there instead of in the go script

 

Could someone give me a clue as to how to implement the "rc.d structure".  I added the /boot/custom/etc/rc.d directory and unmenu added the necessarry file to that directory, but the packages don't auto install.  I'm missing a step, aren't I?  I searched the forum as I figured this was something discussed prior but I couldn't find anything.

 

What is missing was discussed in this thread: http://lime-technology.com/forum/index.php?topic=1953.0

 

I have the following script (shown below in blue, and also in the attached zip file), similar to the one discussed in that thread, located in the /boot/custom/etc/rc.d directory:

 

root@Tower:/boot/custom/etc/rc.d# cat rc.local_startup

#!/bin/bash

logger -trc.local_startup -plocal7.info -is "Initiating Local Custom Startup."

scripts=`ls /boot/custom/etc/rc.d/ | egrep -v "rc.local_startup|rc.local_shutdown"`

cd /boot/custom/etc/rc.d

for script in $scripts

do scriptbase=${script##*/}      # Strip pathname

   ( echo "Processing $script"

        sh -xv $script

   ) 2>&1 | logger -t$scriptbase -plocal7.info -is

done

 

It is invoked by a single line at the bottom of my "go" script

#!/bin/bash

# Start the Management Utility

/usr/local/sbin/emhttp &

fromdos < /boot/custom/etc/rc.d/rc.local_startup | sh

 

I expect the script will need to change when we finally get hooks into the start-up and shut-down functions of the unRAID management interface.

 

For now, the one line in the "go" script invokes the rs.local_startup script located in /boot/custom/etc/rc.d

It in turn invokes all the other scripts located in /boot/custom/etc/rc.d

 

I have this script in the same folder so it will execute first and make sure the array is started before I continue:

root@Tower:/boot/custom/etc/rc.d# cat S05-sleep

sleep 30

 

Don't forget to make the scripts execuitable. 

cd /boot/custom/etc/rc.d

chmod +x rc.local_startup

chmod +x S05-sleep

 

To make it easy, I've attached a zip file with the two files.

Do not leave the .zip file in the rc.d folder.  Extract the two files and put them there.

You will then have three files, these two and the S10-install_custom_packages file put there by the unmenu.awk "Package Mgmt" plug-in.

 

Joe L.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.