An Improved unRAID web-interface, extensible, and easy to install


Recommended Posts

  • Replies 552
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Posted Images

Grab the three 1.2 version zip files from the google.code link

Awesome man, thanks.  Downloaded and "installed" yesterday.  Ran fine with uu.  Quick way to make it run automatically on boot?

 

Cheers!  Great work!

 

Robbie

Most people just add two lines to the end of their config/go script.

 

sleep 30

/boot/unmenu/uu

 

By the way, I added a SSH package to the unMENU set of packages.  I figure you might give it a try.  You will need to define ids/passwords as I don't think it will allow a null password.   I know the openssl library it needs is in 4.5beta6.  I don't know if it is in 4.4.2, but I also have the download package in unMENU for openssl if it is needed.  Just install it first.

 

We did not have a web-server, other than the minimal one built into the emhttp process, and that one was closed to us and not extensible, so I wrote a web-server using only what was on the unRAID distribution... unMENU is mostly all written in "GNU awk"

 

If you ever need a true web-server for php pages, one member wrote an improved front-end to unRAID named "unRaid-Web" using lighttpd and it also supports "extensions."  I wrote an "unMENU" extension to allow use of the unMENU plug-ins from within it.  So... most of everything was re-usable.  Wait till you see the file-explorer in it, written using ajax,  ajaXplorer is really nice.  (login id/password = admin/admin till you change it)

Find it here: http://lime-technology.com/forum/index.php?topic=3354.0

 

As you yourself said, this is Linux... and we can make it do almost anything we need with a bit of configuration.   Version 5 of unRAID (some day) will probably also be using lighttpd as the server, with an API to emhttp to control the array itself.

 

In the interim, you can see unRAID-Web, and once it is installed, have access to php as an interpreter.

 

Joe L.

Link to comment

Very cool, thanks Joe.

 

Wow, if they added PHP as an interpreter, I could provide contributions!  :)

 

Some super-sexy CSS would be a nice touch too.

 

I create a lot of web-based Operating System layouts... they could beautifully transcend to an interface for unRAID.

 

Hmm... now you got me thinking I'm gonna start some coding  ;)

 

Any collaborators?  :)

 

Thanks for adding ssh Joe.  I'll positively adore using that on my main Ubuntu box.  I'm at some point going to need to setup user-level samba as well... the "root" 777 access really doesn't cut it for me.

 

Adding uu to my go script now, thanks!!  Looking forward to looking around!

 

Robbie

Link to comment

Hey Joe,

Just a quick mention... well, first off - LOVE the package manager idea, and it looks well implemented.  Makes it so easy for someone [like me] who doesn't really want to learn slackware  :)

 

Here's my note:

I installed the update thinger for rsync.  I told it yeah, I want it to reinstall at boot...

 

So it added:  cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

However, it did not first add an \r\n.  Therefore, it's expecting (potentially erroneously) that the last line of the go script is blank.

 

In my case, this broke the autostart of unmenu, because the last line of my go script now looked like this:  /boot/unmenu/uucd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

I'm still in the process of growing my array, but will really get in-depth into everything (including unmenu) once that's all done.

 

Thanks for an outstanding (over the top) contrib!!

 

Robbie

Link to comment

I fully expected to see something like "loop"

 

Are you doing anything unusual configured with your "devices"? any loop-back devices mounted?

 

If you type this command, anything like "loo???" listed?

cat /proc/partitions | grep -v '^major' | grep -v '^$' 

 

When I run the above command I get this as the printout:

   7     0      10240 loop0
   8     0     990720 sda
   8     1     990704 sda1
   8    16  732574584 sdb
   8    17  732574552 sdb1
   8    32  488386584 sdc
   8    33  488386552 sdc1
   8    48  156290904 sdd
   8    49  156290872 sdd1
   8    64 1465138584 sde
   8    65 1465138552 sde1
   8    80  488386584 sdf
   8    81  488386552 sdf1
   8    96 1465138584 sdg
   8    97 1465138552 sdg1
   8   112  732574584 sdh
   8   113  732574552 sdh1
   8   128  976762584 sdi
   8   129  976762552 sdi1
   9     1  732574552 md1
   9     2  488386552 md2
   9     3  488386552 md3
   9     4  732574552 md4
   9     5  976762552 md5
   9     6 1465138552 md6

 

Sorry it took so long to get this back to you.  I decided to shut down the server just in case, but was not able to grab a syslog (does not really matter anyway, as this unmenu line is the only "problem" I have seen with it).

 

EDIT: I am fairly certain that I am not doing anything special with my setup.  I have the Third party Boot Flash Plugin Architecture setup, unmenu installed, BubbaRAID running (with rtorrent & wtorrent, apcupsd, and u-notify), openssh, openssl, crashplan, and a few other extras but nothing that I know of that is causing this problem.

 

I never have this problem with the previous version of unmenu and the my-main plugin.

Link to comment

Ok... now I see where the "loo" is coming from.  The myMain is taking the first three characters of each partition to get "sda", "sdb", etc.

 

It was not coded to handle a loop device.

 

It looks like something you are running is mounting a file-system via the /dev/loop0 device.

 

The error message in the syslog is still harmless... but I'll see if I can figure out the best way to get rid of it.

 

Joe L.

 

 

Link to comment

Ok... now I see where the "loo" is coming from.   The myMain is taking the first three characters of each partition to get "sda", "sdb", etc.

 

It was not coded to handle a loop device.

 

It looks like something you are running is mounting a file-system via the /dev/loop0 device.

 

The error message in the syslog is still harmless... but I'll see if I can figure out the best way to get rid of it.

 

Joe L.

 

yeah, I figured it was harmless but if you can do something to get rid of it that would be great.  I would prefer to not see it in my syslog and see it repeating every 5 minutes.

 

If you want any more information or can give me some pointers on what might be causing the "loop" that would be great

 

EDIT: I do have these three lines in my syslog that report something about loop0

Sep 26 20:39:52 Tower kernel: kjournald starting. Commit interval 5 seconds
Sep 26 20:39:52 Tower kernel: EXT3 FS on loop0, internal journal
Sep 26 20:39:52 Tower kernel: EXT3-fs: mounted filesystem with ordered data mode.

 

I think this has something to do with BubbaRAID but I am not positive.

Link to comment

EDIT: I do have these three lines in my syslog that report something about loop0

Sep 26 20:39:52 Tower kernel: kjournald starting. Commit interval 5 seconds
Sep 26 20:39:52 Tower kernel: EXT3 FS on loop0, internal journal
Sep 26 20:39:52 Tower kernel: EXT3-fs: mounted filesystem with ordered data mode.

 

I think this has something to do with BubbaRAID but I am not positive.

Yes, it is BubbaRAID mounting an EXT3 file-system via a loop device.

 

A quick fix for the error is for you to edit the unmenu.base.lib.awk file and to change one line.

 

That line is near the very end of the file in a function named GetReadWriteStatsOther()

 

It currently looks like this:

function GetReadWriteStatsOther(theDevIndex, theDevice, cmd) { #bjp999
  cmd="cat /sys/block/" theDevice "/stat"                      #bjp999
  cmd | getline                                                #bjp999
  other_reads[theDevIndex] = $1                                #bjp999
  other_writes[theDevIndex] = $5                               #bjp999
  close(cmd)                                                   #bjp999
}                                                              #bjp999

You need to change the second line in it (line 212 of the file) from

  cmd="cat /sys/block/" theDevice "/stat"                      #bjp999

to

cmd="cat /sys/block/" theDevice "/stat 2>/dev/null "     #bjp999

 

That should eliminate the error from your syslog.   

Simplest way to do edit the file is to paste the following line at a command prompt.  It will use the "stream editor" in Linux to change the file to add the extra " 2>/dev/null" on the correct line.

 

sed -i "sXtheDevice \"/stat\"XtheDevice \"/stat 2>/dev/null \"X" /boot/unmenu/unmenu.base.lib.awk

Of course, if you want, you can edit the file in a editor of your choice and not use the line above, but make sure you use an editor that properly deals with Linux files and does not add MS-DOS style carriage returns on the ends of the lines..  Easiest is probably the "sed" script I gave you above.

 

Joe L. 

Link to comment

that seems to have worked just fine, thank you.

 

one other thing that I have noticed after installing this version is the emhttp is going by by after a while. I have no idea why, and nothing is reported in the syslog either about it being killed.

 

A ps -ef shows that it is not running and a free returns with:

               total               used            free      shared         buffers          cached
Mem:       2074592    1989128        85464              0            8648        1884732
-/+ buffers/cache:        95748    1978844
Swap:            0          0          0

 

Thanks

Link to comment

Hey Joe,

Just a quick mention... well, first off - LOVE the package manager idea, and it looks well implemented.  Makes it so easy for someone [like me] who doesn't really want to learn slackware  :)

 

Here's my note:

I installed the update thinger for rsync.  I told it yeah, I want it to reinstall at boot...

 

So it added:  cd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

However, it did not first add an \r\n.  Therefore, it's expecting (potentially erroneously) that the last line of the go script is blank.

 

In my case, this broke the autostart of unmenu, because the last line of my go script now looked like this:  /boot/unmenu/uucd /boot/packages && find . -name '*.auto_install' -type f -print | sort | xargs -n1 sh -c

 

I'm still in the process of growing my array, but will really get in-depth into everything (including unmenu) once that's all done.

 

Thanks for an outstanding (over the top) contrib!!

 

Robbie

Actually, it expected the prior line to have a trailing newline.  Apparently when you added the one line to start unMENU, you did not press "enter" at the end of the line.

 

But... you bring up a really good point.  I'll add in a leading newline when I append my line to invoke the "auto_install" scripts, and it will work either way.  That will have to wait for the next release of the "package manager" but I've got some ideas for improving it... so it won't take that long before I put out a new version.

 

Glad you like the package-manager.  Not too shabby for a web-server AND set of pages written using "awk"  Not exactly "apt-get," but perhaps a little better in some ways for real beginners.  The .conf files for the packages are pretty easy to create, and a lot of people have contributed them in the effort. 

 

Wait till you see how easy it can be to write a "plug-in" or a "user-script" under unMENU..  until then... have fun.

 

Joe L.

Link to comment

that seems to have worked just fine, thank you.

 

one other thing that I have noticed after installing this version is the emhttp is going by by after a while. I have no idea why, and nothing is reported in the syslog either about it being killed.

 

A ps -ef shows that it is not running and a free returns with:

               total               used            free      shared         buffers          cached
Mem:       2074592    1989128        85464              0            8648        1884732
-/+ buffers/cache:        95748    1978844
Swap:            0          0          0

 

Thanks

Interesting...  You can re-start it with

nohup /usr/local/sbin/emhttp &

 

I have no clue about this... unless I'm killing it when attempting to un-mount a disk since it has an open file-descriptor to it. 

Were you doing any reiserfsck tests of your disks?  (that is when it would be using that logic)

 

Joe L.

Link to comment

Interesting...  You can re-start it with

nohup /usr/local/sbin/emhttp &

 

I have no clue about this... unless I'm killing it when attempting to un-mount a disk since it has an open file-descriptor to it. 

Were you doing any reiserfsck tests of your disks?  (that is when it would be using that logic)

 

Joe L.

 

I have been restarting it with the above command so that is working fine.  I wil try and see if there is anything odd with the machine when I next see emhttp die.  I had never noticed emhttp disappearing before the 1.2 version of unmenu but weirder things could be happening.  I now for a fact that I was not doing an reiserfsck related stuff on the drives at any point in time.

Link to comment

Joe,

 

Many Thanks for a great addon. As a new user of unraid, i found it easy to install and use.

 

I made the following changes as you mentioned in previous posts, and all seems to work well so far.

 

Downloaded libxml-unmenu-package.conf attached to your post earlier in the thread.

 

Around line 115 of 08-unmenu-array_mgmt.awk, change the word "disk\'s" to "disk"

 

Around line 47 of 600-unmenu-file_edit.awk, add the "2>/dev/null" as shown below.

 

Change it from:

editable_files = GetEditableFiles(editable_files, CONFIG["AUTO_INSTALL_DIRECTORY"], "* | grep -v '*.zip'" )

to:

editable_files = GetEditableFiles(editable_files, CONFIG["AUTO_INSTALL_DIRECTORY"], "* 2>/dev/null | grep -v '*.zip'" )

 

One thing i had a problem with was the link in the ntfs-3g-2009-3.8-unmenu-package.conf packaage was wrong. I tryed to find a working link, after 6 or 7 trys, i gave up and just copied the one i downloaded previously into the packages folder on the flash drive.

 

Thanks again for all your hard work on unMenu.

 

Gary

Link to comment

Joe,

 

Many Thanks for a great addon. As a new user of unraid, i found it easy to install and use.

 

I made the following changes as you mentioned in previous posts, and all seems to work well so far.

 

Downloaded libxml-unmenu-package.conf attached to your post earlier in the thread.

 

Around line 115 of 08-unmenu-array_mgmt.awk, change the word "disk\'s" to "disk"

 

Around line 47 of 600-unmenu-file_edit.awk, add the "2>/dev/null" as shown below.

 

Change it from:

editable_files = GetEditableFiles(editable_files, CONFIG["AUTO_INSTALL_DIRECTORY"], "* | grep -v '*.zip'" )

to:

editable_files = GetEditableFiles(editable_files, CONFIG["AUTO_INSTALL_DIRECTORY"], "* 2>/dev/null | grep -v '*.zip'" )

 

One thing i had a problem with was the link in the ntfs-3g-2009-3.8-unmenu-package.conf packaage was wrong. I tryed to find a working link, after 6 or 7 trys, i gave up and just copied the one i downloaded previously into the packages folder on the flash drive.

 

Thanks again for all your hard work on unMenu.

 

Gary

This link seems to work... at least for now, but since it is for "ftp" it will not work in the package manager in unMENU.  You can use it to download the .tgz file to your packages directory and then proceed from there:

ftp://ftp.biochem.uthscsa.edu/pub/linux/slackware/slackware-current/slackware/a/ntfs-3g-2009.3.8-i486-1.tgz

 

Since it has "slackware-current" as part of its path, it too will go out of date once they upgrade to a more recent 'current' version.

 

Joe L.

Link to comment

I have quite a few ISO files on my unRAID servers.

However, I do have an old media player that can only play VOB files.

Is there anyway I can mount the ISO file using unMenu and then share it out? Either through a script or some other means...appreciate the help!  :)

 

It certainly is possible to do what you ask at the linux command line, but there is no plug-in in unMENU written to do it.

 

Are all your ISO images in a single folder?  or are they in sub-folders?

 

The biggest questions are

  How to select a specific ISO you wish to mount

  Where to mount the ISO.  do we name the mount-point after the ISO file name?

  Where do we "share" the mounted directory.  What name do we give it on the LAN.  (I think a specific fixed name is best, but that would allow only one movie to be mounted and shared at a given time... it that sufficient?)

 

  How do we un-share and un-mount the ISO image... If we do not, the unRAID array will not be able to stop properly, and all those disks that can be un-mounted will show as un-formatted.  (No harm unless you press the "Format" button, but still is frightening when you first see it)

 

If you wish to experiment to see if it will work at all, I can give you the linux commands to try, but first I could use some feedback by answering the questions above.

 

Joe L.

 

 

Link to comment

It certainly is possible to do what you ask at the linux command line, but there is no plug-in in unMENU written to do it.

 

Are all your ISO images in a single folder?  or are they in sub-folders?

 

For ease of use and testing, I will consolidate all ISOs in a single folder.

 

The biggest questions are

  How to select a specific ISO you wish to mount

 

Ideally, it should list all the ISOs in the single folder I have fixed/designated.

Then allow me to choose one to be mounted.

 

  Where to mount the ISO.  do we name the mount-point after the ISO file name?

  Where do we "share" the mounted directory.  What name do we give it on the LAN.  (I think a specific fixed name is best, but that would allow only one movie to be mounted and shared at a given time... it that sufficient?)

 

For a start, it will be enough to be able to mount one ISO at a given time.

Both the mount-point and shared name can be fixed in this case.

 

  How do we un-share and un-mount the ISO image... If we do not, the unRAID array will not be able to stop properly, and all those disks that can be un-mounted will show as un-formatted.  (No harm unless you press the "Format" button, but still is frightening when you first see it)

 

Hoping that I can do it eventually using a web interface similar to un-sharing and un-mounting drives using Disk Mgt in unMenu.

 

If you wish to experiment to see if it will work at all, I can give you the linux commands to try, but first I could use some feedback by answering the questions above.

 

I'm a sucker for experimentation. Bring it on!  ;D

 

 

 

Link to comment

I'm a sucker for experimentation. Bring it on!  ;D

You asked for it...

 

Here is a new plug-in.  Designed to share ISO images as their mounted contents. (As read-only)

The interface is similar to the file-browser, but it only lists ISO files.  At the right is a link per ISO file to "Share" it, or "UnShare" it.

(If currently shared, the link will be to un-share it)

 

It looks like this:

13ydpiw.jpg

 

When you "Share" an ISO image, it is first mounted via a loop device.   Then, the samba configuration is automatically edited to add the new share.

The new share is named after the ISO image, but without the .ISO suffix, and without any single quotes.

 

Any currently "loop shared" ISO images are printed at the top of the screen.

 

I've not tested all the possible weird characters you can put in an ISO image file name, so you'll need to let me know if something specific does not work.

 

At the linux command line you can see the loop device status by typing:

losetup -a

It will print the currently loop mounted ISO images.

 

You can also see where they are mounted by typing

mount

I'm creating the mount points (empty directories) named after the ISO files under /var/tmp/mnt

 

You can only mount 8 ISO images, as there are only 8 loop devices.  If you are using a loop device for something else, the limit might be even less.  (I know BubbaRAID uses a loop device for itself, so that would leave 7 for ISO image use)

 

Lastly, if you attempt to stop the array using the unRAID management console while having Shared ISO images in place you will find it unable to stop the array and it will likely show drives it can un-mount prior to stopping the array as Unformatted.  Do not panic, they are not un-formatted.  Do NOT press the format button it then displays... or you will reformat your own disk, with your data... ouch... you have been warned.

 

So... before stopping the array, un-share any ISO images you have shared.  Then the stop button should work normally.  If you forget, just un-share and you should be able to press the "Stop" button once more.   

 

To install, just download and un-zip the attached file to your unMENU folder, then stop and re-start unMENU so it will be added to the top menu.

 

Let me know how it works for you.  An unknown is that some older SMB clients cannot deal with share names longer than 14 characters... you might need to name your movies accordingly if none show up in your old media-player file-browser.

 

I named this a Share ISO plug-in, as someone may also use it to mount any ISO image, including file-system-backups, and Linux live images.

 

Edit: version 1.1 posted... fixed first set of bugs... added ability to UnShare from list of Shared ISO at top of screen.

 

Joe L.

 

 

Link to comment

This is very cool.  My only difficulty with it is having to dig down into the file structure to unmount iso's prior to stopping the array.

 

I tried to add a link to unmount iso's from the front page listing of mounted iso's by changing line 520 to:

 

out_str = out_str loop_iso[a] "<a href=\"http://" MyHost ":" MyPort "/share_iso?dir=" loop_iso[a] "&unshare_iso=" loop_iso[a] "\"> UnShare ISO</a><br>"

 

It only kinda works.  When you click the link it throws the following error: "Sorry: Access to invalid-directory/ is not permitted."

However, it does appear to unmount the iso.

 

I admit... I don't know AWK syntax at all.

 

It would be nice if the Array Management plugin could close all loop devices when the 'Stop Array' button were pressed.

 

In my testing I accidently mounted "backup.iso.uibak".  It mounted it as "backup.iso" but since that file doesn't really exist (notice the file extension) I am unable to unmount it!!

 

Link to comment

This is very cool.  My only difficulty with it is having to dig down into the file structure to unmount iso's prior to stopping the array.

 

I tried to add a link to unmount iso's from the front page listing of mounted iso's by changing line 520 to:

 

out_str = out_str loop_iso[a] "<a href=\"http://" MyHost ":" MyPort "/share_iso?dir=" loop_iso[a] "&unshare_iso=" loop_iso[a] "\"> UnShare ISO</a><br>"

 

It only kinda works.  When you click the link it throws the following error: "Sorry: Access to invalid-directory/ is not permitted."

However, it does appear to unmount the iso.

 

I admit... I don't know AWK syntax at all.

 

It would be nice if the Array Management plugin could close all loop devices when the 'Stop Array' button were pressed.

 

In my testing I accidently mounted "backup.iso.uibak".  It mounted it as "backup.iso" but since that file doesn't really exist (notice the file extension) I am unable to unmount it!!

 

You  can use losetup to take down the loop device.  ( I think it is losetup -d device )

 

I know I check for files open on the loop devices in the unMENU Admin page when I go to un-mount the disks.  I did not test if it would take down the loop devices set up by the new plug-in.  Odds are it will.  You can test it and let me know. 

 

I'll add something at the top of the page to unShare the ISO images.   As you said, it is not as easy...   Just think of this as the "alpha" version.

 

Joe L.

Link to comment

I am not quite sure what I did but I am getting this output when I try to start unmenu via the uu

 

unmenu[19548]: awk: ./unmenu.awk:72:   main_page_user_content = CONFIG["main_page_user_content*] ? CONFIG["main_page_user_content"]$: ""
unmenu[19548]: awk: ./unmenu.awk:72:                                                                                                   ^ unterminated string
unmenu[19548]: exit status 1

 

The only thing I think I have done is add the plugin just put up, and edit the file specified a few posts ago.  Any help is appreciate!

Link to comment

I am not quite sure what I did but I am getting this output when I try to start unmenu via the uu

 

unmenu[19548]: awk: ./unmenu.awk:72:   main_page_user_content = CONFIG["main_page_user_content*] ? CONFIG["main_page_user_content"]$: ""
unmenu[19548]: awk: ./unmenu.awk:72:                                                                                                   ^ unterminated string
unmenu[19548]: exit status 1

 

The only thing I think I have done is add the plugin just put up, and edit the file specified a few posts ago.  Any help is appreciate!

You probably introduced an error when you edited the file.  Did you use an editor that used unix newlines at the ends of the lines, or MD-DOS style newline/carriage return pairs?

 

I'd start with a fresh copy of unmenu.awk.

Link to comment

I am not quite sure what I did but I am getting this output when I try to start unmenu via the uu

 

unmenu[19548]: awk: ./unmenu.awk:72:   main_page_user_content = CONFIG["main_page_user_content*] ? CONFIG["main_page_user_content"]$: ""
unmenu[19548]: awk: ./unmenu.awk:72:                                                                                                   ^ unterminated string
unmenu[19548]: exit status 1

 

The only thing I think I have done is add the plugin just put up, and edit the file specified a few posts ago.  Any help is appreciate!

You probably introduced an error when you edited the file.  Did you use an editor that used unix newlines at the ends of the lines, or MD-DOS style newline/carriage return pairs?

 

I'd start with a fresh copy of unmenu.awk.

 

Yup, that did indeed fix it.  I assumed it would but I wanted to see what you thought before I did anything, thanks JoeL

Link to comment

TEST RESULTS: I don't think the 'stop array' button closes loop devices.  When I tried to stop the array while an ISO was mounted the screen went blank with "e 0 " in the top left corner.  Then when I went back into unmenu the ISO was still mounted. 

Link to comment
  • Squid locked this topic
Guest
This topic is now closed to further replies.