How does the plugin system work? Documentation Added - WIP


dlandon

Recommended Posts

Next, I would like to make a feature request to the "plugin" manager which allows plugins to be installed as "system plugin", meaning they get installed in the directory '/boot/plugins' instead of '/boot/config/plugins'.

 

What is the significance of installing in /boot/plugins?  Does it affect the way that the plugin operates?

Link to comment
  • Replies 195
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Posted Images

Next, I would like to make a feature request to the "plugin" manager which allows plugins to be installed as "system plugin", meaning they get installed in the directory '/boot/plugins' instead of '/boot/config/plugins'.

 

What is the significance of installing in /boot/plugins?  Does it affect the way that the plugin operates?

As far as I can tell, it is only a simple way to control timing.  Some things have to be installed before others.  So far, it only appears to have been used to load base plugins, the platforms upon which other plugins will install.  The distinction 'system' has so far just meant that it should load before other plugins.  Tom, dlandon, and others may know more though.

Link to comment

Your explanation is right. A system plugin gets installed before a user plugin. This becomes important when a dependency exists and certain plugins (e.g. Dynamix webGui) need to be installed first to create the correct foundation for other plugins to operate.

 

Other than that there are no differences between both in terms of execution or privileges.

Link to comment

Your explanation is right. A system plugin gets installed before a user plugin. This becomes important when a dependency exists and certain plugins (e.g. Dynamix webGui) need to be installed first to create the correct foundation for other plugins to operate.

 

What sort of dependencies exist at "install time" for a plugin?  I really would like to avoid having to introduce plugin install ordering mechanisms.  I want to eliminate the 'system' plugin concept, and it's only left in unRaid-6 as a way (if necessary) to update the plugin  manager itself.  For example, a new 'plugin' script could be installed using system plugin so that all subsequent user plugins get installed with this new script, but I've never had to do this.

 

I agree the 'Stale Plugins' idea is not very useful and is a source of confusion.  Probably this will get eliminated from the webGui but the 'plugin' script will still maintain the 'plugins-stale' directory on the flash.

Link to comment

The plugin dependency which exists in Dynamix comes from the fact that the Dynamix webGui adds several javascript libraries which do not exist in the stock webGui. The same is true for additional API functionality to store and retrieve settings.

 

The optional Dynamix plugins rely on these libraries and API calls to function properly, hence a check is performed by each optional plugin to see whether the Dynamix webGui is present. This system can only work when the Dynamix webGui is installed before any other plugin.

 

Link to comment

Solved by using the 'at' command to schedule the script to run.  The rc.snap start calls a script that puts an inotify script in the background.

 

<!--
Snap start up script.
-->
<FILE Name="/tmp/start_snap.sh">
<INLINE>
<![CDATA[
#!/bin/bash
/etc/rc.d/rc.snap start

rm /tmp/start_snap.sh
]]>
</INLINE>
</FILE>

<!--
Snap install.
-->
<FILE Name="/tmp/SnapInstall.sh" Run="/bin/bash">
<INLINE>
<![CDATA[
#!/bin/bash
#
# Start SNAP and mount any SNAP (usb) drives found.
#
echo "Starting Snap..."
at -f/tmp/start_snap.sh now

echo "Snap installation complete..."
rm /tmp/SnapInstall.sh
]]>
</INLINE>
</FILE>

 

I ran in exactly the same issue, plugins which need to start a background process get stuck upon installation. This was working previously under v5 when plugins were installed directly from the CLI.

 

The proposal of dlandon is working, but I find it a bit cumbersome, ideally I like to have the one liner working again, so the example above really should be just the line:

 

/etc/rc.d/rc.snap start

 

Link to comment

For those who are not familiar with setsid, here's some information for starting the program in the background in a new session id.

http://linux.die.net/man/1/setsid

 

root@unRAID:~# setsid

usage: setsid program [arg ...]

 

root@unRAID:~# setsid sleep 30

 

root@unRAID:~# pgrep sleep

21635

 

root@unRAID:~# ps -fp 21635

UID        PID  PPID  C STIME TTY          TIME CMD

root    21635    1  0 11:14 ?        00:00:00 sleep 30

Link to comment

Tried "setsid" but it holds plugin installation, like when executing the command directly.

 

The "at" approach makes the plugin installation continue.

 

:o  It shouldn't.  It does a fork() then setsid() and execvp(). 

 

root@unRAID:~# strings /usr/bin/setsid | egrep 'fork|setsid|exec'

fork

execvp

setsid

fork

setsid

execvp

 

*shrug* if /usr/bin/at works, so be it!

Link to comment
  • 2 weeks later...

I can't seem to get my icon or description from the README.md to show on the plugin manager page.  Is anyone able to have a look at this plugin to see what I'm doing wrong? 

 

Oddly enough, I had the icon working at one point, but now it just doesn't want to.

 

As a start I think these:

INSTALLDIR="/usr/local/&name;"
CONFIGDIR="/usr/local/&name;/config"

 

Should be:

INSTALLDIR="/usr/local/emhttp:/plugins/&name;"
CONFIGDIR="/usr/local/emhttp/plugins/&name;/config"

 

The plugins are installed in:

/usr/local/emhttp/plugins/

 

EDIT: I also see an issue here:

<!-- Create menu under settings tab -->
<FILE Name="/usr/local/emhttp/plugins/webGui/&author;.page">
<INLINE>
Author="&authorName;"
Version="1.0"
Title="&author; Plugins"
Menu="Settings"
Type="menu"
</INLINE>
</FILE>

 

Change to:

<!-- Create menu under settings tab -->
<FILE Name="/usr/local/emhttp/plugins/&name;/&name;.page">
<INLINE>
Author="&authorName;"
Version="1.0"
Title="&author; Plugins"
Menu="Settings"
Type="menu"
</INLINE>
</FILE>

Link to comment

I can't seem to get my icon or description from the README.md to show on the plugin manager page.  Is anyone able to have a look at this plugin to see what I'm doing wrong? 

 

Oddly enough, I had the icon working at one point, but now it just doesn't want to.

 

The answer turned out to be much easier that I thought.  I loaded the plugin on my test server.

 

Here is the change you need to make:

<PLUGIN  name="&displayName;"
         author="&author;"
         version="&plgVersion;"
         category="&category;"
         pluginURL="&pluginURL;">

 

Change to this:

<PLUGIN  name="&name;"
         author="&author;"
         version="&plgVersion;"
         category="&category;"
         pluginURL="&pluginURL;">

 

And it works.

Link to comment

I can't seem to get my icon or description from the README.md to show on the plugin manager page.  Is anyone able to have a look at this plugin to see what I'm doing wrong? 

 

Oddly enough, I had the icon working at one point, but now it just doesn't want to.

 

After the plugin is removed, you leave a lot of downloaded Slackware packages behind.  I'm not sure that is a good idea.

Link to comment

Thanks! The installdir and configdir are the default locations where the apps are installed, not the plugin install location.

 

I forgot I changed that to displayname to test (doh).

 

As for the packages, they are shared throughout a lot of my plugins, and deleting it with one plugin will make them unavailable to another plugin. For example, the python package is used by 3 or 4 plugins and the git package is shared by at least 6. I'm not sure how to handle that situation yet.

 

Thanks for finding the problem so fast!

Link to comment

Thanks! The installdir and configdir are the default locations where the apps are installed, not the plugin install location.

 

I forgot I changed that to displayname to test (doh).

 

As for the packages, they are shared throughout a lot of my plugins, and deleting it with one plugin will make them unavailable to another plugin. For example, the python package is used by 3 or 4 plugins and the git package is shared by at least 6. I'm not sure how to handle that situation yet.

 

Thanks for finding the problem so fast!

In appsupport you could host all packages  so a link never gets broken.  I know you already have the unraid5 and 6 repos.  You could create a separate repo for each plugin esp for unraid6 versions and drop the 64 from the plugin name (makes variables easier). You then strip out all the files created by the plg script and just host them in the plg repo. It makes managing them a little easier.  Look in my github e.g.  virtMan (the easiest), webvirtmgr, apache, vmMan maybe for some ideas. Recently it took me a few minutes to make a plg for vmMan since I already had a template from the others. I know you and dlandon's plugins have given me lots of ideas.

Link to comment

Also I use a linux app git-cola but I'm sure there's windows apps. I have  each plg's repo cloned to my laptop hard drive. Then after I make changes and commit each file, I create tag e.g. 2014.10.24 then push commits plus tag.  I usually forget to change date in plg or check push tags. :) Sorry if I'm explaining stuff you guys already know but I had to figure it out for myself so maybe it helps someone.

Link to comment

Yea but the problem still remains that if someone installs two of my plugins that use java, then uninstalled one of them, should each plugin remove their dependencies? If so, removing one plugin will erase the java install file which means on next reboot it needs to be downloaded again for the remaining plugin and it's a fairy hefty file size.

Link to comment

Yea but the problem still remains that if someone installs two of my plugins that use java, then uninstalled one of them, should each plugin remove their dependencies? If so, removing one plugin will erase the java install file which means on next reboot it needs to be downloaded again for the remaining plugin and it's a fairy hefty file size.

Do you mean in /boot/packages?  I  debated things like this too.  Like should you remove /boot/config/plugins/plg_folder?  Its too easy to accidentally hit the remove plugin link. If your talking a few MB then I say just leave it. Let the user delete those if they need flash drive space that bad. I do have it remove the old plg tar when updating. Then it gits the new tar.

 

The way I look at it is you should not need internet to start your server.  If you installed the plg from the webgui then you had internet and it installed all you needed then.  That's why if you use method 2 for a plg then the tar is there on your flash ready to install.  No network needed. You can remove the network check from your plg and a lot of other unnecessary steps.

 

Look at my webvirtmgr.plg it's just a python app. It started off with your headphones plg as a template so I could use  what you added to infuencers version.

 

Link to comment

Yea but the problem still remains that if someone installs two of my plugins that use java, then uninstalled one of them, should each plugin remove their dependencies? If so, removing one plugin will erase the java install file which means on next reboot it needs to be downloaded again for the remaining plugin and it's a fairy hefty file size.

Do you mean in /boot/packages?  I  debated things like this too.  Like should you remove /boot/config/plugins/plg_folder?  Its too easy to accidentally hit the remove plugin link. If your talking a few MB then I say just leave it. Let the user delete those if they need flash drive space that bad. I do have it remove the old plg tar when updating. Then it gits the new tar.

 

The way I look at it is you should not need internet to start your server.  If you installed the plg from the webgui then you had internet and it installed all you needed then.  That's why if you use method 2 for a plg then the tar is there on your flash ready to install.  No network needed. You can remove the network check from your plg and a lot of other unnecessary steps.

 

Look at my webvirtmgr.plg it's just a python app. It started off with your headphones plg as a template so I could use  what you added to infuencers version.

I'll have a look at that.  Yeah the issue is that, lets take the Media Browser plugin i'm working on, it has like 20 dependencies thanks to libgdiplus, so are you saying I should include all of those dependency files in a tar.gz file and host it with the pluigin files?

 

From one perspective, doing that means a much larger file (the tar.gz) but from the other it means no downloading on reboot if a dependency txz file is missing.  But if I have a plugin that uses java, every time the user updates the plugin they will technically be re-downloading that java install file as part of the tar.gz file for the application.  If multiple plugins use java, and the same version at that, then you get multiple copies of that install stored on the USB in each applications tar.gz file taking up more space.  Utilizing the packages folder let apps have access to the install files and share them without duplication of the files. 

 

This also poses a problem for some of the apps that clone from GIT on startup.  I'm not sure how is best to handle those when a server has no network.

Link to comment

Yea but the problem still remains that if someone installs two of my plugins that use java, then uninstalled one of them, should each plugin remove their dependencies? If so, removing one plugin will erase the java install file which means on next reboot it needs to be downloaded again for the remaining plugin and it's a fairy hefty file size.

Do you mean in /boot/packages?  I  debated things like this too.  Like should you remove /boot/config/plugins/plg_folder?  Its too easy to accidentally hit the remove plugin link. If your talking a few MB then I say just leave it. Let the user delete those if they need flash drive space that bad. I do have it remove the old plg tar when updating. Then it gits the new tar.

 

The way I look at it is you should not need internet to start your server.  If you installed the plg from the webgui then you had internet and it installed all you needed then.  That's why if you use method 2 for a plg then the tar is there on your flash ready to install.  No network needed. You can remove the network check from your plg and a lot of other unnecessary steps.

 

Look at my webvirtmgr.plg it's just a python app. It started off with your headphones plg as a template so I could use  what you added to infuencers version.

I'll have a look at that.  Yeah the issue is that, lets take the Media Browser plugin i'm working on, it has like 20 dependencies thanks to libgdiplus, so are you saying I should include all of those dependency files in a tar.gz file and host it with the pluigin files?

 

From one perspective, doing that means a much larger file (the tar.gz) but from the other it means no downloading on reboot if a dependency txz file is missing.  But if I have a plugin that uses java, every time the user updates the plugin they will technically be re-downloading that java install file as part of the tar.gz file for the application.  If multiple plugins use java, and the same version at that, then you get multiple copies of that install stored on the USB in each applications tar.gz file taking up more space.  Utilizing the packages folder let apps have access to the install files and share them without duplication of the files. 

 

This also poses a problem for some of the apps that clone from GIT on startup.  I'm not sure how is best to handle those when a server has no network.

 

Host all depends in appsupport or a separate repo. The depends still get put in /boot/packages or wherever you want. You still have them downloaded in the same way with file declarations in the plg. But I was suggesting if you host them yourself in a separate repo from the plg then the link never gets broken. Granted for the utah slackware that's usually not a problem. But if you had some from a different less reliable repo.

 

The plg tar would just have files for emhttp and config directories files not all the depends too.

 

Link to comment

Here's a screenshot of what my webvirt git looks like.  And the plg.  Packages are in a separate repo.

 

<?xml version='1.0' standalone='yes'?>

<!DOCTYPE PLUGIN [
<!ENTITY name      "webvirtmgr">
<!ENTITY author    "dmacias72">
<!ENTITY version   "2014.10.12">
<!ENTITY category  "VM Support">
<!ENTITY pluginURL "https://raw.githubusercontent.com/dmacias72/&name;/master/&name;.plg">
<!ENTITY pkgURL	   "https://raw.githubusercontent.com/&author;/unRAID_6.0/master/">
<!ENTITY pkgs 	   "/boot/packages/">
<!ENTITY plugin	   "/boot/config/plugins/&name;">
<!ENTITY emhttp	   "/usr/local/emhttp/plugins/&name;">
]>

<PLUGIN  name="&name;"
         author="&author;"
         version="&version;"
         category="&category;"
         pluginURL="&pluginURL;"
 pkgURL="&pkgURL;"
 pkgs="&pkgs;"
 plugin="&plugin;"
 emhttp="&emhttp;">
>
<!--
This Plugin installs webvirtmgr unRaid 6.0 ONLY. All dependencies are installed as needed and everything is controlable from the webgui.

Credits:
Troy Wagner: Heavily updated this plugin originally created / modified by Tim Alexander.
This now contains more options and some validity to information added as to not break the app with invalid options.
When invalid options are found, default values are loaded in its place.
Spaces will be replaced with _ as passing anythign with a space breaks these plugins.
Empty values which also break plugins, are replaced with default values also.
Infuencer
Benni-Chan: Much of the plug-in structure from his original plug-ins. Much of that code is still in place.
Anatoliy Guskov aka retspen: for creating webvirtmgr https://github.com/retspen/webvirtmgr
Rafi Khardalian: adding local socket support to webvirtmgr and help
Peter: compiling libvirt packages, help and testing
-->

<FILE Name="&pkgs;libvirt-python-1.2.8-x86_64-1_SBo.tgz" Run="upgradepkg --install-new">
<URL>&pkgURL;libvirt-python-1.2.8-x86_64-1_SBo.tgz</URL>
<MD5>df99c2de6e6c1df73ab740878922bfbe</MD5>
</FILE>

<FILE Name="&pkgs;tcl-8.6.1-x86_64-1.txz" Run="upgradepkg --install-new">
<URL>&pkgURL;tcl-8.6.1-x86_64-1.txz</URL>
<MD5>81a2ee6b801ff63a0b96ac8239fc9f26</MD5>
</FILE>

<FILE Name="&pkgs;expect-5.44.1.15-x86_64-2.txz" Run="upgradepkg --install-new">
<URL>&pkgURL;expect-5.44.1.15-x86_64-2.txz</URL>
<MD5>ba1415d7928ade48de4d17c74d65a7a1</MD5>
</FILE>

<FILE Name="&pkgs;webvirtmgr-site-packages-x86-64-1.txz" Run="upgradepkg --install-new">
<URL>&pkgURL;webvirtmgr-site-packages-x86-64-1.txz</URL>
<MD5>7de46f65b5b0065b49d3b19fa00ab5e0</MD5>
</FILE>

<FILE Name="&pkgs;pysetuptools-3.6-x86_64-1ponce.txz" Run="upgradepkg --install-new">
<URL>&pkgURL;pysetuptools-3.6-x86_64-1ponce.txz</URL>
<MD5>8ff5271f08e258ce46e8e96bebec66aa</MD5>
</FILE>

<FILE Name="&pkgs;git-1.8.4-x86_64-1.txz" Run="upgradepkg --install-new">
<URL>&pkgURL;git-1.8.4-x86_64-1.txz</URL>
<MD5>e24d44ec30e7afc16c5509c4d9446140</MD5>
</FILE>

<!--
get from github as tarball
-->
<FILE Name="&plugin;/&name;-&version;.tar.gz">
<URL>"https://github.com/&author;/&name;/archive/&version;.tar.gz"</URL>
</FILE>

<!--
The 'install' script.
-->
<FILE Run="/bin/bash">
<INLINE> 
if [ ! -d &emhttp; ]; then
mkdir -p &emhttp;
fi
tar -zxf &plugin;/&name;-&version;.tar.gz --strip=1 -C &emhttp;/
find &plugin; -type f -iname "*.tar.gz" ! -iname "&name;-&version;.tar.gz" -delete
mv -f /usr/local/emhttp/plugins/&name;/rc.&name; /etc/rc.d/rc.&name;
chmod 0770 /etc/rc.d/rc.&name;
cp -nr /usr/local/emhttp/plugins/&name;/&name; /boot/config/plugins
</INLINE>
</FILE>

<!--
The 'remove' script.
-->
<FILE Run="/bin/bash" Method="remove">
<INLINE>
/etc/rc.d/rc.&name; stop
rm -rf /usr/local/emhttp/plugins/&name;
rm -f /etc/rc.d/rc.&name;
rm -f &plugin;/&name;-&version;.tar.gz
</INLINE>
</FILE>

<FILE Name="/tmp/hp-chkconf" Run="/bin/bash">
<INLINE>
<![CDATA[
# This will check each entry in the config so nothing is missing, and if missing, sets to default
CFGFILE=/boot/config/plugins/webvirtmgr/webvirtmgr.cfg
[ ! `cat "$CFGFILE" | grep SERVICE` ] && echo "SERVICE=\"disable\"" >> "$CFGFILE"
[ ! `cat "$CFGFILE" | grep INSTALLDIR` ] && echo "INSTALLDIR=\"/usr/local/webvirtmgr\"" >> "$CFGFILE"
[ ! `cat "$CFGFILE" | grep ^PORT` ] && echo "PORT=\"8000\"" >> "$CFGFILE"
[ ! `cat "$CFGFILE" | grep RUNAS` ] && echo "RUNAS=\"nobody\"" >> "$CFGFILE"
[ ! `cat "$CFGFILE" | grep USERNAME` ] && echo "USERNAME=\"yes\"" >> "$CFGFILE"
[ ! `cat "$CFGFILE" | grep PASSWORD` ] && echo "PASSWORD=\"yes\"" >> "$CFGFILE"
rm /tmp/hp-chkconf
]]>
</INLINE>
</FILE>

</PLUGIN>

uploadfromtaptalk1414239120278.jpg.df2b83aa6bbde79b94596bc7a32f21be.jpg

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.