I originally posted this in response to another thread, but WeeboTech suggested splitting it into its own thread for better visibility and hopefully discussion....

 

The latest OpenSSH requires openssl 1.0 libraries, why not use the latest version? There is a pre-compiled Openssh slackware package on pkgs.org.

 

This is the problem with unRAID/Slackware not having a unified package manager. apt-get/yum would be very useful right about now......

Perhaps something like slapt-get (http://software.jaos.org/) could be useful for managing packages with regard to plugins. I've had a little play around with it and it seems to work ok on unraid.

 

There would be 1 or 2 things to be thought about before it could be adopted as part of unraid though.

 

First being the location where it stores it's cache (downloaded packages, details of sources etc), by default this is in /var and for unraid it probably ought to be on the flash in /boot. Changing that location is just a tweak of a config file, but when I tried changing that and then moving the cache from /var I encountered problems relating to the names of some of the files in the cache. It uses the url to a package source as a file name, while it replaces/escapes some invalid characters it leaves colons in, which aren't valid on a fat32 flash drive.

 

So a small tweak to the source would be needed.

 

Some thought would probably also need to go into the location of the config file. By default it's in /etc/slapt-get, but again it ought to be somewhere which is persistent and user editable. So that users can add/change package sources etc.

 

I'd envisage that slapt-get would be included as part of unraid, and that during boot unraid would check for and create if necessary the folder on the flash to use as the cache and setup the default config file - if it doesn't already exist.

 

Plugins could then be adapted to include a dependency section which specified which packages the plugin requires to be installed by slapt-get.

 

Anybody got any thoughts?

Perhaps the links should not be on /boot but perhaps on a hidden folder on a cache drive or on the array.

 

Another choice is to create two partitions on a flash key with the second one being a reiserfs partition.

Although that's a bit more complicated.

 

In the past I also created a loop back filesystem on the flash key.

That's easier because you do not have to re-partition, but it has it's own complexities in initial setup.

 

 

I'm no expert in plugins, but I do use a lot of them, and I think this is a brilliant idea; tho some may argue it is against the core values of slackware(?!?).

 

Using a package manager would be a much easier way to install required packages for plugins, and only especially compiled packages would require separate hosting! Possibly on a dedicated part of the unRAID site?!

 

 

I think they should definitely be somewhere that is available even without the array being started.

 

That way the packages are available for any plugins which aren't dependant on the array running - overbyrns openssh plugin for example.

 

Patching the source to remove/escape colons in filenames seems the easiest option. From a quick look it seems to be adding a line to a single function in the code. I would have tested it but I don't have any form of development system setup for compiling for unraid.

 

Peter

Just installed a few packages in a virtual version of unraid and done some tests.

 

Had to make 2 changes to the source for slapt-get - one to replace the colon in filenames the other to remove a check where it checks for free space on /

 

Package built and installed fine (once dependencies were installed) and I was even able to make minor modifications to a version of overbyrns openssh plugin to install openssh using slapt-get.

 

Might be a promising tool

Sounds good!! Anyway we can get Tom (aka Limetech's) view to possibly implement this as a tool?!?

Sounds good!! Anyway we can get Tom (aka Limetech's) view to possibly implement this as a tool?!?

 

This will probably be put on the back burner until the current issue with 5.0 is resolved and final gets released.

 

Sent from a phone, sorry for any typos

 

 

I don't want to shoot this down, but sadly it's not enough to get the tool running. I had it working fine as I was experimenting with it half a year ago. But very often it failed to recognize dependencies due to missing information in the slackware packages. In the end I had to find the missing deps by hand in far more cases than I wished for. So until the bulk of available slackware packes is changed to contain this info, slap-get will sadly have no real benefits.

 

I agree wholeheartly however, that it would be great if it worked as expected.

Disappointing to hear of issues, but they may or may not be an issue in the context of a plugin. As it stands now each plugin contains details of the packages it requires and links to specific versions of those packages.

 

With slapt-get it should be possible to still declare the required packages, but just specifying the package name rather than a specific version e.g. specifying just "openssh" rather than "openssh-x.x.x-i486..."

 

So the plugin author would still have to include all of the dependencies in a plugin just without including the version specific links. It would also be possible to specify a specific version e.g. python.2.6.7 should that be required.

 

Another possibility would be to follow something similar to the XBMC add-on model. Where each plugin specifies which other plugins (and in this context I'd imagine that each slackware package would have to have it's own plugin defined), and which versions of those plugins it relies on.

 

For example you might have a "package plugin" for the openssh package. Then you would have the "unraid plugin" (as in overbyrns current plugin) which specifies that it depends on the package plugin and also contains all of the other plugin related bits and pieces - start up scripts etc.

 

The downside would be that there would potentially be an awful lot of package plugins and there would need to be some infrastructure to support those - think XBMC's add-on repository. As well as some way to manage them on the unraid side.

 

All in all, whichever way it's done there's no easy way on slackware. I think slapt-get could be worth further investigation though.

 

I'm not a plugin developer so I'm not sure on whether it would entirely work. But starting a discussion is at least a jumping off point and it would be good to get the thoughts of some people who have developed plugins, or who may have other thoughts on possible solutions.

 

Peter

@ Peter, you mention making a couple of mods to the source.  Mind sharing or providing patches to save some time please?

 

 

 

Thanks,

overbyrn

 

 

See this:

 

http://lime-technology.com/forum/index.php?topic=21260.msg242294#msg242294

 

Seems like piotrasd has gotten PIP to work ok for Python packages

@ Peter, you mention making a couple of mods to the source.  Mind sharing or providing patches to save some time please?

 

 

 

Thanks,

overbyrn

Will do - probably tomorrow as I won't have access to the vm I did the testing on until then.

 

Peter

As requested I'm posting a patch with the modifications I made to the slapt-get source. I'll also detail the changes I made to the openssh plugin to get it to.

 

It was only a quick trial so please forgive the quick and dirtiness of the patch and test methods.

 

As previously mentioned, there were 2 source changes I had to make. One is to replace colons in filenames derived from urls. In this one I simply replaced ':' with '^' - no real reason for that particular, it was just a random valid character that I chose.

 

The second change was in related to a part of the code which checks for available space on / - that was always returning 0 and stopping slapt-get from running. So I simply commented that section out to see what would happen and it seems to work fine.

 

I also included a default config file for unraid. The default config is to store the slapt-get cache (including downloaded packages) in /boot/slapt-get.

 

To get the config file included in the slackware package which gets built I had to modify the makefile to include an environment variable which I called UNRAID. The contents of this variable get appended to the name of the default config file to specify which default file gets included in the package.

 

I also added a line to the end of the package installation script, which gets generated, to call 'slapt-get --update' after the package has been installed to refresh the package cache so that the application is ready to be used. However in my testing this hasn't always worked so well as I'm loading the package from /boot/extra and it relies on a curl package to be installed, which I also have in /boot/extra but the 2 packages don't always get installed in the same order.

 

To build I do the following:

export UNRAID=.unraid
make pkg

 

I've been using the slapt-get-0.10.2p.tar.gz source package. It is specified on the slapt-get homepage that it requires gpgme to be installed, but I had problems getting it to build with that installed (probably wrong gpgme version or something), so I built it without gpgme and again it seems to be fine for test purposes.

 

To get the openssh plugin to install the openssh package using slapt-get I had to change a few lines from this:

<FILE Name="&pkg;/openssh-5.9p1-i486-2.txz" Run="upgradepkg --install-new">
<URL>-q --no-check-certificate &url2;/openssh-5.9p1-i486-2.txz</URL>
</FILE>

to this:

<FILE Name="openssh" Run="slapt-get --install">
</FILE>

Note: I'm not 100% sure which version of overbyrn's openssh plugin this was. It was just a version that I had on the vm I was testing on.

 

As I said - quick and dirty, but just a proof of concept.

 

Peter

unraid.patch.txt

Good stuff.  Thank you for that.  Looking at the slapt_disk_space_check function, passing a value less than 0 effectively disables the check as it forces the function to return TRUE.  I've not tried it yet, but it should be ok to leave in the code and change "uncompressed_size" to -1.

 

For gpgme, it also requires gnupg.  I saw it failing on checksum validation in the initial "slapt-get --update" until I installed a gnupg package.  I used gnupg-1.4.10-i486-1.txz and gpgme-1.2.0-i486-1.txz.

 

I'm not sure how much mileage there is in using the application but it's always nice to tinker with new things 

 

 

Regards,

overbyrn

Yes I had it working with gpgme and gnupg when trying the pre-built application - up to a point anyway as it wouldn't install anything because of the root space check. But when I tried making changes and building it, with gpgme installed, it didn't like it. No idea what the problem was and I didn't have time to spend too long on it at the time so I just removed gpgme and built without.

 

Good spot on the disk space function

 

At this point I'm not really sure how best to test the validity of this as a proposed solution to the package management problem. The only thing I can think of is to grab a bunch of plugins, modify them to use slapt-get and try them out. Would be rather an artificial test though.

For anyone that's interested, here is a link to a slap-get package which has been modified to store its data at /mnt/cache/slapt-get by default.  This is configurable by editing /etc/slapt-get/slapt-getrc and changing the WORKINGDIR variable.

 

The package includes the changes documented above by Peter (many thanks!), so the files that slap-get create are FAT32 friendly.

 

slap-get package:

https://dl.dropboxusercontent.com/u/572553/unraid/packages.custom/slapt-get-0.10.2p-i386-1-unraid.tgz

 

The following packages are necessary dependencies::

http://slackware.cs.utah.edu/pub/slackware/slackware-13.1/slackware/n/curl-7.20.1-i486-1.txz

http://slackware.cs.utah.edu/pub/slackware/slackware-13.1/slackware/n/gnupg-1.4.10-i486-1.txz

http://slackware.cs.utah.edu/pub/slackware/slackware-13.1/slackware/n/gpgme-1.2.0-i486-1.txz

 

Once installed, you will need to issue the following two commands:

slapt-get --add-keys

Needed for successful GPG checksum verification

 

slapt-get --update

To download the initial package list

 

 

Regards,

overbyrn

Missing file...

slapt-get: /usr/lib/libidn.so.11: no version information available (required by /usr/lib/libcurl.so.4)

  but previously I had libidn-1.25-i486-2.txz in my OpenVPN plugin just for curl, but I removed it, so I installed it again, and now I got

Segmentation fault

when executing

slapt-get --add-keys

 

I'm on VM with only OPenVPN plugin installed on RC15a

 

End of my syslog.....

Jun 24 20:01:58 Tower-VM kernel: slapt-get[4322]: segfault at 0 ip b73f9760 sp bfd6bec0 error 4 in libc-2.11.1.so[b7380000+15c000]
Jun 24 20:05:40 Tower-VM kernel: slapt-get[4824]: segfault at 0 ip b7482760 sp bf948db0 error 4 in libc-2.11.1.so[b7409000+15c000]
Jun 24 20:06:13 Tower-VM kernel: slapt-get[4981]: segfault at 0 ip b749c760 sp bff1d0e0 error 4 in libc-2.11.1.so[b7423000+15c000]
Jun 24 20:10:08 Tower-VM kernel: slapt-get[4982]: segfault at 0 ip b7493760 sp bff9cd60 error 4 in libc-2.11.1.so[b741a000+15c000]
Jun 24 20:10:13 Tower-VM kernel: slapt-get[4983]: segfault at 0 ip b7464760 sp bfe1f2f0 error 4 in libc-2.11.1.so[b73eb000+15c000]

Hmm.  Well there is certainly a library mismatch somewhere eh.  For what it's worth my tests were done using rc15a in a VM with only SimpleFeatures web core plugin installed, vanilla Go file.

would it be entirely too improbable to have unraid host it's own repository or is that just unnecessary? i dont imagine the extra traffic would cause too much of a strain, although i suppose it could.

I guess it would depend on what route was followed. If the slap-get (or similar) path was taken then I don't think it would really be required as, for the most part, the packages being installed would be coming from an already existing repository. The repositories used for slapt-get are easily added or edited by modifying the slapt-get config file.

 

If something more akin to the XBMC add-on route was taken then there would need to be a repository created to host the plugins and packages. Whether that would need to be a dedicated system, or whether something like github could be used (as many plugin developers do now) I don't know.

 

well, my main thinking was to prevent issues such as different plugins having different versions of packages and avoiding situations such as one plugin installs python 2.6 and the next one turn right around and uninstalls it only to install python 2.7. or even worse, just one plugin breaking another. if unraid had it's own repository then the plugins, for the most part, need only specify the package needed and just fetch the latest one from the unraid repo.

 

this would help standardize plugin writing a little more and help avoid "inter-plugin compatibility" issues. things arent too bad right now in that respect but there is a fair bit of fragmentation already in regards to the package version(s) each plugin uses which results in users having to manually edit their plugins. this would in one sense make updating plugins easier for devs and users.

 

by simply updating the repo, all plugins will have, using the previous example, the most currently available version of python. packages for any plugins that is no longer being developed will still be updated and users will not have to edit it to get it to work with other plugins that use the same packages.

 

the downside of this of course is the plugin developers will have to verify any new versions of packages do not break their plugins. a solution to this could be that plugins can be 'verified working for unraid' or have some other similar status indicating all of the packages in the plugin come from the unraid repo and are verified working with this plugin. plugin developers can have an official 'plugin developer' status that puts them on a mailing list where they can be alerted to any impending repo package updates and be given time to verify compatibility. once devs verify their plugin works with the new package, the package will be update in the repo. if others need more time or the plugin has been abandoned they will loose that 'verified for unraid' status until the situation changes and the update will roll out anyway.

 

additionally any package that is not in the official unraid repo can be submitted for inclusion if needed. in my case where there are a lot of packages that arent available pre-compiled in slackware and must be compiled from source

 

just a thought but i'd be interested in what others think. does this seem feasible? worthwhile? too much of an undertaking? stupid?

I whole heartedly agree with all of the above!!

bobbintb,

 

that scenario should be handled out of the box by slapt-get. If you look at the example of where I modified overbyrns openssh plugin, it's just a case of specifying the name of the package i.e. openssh, or python to use your example.

 

So if you had multiple plugins which all specified python, then only the version available in the configured slapt-get repository would be installed.

 

What would happen if you had multiple repositories configured (maybe one for Slackware 13.1 and one for 14.0 - don't know if anyone actually would it's just an example), which both had versions of python packages available, I'm not sure. I haven't tested it but I'd guess that the latest version of the package would be used. I believe that is how apt-get works on Ubuntu, and slapt-get is supposed to mimic that functionality.

 

Of course the idea of an unraid specific repository, and your ideas about controlled versions of packages, could easily be combined and slapt-get used to provide the package downloading and installing muscle. The unraid repository would just have to be setup as the default/preferred repository in the slapt-get config.

that's true, and i knew that. i just wasnt thinking clearly when i wrote that and just mixed some of my arguments for slapt-get in with those for an unraid repo.

Having read a couple of comments in the XBMC Standalone Updater thread, which touched on similar issues, I'm just giving this a bump so that it doesn't get lost and hopefully attracts more attention/discussion