The (un)official unRAID 6.x plugin discussion thread

[NAS]

...

There will ofc still have to one or two plugins, perhaps the VM GUI plugin for example or whatever.

 

Absolutely but I think we should try and aim to have key plugins like this developed with a view to pushing them upstream to the core OS. Ideally any plugin that is essential a must have should be pushed up I would say.

[nicinabox]

My honest opinion is that plugins like this will become rare and people will be in a VM most of the time. But we are not there yet and there are quite a few hurdles on the way yet.

 

I'd like to see someone make a proof-of-concept of that system.

 

I also agree that hard coding deps into the plugin is in general a bad idea for the very reasons you cited.

 

Boiler solves this by allowing you to specify deps by name and use a version constraint.

 

Honestly, 6.0 would be a good time to deprecate the plg system. The shortcomings with plgs run so much deeper than "should we change the file ext to plg64".

 

[ironicbadger]

My honest opinion is that plugins like this will become rare and people will be in a VM most of the time. But we are not there yet and there are quite a few hurdles on the way yet.

 

I'd like to see someone make a proof-of-concept of that system.

 

I also agree that hard coding deps into the plugin is in general a bad idea for the very reasons you cited.

 

Boiler solves this by allowing you to specify deps by name and use a version constraint.

 

Honestly, 6.0 would be a good time to deprecate the plg system. The shortcomings with plgs run so much deeper than "should we change the file ext to plg64".

 

Listen to the man, people. Despite my earlier protestations which may have indicated otherwise I agree that .plgs SUCK.

 

We need a package manager and boiler is pretty close from what I understand it to be...

[dirtysanchez]

Absolutely but I think we should try and aim to have key plugins like this developed with a view to pushing them upstream to the core OS. Ideally any plugin that is essential a must have should be pushed up I would say.

 

Completely agree. The two that come to mind instantly would be a VM manager and APCUPSD.

[nicinabox]

Absolutely but I think we should try and aim to have key plugins like this developed with a view to pushing them upstream to the core OS. Ideally any plugin that is essential a must have should be pushed up I would say.

 

Completely agree. The two that come to mind instantly would be a VM manager and APCUPSD.

 

I think it's unlikely that this will happen. Part of limetech's philosophy on the unRAID product is that the dist remains as light and small as possible. Fewer moving parts means more reliability.

 

That said, folks should be able to tune their machine to their needs. Most folks wont have an APC and won't need that feature. But for those that do, being able to add that functionality from an "official" place would be a huge benefit.

 

 

[NAS]

I would suggest that if most people dont need the APC plugin then it would be a candidate to push upstream.

 

I am talking about things like perhaps a VM Gui or your inventions nicinabox.

[sparklyballs]

As far as plugins go, i have a couple of scripts (DS_STORE cleanup, monthly parity check) , email notifications and cache_dirs.

 

I'd really like email notifications of disk troubles etc... to be baked in the default UnRAID install if at all possible, but cache_dirs i'm easy on...

Script side of things, DS_STORE cleanup may be a niche for just OSX users so can't see that getting in the main OS, but monthly parity checks would be a nice addition to the main OS.

[BLKMGK]

It's funny, I've been using the monthly parity check code for so long I DID think it was part of the base distro! I believe we need that as base, that email alerts should be base, and that UPS interfacing should be base. What we'll be using to manage VMs should be a plugin as I expect it'll be heavier and not needed by all.

 

I'm very interested in what folks think would work well for that management. Roll our own? Something web based? I've been trying to research this myself as I'm most used to VSphere and so far I'm real unsure as to what might work best. The cli commands for some things are fairly lengthy and not user friendly at all however. Sadly I haven't yet come across support for OSX in Xen :-(

 

 

Sent from my iPad using Tapatalk

[sparklyballs]

It's funny, I've been using the monthly parity check code for so long I DID think it was part of the base distro! I believe we need that as base, that email alerts should be base, and that UPS interfacing should be base. What we'll be using to manage VMs should be a plugin as I expect it'll be heavier and not needed by all.

 

I'm very interested in what folks think would work well for that management. Roll our own? Something web based? I've been trying to research this myself as I'm most used to VSphere and so far I'm real unsure as to what might work best. The cli commands for some things are fairly lengthy and not user friendly at all however. Sadly I haven't yet come across support for OSX in Xen :-(

 

 

Sent from my iPad using Tapatalk

 

I think asking people which management option they would like would be something akin to "browser wars" on a smaller scale and you'd be hard pushed to find a general consensus.

Agreed with the lack of XEN support in OSX being a bugbear, but i have a windoze instance that I use for XBMC that doubles as my XENSERVER remote admin rig.

[nicinabox]

It's funny, I've been using the monthly parity check code for so long I DID think it was part of the base distro! I believe we need that as base, that email alerts should be base, and that UPS interfacing should be base. What we'll be using to manage VMs should be a plugin as I expect it'll be heavier and not needed by all.

 

I'm very interested in what folks think would work well for that management. Roll our own? Something web based? I've been trying to research this myself as I'm most used to VSphere and so far I'm real unsure as to what might work best. The cli commands for some things are fairly lengthy and not user friendly at all however. Sadly I haven't yet come across support for OSX in Xen :-(

 

 

Sent from my iPad using Tapatalk

 

I think asking people which management option they would like would be something akin to "browser wars" on a smaller scale and you'd be hard pushed to find a general consensus.

Agreed with the lack of XEN support in OSX being a bugbear, but i have a windoze instance that I use for XBMC that doubles as my XENSERVER remote admin rig.

 

If there's one thing you can know for sure about the unraid community, it's that there will be no consensus.

 

There's more than one viable solution, but design-by-committee never works out well for anyone. Lots of people have opinions about how x should be done, but very few are qualified opinions.

[Bags]

Would it be out of line for someone to start a poll for the top plugins? What I liked about Unmenu was I could chose what I wanted to install. There are more plugins I would like to Install but I am waiting to see how this turns out.

[nicinabox]

Would it be out of line for someone to start a poll for the top plugins? What I liked about Unmenu was I could chose what I wanted to install. There are more plugins I would like to Install but I am waiting to see how this turns out.

 

What the plugins are isn't as important as how the system works. The plg system was not designed for how people want (or expect) to use it today. In order to move forward, we need to identify the core problems with the system and work towards a solution. Like I said before, the problems are extensive and multifaceted.  We need standards, infrastructure, and tooling.

 

Real people need to put their heads together and be empathetic towards the problem instead of aggressively pulling in different directions. You think virtualized plugins are viable? Great, build a distributable prototype and get feedback!

 

 

 

[SchoolBusDriver]

Real people need to put their heads together and be empathetic towards the problem instead of aggressively pulling in different directions. You think virtualized plugins are viable? Great, build a distributable prototype and get feedback!

 

Take off your programmer hat and put on a marketing one. Stop talking / educating, asking for permission and provide a solution that users can use TODAY. Users do not care / need to know how a plugin works.

 

Create a separate thread for every plugin that you have "converted" and indicate in the title it works for unRAID 5.X and 6.X. (and keep adding more plugins as you convert them)

 

That is how users and the plugin guys are going to know / see / use your program. How many people do think know how unmenu works? How it installs packages? That it's a awk? 95% have no clue or care.

 

Get after it!

[nicinabox]

Real people need to put their heads together and be empathetic towards the problem instead of aggressively pulling in different directions. You think virtualized plugins are viable? Great, build a distributable prototype and get feedback!

 

Take off your programmer hat, put on your marketing one. Stop talking / educating, asking for permission and provide a solution that users can use TODAY. Users do not care / need to know how a plugin works.

 

Create a separate thread for every plugin that you have done and indicate in the title it works for unRAID 5.X and 6.X. (and keep adding the ones other than the few you have)

 

That is how users and the plugin guys are going to know / see / use your program. How many people do think know how unmenu works, how it installs packages, that it's a awk?

 

Get after it!

 

I see we're going to have problems.

 

1. I did build a solution that works TODAY. It is natively rooted in the OS.

2. No one REALLY wants to spend their time scouring forum threads looking for a plugin that may or may not work. They do it because the are HAVE to.

3. Users do not truly care about the DIFFERENCE between 5.0 and 6.0. They just want their software to work and not lose their data.

4. UnMENU's slackware packages out outdated and incompatible with many plugins. And those are SLACKWARE packages. Not the same thing. Besides, UnMENU has a fundamental gatekeeper problem.

5. They way people think about plugins is because of the way the system was (not) designed. That doesn't prevent it from growing/changing/evolving.

 

> Users do not care / need to know how a plugin works.

 

No, they probably don't care about the technical underpinnings. They DO CARE that their experience isn't a shitty one.

[BLKMGK]

Honestly I've not looked at Boiler because the last I looked at it I couldn't figure out how to install it and what it would buy me. With 6.0 requiring me to start with a clean install now is a much better time to take the plunge!

 

Yes, choosing one VM manager to rule them all isn't easy. We won't get consensus but we gotta' start somewhere! Which one is easiest to package? Does anyone have experience with one, could name one worth looking at? If the interface is sentence long command strings I cannot believe there will be much adoption :-( I'm excited by the prospect of using Xen to replace ESXi but not if its like driving nails with my forehead!

 

As for OSX, I want to virtualize OSX not run management tools under it :-) I've done this on ESXi and the system isn't really up to snuff.

 

 

Sent from my iPad using Tapatalk

[SchoolBusDriver]

I see we're going to have problems.

 

Why is that?

 

1. I did build a solution that works TODAY. It is natively rooted in the OS.

2. No one REALLY wants to spend their time scouring forum threads looking for a plugin that may or may not work. They do it because the are HAVE to.

3. Users do not truly care about the DIFFERENCE between 5.0 and 6.0. They just want their software to work and not lose their data.

4. UnMENU's slackware packages out outdated and incompatible with many plugins. And those are SLACKWARE packages. Not the same thing. Besides, UnMENU has a fundamental gatekeeper problem.

5. They way people think about plugins is because of the way the system was (not) designed. That doesn't prevent it from growing/changing/evolving.

 

> Users do not care / need to know how a plugin works.

 

No, they probably don't care about the technical underpinnings. They DO CARE that their experience isn't a shitty one.

 

All of the above is great and all (5% of the people know what all of that means)

 

Unless the users see a "[Plugin] Universal Sickbeard 5.0 / 6.0 installer" thread that you create and the users can download / install / use... the fact that it has boiler or not is immaterial.

 

You see plugin people fixing / updating old 32-bit plugins to 64-bit now... Are they using boiler? No.

 

Did they even think of boiler when creating this thread? no

 

Love / Hate it... Those are the facts.

 

My advice... stop selling your "technology" and trying to convince people who have no idea what boiler does / is to use it. Create a product that customers can use (Individual threads for all the converted plugins you made so far). This is how users know / see / get plugins (they have done it for years this way). Once you have a bunch out there, plugin people will create more and those  plugin people will see the glory of your "technology" and incorporate going forward. It also wouldn't hurt to PM all the plugin guys and send them a link to boiler and how to use it.

 

I hope you understand that I like boiler and even suggested that Lime Technology get behind it but that has not happened and the plugin guys do their own thing (part of the problem). So head it off and get a product out there with your technology in it.

 

I have been in your shoes and I did the same thing with Xen and KVM. I didn't sell what it does, create screenshots and hope people would use it or ask questions. I just created guides, bzroots / bzimages, vhd files, etc. and from that users asked tons of questions and installed / use it. Now look where it's at. It's been accepted / adopted by the community and even Lime Technology is rolling it into unRAID.

 

I think the same will happen with boiler if you convert plugins and post them individually so users can use them.

[Kryspy]

I gotta agree with Grumpy. No one cares what a plugin is comprised of. Best case scenario would be the way OMV and Nas4Free do it and Synology.  Plugins installing via one click from an installed repo.

 

Personally, until that happens I see no use in coming back to unRAID. Ubuntu is far more flexible as is. unRAID as a storage solution only is stable and solid but near dead to anyone looking for something more in it's current configuration.

 

Kryspy

[zoggy]

It seems to me that the 'plugin/package' situation is going to fall into two main scenarios.

 

1) you install packages like you do today (manually/plg/boiler/unmenu/whatever)

2) you do a virtual os to run your apps in (thus you dont mess with unraid's os)

 

there are certain 'scripts/plugins' that really you have to install with unraid..  cache_dir / powerdown / smtp / ssh / zip / vim / preclear / unmenu / boxcar / status..

there are certain 'plugins' that you really dont want to be on the same os as unraid in case something went bad (used up too much ram?/killed cpu?)

 

I can see the reason you would want both scenarios... each one though has its own set of hurdles.

 

The vitalization route sounds great.. but what if the user doesn't have that much ram..  what if they dont have that fast of a cpu? what if the user doesnt want to deal with the extra complexity for that 'saftey net' of having a vm silo those apps? I know freenas does the whole jail method and somewhat forces it upon the user.. maybe thats the best approach? but then you have to start changing the min requirements for unraid.

 

There are fundamentally core problems with using slackware as unraid's os choice. This is well known and talked about.

The vm route does not magically fix everything, it just masks the problem as you rely on a better package manager / dependency manager -- for the apps that dont need to directly work with unraid.

 

To address legacy apps from just 'working' which would allow users to do dumb things.. we should change the installer tool/method for plugins.

That way it forces scripts/plugins/etc to be updated.

 

nicinabox so far is the only person I've seen put some real effort behind a solution to shortcomings of slackware/plugin management.

He took how synology does community packages to a more modern approach.

I don't think Tom has looked into this solution, or maybe he doesn't like that this solution requires several of its own packages/dependencies?

 

Anyways, I'm sure this post will get flamed like many others have in here. People want to be bullish and attack others rather than show respect and discus things like adults. Were not paid to use unraid, we just want to help volunteer our time/ideas to make something better.

 

 

[neilt0]

There's a lot of talk here advising users to run apps in a VM and not under unRAID.

 

The main reason why I'm interested in (and currently testing) 6.0 is because cache_dirs crashes my box when it uses too much lowmem. I understand 64-bit essentially removes that problem and I will be able to use most of my 8GB in one unified memory model?

 

OK, but cache_dirs can't run under a VM can it? So running a VM won't make any difference there?

 

I run 3 other apps currently under 5.x -- nzbget, Sick Beard and Transmission. They don't crash the box.

 

nzbget needs to be able to read and write quickly to drives.

So, my question is, if I run it in a VM, how does it write to the array? Does it see array drives as (fast) shared drives? Will that slow down writes? Will running nzbget in a VM mean the writes within the VM's own drive are slower?

I'm already now running nzbget in 6.0, thanks to overbyrn.

 

I came to unRAID about 5 years ago from using Buffalo NAS boxes. The great thing about using those was that when rooted, you had a package manager (ipkg), so if you want to install nzbget, you'd type "ipkg install nzbget". And it worked!

 

I don't see the point of running a VM for a couple of apps, especially if performance will be reduced, but perhaps people can enlighten me here.

[ironicbadger]

There's a lot of talk here advising users to run apps in a VM and not under unRAID.

 

The main reason why I'm interested in (and currently testing) 6.0 is because cache_dirs crashes my box when it uses too much lowmem. I understand 64-bit essentially removes that problem and I will be able to use most of my 8GB in one unified memory model?

 

OK, but cache_dirs can't run under a VM can it? So running a VM won't make any difference there?

 

I run 3 other apps currently under 5.x -- nzbget, Sick Beard and Transmission. They don't crash the box.

 

nzbget needs to be able to read and write quickly to drives.

So, my question is, if I run it in a VM, how does it write to the array? Does it see array drives as (fast) shared drives? Will that slow down writes? Will running nzbget in a VM mean the writes within the VM's own drive are slower?

I'm already now running nzbget in 6.0, thanks to overbyrn.

 

I came to unRAID about 5 years ago from using Buffalo NAS boxes. The great thing about using those was that when rooted, you had a package manager (ipkg), so if you want to install nzbget, you'd type "ipkg install nzbget". And it worked!

 

I don't see the point of running a VM for a couple of apps, especially if performance will be reduced, but perhaps people can enlighten me here.

 

VM's seem to have a bad rep amongst some people, perhaps because of the old days where things like virtualbox didn't have direct hardware access, and had to use virtual devices and NICs etc; They were therefore very slow compared with 'bare metal'. What we are talking about here is a 'type 1' hypervisor such as KVM or Xen, this changes things dramatically.

 

Type 1 hypervisors are used extensively in the enterprise world and are a proven, reliable technology. ESXi is another example of a type 1 hypervisor, but isn't relevant to our discussions as you cannot 'install it as a package' on top of an existing OS like you can with Xen or KVM.

 

If you want an idea of what performance you can expect from a VM then see my youtube video below. Spolier: It's neglible the difference between a Xen VM with the correct drives installed vs a bare metal installation of the same OS. As for how would this apply to a 'plugin specific VM', well that's easy. We'd use the virtio-9p protocol previously discussed here to give the VM access to the host's disks. Now remember the host is unRAID, therefore that's how you get access from let's say Transmission through to unRAID. The VMs would run on an SSD or cache drive.

 

My plan is, once the beta phase has settled down a bit to get some KVM / Xen guides done and release an image of Arch (or something I don't know yet) that you can download and import yourself. Then to upgrade any package would be done via a repo hosted by the community, as previously mentioned if it's on Arch User Repo (AUR) then we can easily include it our community repo. Example (if VM was in Arch): pacman -S sabnzbd would install sabnzbd, pacman -Syu would upgrade anything that needed it from our repo. Simple stuff!

 

Youtube video showing gaming performance using Xen hypervisor running Windows 8 with VGA passthrough

 

Did you know that the Xbox One utilises this technology as well? It runs 3 OS's concurrently. OS#1 is for the hypervisor only (the host), then OS#2 does gaming (a VM no less), then OS#3 handles dashboard / media duties (another VM). This is how it can switch so quickly between activities etc. If it's good enough for an Xbox and enterprise companies you can bet your arse it's good enough for you and I at home.

[neilt0]

OK, well I have a test server, so once this is made simple enough for us humans to try out, I'll get on it.

 

Sent from Moto G using Tapatalk

 

[Ockingshay]

This is a great discussion to have, brought about by this very forum and user input, so let's remain polite to keep this momentum going.

 

There seems to be 4 groups of people using unRAID:

 

1) legacy users on 4.x who want to use any and all hardware available to them to create a server

 

2) Purist users who simply want to use the unRAID software and what it offers as is and on whatever version that's suitable.

 

3) Power users that want to take the basics of unRAID and add additional functionality through modern hardware and plugins. <---- this is me

 

4) Nerds (meant in a nice way!) who want to make the most out of their server and add all the functionality they want, including plugins and/or VMs to provide greater versatility.

 

1&2 have pretty much been catered for by Limetech.

 

Trying to keep the rest happy through a set standard and framework is really what the rest is all about, as simply Adding VM support does not alleviate the fundamental flaws in the current package management system/dependencies and the need to rely on forum volunteers to maintain plugins for every user class.

 

The issues then that need addressing;

 

A) A way to standardise code/maintain/future proof those that want to use plugins. Boiler springs to mind

 

B) A way to bake into unRAID the ability to use virtualisation. The work Tom is doing with v6 betas.

 

 

[Bags]

What's the minimum hardware I need to run a VM?

[vl1969]

What's the minimum hardware I need to run a VM?

 

Do you mean run unraid in vm or run vm on the new unraid?

 

If 1. Than most important thjng you need is the hardware with full vt-d iommu support as it must support pass through option. 

If 2. Than it all depends on what you plan to do with that vm.

I would say for best result a quad core with 8gb ram is a must minimum. You can start with less but you will be upgrading soon enough :-)

 

 

Sent from my SGH-T889 using Tapatalk

 

 

[neilt0]

Whar happened to all the talk about not needing vt-d support with 64-bit?

 

Sent from Moto G using Tapatalk