Xen/unRAID-6 Discussion

[Chris Pollard]

Geeks and tinkerers aside I think most people would just like to click a button or upload a file which installs package X and guides them through setting it up.  How this is achieved, I suspect, will be of little interest as long as the outcome is they have the thing they wanted working.

 

Personally I'll just install one lightweight dist with a package manager and I wont need to worry about plugins anymore.

[dalben]

Geeks and tinkerers aside I think most people would just like to click a button or upload a file which installs package X and guides them through setting it up.  How this is achieved, I suspect, will be of little interest as long as the outcome is they have the thing they wanted working.

 

Yep, that's the mass market for this type of device.

[SchoolBusDriver]

I'd like to learn more about this, but Google fails me. Would you be so kind as to point me to a white paper or two that describes the technical rationale for not passing through disks to a VM? Or if you are inclined, share one or two of those technical reasons?

 

I will post several tomorrow.

 

Without getting into the 1 and 0s like the whitepapers do...

 

You are adding ANOTHER layer on top of the things your data has to go through before it writes the data to your drives. Even when passing through your SATA / SAS / RAID Controller... Your VM DOES NOT have direct access / control over it. It is talking to it through the VM to the Host, Host to the Kernel and the Kernel to Block Device.

 

This blog post from the Hyper-V team gives instructions on how to use passthrough disks, and doesn't warn about it: http://blogs.technet.com/b/askcore/archive/2008/10/24/configuring-pass-through-disks-in-hyper-v.aspx

 

I found this white paper: http://blogs.msdn.com/b/taylorb/archive/2010/02/26/hyper-v-virtual-hard-disk-vhd-performance-white-paper.aspx

 

1. You cannot do PCI Passthrough in Hyper-V 2008, 2012 or 2012 R2.

 

2. You cannot do USB Passthrough in Hyper-V 2008, 2012 (You can in 2012 R2 which isn't even released yet).

 

3. They are talking about Virtual Hard Drives (what your VMs will be running on in your unRAID / Cache drive) not drives that are passed through via PCI Passthrough.

 

Considering you didn't know what a VHD is and the Hyper-V can't do PCI Passthrough... Do I really need to post the whitepapers that you probably won't understand and will only confuse you and others?

[sacretagent]

Geeks and tinkerers aside I think most people would just like to click a button or upload a file which installs package X and guides them through setting it up.  How this is achieved, I suspect, will be of little interest as long as the outcome is they have the thing they wanted working.

 

Yep, that's the mass market for this type of device.

 

Only way that will happen is when Tom moves away from slack

if he makes unraid on ubuntu for example which has a package manager then all your problems will go away

 

Even with Trolley which is a nice effort from nicna , slackware is just not meant to deal with dependencies see the thread here http://lime-technology.com/forum/index.php?topic=29641.msg286938#msg286938

 

only other option is that Tom hires a guy to make plugins and opens a plugin shop

 

till then you depend on the goodwill of the community here to help you out with a plugin

 

[dalben]

Only way that will happen is when Tom moves away from slack

if he makes unraid on ubuntu for example which has a package manager then all your problems will go away

 

I'm not sure I have any problems.  I started life as an assembler programmer so I'm not that scared of a CLI.  But it is more convenient to click a link then delving into config scripts.

 

till then you depend on the goodwill of the community here to help you out with a plugin

 

Yes, that's what is being discussed here, will that goodwill that's here now continue to strive to single point and click or upload and click solution to expand unRAID.  If Tom wants to chase the mass market for unRAID, then that's what his product needs.  If all he wants is the more technically inclined and the other end, those that want unRAID for file storage only, and nothing else, then it doesn't much matter what solution he provides as unRAID itself can penetrate this market already.

[gtroyp]

I just went through the process of installing Xenserver on my new build, and installing VM's to do all my tasks that used to be plugins on unraid, while running unraid as a VM itself on the system. There was quite a learning curve (I was a linux noob) but it all works really well now.

 

Not having to worry about my apps becoming lost (ahem, Transmission as a plugin was a FEW versions behind), and being able to try something out without worrying about crashing my mission critical file-server-system make up for all of the effort. As a hobbyist that built a unraid because I wanted to have some control over what went into my system, being virtual extends that control to my software, and I like that a lot.

 

Making unraid the dom0 is the best idea I've heard for the movement and development of the product, even it means reworking all of the things I just did setting up my system.

 

Tom, I am on board with you and can't wait to help in any way I can to make it easier for the even noobier users than me.

[SchoolBusDriver]

Just as a counter-argument, I would get lots of blowback at work for installing things into the Xen dom0. In fact, the Xen Server docs advise that you treat dom0 like an appliance. They don't even support simple RPM-based upgrades any more. You have to reinstall dom0 from the ISO to upgrade. So mixing up unraid and Xen in dom0 could send enterprise folks into conniptions too.

 

You aren't making a counter-arguement... You are actually agreeing with ours.

 

You are quoting the "Best of Breed" Practice that Cirtix, VMWare, etc. and their customers follow.

 

Perhaps those Billion Dollars companies, and their Billion Dollar Customers who spend Billions of Dollars on IT Professionals / Consultants... Might be onto something.

[SchoolBusDriver]

Peter virtualizing OpenElec should be possible although I'm not clear on some of the paravirtualizing drivers Grumpy mentioned before so it might not be as clear cut as on ESXi.

 

On your OpenElec VM in ESXi, goto system, system info and tell me what Video Card Driver is showing. Is it VMWare?

 

If it isn't showing AMD or nVidia (if you a Quattro Series)... You are not using Hardware Video Acceleration. Instead your CPU in the VM is doing all the processing / decoding of your video files.

[SchoolBusDriver]

But it is more convenient to click a link then delving into config scripts.

 

There are 9,000+ threads in the Support Forum. Having been around here for many years... I am willing to bet that 7,000+ of them are directly related to Plugins.

 

Anyway you slice it, plugins are a FAILURE and the further you get away from a Slackware Releases (13.1 or 14.1)... They only get worse. Most of the plugin developers have "burned out" and gone and they are no longer maintain. It's gotten to the point where users are begging / pleading / offering rewards / money not for new ones... but to modify or fix old ones.

 

Yes, that's what is being discussed here, will that goodwill that's here now continue to strive to single point and click or upload and click solution to expand unRAID.

 

The "Goodwill" was exhausted when the Original Plugin Developers got "burned out" through all the 5.0 beta / Release Candidate that we went through for 2+ years. Can anyone else think of a new plugin that was written in the last year or so? All it's been is patchwork or creating "new" plugins for forks of old plugins?

 

The only way I see plugins getting fixed / maintained / created is if Tom sets up a subscription model where we all pay a monthly / yearly fee to fund some developers or Tom provides a way for the plugin guys to charge / ask for donations. This isn't a foreign concept and on many Open Source projects add-ons, skins, themes, customizations are things you pay for.

 

To Prove The Point...

 

For Tom to get KVM working in unRAID he is going to have compile about 30 - 40 more dependiances (and the dependiances of the dependiances). Ubuntu / Debian has a TEAM (7+ people) just maintaining QEMU alone and KVM has their own TEAM of Package Maintainers too. That doesn't even include all the various Package Maintainers for the other 30+ packages / libraries.

 

Do you honestly think that Tom or a Plugin Developer is going to write a plugin that installs 30 - 40 packages that he has to compile from scratch and keep track of all the various updates / bugs / patches / etc. for each one of them for FREE in his spare time? There isn't a snowballs chance in hell. Not to mention, having to provide need packages for beta / release candidates isn't going to be what I call a good time.

 

Sure, I could do it but you are out of your mind if you think I would spend months writing a plugin that is 2,000+ lines long and then keep track / update / maintain 30 - 40 packages that constantly change / need to upgraded for FREE. Would any of you?

[sacretagent]

I just went through the process of installing Xenserver on my new build, and installing VM's to do all my tasks that used to be plugins on unraid, while running unraid as a VM itself on the system. There was quite a learning curve (I was a linux noob) but it all works really well now.

 

Not having to worry about my apps becoming lost (ahem, Transmission as a plugin was a FEW versions behind), and being able to try something out without worrying about crashing my mission critical file-server-system make up for all of the effort. As a hobbyist that built a unraid because I wanted to have some control over what went into my system, being virtual extends that control to my software, and I like that a lot.

 

Making unraid the dom0 is the best idea I've heard for the movement and development of the product, even it means reworking all of the things I just did setting up my system.

 

Tom, I am on board with you and can't wait to help in any way I can to make it easier for the even noobier users than me.

 

You have a link to that manual how you installed this ?

and without a network bridge ?

are you using a gui to manage the vm's ?

[SchoolBusDriver]

You have a link to that manual how you installed this ?

 

[GUIDE] Virtualizing unRAID in XenServer 6.2 - Updated to 5.0.4

 

are you using a gui to manage the vm's ?

 

Yes.

[PeterB]

Especially when you can install a VM Appliance that many of us will make / share here in less than 5 minutes? Or create your own VM Appliance in 30 minutes or less following 7 easy steps I listed above using any of the 1,000+ blogs, forum posts, wikis, etc. which walk you through the process?

 

What we have to bear in mind is that not everyone has hardware capable of supporting virtualisation, so that wouldn't be an option for them.  Why, there are some people still running unRAID systems on 32-bit only hardware.  There is still a place for plugins, especially those that relate to core functionality, such as apcupsd.

 

This 8-track player I have in my car is awesome.

8-track? Pah - new fangled technology.  I'm looking for wax cylinders for my home audio system!  I understand that wax cylinders were abandoned when they moved to shellac.  I guess that I might have to surrender and move in that direction myself.

[sacretagent]

You have a link to that manual how you installed this ?

 

[GUIDE] Virtualizing unRAID in XenServer 6.2 - Updated to 5.0.4

 

are you using a gui to manage the vm's ?

 

Yes.

 

LOL

 

my fault ... not natively English speaking

 

i was under the impression he managed to get a vm running in unraid 6.0

but after rereading it became clear he setup xen and is running unraid as a guest

 

Oh well waiting for the next Beta to get ubuntu vm running with plugins

[spants]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

[Harpz]

LOL

 

my fault ... not natively English speaking

 

i was under the impression he managed to get a vm running in unraid 6.0

but after rereading it became clear he setup xen and is running unraid as a guest

 

Oh well waiting for the next Beta to get ubuntu vm running with plugins

 

Eagerly awaiting the next beta also (may even wait abit longer), been playing / learning basics of Ubuntu server in a virtualbox vm with lots of snapshots, been an experience but a fun one, not used to all this command line stuff got webadmin running and monitorix running once i figured out a permission problem next is mysql for my xbmc database serviio, tansmission, Microdc2 also could be fun also wanting to learn php with mysql so phpmyadmin, all stuff Ive never really played with.

 

Just hope i can get my head around setting a vm up with unRAID, hope Tom releases an idiots guide sure he said something saying he would.

[butlerpeter]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

[ironicbadger]

What we have to bear in mind is that not everyone has hardware capable of supporting virtualisation, so that wouldn't be an option for them.  Why, there are some people still running unRAID systems on 32-bit only hardware.  There is still a place for plugins, especially those that relate to core functionality, such as apcupsd.

 

I think you're missing a fundamental piece of knowledge. Virtualisation doesn't necessarily require any special hardware. It only does if you want to 'passthrough' something, such as usb port or a graphics card.

 

Where I suspect you're confused is that unraid and virtualisation thus far has required 'special hardware' because you need to passthrough the USB port for licensing purposes. With unraid as the host, it is a game changer.

 

32 bit only people are not likely the people who care about plugins, certainly not in a great numbers, so I will discount them from here. The only 32 bit processor I have bought in the last few years was in a Netbook back in 2008! Therefore I am assuming 95%+ of people here have a 64 bit, KVM / Xen capable, CPU.

 

We are not denying that plugins have a place, but if they do they require an officially sanctioned, designed and supported plugin system. Why reinvent the wheel? 1000's of other Linux distros have dozens of people assigned to their package managers, Tom is just one guy who focuses on storage. Let unRAID do what it's good (storage and minimal footprint) and pass off everything else to a VM. I do concur about UPS support in the core OS as at least an optional, official plugin somehow...

[ironicbadger]

Just hope i can get my head around setting a vm up with unRAID, hope Tom releases an idiots guide sure he said something saying he would.

 

If he doesn't, I will. 

[ironicbadger]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

Feel free to compile your own kernel using the instructions posted here matey!!

 

http://blog.ktz.me/?p=116

[NAS]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

I am a big advocate here of Docker but I would prefer to push looking at it back to after the VM stuff is working.

 

For no reason other that it is better to get one thing done right than get 2 not done at all

 

 

There also seems to be some confusion about VM and how complicated it will be for new users etc. This stems from the fact that we have some pretty heavy hitters here and every VM thread gets pretty complex fast.

 

So we need a simple one liner for people not so savy to understand.

[sacretagent]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

NOt sure anymore if docker is even a good possibility with unraid

 

what i understand from docker is that it makes a snapshot of your OS and then uses that to install your app

now a snapshot of unraid still doesn't have a package manager

so it would depend again on plugin developpers

 

or can you run an ubuntu os in a dock with unraid as the main os ?

 

 

[peter_sm]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

 

 

Feel free to compile your own kernel using the instructions posted here matey!!

 

http://blog.ktz.me/?p=116

 

Can you update your packages to x64 version ?

[ironicbadger]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

 

 

Feel free to compile your own kernel using the instructions posted here matey!!

 

http://blog.ktz.me/?p=116

 

Can you update your packages to x64 version ?

 

Wow, hadn't even considered that! Feel free my good man. I'm a little busy this week.

[NAS]

I would love to see the use of docker for "major" plugins rather than VMs. It just seems more elegant.

Surely must be worth exploring. Having had a look at the docker site it seems to need some stuff enabling in the kernel. I'm not sure which, if any, are already enabled in the v6 kernels.

 

I wonder if those options could be enabled - that way people could investigate independently whether docker is a viable proposition.

 

Especiialy whilst v6 is in beta phase, having the options enabled in the kernel config can't hurt too much can it?

 

NOt sure anymore if docker is even a good possibility with unraid

 

what i understand from docker is that it makes a snapshot of your OS and then uses that to install your app

now a snapshot of unraid still doesn't have a package manager

so it would depend again on plugin developpers

 

or can you run an ubuntu os in a dock with unraid as the main os ?

 

luckily thats not how it works at all

 

dont worry it is a good option for unRAID

 

please fork any more docker chat and link back here are this is a thread for Xen

[sacretagent]

There also seems to be some confusion about VM and how complicated it will be for new users etc. This stems from the fact that we have some pretty heavy hitters here and every VM thread gets pretty complex fast.

 

So we need a simple one liner for people not so savy to understand.

 

Lol

 

what about :

 

If you have a machine that supports x64 then you can run vm's

if you have 4gb or more ram then you can run a vm (guess 3 would even do )

you can run windows as vm  and install all programs the way you are used too, just don't expect anything fast or a good screen resolution without special hardware

preferably you setup a vm with a liux OS where you need to remember 2 commands depending what flavor

 

ubuntu/debian you remember apt-get install and apt-get upgrade

red hat/centos/fedora you remember yum install and yum upgrade

 

all dependencies of whatever program you want to install are included with those

and upgrading an application will also upgrade the dependencies

 

you can run a desktop version of the linux OS if you want to point and click

 

you can upgrade most of the time the complete OS with all programs with one command