64 Bit unRAID running natively on Arch Linux with full hypervisor support

[jumperalex]

Well, IF you were to choose Linux as the Plex OS in your VM...you can add any SMB share as a mount-point when the VM boots. Plex application/server won't see the difference between these being local or networked.

 

Yes that does sound like it should work. 

 

Edit: with the appdrive mounted/inserted in the host, it will have the same availability as the local disks in VM...no risk involved to break anything.

Edit2: I don't know Plex or the concept of an appdrive...if the intention is to share that data with more than one box or make it portable, there might be scenarios where Plex needs to have some kind of support for that or be aware of that...for example locking access for concurrent access during writing of data.

 

re: edit2: no it is nothing like that, and as you will see below I've confused the situation with my ignorance I think.

 

OK I now realized I was basing everything I've said re: an appdrive etc on the UnRaid plugin for PMS.  That works by installing the main package and then hitting a config page in unraid to tell PMS where to store metadata and temp files (either the array, cache drive, or a so called appdreive mounted external to the array).  HOWEVER a normal Linux/iOS/Windows install defines that on its own and is located in the same place as the PMS install (unraid can't due to being stored in ram of course).  According to PMS docs on Linux it is stored at "~/Library/Application\ Support/Plex\ Media\ Server/"

 

So I think I got myself all wrapped around the axle on the wrong issue.  The question I think I should be asking is, absent controller pass through, I assume the Linux-VM can run without being given passthrough access to a controller, and any app that installs will have access to whatever storage space the VM has available to it?  I think I should feel silly for asking a question that has the likely answer of, "well duh of course," but I'd rather not assume since I've already shown I'm liable to confuse myself.

 

Of course this goes to a larger question I guess I have: under what circumstances does a VM need controller passthrough? Is it just when the VM OS needs direct full access to something at the hardware level, like the SATA controller or video card?  And what does it mean to passthough the video card as opposed to what I assume is my ability to "see" the VM OS through the hypervisor?

 

You don't even need a VM.

 

You boot OpenElec over the network using PXE and could (not needed) use NFS, iscsi, AoE, virtFS, etc

 

Let's assume its straight unRAID...

 

You boot your openELEC via PXE. It loads Openelec over IP (in seconds) all the configuration is stored (per openelec box) on the unRAID server. You can either use the internal XBMC database or put all your XBMCs on a central one using the mysql plugin. That way if you watch 15 minutes of a movie and switch TVs when you go to another TV to watch it... it starts where it was.

 

You would access your movie or TV folders on your unRAID via NFS or Samba.  Everything works like it normally does.

 

You do not need a Hard Drive or VM for Openelec (Or Linux and Windows) if you do it this way. Its extremely fast and even sleep works. You turn on the openelec box, it boots into the server via PXE, loads the OS and XBMC and away you go. It happens almost as fast as if you had a hard drive in it.

 

Thanks grumpybutfun.  I'm pretty invested in Plex Media Server + Roku clients at the moment because it can best be described as the turn-key solution of what you just described.

 

BUT, the implications of what you're saying, I think, are not lost on me and probably reinforce what I think Ford is saying coupled with my realigned understanding of how PMS installs on a full Linux install vice as a package in UnRaid right now.

[grumpybutfun]

BUT, the implications of what you're saying, I think, are not lost on me and probably reinforce what I think Ford is saying coupled with my realigned understanding of how PMS installs on a full Linux install vice as a package in UnRaid right now.

 

Right. It's why several of us have said to drop the plugin system and let Linux do the heavy lifting.

 

The reason why we have to do plugins is due too...

 

The bzroot is Slackware Linux compressed. When you turn on your unRAID machine, it boots into the Linux Kernel (the bzimage) which then extracts the bzroot (Slackware Linux) into memory. That is why you need plugins. Whatever packages you install, they are not saved in the bzroot. Each powerup, it downloads and installs the various linux packages (into memory), reads the config files for those packages / plugins either on the flash drive or a cache drive.

 

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE. You could still have it read the config files from the Flash Drive or the cache drive. If you wanted to install Webmin for example... You could use the Package Manager in Slackware, Slackbuilds, Slapt-get, sbopkg and install it ONCE (Slackbuild Webmin). Let Slackware do the heavy lifting instead of reinventing the wheel. Everyone knows how to do "slackpkg install xbmc" (Slackware version of "apt-get install xbmc" in Ubuntu). They don't know how to write / manage / fix plugin issues. Sure, someone could write a plugin that installs webmin but that takes FOREVER and still is no guarentee it will work consistently. However, typing "slackpkg install webmin" takes a few seconds and only has to be done ONCE. You could have an unRAID software repository where we all get the same version of MySQL or Python or whatever.

 

If you wanted phpmyadmin, Mariadb, Flexget, etc... It's very easy to do when bzroot is extracted onto the flash drive itself and could be added into that unRAID repository I was referring too. Users could add packages A LOT easier than writing a script. I could add owncloud or Plex Media Server in minutes for example but would take me months if I had to write a plugin.

 

For the record, this is not an endorsement for Slackware. If I was Tom, I would port unRAID over to CentOS, Debian, openSUSE, etc. Slackware is probably to most difficult Linux Distro I have used. Gentoo is technically more difficult but thanks to the website, wikis, forums, user base, all the info you find on the web on it... it's way easier than Slackware. Also, the package manager in Slackware sucks ass! It only has 100s of packages where CentOS, Debian, openSUSE, etc. have 10,000+ with each one having Package Managers dedicated to each. Each package goes through rigorous testing before the next version is released. The various packages all pull from the same database and place via the package manager. If a package needs python (or curl or openssl, etc.) it installs it if needed or if another package already downloaded it, installed it... it doesn't.

 

They way plugins work, they are dumb. The Plugin Managers go grab what they need for their plugin to work. The tvheadend may pull one version of Python and the Plex plugin may pull another and sometimes they both pull it from different places. This can break their plugin or other ones. It's a total mess if you ask me and totally reinventing the wheel of a function that Linux can handle all on its own and in a much better way.

[Ford Prefect]

OK I now realized I was basing everything I've said re: an appdrive etc on the UnRaid plugin for PMS.  That works by installing the main package and then hitting a config page in unraid to tell PMS where to store metadata and temp files (either the array, cache drive, or a so called appdreive mounted external to the array).  HOWEVER a normal Linux/iOS/Windows install defines that on its own and is located in the same place as the PMS install (unraid can't due to being stored in ram of course).  According to PMS docs on Linux it is stored at "~/Library/Application\ Support/Plex\ Media\ Server/"

 

So I think I got myself all wrapped around the axle on the wrong issue.  The question I think I should be asking is, absent controller pass through, I assume the Linux-VM can run without being given passthrough access to a controller, and any app that installs will have access to whatever storage space the VM has available to it?  I think I should feel silly for asking a question that has the likely answer of, "well duh of course," but I'd rather not assume since I've already shown I'm liable to confuse myself.

Yes, on the host you will define the VM virtual hardware. One part of that is the virtual disk (one or more) which normally is a file on the VM-host.

The VM will see it as attached to what virtual controller you have given/configured to it (scsi, ide, sata or accelerated ones, like virtio-disk, given you have installed drivers in the VM-OS).

In addition, if the VM is connected, it can access to any network resource you (or your firewall) wll allow it to....this includes a network-Fileshare, which could be unRAID...even on the same host or in another VM.

 

Of course this goes to a larger question I guess I have: under what circumstances does a VM need controller passthrough? Is it just when the VM OS needs direct full access to something at the hardware level, like the SATA controller or video card?  And what does it mean to passthough the video card as opposed to what I assume is my ability to "see" the VM OS through the hypervisor?

You do passthrough a controller in order to give direct, native access to that resource inside the VM...to the OS/drivers in the VM-OS, that is.

This is what you want, if only the VM-OS has drivers for this resource or when you need native performance or when the access path is critical.

For example with your disks / raid you want the access layer as thin as possible, because when something goes wrong, your data is in danger.

Risk is that any software layer will add a certain lag, complexity and potential cause for an error.

 

The same applies to a video card.

The VM will have one, or more, virtual ones.

However for native FullHD or 3D gaming performance, these are not suitable...*and* they don't have/offer a physical connector to your TV/Monitor.

You normally access the Display/Screen of your VM via a remote desktop protocol, like RDP or VNC from another computer (or the same host).

[limetech]

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE.

 

Yes that is true, however it becomes a real chore to keep things up-to-date and for me, and other people helping users, to have a common base of understanding what the system config is.

 

I'll put it this way.  After I've prepared a new release of unRaid OS, the user generally needs only to copy 'bzimage' and 'bzroot' to their flash.  I *know* what their root file system looks like and what packages are installed.  It's also very easy to boot without "plugins" or "extra" packages installed to further simplify the configuration.  If someone has handed over their hard earned cash for a key, then I am obligated to provide the best support I can.  If someone has installed numerous other packages on top of the base "unRaid OS", and then has a problem in say, Samba, well to what extent am I obligated to fix the problem?  Related, how much time am I obligated to spend analyzing their particular configuration which may have grown quite a bit since initial install?  I  may not even be able to fix their problem.

 

Another advantage to loading the OS from flash, and then leaving the flash alone, is that it reduces wear-and-tear on the flash devices.

 

I think there are two ways to approach this new use of unRaid.  First is to ditch slack and integrate unRaid core with a more "modern" distro.  This makes it very difficult to maintain the model of "bzimage/bzroot" on the flash.  Now we get into the realm of assigning and maintaining boot/system disks - more configuration.  Also, now anyone who wants to use unRaid needs to be somewhat proficient at linux and the "appliance" model kinda goes away.  A great deal of current unRaid users don't want to do this.  They don't care what's under the hood and they don't have time to learn it.  Any why should they?  If unRaid imposes this, it's easier to just go buy a Buffalo.

 

The second approach is to run a VM-friendly unRaid image in a VM hosted by whatever you want: Arch, CentOS, even Windows.

 

So look, I'm not imposing anything here or making any final decisions.  This is just how I see it at the present time.  My own background is in Enterprise Storage, e.g., used to work at Sun's Network Storage Division.  The golden rule is: "Don't lose the customer's data."  I can be rightly criticized for seeming slow pace of releases at times, but I have very rarely lost all the customers data (I admit it has happened though).  I have found that always upgrading to latest components is both necessary (obviously), but also not without risk, and I try to introduce major changes one component at a time.  You can point to "security risks" identified in older components, but really, in a home LAN or small business LAN, this is not practically an issue.  But do I want unRaid to expand into other markets? YES, which is why development continues and I'm grateful for the ideas being discussed here.

[jumperalex]

BUT, the implications of what you're saying, I think, are not lost on me and probably reinforce what I think Ford is saying coupled with my realigned understanding of how PMS installs on a full Linux install vice as a package in UnRaid right now.

 

Right. It's why several of us have said to drop the plugin system and let Linux do the heavy lifting.

 

.....

 

[nodding emphatically]  You had me at hello ;-)

 

Yes, on the host you will define the VM virtual hardware. One part of that is the virtual disk (one or more) which normally is a file on the VM-host.

The VM will see it as attached to what virtual controller you have given/configured to it (scsi, ide, sata or accelerated ones, like virtio-disk, given you have installed drivers in the VM-OS).

In addition, if the VM is connected, it can access to any network resource you (or your firewall) will allow it to....this includes a network-Fileshare, which could be unRAID...even on the same host or in another VM.

 

Ok that is indeed good to hear.

 

You do passthrough a controller in order to give direct, native access to that resource inside the VM...to the OS/drivers in the VM-OS, that is.

This is what you want, if only the VM-OS has drivers for this resource or when you need native performance or when the access path is critical.

For example with your disks / raid you want the access layer as thin as possible, because when something goes wrong, your data is in danger.

Risk is that any software layer will add a certain lag, complexity and potential cause for an error.

 

The same applies to a video card.

The VM will have one, or more, virtual ones.

However for native FullHD or 3D gaming performance, these are not suitable...*and* they don't have/offer a physical connector to your TV/Monitor.

You normally access the Display/Screen of your VM via a remote desktop protocol, like RDP or VNC from another computer (or the same host).

 

Ok so what I'm hearing is, for a CentOS-VM with Plex installed in it (because that is one of the Linux installs Plex offers in addition to Ubuntu and Fedora), I should not really have to worry about pass-through since Plex just needs a place in the VM-OS to store its database and network access.  And the same can be said for just about all the other typical plug-ins we run, or want to run, right now like SANZB, Couchpotato, uTorrent, MakeMKV (ok this might need direct access to the BD-drive?), Handbrake ... 

 

But things like using it as an HTPC with XBMC connected to a TV will require video card pass-through.

 

And of course this one reason why we'd rather have UnRaid as part of the host vice running as a VM since it opens up the hardware we can reliably use.  Right?

 

 

[grumpybutfun]

Tom, thanks for sharing your thoughts / views on the various things we have discussed in this thread.

 

We will continue to make suggestions and if there is anything we can do to assist, let us know.

[ironicbadger]

Tom...

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE.

 

 

So look, I'm not imposing anything here or making any final decisions.  This is just how I see it at the present time.

 

dang, that's a shame...

 

perhaps we will see this one day in unraid's future even if not for now...

 

[WeeboTech]

Tom...

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE.

 

 

So look, I'm not imposing anything here or making any final decisions.  This is just how I see it at the present time.

 

dang, that's a shame...

 

perhaps we will see this one day in unraid's future even if not for now...

 

 

Has anyone given consideration to using UNIONFS or AUFS then using the full slackware install? Kinda like Knoppix?

I compiled a kernel a long time back and it was working for my tests. This could provide the best of both worlds.

 

 

Bare unRAID or XunRAID(exanded).

 

 

While I'm a centos voter(yes I admit it), slackware provides a really simple way to do things in a minimalist approach.

[Ockingshay]

Tom, with that being your view, where do you see unRAID going?

 

Clearly there is demand for extra programs through whatever means (that currently being plugins). Running virtual machines and software that is already compiled. Separating core unRAID and allowing plugins that if break only break themselves is possible? This is what separates you from synology/readynas.

 

Secondly there is demand for a more feature rich webgui to allow monitoring, pre clearing, plugin management etc. utilising GitHub for issue management, user contributions etc

 

Thirdly to keep core unRAID up to date in terms of share connectivity, AFP, Samba etc

 

Personally those are the only things missing, a framework at least so that everyone writes code in the same direction.

[jumperalex]

Tom...

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE.

 

 

So look, I'm not imposing anything here or making any final decisions.  This is just how I see it at the present time.

 

dang, that's a shame...

 

perhaps we will see this one day in unraid's future even if not for now...

 

@ironicbadger, In full context I don't think that sounds all that dire.  His concerns seem reasonable and workable.  Its only been a few days, things have to sink in, you guys need to talk off-line.  Maybe with a few beers in hand

 

@Tom, Would it be infeasible to still maintain the bare-metal flash-only version as well as the Host-OS version for the two different classes of use?  Wouldn't the core functionality be reusable anyway?  I don't know how much harder this would be on the front end but on the back-end support side it should keep the more appliance minded folks from being scared off while giving flexibility to those of us who are willing to work for it.

 

As to dealing with configuration drift ... I don't know. I have a lot of vague ideas in my head that are really nothing more than the uninformed ramblings of someone who has never had to be on your side of a customer support call.  But one thing is for sure, if you want to expand your customer base you WILL have to deal with more support issues and that might mean a business decision that involves another technically savvy person to help you out [shrug]

[limetech]

Tom...

You could extract the bzroot and put it on the Flash Drive itself (there are few steps involved but fairly easy) and install those linux packages ONCE.

 

 

So look, I'm not imposing anything here or making any final decisions.  This is just how I see it at the present time.

 

dang, that's a shame...

 

perhaps we will see this one day in unraid's future even if not for now...

Badger, what's a shame exactly?  This is a discussion of the best approach moving forward and I'm open to all input.  Everyone is chiming in with their ideas and concerns, and I'm simply laying out some things you have not thought out, that is all.

[limetech]

Tom, thanks for sharing your thoughts / views on the various things we have discussed in this thread.

 

We will continue to make suggestions and if there is anything we can do to assist, let us know.

 

Something that would be helpful is perhaps a summary of the approaches possible with pros/cons.

 

I think the advantage of having the unRaid "distro" be the host is that it gives the unRaid components direct access to the hardware, is that correct?

 

How about this: couldn't unRaid OS function exactly as before: boot from flash with preconfigured bzimage/bzroot with VM support built in (which one?).  This makes unRaid be a thin "minimum" layer which can be run simply as it is now, or one could load up multiple images of other systems in individual VM's.  Would this be a good approach?

[limetech]

Is this possible to do?  Add a crypto layer between md-devivces and mounted disks, and it's the mounted disks (or shares) that get exported to client VM's.  This way all client VM disk contents become encrypted "automatically".

[jumperalex]

FWIW, some quick googling tells me the GIGABYTE GA-990FXA-UD3 supports IOMMU/AMD-Vi.  $142 at amazon.  Not too bad since I was planning on probably spending $119 on another board that it seems does not support IOMMU despite having a chipset that does.

 

They also talk about the USB being forwardable but that is in context of ESXi: http://hardforum.com/showthread.php?t=1757984

[grumpybutfun]

Something that would be helpful is perhaps a summary of the approaches possible with pros/cons.

 

I think the advantage of having the unRaid "distro" be the host is that it gives the unRaid components direct access to the hardware, is that correct?

 

If you read through the thread. I explain in great details the good / bad throughout it plus all the new / exciting features / programs / etc.

 

Everything from package management, updates, VirtFS, 2,000+ kernel modules, PCI Passthrough for Nvidia, etc.

 

Everyone on here has installed a Ubuntu CD Live via an ISO onto a Flash Drive. You incorporate all the things we have talked about and the Flash Drive will last YEARS and being you can buy them for $5 dollars who cares. If you want to install openSUSE with XBMC, unRAID, Owncloud on a Flash Drive and run it for years before it goes bad... I don't think anyone cares. You still have the stock bare metal unRAID Flash Drive where the key sits and config files remain which is hardly accessed. They just replace their openSUSE one in 5 to 10 years.

 

If someone loaded XBMC and it caused a problem or screws up their version of the "unRAID CD Live" on their flash drive. Reimage it and start over. I could make all the config files be saved on the unRAID USB Flash Drive. Their VMs are still on their datastore drives, the unRAID drives are still intact and unharmed. Just reimage the flash drive with the ISO. It would go read the config files on the unRAID flash drive and look just like it did. Not to mention, they could just boot unRAID "bare metal" and it wouldn't notice the difference. Also, all the files, programs, etc. would be exactly what you expect since it's a new "image" without the customizations. You can also have a "recovery mode" in the grub boot menu that doesn't load plugins. Which is pointless and not needed because plugins wouldn't exist anymore. There isn't a plugin manager who can write a plugin that will be better than the Linux Distro Package Manager. It a waste for someone to attempt it and not needed.

 

If you take my openSUSE 13.1 with KVM thread as an example. I could put all of the stuff people want in this tread into a Live CD with unRAID backed in. It would take me 15 minutes and most of that is creating the ISO and uploading it which people would install.

 

New verison of unRAID comes out, I put up a new ISO. It still pulls the config files from the unRAID USB Stick. As far as the user / unraid / vms are concerned... Nothing change besides updates to PHP or NFS, webGUI, etc.

 

If they installed it onto a hard drive... I could point it to my repo. When there is an update to unRAID or a securtiy update to PHP, NFS, or SSH... "yast update" handles that. The updated unRAID package would install a new kernel if needed, upgrade emhttp if needed, update samba if needed, etc.

 

Or they could use the new "release" and create a new ISO and use that like I mentioned earlier.

 

If you don't grasp the concept of that, which all of us have seen, used on pretty much every Linux Distros there is... I dunno. Go Download Linux Mint Live CD and see it in action for yourself. I think the lightbulb will come on then and it will make more sense.

[ogi]

Something that would be helpful is perhaps a summary of the approaches possible with pros/cons.

 

I think the advantage of having the unRaid "distro" be the host is that it gives the unRaid components direct access to the hardware, is that correct?

 

If you read through the thread. I explain in great details the good / bad throughout it plus all the new / exciting features / programs / etc.

 

Everything from package management, updates, VirtFS, 2,000+ kernel modules, PCI Passthrough for Nvidia, etc.

 

Everyone on here has installed a Ubuntu CD Live via an ISO onto a Flash Drive. You incorporate all the things we have talked about and the Flash Drive will last YEARS and being you can buy them for $5 dollars who cares. If you want to install openSUSE with XBMC, unRAID, Owncloud on a Flash Drive and run it for years before it goes bad... I don't think anyone cares. You still have the stock bare metal unRAID Flash Drive where the key sits and config files remain which is hardly accessed. They just replace their openSUSE one in 5 to 10 years.

 

If someone loaded XBMC and it caused a problem or screws up their version of the "unRAID CD Live" on their flash drive. Reimage it and start over. I could make all the config files be saved on the unRAID USB Flash Drive. Their VMs are still on their datastore drives, the unRAID drives are still intact and unharmed. Just reimage the flash drive with the ISO. It would go read the config files on the unRAID flash drive and look just like it did. Not to mention, they could just boot unRAID "bare metal" and it wouldn't notice the difference. Also, all the files, programs, etc. would be exactly what you expect since it's a new "image" without the customizations. You can also have a "recovery mode" in the grub boot menu that doesn't load plugins. Which is pointless and not needed because plugins wouldn't exist anymore. There isn't a plugin manager who can write a plugin that will be better than the Linux Distro Package Manager. It a waste for someone to attempt it and not needed.

 

If you take my openSUSE 13.1 with KVM thread as an example. I could put all of the stuff people want in this tread into a Live CD with unRAID backed in. It would take me 15 minutes and most of that is creating the ISO and uploading it which people would install.

 

New verison of unRAID comes out, I put up a new ISO. It still pulls the config files from the unRAID USB Stick. As far as the user / unraid / vms are concerned... Nothing change besides updates to PHP or NFS, webGUI, etc.

 

If they installed it onto a hard drive... I could point it to my repo. When there is an update to unRAID or a securtiy update to PHP, NFS, or SSH... "yast update" handles that. The updated unRAID package would install a new kernel if needed, upgrade emhttp if needed, update samba if needed, etc.

 

Or they could use the new "release" and create a new ISO and use that like I mentioned earlier.

 

If you don't grasp the concept of that, which all of us have seen, used on pretty much every Linux Distros there is... I dunno. Go Download Linux Mint Live CD and see it in action for yourself. I think the lightbulb will come on then and it will make more sense.

 

Gumpy,

 

I know you've been neck deep in this for the last while, going pedal to the metal.  Keep in mind that this involves the modification of Tom's baby, and he hasn't been as involved in this project as you have been, and is just now trying to get up to speed with what is being proposed.  He is being cautious is all, but he is clearly open to modification, he's just putting what things look like from his perspective.

 

There is an optimum solution to be found, just give it time and keep the communication light hearted, and it will all work out.

 

Ogi

 

P.S. I'm very very very interested in setting up a VM for unraid, and given my lack of linux knowledge, I don't feel qualified to suggest a distro to work with to achieve that, which a lot of other posters have echoed the same sentiment.  I plan on implementing this project the moment I think it is so idiot proof that even I can't mess it up. 

[limetech]

If you don't grasp the concept of that, which all of us have seen, used on pretty much every Linux Distros there is... I dunno. Go Download Linux Mint Live CD and see it in action for yourself. I think the lightbulb will come on then and it will make more sense.

Grumpy I know what a live CD is.  There are pages and pages of posts, most long on arm waving, short on details.  I'm just looking for a shortcut to get to the details.  I have quite a bit on my plate at the moment.  It sounds like you want to create an "unRaid distro" with all that entails - aren't there enough linux distros in the world already?

[jumperalex]

[sigh] is there any chance an invite only thread could be opened up for the likes of Tom, Grump, Ironic, and Ford to have a discussion.  It'd be great if the community could see it but not post.  However beyond satisfying my inner voyeur I'd just be happy to know such a thread existed so that important details can be discussed without other distraction and fluff.

[ogi]

[sigh] is there any chance an invite only thread could be opened up for the likes of Tom, Grump, Ironic, and Ford to have a discussion.  It'd be great if the community could see it but not post.  However beyond satisfying my inner voyeur I'd just be happy to know such a thread existed so that important details can be discussed without other distraction and fluff.

 

I think that's a fantastic idea; despite knowing little about linux and even less about virtualizing OSs, I can tell there is a lot of arm waving going around, and I think it would be best for the community that there is a place that the technical details can be sorted out with a minimum of noise/ruckus from the everyone (such as myself).

[apgood]

[sigh] is there any chance an invite only thread could be opened up for the likes of Tom, Grump, Ironic, and Ford to have a discussion.  It'd be great if the community could see it but not post.  However beyond satisfying my inner voyeur I'd just be happy to know such a thread existed so that important details can be discussed without other distraction and fluff.

 

That sounds like a good idea to me. A developer thread of sorts...

 

[grumpybutfun]

It sounds like you want to create an "unRaid distro" with all that entails - aren't there enough linux distros in the world already?

 

I was giving you another method of having a "static" install of unRAID so it makes it easier to support.

 

As far as the unRAID Distro...

 

If I was you, would I jump through all the hoops for 51 people who responded to the vote in this thread?

 

No.

 

Are there more than 51 people who would be willing to pay for features / functionality that we discussed in this thread?

 

I dunno. You are more on top of your Marketplace, Trends and Competition than I am.

 

You laid out your concerns and support issues. Which was very good and something to consider. You even suggested people continue to let the NAS be a VM as we have done. Works like a champ and is is a great solution.

 

So far, FreeNAS, NAS4Free, etc. all work just like you suggested too.

 

As I have said, it could be a very small user base that cares about the stuff in this thread. You have a better grasp / idea / understanding of what that number is / could be, not me.

[garycase]

How about this: couldn't unRaid OS function exactly as before: boot from flash with preconfigured bzimage/bzroot with VM support built in (which one?).  This makes unRaid be a thin "minimum" layer which can be run simply as it is now, or one could load up multiple images of other systems in individual VM's.  Would this be a good approach?

 

I like this concept ... it allows a basic boot configuration that's as simple as the current setup; but provides a virtualization architecture that can be used for running additional OS's under control of a hypervisor and fully isolated from UnRAID.    This has, of course, already been done with the Virtual Box plugin that's available, but that's a Type 2 hypervisor and doesn't provide the performance that's possible with one of the Type 1 hypervisors that are being discussed.  I think the KVM approach -- which is kind of a "type 1.5" hypervisor (still under control of an OS, but much closer to the hardware, since KVM is a kernel level module) -- sounds very promising.    But whatever is done, the #1 goal should be to always obey what Tom so accurately stated earlier (and which I think he's done a very good job of maintaining the focus on as he's evolved UnRAID to where it is now):

 

The golden rule is: "Don't lose the customer's data."

[NAS]

.....

How about this: couldn't unRaid OS function exactly as before: boot from flash with preconfigured bzimage/bzroot with VM support built in (which one?).  This makes unRaid be a thin "minimum" layer which can be run simply as it is now, or one could load up multiple images of other systems in individual VM's.  Would this be a good approach?

 

This is interesting. Personally I have no interest in moving away from the appliance model as that is a main feature of what drew me here originally. But VMs running on top of the appliance is pretty interesting. Specifically not a full OS I dont need the headache that comes with that I want my NAS appliance to run and be almost maintenance free with upgrades working the traditional unRAID way. (Caveat i have multiple ESX boxes so i full fill the requirements elsewhere)

 

Out of curioisity what are the benefits of either a full HV OS/unRAID appliance HV versus say Docker.

[Ice_Black]

Interesting discussing and ideas.

 

Feel free to discuss in IRC too on freenode network, #unraid channel

[grumpybutfun]

The golden rule is: "Don't lose the customer's data."

 

A lot of people also believe in the following rule:

 

unRAID running in VM with PCI Passthrough on ESXi / Xen / KVM / etc. is SAFE but Xen / KVM running on unRAID is NOT SAFE.

 

Not sure how such a rule every started. I haven't been in a Fortune 500 Enterprise Environment in 30+ years where they would dream of doing it that way. In fact, they spend / invest a lot of money to do the exact opposite.

 

Guess users with little to no Linux / IT / Storage Experience with their home NAS devices full of Movies, TV Shows, etc. have it all figured out. Clearly, Fortune 500 companies who spend millions on IT professionals, services, consultants, have 10+ Million dollar ERP Applications and Billions in Revenue to lose need to get with the times and put all their Storage in VMs with PCI Passthrough. Otherwise they will lose their data.

 

Last project I worked on, we consolidated 1,200+ Mission Critical Servers (most of with had their own storage) that were spread-out throughout the Enterprise and located at various branches and Data Centers. Maintaining compliance with Sarbanes Oxley and all the other Financial Regulations we had to be in compliance with...  We ended up using ESXi and EMC as a solution. Not a single one of the VMs was it even discussed or thought of to use PCI pass-through for storage. Perhaps it had something to do with the data that was mission critical and worth millions of dollars the customer and the consultants didn't want to put it at risk of being lost, corrupted, etc. when using the PCI Passthrough method.

 

To listen to some of the people here... Maybe the company and us consultants had it all wrong.

 

By having your Data / Drives sit outside the VM... Technically it is considered more secure, safe, reliable for the integrity of the data. It's easier to support, better uptime, etc. If you think otherwise, I can post some whitepapers on the subject and we can discuss those. I think it's common sense but maybe I am in the minority.

 

Point is, If we are going to use safety and integrity of the data as the most important part of the equation... Let's at least be honest. Running unRAID in a VM with PCI Passthrough is considered less safe than doing it the other way around.