64 Bit unRAID running natively on Arch Linux with full hypervisor support



Recommended Posts

  • Replies 451
  • Created
  • Last Reply

Top Posters In This Topic

I though what attracted people to PCI pass through was accessing the drives with complete control.

spin and smart control without interfering with other hardware.

No hard locked drive configuration at the hypervisor level. For most of the hobbyists, that works well for them.

 

[scratching head] ok I'm probably a bit confused (no duh).  But I thought the order of "goodness" was:

 

1) Unraid as host with direct access to SATA, all other VM's go under that.

2) Unraid as VM with pci pass through access to SATA

3) Unraid as VM with some other virtualized access to SATA that either doesn't actually work, is slow, unreliable, or all three

 

The only reason pci pass through was ever considered "desirable" was because it was better than 3) and 1) was not even an option.

 

I ask because trying to follow everything that followed I feel like either the above was lost in the noise, or I've been woefully misunderstanding what Ironic has proposed, and grumpy has been trying to get us to understand, from the very start.

I'm leaning toward 1) now.  Makes the most sense.

 

i'm just thinking about my use case ... two unraids  ;D

would it be possible to run two "instances" of unraid in the host with bare metal access to sata/m1015/etc ?

or perhaps a single unraid managing two/multiple arrays (with appropriate licensing) ?

Link to comment

1) Unraid as host with direct access to SATA, all other VM's go under that.

 

That is what we have been talking about.

 

2) Unraid as VM with pci pass through access to SATA

 

That is how we do it today. It works and works well and people can continue to do it that way.

 

3) Unraid as VM with some other virtualized access to SATA that either doesn't actually work, is slow, unreliable, or all three

 

I'm not even going to try to explain this. If you are having trouble sleeping at night... Google ESXi and RDM and you will be asleep in 10 minutes or less. To answer your question though, if you read the ESXi and RDM and stayed away... that is what Ford was talking about.

 

 

The only reason pci pass through was ever considered "desirable" was because it was better than 3) and 1) was not even an option.

 

You are correct about 1.

 

Forget 3. It's a fridge group of people who do that and that ability will remain anyway.

 

I ask because trying to follow everything that followed I feel like either the above was lost in the noise, or I've been woefully misunderstanding what Ironic has proposed, and grumpy has been trying to get us to understand, from the very start.

 

You got it right. In a nutshell, we passthrough drives because unRAID at the moment doesn't have a Type 1 Hypervisor enabled in it now.

 

We are not the only ones having fun. The FreeNAS / NAS4Free are all going nuts too. They run on BSD and since they have a "crippled" version of Xen and a "crippled" Linux Port of KVM... there is a huge group / discussion going on over there trying to "jury rig" what we are talking about here.

 

Good News though, ours WORKS and has been proven / tested / implemented in Huge Large Scale Deployments. Just a matter of figuring out how best to do it in unRAID.

 

 

Link to comment

Thanks Grumpy.  That all makes sense.  For sure I do NOT have any desire to learn about 3) beyond knowing it is NOT how we want to do this thing. I only mentioned it in contrast to 2) as  ring "desirable"  absent 1)

 

And even better now it sounds like Tom might be coming on board with 1) :-D

 

Sent from my Nexus 7 using Tapatalk 4

Link to comment

While this is somewhat in the right direction, this type of gatekeepery is not conducive to distributed growth. It keeps software in silos, unable to be reviewed, changed, or improved by anyone but the creator.

 

With all due respect, this is exactly how Linux Package Mangers handle it today.

 

Using Samba and Debian as an example:

 

http://packages.debian.org/wheezy/samba

 

You will notice there are in this case a team of people (Maintainers) who are solely responsible for their package and integration / consistency / updates / bug reports / reporting issues to the Linux Kernel and Samba folks / testing of Samba in Debian and all the various versions of Debian too. These people are experts on Samba, what's new, what bug fixes are needed, what patches may or may not be out there for security flaws, preparing it for the next version for release, preparing it for the next release of Debian, etc.

 

That's right. Not only is the release backed by an organization, but the packages are backed by teams of maintainers.

Debian's packages aren't siloed though-- anyone can theoretically publish a package there (simply put), and that's key.

 

We may not have the scale that Debian is dealing with, but we definitely want to avoid the gatekeeper pattern. It sucks that Slackware has such a poor package system to begin with. (This topic probably warrants it's own thread?)

Link to comment

LOL you'd be surprised....  ;)

 

You picked Slackware... not me.  :o

 

I'm sure you are aware of this but you can create a "bzroot" in CentOS too (and it be just as small as).

 

When you switch to the unRAID 64 bit... Plugins are going to be an issue as you are well aware.

 

If you change distros to a REAL one that has a HEARTBEAT like CentOS... You could leave the unRAID 5.0 plugins as is. Avoid all the crap that can happen with the 32 Bit / 64 Bit switchover.

 

Have unRAID 64 run on CentOS and the plugin guys have NEW (modify their old plugins to download the CentOS packages) plugins. Heck they could even get their library packages like python from the default repo.

 

It would be a good time to have a hard stop to 32 Bit, those Plugins and Slackware and have unRAID 6.0 be a fresh / new start for 64 Bit and new 64 Bit Plugins.

 

Kill a lot of birds at one time and remove the 32 Bit Plugins bomb their 64 Bit unRAID festivities that are sure to happen.

Link to comment

It sucks that Slackware has such a poor package system to begin with. (This topic probably warrants it's own thread?)

 

That is moral of the story.

 

I would use Gentoo 10 out 10 times over Slackware and it's suppose to be A LOT harder (To me it seems easier because it's not DEAD and you find everything on their website, forum, wikis, out on the web).

 

Good luck doing a search on Slackware where somebody does anything in it that isn't dated before 2009.

 

Took me half a day to compile all the dependencies (and the dependencies of the dependencies) needed before I could compile XBMC itself. I feel like I should sell my XBMC package to the 5 people who actually run Slackware and ask them if they want it. I could probably make $100 from each of them so they don't have to go through the same crap I did.

 

I later installed it in CentOS.

 

yum install xbmc

 

Never felt so good!

Link to comment

The Pros of CentOS are reliability and a solid reliable repo of additional packages.

 

 

The Cons are going to be people who want to write their own RPMS.

While I do it all the time, it's going to take some work for people who are unfamiliar with it.

What I've always liked about RPMS is that you can download an RPM via HTTP and install/update it.

Link to comment

The Pros of CentOS are reliability and a solid reliable repo of additional packages.

 

 

The Cons are going to be people who want to write their own RPMS.

While I do it all the time, it's going to take some work for people who are unfamiliar with it.

What I've always liked about RPMS is that you can download an RPM via HTTP and install/update it.

 

You mean for plugins? They don't necessarily need be be distributed via rpm.

Link to comment

The Cons are going to be people who want to write their own RPMS.

While I do it all the time, it's going to take some work for people who are unfamiliar with it.

What I've always liked about RPMS is that you can download an RPM via HTTP and install/update it.

 

I think a large majority of the users would continue to use plugins.

 

If Tom would create an unRAID repo, add it too the CentOS bzroot. The plugin guys would pull (if their package needed it) python, python-cheetah, openssl, whatever... All from ONE place and you wouldn't have one plugin trying to use X version and another trying to use Y version.

 

Way it works now, the plugins guys can mess each other up should one select a different version of a package that is common between plugins. It happens all the time and most of the posts / issues on here are a direct result of this very issue.

 

Also, let's say I put together a package that doesn't exist in CentOS like PlexMediaCenter. I upload the package or give it to Tom to put into the unRAID repo and provide a simple Plugin for it. Otherwise I have to write a VERY complex plugin that has to do A LOT of things just to install it correctly and could screw up other Plugins. Should I get hit by a bus, there are plenty of users who could create an upgraded PlexMediaCenter package and update the plugin when a new version comes out.

Link to comment

The Cons are going to be people who want to write their own RPMS.

While I do it all the time, it's going to take some work for people who are unfamiliar with it.

What I've always liked about RPMS is that you can download an RPM via HTTP and install/update it.

 

I think a large majority of the users would continue to use plugins.

 

If Tom would create an unRAID repo, add it too the CentOS bzroot. The plugin guys would pull (if their package needed it) python, python-cheetah, openssl, whatever... All from ONE place and you wouldn't have one plugin trying to use X version and another trying to use Y version.

 

Way it works now, the plugins guys can mess each other up should one select a different version of a package that is common between plugins. It happens all the time and most of the posts / issues on here are a direct result of this very issue.

 

Also, let's say I put together a package that doesn't exist in CentOS like PlexMediaCenter. I upload the package or give it to Tom to put into the unRAID repo and provide a simple Plugin for it. Otherwise I have to write a VERY complex plugin that has to do A LOT of things just to install it correctly and could screw up other Plugins. Should I get hit by a bus, there are plenty of users who could create an upgraded PlexMediaCenter package and update the plugin when a new version comes out.

 

Wouldn't packages just come from official sources? Plex might not be a good example as it already has an rpm for CentOS. Otherwise, plugins could use version constraints to avoid dependency conflicts.

Link to comment

I think a large majority of the users would continue to use plugins.

 

I for one wouldn't.  I'm worried that plugins for the most part rely on one person and when real life catches up to them the plugins get stale.  Too many examples of that here to mention.

 

The thought of a baremetal unRAID that allows me to run VMs is intersting.  I could then load a Headless XBMC and MySQL DB in one, SABzbd, SB, CP, Transmisson & Flexget in another, and what ever else tickles my fancy.  I really don't need the bare metal host to do much more than host VMs and act as my NAS.

 

Having a full fledged Distro with unRAID as the host might entice me to load up a hom DHCP/DNS and maybe Firewall, but I don't think I'd add too much more to the Host but rely on VMs (resources permitting).

 

Actually, why would people want to use plugins when thay can install the native packages directly into a VM ?

Link to comment

Wouldn't packages just come from official sources? Plex might not be a good example as it already has an rpm for CentOS. Otherwise, plugins could use version constraints to avoid dependency conflicts.

 

Plex was a bad example but you know what I am talking about.

 

If Tom doesn't take advantage of a Linux Distros repo / package manager capability... What you have there would be most useful.

 

Otherwise we can continue to let Plugins pull whatever from random places on the internet like they do now and hope for the best.

Link to comment

I for one wouldn't.  I'm worried that plugins for the most part rely on one person and when real life catches up to them the plugins get stale.  Too many examples of that here to mention.

 

From what Tom and the mods were saying... Plugins are important, needed and continue to be supported for the users who use unRAID as a straight up NAS with a couple of plugins. Which makes sense to me and has no effect on how I use unRAID.

 

However, for "Power Users"...

 

The thought of a baremetal unRAID that allows me to run VMs is intersting.  I could then load a Headless XBMC and MySQL DB in one, SABzbd, SB, CP, Transmisson & Flexget in another, and what ever else tickles my fancy.  I really don't need the bare metal host to do much more than host VMs and act as my NAS.

 

Having a full fledged Distro with unRAID as the host might entice me to load up a hom DHCP/DNS and maybe Firewall, but I don't think I'd add too much more to the Host but rely on VMs (resources permitting).

 

That is basically how I do it now and pfSense in a VM works like a champ.

 

If anyone who has install a Linux Distro and spend 15 minutes with it that will forgo the plugins and use the package manager. The documentation on the web for installing, configuring, upgrading, etc. is going to far exceed anything Tom and company will put together / maintain. If someone really wants to install Mariadb in their CentOS unRAID for example... They will go learn exactly how it's done via 100+ guides on the web and use the CentOS package manager (which only has to be done once).

 

Actually, why would people want to use plugins when thay can install the native packages directly into a VM ?

 

The people that only run a plugin or two and don't really do anything "fancy" with their NAS.

 

I don't actually happen to be one of those people. However, people tell me they do exist.

Link to comment

Wouldn't packages just come from official sources? Plex might not be a good example as it already has an rpm for CentOS. Otherwise, plugins could use version constraints to avoid dependency conflicts.

 

Plex was a bad example but you know what I am talking about.

 

If Tom doesn't take advantage of a Linux Distros repo / package manager capability... What you have there would be most useful.

 

Otherwise we can continue to let Plugins pull whatever from random places on the internet like they do now and hope for the best.

 

The distro package manager should definitely be leveraged, I'm just not sure that it should be the primary mechanism for plugins. They could easily take advantage of it though. Boiler essentially functions as a url shortener--very simple to implement, but gives tons of convenience. I had to build trolley in order to get basic distro package manger functionality. I'd love to be able to offload that to the distro.

Link to comment

The distro package manager should definitely be leveraged, I'm just not sure that it should be the primary mechanism for plugins. They could easily take advantage of it though. Boiler essentially functions as a url shortener--very simple to implement, but gives tons of convenience. I had to build trolley in order to get basic distro package manger functionality. I'd love to be able to offload that to the distro.

 

I'm sold and it could help deal with the havoc the updates to packages, plugins and unRAID causes. Get Tom and the plugin folks on-board.

 

Like I said, a majority of the issues / posts on here are a direct result of Slackware, the lack of a package manager and the way plugins are written / done today.

 

Your program would go a long way with preventing the issues people experience.

Link to comment

The distro package manager should definitely be leveraged, I'm just not sure that it should be the primary mechanism for plugins. They could easily take advantage of it though. Boiler essentially functions as a url shortener--very simple to implement, but gives tons of convenience. I had to build trolley in order to get basic distro package manger functionality. I'd love to be able to offload that to the distro.

 

I'm sold and it could help deal with the havoc the updates to packages, plugins and unRAID causes. Get Tom and the plugin folks on-board.

 

Like I said, a majority of the issues / posts on here are a direct result of Slackware, the lack of a package manager and the way plugins are written / done today.

 

Your program would go a long way with preventing the issues people experience.

 

Agreed. It's a huge undertaking, and only tangentially attached to the bigger picture in this thread. It'll be easy enough to update boiler to support whatever distro. Plugins themselves...maybe we needed a good purge anyway?

Link to comment

I have a question, if unraid baked into a distro was realised, would the amount of memory for unraid be configurable by the end user ?

 

Yes.

 

Take a look at the first screen shot on the first post. Also, run the following command on your unRAID Server:

 

free -m

 

How much were you thinking of giving it?

 

I run all kinds of crap on my Server (Pxe, mysql, webmin, XBMC, etc.) and it uses less than a 1GB of memory. I allow it to ballon to 2GB. The rest is for my VMs. I haven't looked lately but I will bet with everything going and a parity check it didn't go over 1.5 GB.

Link to comment

My thoughts on this

 

Give unraid full KVM support, and perhaps even move away from slack to as eg CentOS with bzimage / bzroot support that we have today.

 

Then it's up to each one to virtualize what they want ......

 

So how do we control and creating our VM? let's say if we could controller and create our VM from a web page, it would be perfect.

 

I have my server in my basement, I think most people have there server headless, either in a basement or in a closet :-)

 

 

//Peter

Link to comment

The distro package manager should definitely be leveraged, I'm just not sure that it should be the primary mechanism for plugins. They could easily take advantage of it though. Boiler essentially functions as a url shortener--very simple to implement, but gives tons of convenience. I had to build trolley in order to get basic distro package manger functionality. I'd love to be able to offload that to the distro.

 

I agree on using a more lightweight approach for this...but it also should not open up all gates to let a user start a mess with his/her core system.

've already seen it mentioned here somewhere...what about something like this: https://www.docker.io/ to deliver apps that are not  part of the main distro?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.