64 Bit unRAID running natively on Arch Linux with full hypervisor support

[Ford Prefect]

Both work fine and both make sense to me. It would to you too if spend 15 minutes and understood the structure / syntax.

 

Don't you think people would be off the races posting here how they installed / configured Ubuntu / Debian / Arch / FreeBSD / illumos / etc. and provide the command line? I know I would.

 

I wouldn't let this be the roadblock that stops this.

 

If a "power user" wants it bad enough, he will figure it out, have plenty of help here, on libvirts site, in other forums, etc. It's EVERYWHERE else on the web (libvirt is a Red Hat creation and been around a long while. It's VERY VERY popular and tons of info / resources / examples on the web).

 

Grumpy,

 

excuse me...maybe I need a brake but I can't connect that back here in the thread ATM.

...you are referring to "you"...who?"

 

Personally I agree.....to me the whole feature of virtualization is not prio 1..which is about unRAID-NG itself.

[grumpybutfun]

Personally I agree.....to me the whole feature of virtualization is not prio 1..which is about unRAID-NG itself.

 

Sorry... Let me summarize.

 

unRAID

 

Tom continues with unRAID 5.X as he always has.

 

Plugins, bzroot, bzimage, etc.

 

Possible 64 Bit version and normal updates as he always as done.

 

unRAID-NG (Next Generation or whatever Tom calls it)

 

Tom releases unRAID-NG, a separate version dedicated to "Power Users". Still has the same Linux Kernel Version, emhttp, etc. as regular unRAID.

 

1. Enables unRAID in a Linux Distro.

 

2. Releases it on an ISO that users install on a separate USB Flash Drive (aside from the unRAID USB Flash Drive) or HDD.

 

3. What the users do with their unRAID Server from there... Is solely up to them.

 

If you want KVM, install it. If you want MySQL, install it. If you want XBMC, install it. If you want WordPress, install it.

 

If you do not want the freedom /  functions / capability of running unRAID in [insert Linux Distro]... Use the other version of unRAID.

[vl1969]

Personally I agree.....to me the whole feature of virtualization is not prio 1..which is about unRAID-NG itself.

 

Sorry... Let me summarize.

 

unRAID

 

Tom continues with unRAID 5.X as he always has.

 

Plugins, bzroot, bzimage, etc.

 

Possible 64 Bit version and normal updates as he always as done.

 

unRAID-NG (Next Generation or whatever Tom calls it)

 

Tom releases unRAID-NG, a separate version dedicated to "Power Users". Still has the same Linux Kernel Version, emhttp, etc. as regular unRAID.

 

1. Enables unRAID in a Linux Distro.

 

2. Releases it on an ISO that users install on a separate USB Flash Drive (aside from the unRAID USB Flash Drive) or HDD.

 

3. What the users do with their unRAID Server from there... Is solely up to them.

 

If you want KVM, install it. If you want MySQL, install it. If you want XBMC, install it. If you want WordPress, install it.

 

If you do not want the freedom /  functions / capability of running unRAID in [insert Linux Distro]... Use the other version of unRAID.

This is perfect.

But maybe unraid should be made a module that can be installed on plain distro. Just like any other package.

 

 

Sent from my SGH-T889 using Tapatalk

 

[limetech]

A lot of work behind the scenes right now.

 

As an aside re: the poll: CentOS is still on the 2.6 kernel AFAIK, this could be an issue.

[apgood]

A lot of work behind the scenes right now.

 

As an aside re: the poll: CentOS is still on the 2.6 kernel AFAIK, this could be an issue.

 

Good point.

 

I imagine if one of the reasons for going to a heavier OS is for things like XBMC then you would want one of the latest kernels which includes things like multichannel HD audio for the Intel, AMD and nVidia (from what I've read I think kernel 3.12? has this?). Not being a Linux guy is there an issue with just compiling the most recent kernel into whichever distro that's used? Or are there compatibility and dependency issues if you do that?

 

[grumpybutfun]

A lot of work behind the scenes right now.

 

That is GREAT to hear! Thanks for the update.

 

As an aside re: the poll: CentOS is still on the 2.6 kernel AFAIK, this could be an issue.

 

CentOS 6.5 was released a few weeks ago and it using Linux Kernel 3.8.13. You could always bump up to the 3.10 series (Long Term Support) and apply the ReiserFS patch.

 

EDIT: I got my Linux Distros crossed up. Tom is correct, CentOS is using 2.6.32.

 

Red Hat 7 (currently in beta) is using 3.10 and will ship with it. I don't think you are putting us in harms way by using it too. We have been running solid on 3.9.11 for a while.

[WeeboTech]

 

from wikipedia.

RHEL 7[edit]

Red Hat Enterprise Linux 7 (Maipo) will be based on Fedora 19 and upstream Linux kernel 3.10, with the first beta announced on 11 December 2013.

 

Maybe a benchmark or proof of concept could be tested with Fedora 19.

[grumpybutfun]

 

from wikipedia.

RHEL 7[edit]

Red Hat Enterprise Linux 7 (Maipo) will be based on Fedora 19 and upstream Linux kernel 3.10, with the first beta announced on 11 December 2013.

 

Maybe a benchmark or proof of concept could be tested with Fedora 19.

 

I have a working version in CentOS with Linux Kernel 3.12.X and have thrown the Kitchen Sink at it. Running KVM, passthrough video cards to VMs, added drives, removed drives, countless parity checks, rebuilding the arra, using LVMs on drives outside of unRAID for VMs and numerous Linux packages for over a month now. I would consider myself a power user and everything works like a dream.

 

Dropping it down to 3.10 would not be an issue.

 

I have no problems give access to my test machine if people want to take a look around.

[tr0910]

If it works with 3.12, why drop back down?  Just keep testing...

[grumpybutfun]

If it works with 3.12, why drop back down?  Just keep testing...

 

I'm going to go out on a limb and guess that Tom wants a "server grade" Linux Kernel.

 

3.10 is considered one.

 

I was just testing some things in 3.12.X.

 

1. AMD FINALLY has HD Audio and you can do Video Hardware Acceleration via VDPAU (requires Mesa 10). Before 3.12.X if you were using the AMD Drivers or Radeon (open source) ones... You had to run a special version of XBMC (using XVBA) to get HW Acceleration. My CPU utilization dropped to lower than 5% when watching a 1080p with HD Audio on AMD.

 

johnodon mentioned doing a write up on it when he can wrestle some time away from his family. In his case, he did the above with his XBMC VMs no less (where he passes through several AMD Video Cards). He saw HUGE improvements with CPU Utilization and Picture Quality.

 

Intel (uses VAAPI) and nVidia (uses VDPAU) have had this ability for long time.

 

2. Testing passthrough with nVidia Graphic Cards (4XX, 5XX, 6XX) to VMs using VFIO.

 

[EGOvoruhk]

 

from wikipedia.

RHEL 7[edit]

Red Hat Enterprise Linux 7 (Maipo) will be based on Fedora 19 and upstream Linux kernel 3.10, with the first beta announced on 11 December 2013.

 

Maybe a benchmark or proof of concept could be tested with Fedora 19.

 

CentOS 7 should be dropping close after RHEL 7 does: http://www.karan.org/blog/2013/12/15/rhel-7-and-a-centos-plan/

 

Might as well use the RHEL 7 beta for testing

[NAS]

Food for thought.

 

The Debian technical committee have/are voting on the new init now. To my eye eventually Debian would be the perfect Distro. I dont know if that timescale would be soon enough but what we argue here using XBMC as a driver other with argue in the future using SteamOS.

[meep]

Having read this lengthy thread with interest, I'm excited by the possibilities here.

 

For my use case, I'm interested in consolidation but current virtualization implementations seem complex. An unRAID release that provides base unRAID functionality with the ability to run VMs with hardware passthrough on top seems ideal for me.

 

I see the case for having the base install live in a full distro allowing access to onboard graphics hardware and the ability to install additional applications. However, I've been reluctant to install 'other things' on my unRAID as it does one thing extremely well and I buy the KISS approach.

 

I'd be happy to run everything else in VMs offering a good balance of unRAID performance and stability with almost limitless flexibility.

 

Looking forward to developments.

 

Peter

[Ford Prefect]

 

Sorry... Let me summarize.

 

unRAID

 

Tom continues with unRAID 5.X as he always has.

 

Plugins, bzroot, bzimage, etc.

 

Possible 64 Bit version and normal updates as he always as done.

 

unRAID-NG (Next Generation or whatever Tom calls it)

 

Tom releases unRAID-NG, a separate version dedicated to "Power Users". Still has the same Linux Kernel Version, emhttp, etc. as regular unRAID.

 

1. Enables unRAID in a Linux Distro.

 

2. Releases it on an ISO that users install on a separate USB Flash Drive (aside from the unRAID USB Flash Drive) or HDD.

 

3. What the users do with their unRAID Server from there... Is solely up to them.

 

If you want KVM, install it. If you want MySQL, install it. If you want XBMC, install it. If you want WordPress, install it.

 

If you do not want the freedom /  functions / capability of running unRAID in [insert Linux Distro]... Use the other version of unRAID.

 

Yes, thank you for that...cannot agree more.

 

...in the spirit of this, let me fetch some points I made earlier ... as this thread is getting larger and larger, it might be worthwhile to keep things together.

 

A) - intregrating in a distro and reworking the i-factor into unRAID/emhttp

I am still  giving this more thoughts...debating about virtualization is nice but is this the right way for making unRAID more poular?

 

I think unRAID needs another deployment/distribution method...more like google or apple does it.

For this the components architecture needs an overhaul.

Yes, I think the existing parts md-module and emhttp need to be re-worked.

 

1. md-kernel module ...this is the real technical CORE, keep it...make it available to everyone..it is open-source already....everyone can incorporate it into a distro or installation of his/her own.

2. integration app/module - this is new...it does everything emhttp does today, but without a GUI....it needs to provide an open API...maybe based on REST...*this* is the real product.

Make it available with the package mechanisms of all major distros for users to purchase

3. UI - this is interacting with the API of no.2 ...can be a Web-UI, a mobile app, a server agent...whatever you wish....this is nothing more or better/other than a plugin...because of the APi, others can roll their own.

 

B) engagement model  for unRAID as an innovative product

I can understand the requirement to keep the distro clean, in a snapshot with well known content.

Maybe one needs to rethink the hybrid approach.

 

Why not have a "pro", appliance version? ... and an enthusiast/community version?

..with a different lifecycle and licensing and support (...and cost) scheme?

I can see the enthusiast version to be released more often...like being a test-repo for the pro stream...only stable stuff will be moved into pro/appliance stream by TOM.

 

Will need some help from TOM to kick this of..especially with emhttp we need some more generic integration features (not to say a public API), but AFAIU

there are some thought from TOM about this already (seen in the encryption layer discussion thread).

 

...another 2 cents on the huge pile of coins 

[ironicbadger]

Not alienating the existing customer base I believe is top priority for Tom.

 

However, for me, I'm not bound by such constraints and will be working closely with Tom over the coming weeks to bring this "unRAID Extreme" edition to fruition.

 

This is happening guys, it will just take me a little while to get the little details worked out, potentially porting the extreme edition to centOS or whatever is not too hard but I reckon tweaking emhttp will take some effort before its as reliable as it is now.

 

Once I've got the core port done we can look to add extra features.

 

Sent from my Nexus 5 using Tapatalk

 

[dheg]

Not alienating the existing customer base I believe is top priority for Tom.

 

However, for me, I'm not bound by such constraints and will be working closely with Tom over the coming weeks to bring this "unRAID Extreme" edition to fruition.

 

This is happening guys, it will just take me a little while to get the little details worked out, potentially porting the extreme edition to centOS or whatever is not too hard but I reckon tweaking emhttp will take some effort before its as reliable as it is now.

 

Once I've got the core port done we can look to add extra features.

 

Sent from my Nexus 5 using Tapatalk

Gooooooood!  :-)

 

Sent from my GT-I9305 using Tapatalk

 

 

[binhex]

Not alienating the existing customer base I believe is top priority for Tom.

 

However, for me, I'm not bound by such constraints and will be working closely with Tom over the coming weeks to bring this "unRAID Extreme" edition to fruition.

 

This is happening guys, it will just take me a little while to get the little details worked out, potentially porting the extreme edition to centOS or whatever is not too hard but I reckon tweaking emhttp will take some effort before its as reliable as it is now.

 

Once I've got the core port done we can look to add extra features.

 

Sent from my Nexus 5 using Tapatalk

 

wow, i can't believe this is actually going to happen so quickly!, very excited by this news, for me personally if this can alleviate plugins causing issues with shutdown of unraid by basically running the apps on centos(or inside a vm) instead of on top of unraid then i will be one happy camper! 

[ars92]

The first thing that popped in my head after reading this thread is. ..easy installation of nzbdrone. Since there's no one making a plugin for this, and me being a coding noob, I'm basicallly locked out from using that awesome piece of software. But with this, it's gonna be a piece of cake to get it installed! ! And i might even have a vm of Windows on it for vnc usage coz turning on my pc just for some simple task is annoying. Really hope this attempt actually turns into a real product!

 

I do sometimes regret a little getting a core i3 with 8gb of ram for something which barely/rarely gets pushed, so this would somehow justify my purchase while making me explore stuff I've never thought of exploring before! !

 

Sent from my SM-N9005 using Tapatalk

 

[dheg]

I do sometimes regret a little getting a core i3 with 8gb of ram for something which barely/rarely gets pushed, so this would somehow justify my purchase while making me explore stuff I've never thought of exploring before! !

Don't mean to put you down but i don't think an i3 will support vt-d

 

Sent from my GT-I9305 using Tapatalk

 

 

[ars92]

I do sometimes regret a little getting a core i3 with 8gb of ram for something which barely/rarely gets pushed, so this would somehow justify my purchase while making me explore stuff I've never thought of exploring before! !

Don't mean to put you down but i don't think an i3 will support vt-d

 

Sent from my GT-I9305 using Tapatalk

Is vt-x not good enough? 

I dont really need super fast transfer speed from whats on the vm to be transferred to the host unraid as i think thats what directed io means...right?

 

Or am i completely locked out from getting anything from this "unraid-NG"?

 

Sent from my SM-N9005 using Tapatalk

 

 

[ironicbadger]

I do sometimes regret a little getting a core i3 with 8gb of ram for something which barely/rarely gets pushed, so this would somehow justify my purchase while making me explore stuff I've never thought of exploring before! !

Don't mean to put you down but i don't think an i3 will support vt-d

 

Sent from my GT-I9305 using Tapatalk

 

It won't, but that only means he's locked out from pass through. He can still run a windows VM or whatever just can't say passthrough a GPU to a VM.

 

He can still run Linux! Therefore unraid extreme...

 

Sent from my Nexus 5 using Tapatalk

 

[ars92]

I do sometimes regret a little getting a core i3 with 8gb of ram for something which barely/rarely gets pushed, so this would somehow justify my purchase while making me explore stuff I've never thought of exploring before! !

Don't mean to put you down but i don't think an i3 will support vt-d

 

Sent from my GT-I9305 using Tapatalk

Is vt-x not good enough? 

I dont really need super fast transfer speed from whats on the vm to be transferred to the host unraid as i think thats what directed io means...right?

 

Or am i completely locked out from getting anything from this "unraid-NG"?

 

And i use plex rather than xbmc, so this machine will definitely remain headless without any need of vga passthrough or a desktop.

 

But of course, changing to a 4670 is not really impossible

Sent from my SM-N9005 using Tapatalk

 

 

Sent from my SM-N9005 using Tapatalk

 

[Ford Prefect]

Is vt-x not good enough? 

I dont really need super fast transfer speed from whats on the vm to be transferred to the host unraid as i think thats what directed io means...right?

 

Or am i completely locked out from getting anything from this "unraid-NG"?

 

Sent from my SM-N9005 using Tapatalk

 

...vt-x is good enough as long as you don't want to passthrough a hardware-controller...this is what vt-d is for.

In example, for running a DVR with the tuner cards inside a VM.

As long as you can live with the apps in the VM to use networking, you'll be fine.

With using the right drivers in the VM, like virtio-net, you can get as fast as native CPU/Mem-bandwidth...yes, even if the virtual NIC reports as a 1GBit, it can go as fast as the CPU would allow him.

[josywong]

quick question. im nt linux guy.

 

with this unraid-extreme made, hw hard will it b for me to "bake" into SteamOS? im much more interested with SteamOS since, it will get tested with massive amounts of gamers.

this will save me a lot of time(maybe) to passthrough nvidia or just be my gaming rig + unraid in a box.

[ars92]

Is vt-x not good enough? 

I dont really need super fast transfer speed from whats on the vm to be transferred to the host unraid as i think thats what directed io means...right?

 

Or am i completely locked out from getting anything from this "unraid-NG"?

 

Sent from my SM-N9005 using Tapatalk

 

...vt-x is good enough as long as you don't want to passthrough a hardware-controller...this is what vt-d is for.

In example, for running a DVR with the tuner cards inside a VM.

As long as you can live with the apps in the VM to use networking, you'll be fine.

With using the right drivers in the VM, like virtio-net, you can get as fast as native CPU/Mem-bandwidth...yes, even if the virtual NIC reports as a 1GBit, it can go as fast as the CPU would allow him.

 

I remember you saying it might even go as fast as 10Gbit with the right drivers, so basically even the transfer speed of files from the vm to host would be as fast as my hdd( no ssd for now) lets me? So then, for my usage at least, i don't see the cons of not having a vtd cpu

 

Sent from my SM-N9005 using Tapatalk