64 Bit unRAID running natively on Arch Linux with full hypervisor support



Recommended Posts

I really like the idea. I think it would be nice to keep unraid doing what it does best and ditch the plugin system. My only concern is with the amount of updates to the 5.0 RC versions were constantly being changed due to different linux kernel versions etc which broke peoples unraid. Would this not be the same problem since there would be different kernel updates etc? By keeping unraid on a seperate operating system, Tom can ensure stability. This is strictly by watching the changes from 4.7 to 5.0. I am by no means a linux guru. If its past apt-get install unRaid, I would not be able to install it!

Link to comment
  • Replies 451
  • Created
  • Last Reply

Top Posters In This Topic

Your comments about donations etc are true, they would be welcomed of course (being as I'm a poor student!) But before much further progress is made I reckon it'd be sailing a bit close to wind without Tom on board.

 

As I am of "another" or rather ancient age, I am probably looking into a way to find some fun and extra tips for when I retire ...hopefully soon  ;D

But you are right, there needs to be a working setup as good proof upfront.

However, the md-module is one thing, emhttp is another turf.

This http://lime-technology.com/forum/index.php?topic=30690.msg275757#msg275757 does not give away anything about Tom's position in this.

Especially when talking about a "community distro" that ships emhttp (which is proprietary and owned by Tom)... so I'd rather not do anything towards that kind of action without explicit blessing from Tom.

Link to comment

Will someone get Toms attention please and get him in on this? I've sent him a PM, what else next...

 

And there you have it. The fundamental issue you will need to address / deal with.

 

Tom with the next release of unRAID could simply add some additional checks that emhttp does to ensure it's running on Slackware, a particular Linux Kernel version, do chksums of all the files, etc. which would make this whole thing go up in smoke.

 

unRAID is a solid product and very good at what it does. Tom seems happy with where unRAID is at and there are plenty of users who agree. Based on the past product development lifecycles and the pace at which current technologies and innovation are introduced within unRAID. I do not believe Tom sees that as viable customer base or one he is interested in pursuing.

 

As Ford was alluding too... We can jump up and down, scream and shout... Problem is, we do not get a vote or have a say in the matter and if you continue to use emhttp, neither do you. It's solely up to Tom and based on history... I don't see that changing.

 

Way I see it, you are hitching your wagon (unRAID 64 bit with KVM / Xen on Another Linux Distro) to a horse (unRAID) that if I had to bet... Doesn't want to be ridden. That usually doesn't end well for the wagon or the people in it. At best, you and the people in the wagon are in for a very wild and bumpy ride. Worst case, you and the people in the wagon go flying off the cliff.

Link to comment

Well let me say this, I for one would gladly pay limetech and/or ironic/grumpy/ford/whomever for the product described.  I paid for Unraid once as described in the marketing blurb and do expect reasonable updates as described in that same blurb.  Easy or hard, I would not really expect a huge functional change such as you all are describing to be free.

 

Frankly I would see this as another license level or add-on option if it were feasible to segment the software like that without being a maintenance headache.  Call it Pro+, Unraid OS, Virtual UnRaid, UnRaid-VR, whatever.  Many people might not want it and will be happy with the basic functionality available now (albeit just a crippled version of the full version for maintainability).

 

Of course for all current users maybe we get a discount ;-)  But still I can appreciate the work this would take by [insert whomever] and that they should be compensated either directly or by Limetech.  That would be for you all to figure out.

Link to comment

Will someone get Toms attention please and get him in on this? I've sent him a PM, what else next...

 

And there you have it. The fundamental issue you will need to address / deal with.

 

 

Tom responds better with direct email providing the request with all relevant information needed to make a decision, pros, cons and any settings.

You can direct him to a thread if need be.

Chances are if it's a long one. His response will need to be scheduled out with regards to his time and priorities.

 

Allot of this started with a request to add some kernel settings and from what I saw, there were URLS provided with nothing specific enough for him to set up without a bunch of research. (from my understanding).

 

Some of what is being asked for shouldn't be all that hard to do, but the requests need to be clear, concise, direct to the point.

 

Try emailing him with relevant information.

Link to comment

Well let me say this, I for one would gladly pay limetech and/or ironic/grumpy/ford/whomever for the product described.  I paid for Unraid once as described in the marketing blurb and do expect reasonable updates as described in that same blurb.  Easy or hard, I would not really expect a huge functional change such as you all are describing to be free.

 

Here is how I see it...

 

Ford, myself and Ironic would volunteer and offer our skills, expertise, time, effort and energy into this all the while making sure Tom get compensated (use emhttp which is where the registration keys are read and the engine of the unRAID WebGUI).

 

All the things we have talked about doing in this thread we could do in a matter of weeks / one month tops.

 

Question is how many of us out there want this type of functionality?

 

Based on what I see here, in the XBMCs, other Multi Media and Virtualization Forums... It's quite a lot.

 

Excluding ESXi, XenServer, people who run KVM, Xen on Linux, Proxmox, NAS4Free, people who purchase QNAS like appliances, etc... Only using FreeNAS as an example (which doesn't have all the capabilities we describe here), it has 5.5 Million Downloads.

 

If ONLY 500,000 (which is a very small percentage) of FreeNAS people or all the others I excluded above were to pay $50 for the unRAID we are describing... That is $25,000,000.

 

What would concern me if I was Tom...

 

1. Business 101 - Give the customers what they want for a reasonable price.

 

From what I see, there would be a lot of people who want the features and functionality we are talking about it and would pay more than $50 dollars for it.

 

2. There is NOTHING preventing someone from taking the unRAID md_module and developing a version of emhttp with little to no investment.

 

We already know there are people who can develop a nice / sexy WebGUI (simplyfeatures and Boxcar) along with plenty of other web developers out there.

 

Instead we get this...

 

Tom responds better with direct email providing the request with all relevant information needed to make a decision, pros, cons and any settings.

 

You can direct him to a thread if need be. Chances are if it's a long one. His response will need to be scheduled out with regards to his time and priorities.

 

Allot of this started with a request to add some kernel settings and from what I saw, there were URLS provided with nothing specific enough for him to set up without a bunch of research. (from my understanding).

 

Some of what is being asked for shouldn't be all that hard to do, but the requests need to be clear, concise, direct to the point.

 

Try emailing him with relevant information.

 

Option A

 

Do the above, hope you get a response, hope Tom agrees to do it, if he does agree to do it hope it doesn't take as long as it does to get a 64Bit version of unRAID.

 

Option B

 

Let's dwindle the number I used earlier from 500,000 to 50,000 and the user only paying $50. You are still talking about $2,500,000.

 

How long will be before someone invests a few months of their time to put together what Ironic, Ford, myself and others could do in a few days (including a few months to develop the WebGUI, Website, Forum, Documentation, Incorporate, Promote, etc.), package it up, sell it and reap all the profits?

 

This is EXACTLY how FreeNAS / NAS4Free forked and became what they are today. FreeNAS name was acquired and new developers and version 8 and 9 is  forked from the original FreeNAS 7. The original group of developers went forward with FreeNAS 7 development under the new name NAS4Free.

 

Call me crazy... But If I was Tom, I would want to be on receiving end of those potential new customers / profits when it's people who have agreed to do all the work for FREE.

 

That is how I see this playing out and where I see the marketplace going.

 

Perhaps Tom doesn't see it playing out that way, maybe he is swimming in money from new / existing customers who are happy with how unRAID works now and well into the future or maybe he is working on the things we have talked about in this thread and others. Maybe he will respond to one of his PMs, comment in this thread or provide an update elsewhere on the website.

Link to comment

Business decisions of what you describe above do not happen over night.  There is a lot of thought that has to go into something like what you describe.  It is not some flip of a coin decision.

 

As for not being on the forum and responding here, I do understand that.  It is a complete pain in the arse and can be hard to follow unless you sit and read the darn thing from beginning to end.  That is why there is a group of mods, not one of us could do it alone, or would want to.  We all have lives outside our normal 40-50-60 hour work week.  The last thing we usually want to do is sit in front of another computer when we get done doing that all day long.

Link to comment

Wow, I take a day and a half off to hang some Christmas lights and find my inbox and this thread blowing up!!

 

Here's some initial thoughts.

 

First, awesome work!  I truly appreciate the fact that some really smart people want to take unRaid to another level.

 

Second, the idea of integrating unRaid into a mainline linux distro, both 32-bit and 64-bit is not new.  There have been projects at least five years old that did this.  I think the big difference at present is the popularity of virtualization and inexpensive h/w powerful enough to run it on.  Obviously this is the future.

 

Please let me have tomorrow morning to go through this in detail before a more thorough reply.

Link to comment

Business decisions of what you describe above do not happen over night.  There is a lot of thought that has to go into something like what you describe.  It is not some flip of a coin decision.

 

Thanks for the feedback.

 

I agree. Of course it isn't a flip of the coin.

 

Since only 30 people have responded to the poll in this thread so far. Perhaps that is a good indication of how small the users base out there who want to see the added features and functionality we have discussed. I suspect not but I don't know.

 

As for not being on the forum and responding here, I do understand that.  It is a complete pain in the arse and can be hard to follow unless you sit and read the darn thing from beginning to end.  That is why there is a group of mods, not one of us could do it alone, or would want to.  We all have lives outside our normal 40-50-60 hour work week.  The last thing we usually want to do is sit in front of another computer when we get done doing that all day long.

 

I think the above goes without saying.

 

However, If I were you, I would remove it. Should a user who is having real issues or have important questions stumble across this "wish list" thread and read that...

 

Saying it's a pain in your ass to read threads and how it's a hard for you to read one from to start to finish. How the last thing you want to do is be on the site, etc. It's quite possible that users could interrupt that as the unRAID team be uncaring and might take offensive to it.

Link to comment

Just to throw in the view of an average user (me)

 

I have been using unRAID for 2 years, but over the last 6 months i have realised its not running at its full potential and with the issues over plugins conflicts and the state of the GUI, having to run a VM on top of unRAID for various other bits of software.

 

I am not competent or knowledgeable with Linux, hence why I haven't cast a vote in the poll. So long as there is a manual and a user friendly interface,  then I am good with whatever is chosen. There are far more technical people than me that will consider the pros and cons and direct this project. So I would not take the 30+ people that have voted as the target audience, there will be a lot of people watching this thread with interest.

 

I won't  pretend to understand all that is being discussed here, but I do understand the benefits of this idea to a user like me. Keeping the system up to date, more flexibility on additional software, not having to change plg codes to correct the many different versions of slackware or risk breaking the other plgs, and greater speed are a few.

 

Even if it comes as another unRAID licence tier, i would be happy to get a more flexible system that can be used in anyway i want to and more efficiently.

Link to comment

Since only 30 people have responded to the poll in this thread so far. Perhaps that is a good indication of how small the users base out there who want to see the added features and functionality we have discussed. I suspect not but I don't know.

My 2 cents:

I mainly follow this forum with tapatalk, and you can't see the surveys there.

I believe many people don't really understand the potential of this, and those who do might not care much about the OS as long as this is achieved.

 

Sent from my GT-I9305 using Tapatalk

 

 

Link to comment

 

I am not competent or knowledgeable with Linux, hence why I haven't cast a vote in the poll. So long as there is a manual and a user friendly interface,  then I am good with whatever is chosen. There are far more technical people than me that will consider the pros and cons and direct this project. So I would not take the 30+ people that have voted as the target audience, there will be a lot of people watching this thread with interest.

Precisely my point

 

Sent from my GT-I9305 using Tapatalk

 

 

Link to comment

I am not competent or knowledgeable with Linux, hence why I haven't cast a vote in the poll. So long as there is a manual and a user friendly interface,  then I am good with whatever is chosen. There are far more technical people than me that will consider the pros and cons and direct this project. So I would not take the 30+ people that have voted as the target audience, there will be a lot of people watching this thread with interest.

 

This I think is what truly reflects the 30 votes, it is certainly the reason why I haven't voted.

Link to comment

Oh you guys. This is looking very promising indeed.

 

If it wasn't obvious already I'm ready to throw some full time dev work into the ring to make this thing a reality. Integrating emhttp would be the ideal solution to keep this community on board.

 

As for writing our own on top of the existing MD module, this probably isn't the right forum to discuss such a product.

 

 

 

Sent from my Nexus 5 using Tapatalk

 

 

Link to comment

I'm running unRAID for 4 months now and I'm a linux noob.

I'm very impressed of your skills with linux and those things.

Let me describe the situation from the perspective of an user-noob.

 

When building my server I was very pleased to be able to build an inexpensive and economical

machine to consolidate all my drives in one place and have some fault tolerance.

I set up with some decent hardware as suggested in the wiki.

When I started reading through this forum I didn't even know what ESXi was.

Never heard of KVM and XEN an all those virtuality things.

 

I've been following the various threads and I'm wondering where it will be going to when unRAID will have to run on/in a VM by default because there won't be any more plugins that run on stock unRAID but in a dedicated VM.

At the moment I can say I understand "in principle" what you're doing here but I feel like I'm missing something in order to be able to discuss with you.

I can find very well made instructions how to set up things but to understand the basics of virtualization ... nope.

Perhaps one of you guys can put together the "ABC of virtualization" for the noobs in here?

I dare to say that many of us are simply overwhelmed by this topic just because we don't know what it means hence the sparse feedback! Either keep it simple or make sure it's simple to understand.

 

My concerns about this advance:

How is the virtualization affecting hardware requirements for the "bare metal" users? What will be the minimal hardware requirements?

Is the VM host running from a thumbdrive also?

While I like the idea of beeing able to run plugins in another environment without interfering with unRAID and

I like the idea of having plugins from a well maintained source - what is the downside?

How is the decision for an OS affecting this?

 

I'm well aware of the flaws of unRAID e.g. stability of emhttp, plugin support, GUI apparance, security... and I would really appreciate a solution to this and a more consolidated way of development.

But at first sight, running it in a VM brings more complexity to it and calls for more knowledge (ABC of VM) in setting it up and maintaining it.

 

Just like already highlighted in this thread, at the moment many highly skilled people work on many independent projects trying to accomplish more or less the same thing. What a pity! Because it's often redundant work and it's often a one man show and when it comes to long term availability and support it will often end up in a dead end.

 

Edit: noticed some of you feel like me...

 

Link to comment

I can find very well made instructions how to set up things but to understand the basics of virtualization ... nope.

Perhaps one of you guys can put together the "ABC of virtualization" for the noobs in here?

 

I try to explain it in very simple terms on my blog. Specifically see the linked post below which shows what's possible (each virtual machine is listed and I stated it's function and reason why). Admittedly, it's a little out of date now with regard to exact current setup but that's not important. What is, is that you understand the overall big picture of virtualising and why it's so awesome.

 

http://blog.ktz.me/?p=219

 

Basically, with virtualisation you're running many 'computers' in one.

Link to comment

Yes ironic,

to be honest, most of my virtualization knowledge I got from your blog  ;) .

But there are still may things that are supposed to be known.

 

Imagine you have to explain virtualization to your wife...

That's where it should start.

 

My wife understand it's thus:

 

  • She can watch anything on XBMC whilst I'm playing video games (separate TVs obviously).
  • To add another XBMC to a different room its £20 for another GPU, not £200+ for a HTPC.
  • It's less computers in the house, which for a wife is always a good thing

Link to comment

This topic is very interesting eventhough i didnt care about Virtualisation in combination with Unraid till now because i thought it would be too complicated.

 

Just a quick question for my better understanding with what you want to do know it would be possible for me to:

-Have a base OS like Arch or CentOS which can run all Linux Software i want without the need of plugins because it is a fullfledged Linux OS

-unRaid will work just as a Software in Arch or is it also virtualized (like in your signature)

-I could also run my Mediaportal TV Server under Windows 7 as guest on my server passing throught my 2 TV Cards?

 

That would be really interesting for me because that would mean i could eliminate on of my servers and save quite a lot of expensive energy.

 

Btw. what about S3 Sleep, would that work just as good as with my current "normal" Setup or is this approach meant to be a 24/7 running server only?

 

I will definitely keep up following this thread!

Link to comment

I started migrating from my windows 2012 server running storage spaces to unraid a month ago. The reason why I did this was - the raid systems - I hate the idea of crashing disks and in the worst case - all data is gone. That is was unRaid fixed for me - but I also had to lose some flexibility. I liked the option to install software easily and plug in any drive into my windows system and share it or use it for whatever. it is possible on unRaid as well - but by far not that easy than on other linux distributions. you have plugins solving a lot - but no real update system - 100 of WebUi solving this and that - why not having a distro with package managing included.

 

I like unRaid and in the next few month I am gonna to install it on a few other systems as well and than starting to license it as well - after knowing everything works fine.

 

What I like to see from such a "new" unRaid distribution:

 

  • easy updates of packages
    I like Debian but I do not care if it is Arch Linux or Centos - as long as there is a package manager
  • installing plugins using special plg flies or whatever - nobody should need that so get rid of it
    there is everything out there as a linux based install package incl. web based frontends at some points and they are all more well developed or up to date than the unraid plgs
  • usb stick install - I LOVE THE IDEA so keep it
    why - so I have a easy OS backup module - take the USB stick and back it up and if it crashes - copy and paste (my father can do it) and also - I do not need a seperate hdd for it (which is important for me)
  • unRaid OS distribution or package to install
    I would love to see unRaid as a package you can install with the proper source on the linux distri of your choice - that makes updating as easy as installing
  • virtualizatin is not an option for me
    I dont care about virtualization on my system - there is not that much CPU available - but if it is in there - why not - could become option in my company ;) or in the future for me as well

 

I am looking forward to whatever happens - I like unRaid (not slackware but the concept of how raid should work) and thats what I wanna see in a good linux distribution to get rid of all drawbacks I had to take after moving from windows 2012 to unraid and not taking the road of ZFS within a full blown linux distri.

 

Thank you guys to make that happen :) perhaps this triggers me to start my long time idea of learning python and writing some scripts and tools for the new unraid :) lets see :)

 

THANKS

Link to comment

Yes. These last two guys get it.

 

Unraid itself is the host so it is the one doing the virtualising of other stuff (but here's the important bit, that's OPTIONAL).

 

I don't know about sleep its not a function I use. I wouldn't imagine KVM or Xen would play with it well. For those that want that there's always unRAID ;)

 

Sent from my Nexus 5 using Tapatalk

 

 

Link to comment

Looks interesting and will more then likely test it when you release it :) be warned though I'm not Linux minded at all, but always willing to learn what i can.

 

I was looking at running unRAID in a VM and plugins / software in another but this has got me thinking i no longer need to do this.

 

Would i be correct in my understanding that you would install this distro with unRAID baked in to a small drive maybe an ssd or 2.5" drive and you would have all the goodness of unRAID and have a full dristo to play with and install things like serviio, transmission, ownclould and they can access all your unRAID drives?

 

Couple of questions:

  • With using arch as a base can you run this headless, I have no monitor or keyboard connected to server?
  • From what Ive read arch is pretty cutting edge, wouldn't something like debian or another less cutting edge distro be a better alternative, less chance of something messing your system up?
  • Will the pre-clear script still work and would Joe L support it in the distro based unRAID?
  • If i was to want to put the plugins / software in a VM using Xen I take it my hardware would need to support AMD-V Technology with IOMMU, still learning here.

 

Hope this makes sense.

Link to comment

I think the above goes without saying.

 

However, If I were you, I would remove it. Should a user who is having real issues or have important questions stumble across this "wish list" thread and read that...

 

Saying it's a pain in your ass to read threads and how it's a hard for you to read one from to start to finish. How the last thing you want to do is be on the site, etc. It's quite possible that users could interrupt that as the unRAID team be uncaring and might take offensive to it.

Fair point, but I am not in the habit of censoring stuff unless need be so I will leave it. What is done is done and a new user should not interpret that  as we are not willing to help.  We are mods and it is what we do.

 

 

 

So I do have a question, that unless I missed something, has not come up.

 

What will be the hardware requirements to run something like this?

 

ESXi obviously has its requirements so I would imagine this does also.  I doubt my Sempron based build is going to get any benefit from this at all, though I would love to be wrong.

Link to comment

So I do have a question, that unless I missed something, has not come up.

 

What will be the hardware requirements to run something like this?

 

ESXi obviously has its requirements so I would imagine this does also.  I doubt my Sempron based build is going to get any benefit from this at all, though I would love to be wrong.

 

The basic requirements will not change, besides a linux distro wth a GUI and being 64bit will consume a bit more memory (+512M is my best guess).

The advantage of this approach has been discussed already....it will be that of an easier operating model and better variety and freedom of choice.

If you want more flexibility to source applications and features, you have a full distro at hand...this includes the virtualization options of that distro.

No need to rely on someone to re-compile the unRAID kernel for you or to create a plugin of choice..

It will be the unRAID  modules following the distro lifecycle automagically.

 

So If you plan to use virtualization, you have some more options now.

As always, you will need the required hardware and resources...especially more RAM....or even a lot more RAM  ;)

In case of employing vt-d/IOMMU these things have not changed (besides you won't need it for unRAID, as this is now native on your VM-host)

 

A Sempron supports AMD-V or vt-x in intel-lingo..so virtualizing Windoze on the unRAID host is an option with use of KVM or XEN...however vt-d/IOMMU is not an option with that CPU, I am afraid.

In case we can make it for Proxmox.VE, there is the option of lightweight containers (CT or OpenVZ) for running linux based VMs, too....you will not even need vt-x for that, but it's lightweight in terms

of "this needs a lot less RAM than in a real VM" what is the real feature in this case.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.