Xen/unRAID-6 Discussion


limetech

Recommended Posts

 

My most important VM will be a pfSense firewall.  Will this require a passthrough on v6? 

 

I am building similar setup and I don't think you need pass through for that.

as far as I can tell you need to create a dedicated bridge for any interface you want to access from pfsence and use that as  NICs for your VM

 

I am still in research mode on this one though so let you know when I have a working setup.

 

 

 

I have 2 stock Tam's servers just like yours too. Will follow your testing with interest. It would be good if they are more capable with v6.

 

Sent from my mobile

 

Link to comment
  • Replies 199
  • Created
  • Last Reply

Top Posters In This Topic

Hmmm.... Anyone got a slackware 14.1 development vm so we can compile and develop unRAID plugins?  ;D

 

I've got a Slackware 14.1 VM Appliance for you. Let me send it over.

 

That way you can spend the next few weeks and months writing a 2,000+ line plugin that probably will not 100% of the time, crash my other plugins or unRAID itself.

 

On top of that, I will bitch / moan / complain about the work you did, how long it's taking, blame you for crashing my other plugins, crashing unRAID itself and really go off on you when you do not meet my demands and support / maintain it through 15+ beta / release candidates for unRAID 6.0. You should also know, you are going to suffer through all of that and not get a dime or even get a thank you.

 

Installing a VM Appliance or installing Ubuntu in a VM and either clicking Mediawiki in the GUI Package manager or typing "apt-get install mediawiki" is far to complicated and less reliable than plugins. I know this because 3 or 4 people here who have no clue how plugins work (don't work is more like it) or who do not know what Virtualzation is or ever set one up... said so.

 

WeeboTech, since I cannot write a plugin and you can... You are now my little bitch! I'm going to rough you up, smack you around and I'm not even going to leave you money on the nightstand when I leave. Remember, I only beat you because you deserve it and because I love you.... got that?

 

Is it any wonder why the old plug developers faded into the night and why we haven't seen new plugins for apps developed (patched up existing plugins do not count) in forever?

cm-41533-050dcc3394a494.jpg.b6a05a6b16841dc4e59c35d70cdf12e6.jpg

Link to comment

There will always be a need to have a build environment as not every package is available as a binary.

 

Does anyone have a VM of one yet?

 

Wouldn't it just be easier to grab the Slackware 14.1 x86_64 ISO and install it in a VM??  No need for a pre-configured VM then (if you need the kernel source for the version that 6.0 is built on, just grab it from one of the kernel repositories and basically follow the instructions in the wiki on how to run 5.0 on a full-fledged slackware distro install).  Easy-Peasy, and don't have to rely on anybody else to get it done, nor have to worry about what hypervisor they used (virtualbox, hyper-v, esxi, xen, etc).

 

 

Link to comment

Creating a VM Appliance 101

 

1. Create a Ubuntu (or whatever Linux Distro you want) image file (img, cow, vdmk, vhd, etc.)

 

2. Install Ubuntu (or whatever Linux Distro you want).

 

3. Install App(s) (Example: "apt-get install owncloud.")

 

4. Poweroff VM.

 

5. Upload VM Appliance image and simple VM cfg to your google drive or dropbox.

 

6. Create thread with overview of which OS you used, which Apps you installed in the Appliance, instructions on how to access the GUI / WebGUI and links to the VM Appliance in google drive / dropbox.

 

7. If you really wanted to put it over the top... Post any number of 1,000+ links to the Apps website, forum, or blogs for how to use / install / configure that App (or Apps) in the same OS you used if the user needs additional help / info.

 

The following thread should turn on the lightbulb for many of you:

 

unRAID KVM appliance - Arch Linux with SAB, CP, SB, Plex

Link to comment

While the responses to the slackware 14.1 environment were funny, I figured it would be a good way to get one's feet wet.

One perhaps being thyself.  I'm familiar with vmware, I know only a lil about xen.

Since I really have to concentrate on my job, I can't play too much with Xen or unRAID. 

 

Why bother, well there are things such as automount that I use and are not available in unRAID.

There are some custom programs I use in cron and for filesystem/database catalogging that I do not think will work well in another VM.

In addition, I have some tools that are bash loadables to be used in the unraid environment to do certain things in shells.

 

I really posted it to be funny at first, yet as I think about it, it would help me.

I have a slackware 13 dev environment on vmware.

I figured it a slackware 14.1 dev environment as a XEN vm would save me some time.

I want to finish my badblocks hackup that can be used in place of dd for preclearing.

ddrescue will need to be recompiled for unRAID X64,

I have a smartctl replacement filter that captures the output of smartctl, saves it to a file while it's passing the data to emhttp.

Eventually I'll have a monit plugin to monitor these files and email via smtp (monit can email directly without needing anything else).

etc, etc

Well you see the idea here.

Some things like downloading files, compressing formatting presentation can be done in a VM.

Other things I would like done inside unRAID.

 

It's no big deal, I'm in no rush.

Link to comment

RPM Repos for CP,SAB,SB etc seem to be a mess.  I'll need to kickstart a fedora box first, and then figure out how to proceed.

 

To expand on what you talking and explain what you are talking about for the people who do not know...

 

CentOS isn't a good choice at all if you want the various plugins we have / use in unRAID.

 

You either have to find a repo (software repository) that has the application you want. Since CentOS is a Linux Server Distro... Many of the applications do not exist. Therefore, you would have to compile / install it yourself manually.

 

A package manager can't save you if no packages exists or if the applications are 3+ years old. XBMC in CentOS 6.5... installs XBMC 11.0.

 

Fedora, that problem goes away / doesn't exist.

Link to comment

RPM Repos for CP,SAB,SB etc seem to be a mess.  I'll need to kickstart a fedora box first, and then figure out how to proceed.

 

To explain what you are talking about in english for many of the people who do not understand...

 

CentOS isn't a good choice at all if you want the various plugins we have / use in unRAID.

 

You either have to find a repo that has it (not likely) or compile / install it yourself manually.

 

A package manager can't save you if no packages exists or they are 3+ years old. XBMC in CentOS 6.5... installs XBMC 11.0.

 

Fedora, that problem goes away / doesn't exist.

 

Uhm.  I'm planning on Fedora for this, not CentOS.  Fedora 20 specifically.  I just don't have any Fedora hosts in my lab right now, so I'll need to kickstart one.  I'm planning on releasing this as a kickstart file and a script to customize it.  I like that better than a prebuilt VM, because with this an end user can see what's going on.

 

This is mostly human readable, and with the help of a page like http://fedoraproject.org/wiki/Anaconda/Kickstart anyone should be able to work out what's going on and be able to say with some certainty that they're happy with it.  I was hoping to just include repos for sab/sickbeard/whatever as repo entries, and then list packages to install, but it looks like it's going to be more complex than that.

 

# System authorization information
auth  --useshadow  --enablemd5
# System bootloader configuration
bootloader --location=mbr --driveorder=xvda
# Clear the Master Boot Record
zerombr
# Partition clearing information
clearpart --all --initlabel
# Use text mode install
text
# Firewall configuration
firewall --disabled
# Reboot after installation
reboot
# Run the Setup Agent on first boot
firstboot --disable
# Use interactive kickstart installation method
#interactive
# System keyboard
keyboard us
# System language
lang en_US
# Installation logging level
logging --level=info
# Use network installation
url --url=http://mirrors.usc.edu/pub/linux/distributions/fedora/linux/releases/20/Fedora/x86_64/
# Network information
network --bootproto=dhcp --device=eth0 --onboot=on
#Root password
rootpw password
# SELinux configuration
selinux --disabled
# Do not configure the X Window System
skipx
# System timezone
timezone --isUtc America/Los_Angeles
# Install OS instead of upgrade
install
# Disk partitioning

part / --fstype="ext4" --size=1 --grow

%packages
@base
libudev
openssh-server
rsync
wget
openssh-clients
vim-enhanced
man
transmission
transmission-cli
transmission-common

 

I would like it avoid having to require that a user gets into a shell with the installer to go interactive, but I might have to for things like timezone and such.

 

I'll build it on real hardware first, and then try it as a domU once Tom puts out another beta.

Link to comment

Uhm.  I'm planning on Fedora for this, not CentOS.  Fedora 20 specifically. 

 

You might want to consider Ubuntu, Linux Mint, etc. as an alternative since they have most of the apps people run in their package manager. Preseeding isn't much different than Kickstarter.

 

Xen-Tools is something else you should look at. You use that along with preseed, you can install a VM without even having to boot into an ISO and you can configure / install various packages and apply modifications to the OS without user intervention. 

 

Xen-Tools

 

and

 

Contents of the preconfiguration file

 

It requires a lot of upfront work and since you are very good at / have experience and don't mind... The rest of use sure could benefit from it and make our lives easier.

Link to comment

Uhm.  I'm planning on Fedora for this, not CentOS.  Fedora 20 specifically. 

 

You might want to consider Ubuntu, Linux Mint, etc. as an alternative since they have most of the apps people run in their package manager. Preseeding isn't much different than Kickstarter.

 

Xen-Tools is something else you should look at. You use that along with preseed, you can install a VM without even having to boot into an ISO and you can configure / install various packages and apply modifications to the OS without user intervention. 

 

Xen-Tools

 

and

 

Contents of the preconfiguration file

 

It requires a lot of upfront work and since you are very good at / have experience and don't mind... The rest of use sure could benefit from it and make our lives easier.

 

Starting with redhat, because that's what I know, and it doesn't require an ISO, just a mirror that it can pull packages off. 

 

When did eth0 become p5p1?  I had seen em0, but this is just weird.

Link to comment

You might want to consider Ubuntu, Linux Mint, etc. as an alternative since they have most of the apps people run in their package manager...

 

I thought you said in another thread that Ubuntu and Mint were both distros that were using a different system from other releases (systemd vs something else).

 

Wouldn't using a distro with the preferred method be preferred, for the reasons given in your other thread?

Link to comment

I thought you said in another thread that Ubuntu and Mint were both distros that were using a different system from other releases (systemd vs something else).

 

Wouldn't using a distro with the preferred method be preferred, for the reasons given in your other thread?

 

It's in a VM Appliance and not the Host so it's a little different.

 

Users are going to load the specific VM Appliance with the apps they want (or create their own).

 

Ubuntu works fine for Sickbeard, CouchPotato, Owncloud, etc. and they can update it via a GUI (if they have a Desktop), WebGUI (you would need to install one of the many) or command line.

 

Fedora is great too and I wouldn't care if its Fedora, Ubuntu, Mint or Arch... As long as it starts up, my apps works and it's easy to update.

Link to comment

Yeah if I have understood all the past topics and their contexts, the fight for something that used systemd was the fight for what might replace Slackware as the unraid base OS.  Anything that goes into the VM's we run underneath UnRaid (Slackware or otherwise) is 100% user choice. 

 

That said, "appliance" development etc may very well coalesce around whatever distro(s) are easiest for the developers / has the most package support for applications UnRaid users want.  And that might be something with systemd.  For something like Plex, it has packages for Ubuntu, Fedora, CentOS and even FreeBSD.  So any "turnkey" VM of those OS's will support Plex with a simple [$package_manager] install command

Link to comment

There will always be a need to have a build environment as not every package is available as a binary.

 

Does anyone have a VM of one yet?

 

Hmmm ... I just do all the compilations on native unRAID.  With 5.0, I use the unmenu gcc package installer.  With 6.0 I've just downloaded the necessary Slack 14.1 packages and set up a simple script to install them.

Link to comment

I thought that I would chime in and give my thoughts...

 

I have been running unRAID for about 3 years 4.7, and recently upgraded to 5.0 and then 5.05. I run a 7 disk system on a Supermicro C2SEE system with a AOC-SASLP-MV8, Celeron 430 / 2Gb.

 

I chose this hardware for the best support, least problem, low power. This is exactly what I wanted out of a file server.

 

 

More recently I have built a Xeon 1230 / Supermicro / 16Gb hypervisor box (Win2012R2-HyperV). I run a full time Win8.1 VM which I use to record TV, Sickbeard, Couchpotato, SAB and a bunch of other tools.

 

I am most confident with Windows, but like the idea of having a rock-solid file server for my "Production" XBMC clients. It is not the fact that it is Linux based, it is the fact that it is an appliance that does ONE job very very well.

 

The only time unRAID gets rebooted, is when I need to swap a disk.

 

Moving forward, I have had great service out of the hardware and the fact that it will not support x64 based unRAID 6.0 is not a big deal. In real world terms this hardware is old and has already returned it's value to me.

 

I don't run plugins on unRAID. The main reason is because I do not want to sarifice any of the following:

1) My time in keeping up with the quirks

2) Stability of unRAID

3) Upgrade path for unRAID

 

I have read countless threads during the v5 development where uncompatible plugins were the cause of issue.

 

unRAID 6.0 with an included hypervisor would suit me well. I could run a full time Windows "utility" machine, and consolidate my hardware to just 1 box. This is important to my powerbill, my time. and the stability of my home network.

 

Plugins have been underutilised by me because they do not inspire confidence.

Keeping other apps isolated off to an additional VM moves them away from being in the realm of support for Tom, allowing him to focus on what is the core product. This is a good idea.

 

Look forward to seeing where the development goes!

 

Link to comment

-beta3 status: Downloaded a couple ubuntu images from stacklet:

http://stacklet.com/downloads/templates/xen3?distro=Ubuntu&arch=x86-64

 

I created a 'vm' directory on the cache disk:

mkdir /mnt/cache/vm

 

wget one of the images and unpack

 

Made a couple tweaks to the VM cfg file

- to specify the correct bridge (it's "br0")

- to specify the location of the image (/mnt/cache/vm/<img name>

 

Start the VM:

xl create -c <cfg file name>

 

And off we go!!  Was able to use tightvnc to get to the ubuntu's desktop.

 

So everything mostly works.  I have a little more work to do, e.g., add 'bridging' control to the webGui... a few other things.

 

I'm hoping to be able to release tomorrow to give you guys something to do during the Superbowl.  But if an unforeseen problem crops up and I don't get the release out tomorrow, please don't go on a war path  ;)  I just want to keep everyone informed on exactly where development stands.

Link to comment

 

Peter virtualizing OpenElec should be possible although I'm not clear on some of the paravirtualizing drivers Grumpy mentioned before so it might not be as clear cut as on ESXi.

 

On your OpenElec VM in ESXi, goto system, system info and tell me what Video Card Driver is showing. Is it VMWare?

 

If it isn't showing AMD or nVidia (if you a Quattro Series)... You are not using Hardware Video Acceleration. Instead your CPU in the VM is doing all the processing / decoding of your video files.

 

So basically simply drivers for the customized virtualized hardware that the hypervisor presents? Ya, understand that but had thought you meant something special rather than just drivers lol. I don't hypervise Openelec as I have a pair of Haswell NUC I use for frontends, I do use software decoding though and would think ESXi would be dragged down. I've got no way to send video out so bit of interest but the term caught my eye and I wasn't completely sure if Xen had something I wasn't familiar with. (Shrug)

 

 

Sent from my iPad using Tapatalk

Link to comment
Celeron 430 / 2Gb.

Moving forward, I have had great service out of the hardware and the fact that it will not support x64 based unRAID 6.0 is not a big deal. In real world terms this hardware is old and has already returned it's value to me.

 

 

it should do, from intels  arc for the 430 - Instruction Set 64-bit

Intel® 64 ‡ Yes

Link to comment

I thought it needed to be "xenbr0".

 

You can edit the sxp files and make it use something other than xenbr0.

 

Most do not change it because other software, tools, applications, appliances look for xenbr0. Plus every guide a user would see online it has the default bridge as xenbr0. It works but users will just have to remember this.

 

What I mean by that... Someone will want to create a FreeBSD VM and they will go looking on the web. Even the Xen / FreeBSD instructions on their site will have the user create the cfg file but in those instructions it will have you setup the bridge with xenbr0 and not br0 so it will take someone a little bit of time to figure that out.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.