After spending weeks of reading the various virtualization options available with/around UnRAID, I've decided that I definitely want to incorporate virtualization, but am not sure how best to proceed, so am looking for some clarification on questions, and potentially some recommendations.

 

I currently have my UnRAID server on the hardware in my signature:

 

Case: Norco-4224 | PSU: Corsair AX860

MB: Asus M4A88TD-V EVO/USB3 | CPU: AMD Athlon II X2 260 | Memory: 4GB Kingston ValueRAM

PCI-e Controller Card: Supermicro AOC-SAS2LP-MV8 (x2)

Drives: Parity:3TB WD EZRX | Cache: 1TB WD FAEX | Data: 4x 2TB WD EARX, 6x 3TB WD EZRX

 

All the drives I have are managed through the 2 AOC-SAS2LP-MV8 controllers.

 

While I know my current CPU is x64 capable, it's current configuration does not really allow for virtualization as the memory is too low, and realistically it's a low end processor, so I don't want to split up CPU cycles between a number of VMs.

 

After deciding I needed to upgrade my UnRAID server for potential virtualization I ended up buying a Xeon 1230 v3 and SuperMicro X10SLH-F-O motherboard with 16GB of ECC memory, but while it was in transit I got involved in conversations on these forums about virtualizing XBMC, which peaked my interest. While the X10 is a great server motherboard with IPMI, it only has 3 PCI-X slots, of which 2 are taken by the AOC-SAS2LP cards, which severely limits what I could potentially do.

 

Thankfully (in hindsight anyways), when I first plugged in the MB it shot sparks about 10 inches above the motherboard (bad capacitor I think), so I had to swap it, and have ended up with a ASRock Z87 Extreme6, which sacrifices IPMI, but gives me a lot more PCI/PCI-X options.

 

I also have a 3rd potential motherboard/cpu combo with an Intel i5-2500 with an ASUS P8H67-M PRO motherboard and 16GB of ram. The only issue with the i5 is that it does not support hyper-threading.

 

I had been thinking of running Arch and virtualizing UnRAID along with all the plugins in various VMs, and XBMC in a couple VMs, but am now thinking just using XenServer may be the easiest approach.

 

So, I am trying to decide if I should put the Xeon in my UnRAID case, and load up XenServer with UnRAID virtualized along with everything else, or maybe run the i5 for UnRAID on bare metal, and install the Xeon in a second case and do all my virtualization there - leaving UnRAID running on a faster base (which I hope will help with some Plex stuttering on playback), and moving my XBMC clients to another machine.

 

I want to do pass-through of the video cards (at least) for the XBMC clients, which I've read confirmations the ASRock motherboard will do, so it's now a matter of whether I consolidate everything, or not.

 

The ASRock motherboard has 3 x PCIe 3.0 x16, 1 x PCIe 2.0 x1, 2 PCI, 1 x mini-PCIe slots and 10 x SATA3, 1 x eSATA, 8 x USB 3.0, 7 x USB 2.0 ports, so there are a ton of options.

 

What I am wondering is if I consolidate UnRAID with the XBMC clients, I use up 2 PCIe 3.0 ports for the AOC-SAS2LP cards, so I have 1 x PCIe 3.0 x16, 1 x PCIe 2.0 x1, 2 PCI, 1 x mini-PCIe slots left (though I don't know what use the mini-PCI-e is for yet).

 

My questions are as follows:

 

1) If I want to do pass-through for the video cards for XBMC, are there any restrictions or minimum requirements recommended for the video cards? I would think I have 3 slots minimum (1 PCIe x16 and 2 PCI), and don't know if I can get a video card in the PCIe x1 slot. I basically want to buy lower end video cards if I can, but want to ensure a solid video viewing experience for 1080p video.

 

2) If I want to run multiple VMs for SAB/SB/CP, Plex, XBMC/MqSQL centralized database, and 2-4 XBMC clients will a single SSD manage that (from an IOPS perspective)? I am thinking of buying 2 SSD drives and mirroring them via the motherboard so if one SSD died I would have a backup. Alternatively, if I run this all in a separate case (without UnRAID) I could do 4 WD 500GB Black drives in a RAID 10 array to get protection and better IOPS, but if the SSDs would likely do then I would lean towards the consolidated approach.

 

3) Does anyone know if I can pass only some of the on-board SATA ports through to a UnRAID VM? While I have 16 ports managed via the AOC-SAS2LP cards, I have room for up to 24 drives in the future. Since I have 10 on-board SATA 3 ports I would like to use some to expand my UnRAID storage while still using a couple for the SSD drives for the host/VMs

 

4) If I am moving from UnRAID on bare metal to UnRAID in a VM and pass-through the 2 AOC-SAS2LP cards and the USB drive with UnRAID on it, should UnRAID just start working again without issue?

 

5) Is there anything else I haven't covered or thought of that could influence this decision?

 

While I don't have many answers for you, I'm in a pretty similar situation - have shiny new VT-d capable hardware and want to cenralize several HTPCs and an aging 32-bit Dual Xeon server running unRAID.

 

I've actually done a full GPU and USB passthrough to a Windows 8 VM on Arch Linux / Xen but I'm (im)patiently waiting to see what features the 64 Bit 6.0 Beta arrives wit. I'd prefer have unRAID run bare metal and access all the SATA controllers natively rather than pass them through through - it just seems cleaner to me and  saves me a VM.

 

However f, when 6.0 beta arrives' it doesn't support hosting VMs or it looks like it will be a considerable time before that happens, I'll go ahead and virtualize unRAID on top of xen or kvm. In terms of answering your questions, I can only tackle the following;

 

1) I have a pair of AMD HD5xxx series cards in my rig and I passed the through to a Widows 8 VM with 4GB RAM / 2 CPUs. This played 1080p video with audio over HDMI to my TV. The VM even ran perfectly with only 1 CPU and 2GB but I didn't conduct a lot o tests with this config.

 

3) As I understand it, you need to pass through the controller, not individual ports. If all 10 of your ports are attached to the same controller (unlikely) then you'll have to pass through all 10 to the same VM. If you have more than  one controller, you might have better luck, though I read somewhere today that even with separate controllers, some people have had difficulties unless both controllers (all ports) were passed to the same VM.

 

My own new MB has two controllers (6 ports on one, 2 on th other). I have not yet tested SATA passthrough but it's nect on my list of To Dos.

 

 

 

 

A bit of counterpoint on virtualization ...

 

I've also thought about building a high-end Xeon-based setup to virtualize both UnRAID and a bunch of VM's for various other purposes.  ... but after considering that, I've concluded that while I'm indeed going to build a nice virtualization host box for all my VMs, it will NOT include UnRAID.  My UnRAID server is used solely as a NAS -- draws less than 20w at idle (drives spun down) -- and is absolutely rock-solid.    Clearly it doesn't have the "horsepower" to run a bunch of add-ons (especially something like Plex) ... but that's not what I use it for.

 

A dedicated VM host machine with a high-end Xeon can use the server as easily across the Gb network as it could locally, with very little difference in drive performance.    And it can be a much smaller box than it would need to be if it had to host all the drives for UnRAID.

 

My problem is that I see benefits both ways. I do like the idea of keeping everything UnRAID oriented on a single server, but am also thinking if I build a dedicated virtualization server I can add a second one down the road and HA them and all my VMs are protected.

 

I've done HA with Hyper-V as I am a Windows guy by nature, but the whole ESX/Xen/Linux thing is a bit new. I had been thinking this exercise would be a good excuse to get into ESX, but from reading Grumpy's notes on Xen it seems that Xen is a better choice for hardware pass-through, so I will likely go that route.

 

I guess one question on XenServer I don't know the answer to is if you can do HA without shared storage. I know ESX and Hyper-V can now do this, but not sure on XenServer. If so, then it would help me lean towards a dedicated box again (plus I don't have to waste 2 PCIe x16 slots with the AOC-SAS2LP cards).

 

It's nice to actually have options, but too many options can be just as frustrating as having none sometimes.

My problem is that I see benefits both ways. I do like the idea of keeping everything UnRAID oriented on a single server, but am also thinking if I build a dedicated virtualization server I can add a second one down the road and HA them and all my VMs are protected.

 

I've done HA with Hyper-V as I am a Windows guy by nature, but the whole ESX/Xen/Linux thing is a bit new. I had been thinking this exercise would be a good excuse to get into ESX, but from reading Grumpy's notes on Xen it seems that Xen is a better choice for hardware pass-through, so I will likely go that route.

 

Most home users do not need all the "extra" stuff that Xen offers (like HA) so KVM is a good choice and for home users, I think easier to manage / support.

 

I have successfully installed unRAID in a VM on Hyper-V 2012 R2 several months back. If that is what you know / like, you could always do that. Since you are probably are a lot more knowledgeable than I am on Hyper-V... you could it and even post a guide so the rest of us know.

 

I guess one question on XenServer I don't know the answer to is if you can do HA without shared storage. I know ESX and Hyper-V can now do this, but not sure on XenServer. If so, then it would help me lean towards a dedicated box again (plus I don't have to waste 2 PCIe x16 slots with the AOC-SAS2LP cards).

 

You get HA right out the box with XenServer.

 

Click me to see all the various features you get XenServer.

 

Thanks Grumpy.

 

The issue with Hyper-V is that I don't think it will do pass-through of devices like ESX/Xen/etc. I think you can pass disks through (maybe/sort of), but not USB, PCIe cards, etc, so I wouldn't be able to do UnRAID, or multiple XBMC guests which I am hoping to do. I figured XenServer is my best bet.

 

As for HA, I do run Active Directory and Exchange server as well as what I am hoping for here, so I would rather build 2 Xen servers so I could fail-over/back as needed (except for maybe XBMC as I don't see buying video cards for the failover machine).

 

I appreciate you confirming the HA with local storage. Outside of these forums you don't hear a lot about XenServer (Citrix) - everything seems to be either Hyper-V or ESX in the corporate world.

The issue with Hyper-V is that I don't think it will do pass-through of devices like ESX/Xen/etc. I think you can pass disks through (maybe/sort of), but not USB, PCIe cards, etc, so I wouldn't be able to do UnRAID, or multiple XBMC guests which I am hoping to do. I figured XenServer is my best bet.

 

Are we talking about Hyper-V 2012 or Hyper-V 2012 R2?

 

One of the selling points of Hyper-V 2012 R2 is the ability to pass through USB.  You can also do Microsoft's version of what many here call / refer to RDM.

 

Having said that, I wouldn't do that but there are several people here who do RDM and it works well for them.

 

As for HA, I do run Active Directory and Exchange server as well as what I am hoping for here, so I would rather build 2 Xen servers so I could fail-over/back as needed (except for maybe XBMC as I don't see buying video cards for the failover machine).

 

I appreciate you confirming the HA with local storage. Outside of these forums you don't hear a lot about XenServer (Citrix) - everything seems to be either Hyper-V or ESX in the corporate world.

 

ESXi in the home user world based on what I have heard from VMWare employees at VMWorld / What My Clients sales reps have said... days are numbered. If the rumors that I hear (and many others are reporting all over the web) are true... Going forward, versions of ESXi are going to become more and more "limited" / "crippled" with each release. I suppose someone could stay on ESXi 5.0 / 5.1 but how long / how many will do that?

 

With you skills / abilities / talents / knowledge with ESXi / Hyper-V why unRAID for storage? Wouldn't you rather use something else that has more redundancy / scalable / features / functions / free / built-in instead of running unRAID in various VMs in a HA environment? unRAID is great for an appliance, home users, etc. but for what you talking about doing and with your skills... I would be surprised if ended up sticking with unRAID after you finish your research / thinking this over.

I have successfully installed unRAID in a VM on Hyper-V 2012 R2 several months back.

 

Grumpy -- how did you overcome this when you set it up in Hyper-V ??

I've seen several attempts to run UnRAID under Hyper-V, but all have been unsuccessful.  I've tried as well; but could only run Basic, as I don't believe you can pass-through the USB flash drive to read the GUID.

 

 

As for HA ... an admirable goal.  If this can be achieved at home with modestly priced boxes, it would be nice -- although I can't think of much realistic need for it in a home setting.    Other than power-failure-induced shutdowns; my HTPC's and UnRAID servers have been running 24/7 since they were built ... that's certainly reliable enough for me 

 

 

Grumpy -- how did you overcome this when you set it up in Hyper-V ??

I've seen several attempts to run UnRAID under Hyper-V, but all have been unsuccessful.  I've tried as well; but could only run Basic, as I don't believe you can pass-through the USB flash drive to read the GUID.

 

I assisted several who attempted it. However both were using those TAMS servers which I am not sure if anyone got one of those to work in ESXi, Xen or KVM with unRAID in a VM.

 

To answer your question, I don't recall how I did it. I know I got the USB working (I might have had to modify some settings in the bzroot for it to see the USB correctly). It was having to do "RDM" where Hyper-V 2012 R2 lost me and why I never continued / posted a guide for it.

 

If you are interested, I can do it again, refresh my memory, the steps involved and share that with you. I'm not writing up a guide for it because I can't / don't have the bandwidth to support it. Plus, I hope unRAID 6.0 beta comes out soon and I am busy working on a guide for it.

It'd be very interesting to know how to do this if you can actually make other-than-Basic work in Hyper-V.    That would completely eliminate the need for a Linux-based install (except, of course, for the UnRAID VM).    Many of us have extensive Windows experience (I'm a long-time Microsoft MVP) but are Linux novices ... and if we could simply add UnRAID to a Hyper-V setup we'd have a rock-solid NAS "built-in" to our primary Windows box ... which would certainly be VERY convenient 

 

... one caveat:  I'm NOT interested in an illegal mod to the UnRAID base that simply eliminates the check for GUID.    I know that would allow it to run (and for that matter I could do that myself) ... but I have no interest in running hacked code.    Hopefully that's not how you did it; but your comment r.e. "... modifying bzroot ...." made me wonder.

 

Lol... No! I wouldn't post anything like that.

 

I obviously had to add the drivers for Hyper-V support and I think the other was mounting the USB different. When I get some free time I will go through the process again and share what I did. If want to take it and run with... be my guest.

 

Getting Hyper-V to work outside of a domain is a MFer even for someone like me. I think that took me longer than getting unRAID to work.

I'm just curious ... why specifically Hyper-V for a windows environ VM?  Why not Virtual Box?  Is it because Hyper-V is a Type 1 and not a Type 2 virtual machine? 

 

With you skills / abilities / talents / knowledge with ESXi / Hyper-V why unRAID for storage? Wouldn't you rather use something else that has more redundancy / scalable / features / functions / free / built-in instead of running unRAID in various VMs in a HA environment? unRAID is great for an appliance, home users, etc. but for what you talking about doing and with your skills... I would be surprised if ended up sticking with unRAID after you finish your research / thinking this over.

 

My experience is with Hyper-V, but I would like to get a better understanding of ESXi, which is where I thought this project would help, but now it looks like Xen I will be learning.

 

As for Hyper-V, there is an appeal to use it because it's familiar ground, but you introduce a lot of overhead for something like UnRAID. I've always liked that I don't need an OS drive and can just boot from USB key and keep all the drives for data. This is less critical now that I am using the Norco-4224 with 24 bays, but when I was building my media library I really didn't want to waste a bay just for the OS.

 

Now it's not an issue, but Windows still takes up a lot of resources for the base OS. I would rather run something lean (though hyper-v server may have served), and then I can dedicate the resources to the VMs where they are doing something useful.

 

I think UnRAID will continue to meet my basic storage needs, but I definitely want to branch out with additional features - I am just thinking of running them in parallel with UnRAID rather than instead of (at least at this point).

 

I'm just curious ... why specifically Hyper-V for a windows environ VM?  Why not Virtual Box?  Is it because Hyper-V is a Type 1 and not a Type 2 virtual machine?

 

Yes, Hyper-V has notably better performance for the VMs, since it's a Type 1 hypervisor.

 

As for HA ... an admirable goal.  If this can be achieved at home with modestly priced boxes, it would be nice -- although I can't think of much realistic need for it in a home setting.    Other than power-failure-induced shutdowns; my HTPC's and UnRAID servers have been running 24/7 since they were built ... that's certainly reliable enough for me 

 

you are right, HA is really overkill, however since I am running Exchange 2010 at home and host email for my family I would rather this not get taken out by a hardware failure. I could just HA Exchange, but like the solution of HA on the virtualization layer better so I can fail everything over if I need to do maintenance on one of the boxes.

 

I currently have 6 servers in the basement doing multiple functions, including UnRAID, and would like to consolidate to 3 (2 virtualization and 1 UnRAID) and retire the others which are a physical domain controller, XBMC machine and backend SQL/Windows storage server.

 

If I am going to consolidate I would rather include HA since I have the hardware - and it's good experience.

 

Agree if you've already got the hardware you may as well do it.

 

I've got 3 servers, 3 HTPCs, 4 desktops, and 3 laptops for the 2 of us      I definitely understand how a home environment can grow !!   

Really?  huh, I hadn't thought it was that big a hit assuming the right PV drivers/kernel features.  [shrug] well fair enough.  Curiosity satiated   Thanks.

Agree if you've already got the hardware you may as well do it.

 

I've got 3 servers, 3 HTPCs, 4 desktops, and 3 laptops for the 2 of us      I definitely understand how a home environment can grow !! 

 

Phew! I thought I had reached dork status. It's official... I haven't.

 

Demoting myself to back to nerd.

 

Speaking of which....

 

512TB (half a petabyte) in his house.

 

Linux - AHEM! Let me clear my throat...

Object Store 512TB Array

 

Personally I think the guy is INSANE with how he did it and the direction he is heading now. No surprise... Former Slackware admin.

 

512TB of storage, split across 128 disks with no redundancy. Each 4TB disk is further split into 1TB partitions so that's 1024 partitions in all. Oh yeah... each partition is a separate media source.

 

Believe it or not... Several people suggested unRAID in the thread.. LOL!

Yes, 512TB is a bit much -- especially with (apparently) no backups ... although it's not clear that there aren't redundant copies of his data (just that he's not fault-tolerant).

 

<Sigh> ... and I've barely got 100TB      [And that includes all of my backups]

 

Hay guys, can anyone help me with setting up my home network?

I do not need physical help but need some guidance and suggestions.

maybe an explanation or two.

 

here is the round up on what I have and want to do :

 

for several years I have been running UnRaid on a dedicated home grade setup(see my sig.) the plugins I wanted were not there or did not work as I expected so I had plans on go virtual. and run all in dedicated VMs

also I have been having some issues with my router from Optimum so plan to replace it with a pfSense/ClearOs  VM at the same time.

 

at the moment my plans for virtualising unRaid are onHold as the hardware I got is not capable of pass through (I got an AMD Opteron server from TAMs that even though says it supports IOMMU nothing recognize it as such. I have tried KVM,Xen,XenServer,ESXi  no-go) but I had invested considerable time and money into it (did some upgrading etc.) so can not simply scrap the whole thing and start a new.

so want to figure out on how to set up the VM server and run everything on it except unRaid and see if I can add unRaid at the later time(maybe the new version 6 with virtualization capabilities will work.)

 

my main issue is how to setup the whole networking bit.

I have

1. the server with 4 Gigabit NIC interfaces (2xOnboard + intel Pro dual NIC card)

2. 24 port netgear dumb switch

3. 3 windows PC 

4. a netgear wireless router from optimum (this is the one I want to replace)

connected to the optimum modem(bridge)

5. a lincsys wireless router in an AP mode (extends wireless network to second floor)

 

I am using OpenSuse 13.1 with LSDX (minimal GUI KDE install)

how do I setup my server and connect it to the home network so I can run pfsense as my router firewall? mainly how do I configure/use the Nic cards with VMs

 

I understand that I will connect  one NIC to the Modem

and one to the switch and asign them to VM.

but do I need to setup VLAN? or VSWITCH on the host?

how do I do that if I do needed.

 

thanks.

 

 

 

 

 

 

 

 

Yes, 512TB is a bit much -- especially with (apparently) no backups ... although it's not clear that there aren't redundant copies of his data (just that he's not fault-tolerant).

 

The 512TB isn't the issue.

 

No Fault Tolerance and not using a unified distributed storage system (like Ceph) is mind boggling!

 

I cannot fathom what it would be like to manage / use /  back up 1,024 individual 1TB partitions spread across 128 drives and 8 nodes.

 

 

 

 

I cannot fathom what it would be like to manage / use /  back up 1,024 individual 1TB partitions spread across 128 drives and 8 nodes.

 

Me either -- although in this case he's "ONLY" got 512 of those 1TB partitions