pls. help with unRAID on a swiss army knife / jack of all trade, master of none

[Ford Prefect]

Hi folks,

 

for a long time now I am on the Quest to find me a swiss army knife of a server (well, who else isn't  )

 

...in the past, I did try out several virtualization concepts, since this appeared to be the magic bullet for my goals.

However, stable access to DVB cards and other hardware was merely impossible at that time

and serving storage with some extra layers of software between the physical disk and the "raid intelligence"

was not appealing as far as performance and my understanding of a good architecture is concerned.

 

But life goes on and virtualization evolves.

With vt-x and vt-d technologies and support for pci-passtrough (XEN, KVM) or vmDirectPath (ESXi)

I'd think that I'll have another try along my Quest.

 

So, the first task is to find the right combination of Board, CPU (and PSU).

Well that does sound familiar, doesn't it? 

 

Here's what I found so far...

 

possible Board & CPU:

The supermicro X8SIL-F has the minimal no. of expansion slots I need,

and has vt-d available in the chipset&BIOS.

Possible CPUs are listed here

The 32nm lithography CPUs are more efficient that their older 45nm brothers.

The CORE-Part of the CPU will go down to zero power when idle, but the unCORE part will still consume around 3-4watts

However, this article states, that the combination of this Board

and CPU will idle around 40Watts (including around another 4watts for IPMI).

 

Supermicro has some DP Xeon boards (I know, much more expensive, but what the h&%§) with even more slots here.

Possible CPUs here

But I couldn't find an energy efficiency review of that board (maybe starting with one CPU first) and if this is better than the above combination on idle.

Power on idle will be my main sucess szenario, because this is what it will do most of the time... idling around the day...

 

Maybe some of you already have some experience or background info that you are willing to share?

Any input will be appreciated.

 

TIA,

  Ford

 

Here's the background on what I need (well...want):

 

- top one priority is: Everything in one case, one Server-Board only

- use unRAID for storage -> what else can I say 

- be as much power-efficient as possible -> one server, more peak power, makes a 80+gold or platinum PSU feasible

- access through IPMI (KVM over IP) for managing the box -> I will never, ever have anything without that again

- run several family services, each should be "best of breed"

  - mailgateway with POP/IMAP -> postfix/dovecot or iRedOS

  - DVB/video live&PVR to XBMC or other frontends -> VDR & tvheadend do a nice job, mythtv is too complex

  - DLNA&UPnP to media renderers in the house -> ps3mediaserver & mediatomb will do

  - central Win7 instance for legacy support -> I know but for some topics you'll have to go with the times and stay antique 

  - groupware / calendar -> haven't tried one yet

 

 

[BRiT]

Have you read the VMWare ESXi thread? There's some good discussion going on there dealing with what might suit you.

 

I think you'd be well suited for virtualization with one guest dealing with the DVB cards and DVR/PVR, possibly a second one for an HTPC front-end if desired, and one guest dealing with unRAID for storage.

 

[prostuff1]

I agree with what BRIT said above; check out the VMWare ESXi thread in the Customization forum.

 

Hopefully, in the next 6 months I will be doing something similar to my machine.  At the very least I will be setting up my server to run VMWare Server and putting everything into a bigger case (Norco or the like).  I would really like to go ESXi but have to do some more reading to see what I might have to buy to support it on either the Gigabyte board I have or the C2SEE i recently bought.

[Ford Prefect]

Have you read the VMWare ESXi thread? There's some good discussion going on there dealing with what might suit you.

 

I think you'd be well suited for virtualization with one guest dealing with the DVB cards and DVR/PVR, possibly a second one for an HTPC front-end if desired, and one guest dealing with unRAID for storage.

 

 

Thanks for pointing me into that direction, BRiT.

As always, this forum is a goldmine 

[Ford Prefect]

I agree with what BRIT said above; check out the VMWare ESXi thread in the Customization forum.

 

Hopefully, in the next 6 months I will be doing something similar to my machine.  At the very least I will be setting up my server to run VMWare Server and putting everything into a bigger case (Norco or the like).  I would really like to go ESXi but have to do some more reading to see what I might have to buy to support it on either the Gigabyte board I have or the C2SEE i recently bought.

 

Yes, I agree.

As it looks, finding a MoBo that is really capable of unleashing vt-x and vt-d from the chipset is one issue.

Compatibility of Controllers, either on the Host-OS/dom0 or ability to "passtrough", seems to be even more challenging.

 

...I am also not a big fan of non-opensource software, being it free (as in free beer) or not.

I had very good experinece with proxmox-VE.

My DVB setup will work in openvz and recent distros can combine openvz containers with kvm, which I need for win guest.

 

[heffe2001]

I plan on trying the ESXi multi-role server in the very near future (I actually played with it this weekend, but had some stumbling blocks on my SAS controller under unRaid, see the ESXi with Raw disks thread for more info there).  I am currently planning on running ESXi 4.1, with one VM for unRaid (probably a single core and 1gb memory dedicated to it, as well as the SAS controller in PCI Passthrough), and a flavor of Windows running SageTV, with 2 HVR-2250's in passthrough mode as well (the remaining 3 cores, and at least 3gb ram dedicated to that one as well).  If I can find a decent 2 or 4 port PCIE network card that doesn't cost a damn fortune, I'll end up moving my firewall into the box as well (it's currently running IPCop on an Intel Atom).  It's probably going to be sometime near the end of the month, first of December before I can really have time to move the tuner cards (I'm HOPING that won't be an issue, ESXi passed the SAS controller through perfectly with a parity generation of about 60m/sec on 3 2tb WD Green drives).  If everything goes to plan with the hardware I currently have, the system setup will be as follows:

 

ASUS M4A89TD PRO/USB3 AM3

AMD Phenom II 810 AM3 2.6ghz Quad Core

8gb DDR3-10666 G.SKILL Memory (4g of what I have now is bad, so I need to RMA it this week, the system only had 4g when I was toying with it)

LSI Logic SAS3081E-S 8 Port SAS:

    3x Western Digital Caviar Green WD20EADS 2TB

    2x Western Digital WD1001FALS 1TB

    3x Western Digital WD7500AADS 750GB

Onboard SATA:

    1x OCZ 30g 2.5" SSD Core v2 (left over from an old build, should be fine for the ESXi boot drive)

    1x WD 250g 2.5" 5400 (Datastore drive for ESXi, will hold the unraid files, and the Windows install most likely)

    Pioneer DL DVD-RW for any OS installs

 

2x Hauppauge HVR-2250 Dual Tuner Analog/Digital  (Can record 4 digital or 4 analog, or any combination)

Corsaid 850w Power supply

Chenbro SR107 Server/Workstation case, with 2 4 port hot swap modules.  I'll put the 3.5" drives in these, and may get a 1-in-4 2.5" drive bay for the other drives to fit in the 5.25" slot at the top with the DVD.

 

I'm still needing a cheap PCI Video card, and a good, quality PCIE network card that is compatible with ESXi (the realtek 8111c onboard works MOST of the time, but it appears to drop out when large amounts of data are transferred, not good for a server machine).  If the system seems to be a bit sluggish with the 810 proc in it, I can always upgrade to a 6 core faster AM3 proc, but I'm hoping I won't need to do that.  The PVR/HTPC side will be running Win7 most likely, with SageTV for the video serving (I use Sages' HD Extender boxes at my TV, so no real processing power needed on the server-side, with the exception of comskip detection, and Playon transcoding for Netflix/Hulu).  Total storage for Unraid will be around 8.25tb, which should be sufficient for my current needs (my current unRaid box comes in around 5.25tb-ish, and is around 90% full at the moment, but it's in a large case, using lots of power, lol).  I was hoping I could find a board with more PCIe slots, as I'll pretty much have this one maxed out as it stands (I'll have a single PCI slot left over, might put a dual-nic 10/100 card in there dedicated to the firewall, not sure yet).  If I need more storage space down the road, I'll probably use my existing Antec 900 case with 3 5-in-3' chassis in it, converted over to a SAS enclosure with one of those HP SAS expander cards in it, and replace my current 4 port SAS card with something that will do at least 8 ports internal, and 4 external if I can find it..

 

I'd love to get off the dual-box setup I'm running now, it just makes more work for me when I have to manage storage now (my current storage is split between the 2 machines, the HTPC machine runs WHS, with around 2tb of space for recordings, and the rest is in the unRaid box, so some things are on one machine, some are on the other, and keeping track is sometimes a PITA).

 

[Ford Prefect]

I plan on trying the ESXi multi-role server in the very near future (I actually played with it this weekend, but had some stumbling blocks on my SAS controller under unRaid, see the ESXi with Raw disks thread for more info there).  I am currently planning on running ESXi 4.1, with one VM for unRaid (probably a single core and 1gb memory dedicated to it, as well as the SAS controller in PCI Passthrough), and a flavor of Windows running SageTV, with 2 HVR-2250's in passthrough mode as well (the remaining 3 cores, and at least 3gb ram dedicated to that one as well).  If I can find a decent 2 or 4 port PCIE network card that doesn't cost a damn fortune, I'll end up moving my firewall into the box as well (it's currently running IPCop on an Intel Atom).  It's probably going to be sometime near the end of the month, first of December before I can really have time to move the tuner cards (I'm HOPING that won't be an issue, ESXi passed the SAS controller through perfectly with a parity generation of about 60m/sec on 3 2tb WD Green drives).  If everything goes to plan with the hardware I currently have, the system setup will be as follows:

 

ASUS M4A89TD PRO/USB3 AM3

AMD Phenom II 810 AM3 2.6ghz Quad Core

8gb DDR3-10666 G.SKILL Memory (4g of what I have now is bad, so I need to RMA it this week, the system only had 4g when I was toying with it)

LSI Logic SAS3081E-S 8 Port SAS:

    3x Western Digital Caviar Green WD20EADS 2TB

    2x Western Digital WD1001FALS 1TB

    3x Western Digital WD7500AADS 750GB

Onboard SATA:

    1x OCZ 30g 2.5" SSD Core v2 (left over from an old build, should be fine for the ESXi boot drive)

    1x WD 250g 2.5" 5400 (Datastore drive for ESXi, will hold the unraid files, and the Windows install most likely)

    Pioneer DL DVD-RW for any OS installs

 

2x Hauppauge HVR-2250 Dual Tuner Analog/Digital  (Can record 4 digital or 4 analog, or any combination)

Corsaid 850w Power supply

Chenbro SR107 Server/Workstation case, with 2 4 port hot swap modules.  I'll put the 3.5" drives in these, and may get a 1-in-4 2.5" drive bay for the other drives to fit in the 5.25" slot at the top with the DVD.

[...]

 

...many thanks for sharing, heffe2001

 

- I definitely want to employ a real server board with IPMI, since it will be stowed away in a rack, somewhere in the dungeons.

- My plans are to use 2 pieces of Dual-DVB-S2 (HD) cards, this will give me access to 4 transponders on the sat-range of my dish.

  The max range of transponders hit by live-TV use is 3 with my family, which will leave me at least one transponder as backup

  and the chance that a scheduled PVR session will not execute is minimal...4 transponders would give approx. 50 channels

  which would cover 120percent of the channels my family would actively select.

- I definitely will run the server from/off a SSD

- for unRAID I can cope with 8 ports on a single controller...currently use 3

- I am thinking of running another unRAID as "mirror"/backup for the first, so a slot for a second controller would be nice

- video I do not need in that box...I am tempted to go for a Dual-Xeon board, start with one CPU and beef up later...I always wanted to

  to know how much it will take to transcode a bluray on-the-fly 

[heffe2001]

I haven't looked at the Proxmox before, but it looks like it supports mostly the same things as ESXi.  How does it compare performance-wise?  Guess it'll be something else I will at least give a try when I get a chance..

 

 

[Ford Prefect]

I haven't looked at the Proxmox before, but it looks like it supports mostly the same things as ESXi.  How does it compare performance-wise?  Guess it'll be something else I will at least give a try when I get a chance..

 

I haven't looked into vmW**** products for a looong time, so I cannot comment on recent performance improvements.

Reason for this was, that I always wanted to do something with pcipasstrough to get my hardware working and I

just discovered that this feature has been added since ESXi4.

 

KVM performance is comparable to XEN and vmware was little behind at that time, I'd say.

I am also a fan of open source, so I'd prefer any linux & community based solution.

 

What I like with Proxmox-VE is, that it is really lightweight.

All that you need to manage the whole box is a webbrowser. (especially when using a board with IPMI   )

For the guests, it will even supply you a VNC via java webstart.

I can testify that is was very stable on my last setup...ran an openfiler nas, WinXP HVM guest, mailserver,

upnP Server and some extra guests on a small AMD 64 X2 SFF (35W-TDP, 4GB) setup with a cheap

desktop board....never booted the machine or a guest for a year.

But that as well was without vt-d and pcipasstrough, just vt-x (what's it called on AMD?).