Converging 2 Servers into 1 - ninthwalker? ;)

[JimPhreak]

I was reading this thread here and didn't want to hijack it so I figured I'd start my own.

 

I'm in the process of upgrading/rebuilding my home VM server + unRAID server and combining them into one and I must say ninthwalker, you're setup is almost exactly how mine is/will be in terms of what I run on my VM server (pfsense, Plex (lots of remote streaming like you), Server 2012, Windows VMs, etc).

 

Just to give a little summary of what I have now and what I will have once my new server is built (parts are on the way).

 

Current 2 server setup (will be selling all once new server is up and running):

 

Server #1 - vSphere 5.5

 

Case: Xigmatek Gigas Silver mATX cube case

MoBo:  SuperMicro X9SCM-F-O LGA 1155 Micro ATX Server Motherboard

CPU:  Intel Xeon E3-1245v2 3.4Ghz Quad-core CPU

RAM:  24GB Kingston KVR16E11 240-pin DDR3 ECC Server Memory

SSD: Samsung 840 EVO 500GB

PSU:  Corsair CX430w Modular PSU

Add-on NICs:  Intel Gigabit CT Adapter (x2)

 

Server #2 - unRAID v6b15

 

Case:  Fractal Arc Mini

MoBo:  SuperMicro X10SLM-F

CPU: Intel i3-4130 3.4Ghz Dual-core CPU

RAM:  8GB Kingston KVR16E11 240-pin DDR3 ECC Server Memory

HBA:  IBM ServeRAID M1015

HDDs:  6 * 3TB WD Reds + 1 * 3TB Hitachi Desktar + 1 * 500GB Seagate Cache Drive

PSU:  Seasonic Platinum Series 400w

 

---

 

New All-In-One Server

 

Case:  SuperMicro CSE-721TQ-250B

MoBo:  SuperMicro X10SDV-TLN4F

CPU: Xeon D-1540 8-core 2.0Ghz 45w CPU

RAM:  64GB (32GB x 2) Samsung DDR4-2133 RDIMMs

SSDs:  Intel 730 480GB SSDs (x2) - May add an m.2 drive as well if I can find a good use for it

HDDs:  4 x 8TB Seagate Archive drives

 

 

Ninthwalker, I mainly have some questions for you since you have a very similar server configuration based on the diagram you posted of your network.

[*]How does one go about converting their physical unRAID v6 server into a VM?  Do I need to purchase a whole new registration key?  And how are you booting unRAID in a VM (passed through USB boot flash drive)?

[*]Do I have to pass through a storage controller (such as the M1015 I have in my current server) to the unRAID VM in order to get all smart and monitoring functions or are there any workarounds?  The reason I ask is that this board only has one PCIe slot so putting an HBA in there prevents any further expansion and leave my 6 on board SATA ports basically sitting there unused.

[*]I've never combined my storage and VM server so what would you recommend as a good way to configure my SSDs/m.2 drives (datastore/cache) for the best performance of my VMs (most importantly unRAID and Plex)?

[*]I see you noted that you have an SSD cache and SSD datastore.  I assume the SSD datastore is where you have your VMs.  What do you use the cache for?

 

If there's any other advice you could lend or best practices you could point to based on your own experience, I would greatly appreciate it.

[uldise]

Hi Jim, what a new server will be used for? except unRAID of course:)

i have very similar setup as your current server #1(except with Norco 4224 case) and i'm using unRAID on it on Esxi quest.. plus other 4 VM's - two win, two linux ..

and everything working like a charm

[JimPhreak]

Hi Jim, what a new server will be used for? except unRAID of course:)

i have very similar setup as your current server #1(except with Norco 4224 case) and i'm using unRAID on it on Esxi quest.. plus other 4 VM's - two win, two linux ..

and everything working like a charm

 

Will be using the following VMs:

• pfSense (routing/firewall/site-to-site OpenVPN)
  
• unRAID
  
• Plex Media Server
  
• Server 2012 R2 DC (AD, DNS, etc.)
  
• Backup VM (Windows 7 OS)
  
• Torrent VM (Windows 7 OS)
  
• Personal Cloud (TBD)
  
• Few other test VMs (Mac OS X, different Windows clients, etc.)
  

[uldise]

Will be using the following VMs:

• pfSense (routing/firewall/site-to-site OpenVPN)
  
• unRAID
  
• Plex Media Server
  
• Server 2012 R2 DC (AD, DNS, etc.)
  
• Backup VM (Windows 7 OS)
  
• Torrent VM (Windows 7 OS)
  
• Personal Cloud (TBD)
  
• Few other test VMs (Mac OS X, different Windows clients, etc.)
  

 

for me, it looks like your old server #1 Mobo+CPU will work better IMHO..

ok, E3 have less cores but that means nothing when you demand on high CPU usage, plex server transcoding for example

 

can you explain a bit in deep why you chose this new Xeon combo?

[JimPhreak]

Will be using the following VMs:

• pfSense (routing/firewall/site-to-site OpenVPN)
  
• unRAID
  
• Plex Media Server
  
• Server 2012 R2 DC (AD, DNS, etc.)
  
• Backup VM (Windows 7 OS)
  
• Torrent VM (Windows 7 OS)
  
• Personal Cloud (TBD)
  
• Few other test VMs (Mac OS X, different Windows clients, etc.)
  

 

for me, it looks like your old server #1 Mobo+CPU will work better IMHO..

ok, E3 have less cores but that means nothing when you demand on high CPU usage, plex server transcoding for example

 

can you explain a bit in deep why you chose this new Xeon combo?

 

The Xeon D-1540 received a 11,914 on CPU mark compared to about 9,000 for the Xeon E3-1245v2.  I know that's just one benchmark but it's what Plex themselves recommend you use as a rough estimate to determine your CPU needs.

 

Also, I need the extra cores and RAM (the board I'm getting supports 128GB of RAM) as I will be adding a lot more test VMs.  The 2 10Gbe NICs isn't bad either for future use.  Whenever I upgrade I try to consider my future needs as much as I can.  My main priorities here were conserve space (this server is tiny), power (entire system won't draw more than 100w on full load), and future expandability in terms of VM resources.

[uldise]

 

The Xeon D-1540 received a 11,914 on CPU mark compared to about 9,000 for the Xeon E3-1245v2.  I know that's just one benchmark but it's what Plex themselves recommend you use as a rough estimate to determine your CPU needs.

 

Also, I need the extra cores and RAM (the board I'm getting supports 128GB of RAM) as I will be adding a lot more test VMs.  The 2 10Gbe NICs isn't bad either for future use.  Whenever I upgrade I try to consider my future needs as much as I can.  My main priorities here were conserve space (this server is tiny), power (entire system won't draw more than 100w on full load), and future expandability in terms of VM resources.

 

thanks for sharing your choice:) this little board looks very tiny..

 

i'm asking your experience cos i'm on something similar situation - all works fine for me on my setup, but all 4 PCIe slots are with cards and i cant add more - currently have 2xM1015 + Dell PERC H710P + Intel 4port Gigabit network card.

i'm experimenting with 10Gbit network too, may be wanna to add Brocade CNA adapter too.. have two of them already if i add one of them to my setup, then i will need pfsense VM too..

 

so, i'm looking for larger board with more PCIe slots. it's hard to decide where to go. may be i will stay with used older X8 Dual Xeon bord cos of price.. any other advice welcome

[JimPhreak]

 

The Xeon D-1540 received a 11,914 on CPU mark compared to about 9,000 for the Xeon E3-1245v2.  I know that's just one benchmark but it's what Plex themselves recommend you use as a rough estimate to determine your CPU needs.

 

Also, I need the extra cores and RAM (the board I'm getting supports 128GB of RAM) as I will be adding a lot more test VMs.  The 2 10Gbe NICs isn't bad either for future use.  Whenever I upgrade I try to consider my future needs as much as I can.  My main priorities here were conserve space (this server is tiny), power (entire system won't draw more than 100w on full load), and future expandability in terms of VM resources.

 

thanks for sharing your choice:) this little board looks very tiny..

 

i'm asking your experience cos i'm on something similar situation - all works fine for me on my setup, but all 4 PCIe slots are with cards and i cant add more - currently have 2xM1015 + Dell PERC H710P + Intel 4port Gigabit network card.

i'm experimenting with 10Gbit network too, may be wanna to add Brocade CNA adapter too.. have two of them already if i add one of them to my setup, then i will need pfsense VM too..

 

so, i'm looking for larger board with more PCIe slots. it's hard to decide where to go. may be i will stay with used older X8 Dual Xeon bord cos of price.. any other advice welcome

 

It seems like you need a board with lots of PCIe slots for your hard drives not because you need more CPU power.  So yea obvioulsy a mini-ITX board is not a good choice for you.  I picked it because I'm trying to keep everything in one small package and don't need more than 4 HDD's (helps when each drive is 8TB ).

 

What is your current board/CPU combo?

[uldise]

so, i'm trying to setup my rig, at last

 

See bottom..

[ninthwalker]

so, i'm trying to setup my rig, at last

 

See bottom..

 

Hey, Sorry for not responding for awhile. Have been a bit busy with switching jobs and such right now.

It looks like some other folks answered some of your questions. Have you completed your setup and have it running the way you want?

 

Going back to your original post's questions, here are some of my answers:

 

How does one go about converting their physical unRAID v6 server into a VM?  Do I need to purchase a whole new registration key?  And how are you booting unRAID in a VM (passed through USB boot flash drive)?

A: Same Pro key or whatever you have is fine. I pass through the usb drive as that is where it looks for the key and plugins and such.

However, my bzroot/bzimage resides on a vmdk so that the server can boot a lot faster.

 

 

Do I have to pass through a storage controller (such as the M1015 I have in my current server) to the unRAID VM in order to get all smart and monitoring functions or are there any workarounds?  The reason I ask is that this board only has one PCIe slot so putting an HBA in there prevents any further expansion and leave my 6 on board SATA ports basically sitting there unused.

A: While I do use HBA controllers to passthrough 16 disks, I also have 6 sata ports. I currently use 2 of the sata ports. One for the SSD I use for the ESXI datastore. The other is my cache drive. You can map the sata ports with RDM to the unraid server to use if you want to connect them that way instead of using your M1015 if you want.

 

I've never combined my storage and VM server so what would you recommend as a good way to configure my SSDs/m.2 drives (datastore/cache) for the best performance of my VMs (most importantly unRAID and Plex)?

A: SSD's are definetly nice for the datastore for the VMs if you have them and they meet your space requirements.

I also choose to use an SSD for my cache, but you could also use a normal HDD. All my dockers and apps are located on my cache drive, and while i do like the speed I also don't think it would hinder Plex in any way to have it on a HDD.

 

I see you noted that you have an SSD cache and SSD datastore.  I assume the SSD datastore is where you have your VMs.  What do you use the cache for?

A: Cache drives is where my docker.img resides, for unraid cache'ing, as well as any other programs and config directories. I pretty much use dockers exclusively for my unraid apps now and so all their configs are on the cache.

 

If there's any other advice you could lend or best practices you could point to based on your own experience, I would greatly appreciate it.

A: I have actually just started using the new cache pool feature to raid my cache for redundancy and that is pretty neat. However I think it's important to still do backups of that. Would suck to have to redo all the configs and rescrape my plex DB as it is rather large. So I have an Rsync setup so every Sunday it syncs my cache programs to my unraid backup share. Which is then crashplanned.

 

unRAID is awesome, but don't forget that it isn't a backup solution if it gets stolen or a fire happens.

[JimPhreak]

so, i'm trying to setup my rig, at last

 

See bottom..

 

Hey, Sorry for not responding for awhile. Have been a bit busy with switching jobs and such right now.

It looks like some other folks answered some of your questions. Have you completed your setup and have it running the way you want?

 

Going back to your original post's questions, here are some of my answers:

 

How does one go about converting their physical unRAID v6 server into a VM?  Do I need to purchase a whole new registration key?  And how are you booting unRAID in a VM (passed through USB boot flash drive)?

A: Same Pro key or whatever you have is fine. I pass through the usb drive as that is where it looks for the key and plugins and such.

However, my bzroot/bzimage resides on a vmdk so that the server can boot a lot faster.

 

 

Do I have to pass through a storage controller (such as the M1015 I have in my current server) to the unRAID VM in order to get all smart and monitoring functions or are there any workarounds?  The reason I ask is that this board only has one PCIe slot so putting an HBA in there prevents any further expansion and leave my 6 on board SATA ports basically sitting there unused.

A: While I do use HBA controllers to passthrough 16 disks, I also have 6 sata ports. I currently use 2 of the sata ports. One for the SSD I use for the ESXI datastore. The other is my cache drive. You can map the sata ports with RDM to the unraid server to use if you want to connect them that way instead of using your M1015 if you want.

 

I've never combined my storage and VM server so what would you recommend as a good way to configure my SSDs/m.2 drives (datastore/cache) for the best performance of my VMs (most importantly unRAID and Plex)?

A: SSD's are definetly nice for the datastore for the VMs if you have them and they meet your space requirements.

I also choose to use an SSD for my cache, but you could also use a normal HDD. All my dockers and apps are located on my cache drive, and while i do like the speed I also don't think it would hinder Plex in any way to have it on a HDD.

 

I see you noted that you have an SSD cache and SSD datastore.  I assume the SSD datastore is where you have your VMs.  What do you use the cache for?

A: Cache drives is where my docker.img resides, for unraid cache'ing, as well as any other programs and config directories. I pretty much use dockers exclusively for my unraid apps now and so all their configs are on the cache.

 

If there's any other advice you could lend or best practices you could point to based on your own experience, I would greatly appreciate it.

A: I have actually just started using the new cache pool feature to raid my cache for redundancy and that is pretty neat. However I think it's important to still do backups of that. Would suck to have to redo all the configs and rescrape my plex DB as it is rather large. So I have an Rsync setup so every Sunday it syncs my cache programs to my unraid backup share. Which is then crashplanned.

 

unRAID is awesome, but don't forget that it isn't a backup solution if it gets stolen or a fire happens.

 

Thanks for checking in ninthwalker and certainly no need to apologize as those kinds of things that you have going on are much more important.  I appreciate the answers to my original questions too.

 

I do have my server all built now and was able to get unRAID setup pretty easily in a VM with regard to passing through the USB boot drive and creating the VMDK for faster booting. 

 

Unfortunately some of my hardware is so new (Xeon D-1500 platform) that I'm having some issues in regards to compatibility with VMware.  For example two of my four NICs are 10GbE and they don't work in VMware yet (can't find a VIB for them anywhere).  I'm also having some severe issues with regard to my USB flash drives.  Both USB drives I've tried (Cruzer Fit 2.0 and 3.0) perform so slowly with my server it's hard to even work on it.  Just doing something as simple as a change directory from the console/telnet can take 20-30 seconds and the same goes for the WebGui with regard to clicking on different links.  It's almost unbearable.  So much so that I'm going to soon test out some other virtualization options like Hyper-V (mostly just since no one seems to have done it yet so I can report on my results) and possibly a KVM solution like Proxmox to see if I have better results.

 

I was hoping the whole process would be pretty painless with regard to ESXi on this new build but these issues I'm having with unRAID coupled with a few other issues I'm having are making it hard for me to stay with VMware.

[garycase]

Sorry to hear about the issues with the D-1540 ... that looks like a VERY nice CPU with an amazing amount of power for a 45w TDP processor.

 

I suspect it's simply "growing pains" => the downside of being a very early adopter.  I suspect ESXi v6.x will resolve those ... but there's certainly no guarantee of that.

 

The only downside of that nifty little board is the single PCIe port => that's clearly a major limit in terms of what you can pass through to your VMs.    It'll be interesting to see if Hyper-V resolves this with its ability to use the native drives without passthrough ... although as you know (from our other discussion) this has its own potential issues (most notably no spindown).

 

In any event, that little MBD-X10SDV-TLN4F-O is definitely a SLICK board !!

 

 

[garycase]

... by the way, how much memory did you put in the new system?  I notice it supports 128GB of RDIMM, but I also know 32GB RDIMM modules are not exactly inexpensive      [~ $450 each]

 

 

[JimPhreak]

... by the way, how much memory did you put in the new system?  I notice it supports 128GB of RDIMM, but I also know 32GB RDIMM modules are not exactly inexpensive      [~ $450 each]

I bought one 32GB module for about $370 for now and plan to add a second later this summer.

[garycase]

It's amazing how much power you can get on a tiny little ITX board like that -- with up to 128GB of RAM !!

 

We're come a long way since I built my first Altair ~ 40 years ago [and this board plus 4 32GB RDIMM modules costs less than what I spent on that Altair => a LOT less if you factor in the current value of that many 1975 dollars !!]

 

[JimPhreak]

It's amazing how much power you can get on a tiny little ITX board like that -- with up to 128GB of RAM !!

 

We're come a long way since I built my first Altair ~ 40 years ago [and this board plus 4 32GB RDIMM modules costs less than what I spent on that Altair => a LOT less if you factor in the current value of that many 1975 dollars !!]

 

Haha, yea it's incredible.  8 cores, 16 vCPUs, 128GB of RAM, plus 32TB + assorted SSD's in a chassis that is honestly not that much bigger than my toaster oven.  I was amazed at how small it was when it arrived.

[garycase]

... I was amazed at how small it was when it arrived.

 

Yes, I've also become enamored of the mini-ITX form factor.  Have a couple of PC-Q25B setups for servers; and used a PC-Q08 for my wife's new PC.    I've also built a couple systems with the even smaller PC-Q01 case, which is really neat for a desktop that doesn't need more than 2 drives or an optical drive.

 

But I've been looking at the SuperMicro case you bought, that's REALLY nifty with it's 4 hot-swap bays.  I think that'll be my next case !!  (... and I'm also very intrigued with the X10SDV-TLN4F ... this can really make that tiny little case an amazing powerhouse)      I'll be following your experiences pretty closely as you work through the ESXi issues -- if those "go away" with the next version that'll make my choices a lot easier !!

 

By the way, are you using ESXi v5.5 or v6 ?

 

[JimPhreak]

... I was amazed at how small it was when it arrived.

 

Yes, I've also become enamored of the mini-ITX form factor.  Have a couple of PC-Q25B setups for servers; and used a PC-Q08 for my wife's new PC.    I've also built a couple systems with the even smaller PC-Q01 case, which is really neat for a desktop that doesn't need more than 2 drives or an optical drive.

 

But I've been looking at the SuperMicro case you bought, that's REALLY nifty with it's 4 hot-swap bays.  I think that'll be my next case !!  (... and I'm also very intrigued with the X10SDV-TLN4F ... this can really make that tiny little case an amazing powerhouse)      I'll be following your experiences pretty closely as you work through the ESXi issues -- if those "go away" with the next version that'll make my choices a lot easier !!

 

By the way, are you using ESXi v5.5 or v6 ?

 

I'm on ESXi 5.5.  I tried 6 briefly when I was troubleshooting getting my 10GbE NICs working but it made no difference so I went back to 5.5 because I have a Standard license for it which lets do host level backups with Veeam which I like a lot.

 

I will keep you updated with my progress though because even though I'm having some issues now, I know this is a fantastic setup for a home server and like you said, this case is quite amazing too with how much is packed into it.

[s.Oliver]

hey there!

 

trying my luck, as it seems some more experienced users here. the thread hasn't got any new replies within a few months, but maybe some new results in terms of virtualizing are cropped up?

 

well, i'm new to unRAID but i want to get unRAID 6.1.3 up as virtualized solution (kvm based) within Proxmox 4.0. Proxmox seems to be a pretty neat solution (and beside i have worked a little bit with ESXI) i somehow have a more warm feeling for Proxmox, it's licensee  and support. and as a bonus, it's a austrian (german language based company - so europe at least :-)

 

has anyone here tried Proxmox? how about virtio support in unRAID 6.x? (reading alot about unRAID 5.x kvm based images, with non official compiled kernels).

 

thx.a lot guys,

Blacky

 

[uldise]

hey there!

 

trying my luck, as it seems some more experienced users here. the thread hasn't got any new replies within a few months, but maybe some new results in terms of virtualizing are cropped up?

 

well, i'm new to unRAID but i want to get unRAID 6.1.3 up as virtualized solution (kvm based) within Proxmox 4.0. Proxmox seems to be a pretty neat solution (and beside i have worked a little bit with ESXI) i somehow have a more warm feeling for Proxmox, it's licensee  and support. and as a bonus, it's a austrian (german language based company - so europe at least :-)

 

has anyone here tried Proxmox? how about virtio support in unRAID 6.x? (reading alot about unRAID 5.x kvm based images, with non official compiled kernels).

 

thx.a lot guys,

Blacky

 

see this sticky post: http://lime-technology.com/forum/index.php?topic=31321.0

[s.Oliver]

thx. uldise.

 

found that post too. just wanted to be sure, that no one maybe had newer, more up-to-date informations or experiences.