I'm putting together a new build based on the Norco 4224 running unRaid in a VM and a bunch of other applications running on their own VMs. I plan to have all the unRaid drives on the 2xM1015s / 1xRES2SV240 and the ESXI Datastores & cache drive on the motherboard controller (using a reverse breakout to the top drives of the 4224).

 

The question I have is what gives me the best throughput - putting the parity drive on the mb controller on a Sata II or putting it on the M1015s.

 

From the research I have done I believe that using the mb controller gives me a gain due to parity writes happen on a different controller than the actual data writes, and I don't think Sata2 will hold it back.

 

What do you experts believe is the best config to remove bottleneck?

 

Server Part List

Case: Norco 4224

CPU: Intel Xeon E3-1240 (Sandy Bridge)

MB: Supermicro X9SCM-IIF

RAM: 4 x Hynix DDR3-1333 8GB ECC CL9 (on memory list for the X9SCM-IIF)

NIC: Intel Gigabit ET2 Quad (4) Port Server Adapter Card E1G44ET2

PSU: SeaSonic X Series X-850 850W 80 PLUS GOLD

HBA1: IBM M1015 (Port 0 - cascade to RES2SV240; Port 1 - 4X SAS2 ports)

HBA2: Intel RES2SV240 SAS Expander (Ports 0&1 - cascade from M1015s, Ports 2-5 - 16X SAS2 ports)

HBA3 : IBM M1015 (Port 0 - cascade to RES2SV240; Port 1 - 4xSAS2 ports not used)

Datastores:

• Datastore1 - Samsung 840 500GB (VM hosts/guests, ISOs); controller: motherboard Sata III
  
• Datastore2 - Western Digital Black 2TB (storage for VMs); controller: motherboard Sata II
  

unRaid Array disks:

• Cache - Western Digital Black 2TB; controller: M1015 (HBA1)
  
• Parity - Western Digital Black 4TB; controller: M1015 (HBA1)
  

Data:

• 8x Western Digital Red 3TB; controller: RES2SV240 (HBA2)
  
• 5x Western Digital Red 2TB; controller: RES2SV240 (HBA2)
  
• 5x Seagate Barricuda 7200.14 3TB; controller: RES2SV240 (HBA2)
  

 

*** Edited May 5 to record final lineup

You're not going to see a benefit running the parity drive on the motherboard SATA controller. In fact, you'd have to RDM the parity drive through to the unraid VM which I would consider a net negative.

+1 for keeping Parity inside the VM...do not run it on RDM.

 

As for the rest of your setup...

I am not sure if you would get a noticeable performance gain with unRAID from connecting port0-1 of the expander to the first HBA.

AFAIK this kind of setup is intended for multipathing on the expander, serving a purpose for redundancy.

With having 2HBAs at your disposal, I'd try (not sure if unRAID (the driver) will support it) to connect port-0 of HBA1 and port-0 of HBA3 to your expander.

Then connect each port-1 of your HBAs to a backplane directly.

 

For performance with Parity, it should be located on a direct port of one of the HBAs

On the other hand...for redundancy, if one HBA fails...having Parity on a port driven though the expander might be a bonus.

 

...and speaking of your expander...it has a FW too, see this thread on how to take care of that: http://lime-technology.com/forum/index.php?topic=25412.msg221179#msg221179

Thanks for the feedback. I'll keep the parity in the array on the HBA's.

 

Should I move the cache as well?

Should I move the cache as well?

 

Yes, RDM will slow it down and with the way you plan your setup you have the physical ports to spare anyway

My understanding of ESXi is that you can't pass-through a motherboard SATA port, so it's really not an option to have parity on the motherboard unless you do as mrow noted, which would significantly slow down the drive access.

 

Based on the performance others have noted in virtualized servers, however, I think you'll get very acceptable performance with the M1015 ports and expander ... not quite "native" speeds, but plenty fast enough for typical UnRAID use.

 

... and the isolation you'll get for your other applications by putting them in separate VMs should make the UnRAID server much more reliable.

 

My understanding of ESXi is that you can't pass-through a motherboard SATA port, so it's really not an option to have parity on the motherboard unless you do as mrow noted, which would significantly slow down the drive access.

 

Based on the performance others have noted in virtualized servers, however, I think you'll get very acceptable performance with the M1015 ports and expander ... not quite "native" speeds, but plenty fast enough for typical UnRAID use.

 

... and the isolation you'll get for your other applications by putting them in separate VMs should make the UnRAID server much more reliable.

 

My understanding is any disks connected the the M1015 and the expander card should see little slow down as the M1015 will act similar to a network switch, keeping data traffic between disks on the card. Disks connected to another controller card may seem a little slower but unlikely. Since the M1015s are PCIe 2.0 8x cards you've got 4GB/s of throughput on each 8x slot.

I agree -- the M1015 with an expander will provide very good throughput for all normal activity.    When I said "... not quite "native" speeds ..." I was referring to UnRAID running "native" on the hardware vs. running in an ESXi VM.

 

Thanks guys. I appreciate hearing some of the science behind the recommendation.