Looking for a SAS card working with ESXi5 and Unraid

[Jerry1111]

I'm looking for a PCIe SAS card with 2x SFF-8087 ports (for 8 SATA drives, with a small probability of using an expander in future). Currently I'm using all the ports on the motherboard and its time to grow the array (and purchase a Pro license as well). The setup runs ESXi5 on a Z68A-GD65 (so PCIe2.0, not PCIe3.0) with several VMs, one of which is Unraid 5.0-rc6-r8168-test.

 

I would like to pass-through the new SAS card directly to Unraid. For performance reasons with eventual expander I would prefer that PCI bandwidth is bigger on equal than the SATA bandwidth (so I don't want to have issues with saturated PCI bus). This rules out the IBM M1015 card. The question is - what else? After a long evening on lime-tech formus all I know for sure is that AOC-SAS2LP-MV8 hasn't been (yet) proven as working with ESXi5/unraid/passthrough combination.

 

3TB+ disk support is a must. There is a probability that as I need more drives I'll decide to get a SAS expander (my motherboard has only 2 PCIe x16 slots and 3 PCIe x1 slots), which is the main reason not to use M1015 or SASLP-MV8.

 

Money... yeah, if I spend too much then my wife will have me sleeping with the server ;-(

 

[dikkiedirk]

Why does this rule out the M1015? Many use them in ESXi and unraid. Some in combination with an expander. So, lots of user-experience here.

[Ford Prefect]

Hmm.....

 

...maybe I am wrong, but as per design of unRAID architecture, the worst scenario

from a bandwidth POV that you can expect, is that you'll be needing bandwidth for

3 disks (1x Parity, 1x Data and 1x Cache) at maximum....most likely you'll only hit two disks at max.

This will not get you anywhere near the saturation of a PCIe-v2 x8 interface (which is maxed at 8Gbps, isn't it?).

 

There are people running large arrays in ZFS or other software raid solutions in ESXi guests

and have no issues with other VMs jumping on the bus with more cards on passtrough as well.

 

IMHO this is a theoretical discussion of technical features.

I'd rather compare some real live scenarios with my requirements and judge after them.

[Jerry1111]

Why does this rule out the M1015? Many use them in ESXi and unraid. Some in combination with an expander. So, lots of user-experience here.

 

Thanks for pointing it out - it doesn't!

When I was comparing all the possible cards, I've noted down M1015 as having only 4 PCIe lanes - my mistake. From this point of view both cards are equal. I understand that issues with spinup/spindown were fixed in 5.0-rc6-r8168-test2, so there should be no obvious differences between those cards.

 

Hmm.....

 

...maybe I am wrong, but as per design of unRAID architecture, the worst scenario

from a bandwidth POV that you can expect, is that you'll be needing bandwidth for

3 disks (1x Parity, 1x Data and 1x Cache) at maximum....most likely you'll only hit two disks at max.

This will not get you anywhere near the saturation of a PCIe-v2 x8 interface (which is maxed at 8Gbps, isn't it?).

 

I'm assuming worst case scenario of 3 reads (video streams) and 1 write (me copying a large file) at the same time. Parity sync/check... well, if for the same money (prices of both cards are roughly equal in the UK)  it can happen faster then it's worth thinking about. Of course all of this is theoretical, I've made a mistake of assuming that M1015 has only 4 lanes.

 

I have also a plan for a 'backup-like' script which would rsync most important folders to a drive outside of the array, so at least some of the data is protected from 2 drive failures. This could be happening overnight, so shouldn't be included in the worst-case scenario.

 

Thanks again guys for pointing it out!

 

[Ford Prefect]

I'm assuming worst case scenario of 3 reads (video streams) and 1 write (me copying a large file) at the same time.

 

My own, real life scenario with a single M1015 in an ESXi build is as follows....

 

Set-up:

VM with a ZFS based NAS (6 disks, 3x 2-way mirror, M1015 in passtrough)

VM with Win7_64 and a Quad-DVB-S2 card in passthrough

 

...this easily allows me to:

 

- stream two Blu-ray rips to individual clients (xbmc)

- while, at the same time, recording of 6 HD channels from DVB-S2

...and the box doesn't even start to sweat