ATLAS My Virtualized unRAID server

[Johnm]

 

i'd like this card too, but besides not having the external port, there are two things stopping me

1 - seems to go eol in 4Q '12

Odd, It is a fairly new card. EOL really means nothing since it has no flashable ROM that needs updating. It should run forever. SAS3 will along soon. maybe that's why.

 

2 - i still dont understand how to make it support 24 drivers, when i have to use one of the ports to connect with an external hba

It runs 2 ways..

Method #1 using link aggregation for only 4 faster ports.

Plug in both ports from the HBA to the expander for 4 Fast SAS ports (16 drives total).

 

Method #2, using a single channel

You used a single port on from the HBA into the expander to split that port into 5 ports (20 drives).

You then use the other HBA channel for 4 Full speed SATAIII ports for the parity and cache.

This gives you a total of (20 slower drives and 4 fast drives)

 

 

thanks as always for your insights.

 

 

To get to your DAS, you can use some of these

http://www.addonics.com/products/dual_mini_sas_converter.php

Or

http://www.pc-pitstop.com/sas_cables_adapters/AD8788-2.asp

 

technically, you could leave your parity and cache drives in the head and the 20 drives in the DAS. save a hundred bucks and buy the Norco RPC-4220 instead.

[Johnm]

i've just found out about the intel RES2CV360 (webpage, pdf manual, provantage link)

 

it includes six cables, so the price (around $330) is a bit misleading.

 

i imagine screwing it down to the chassis, having a sff-8087 to sff-8088 cable run through an open slot and not even needing a motherboard on the das ... even after using one of the ports as an outbound port, it would still leave me with 8 inbound ports (32 drives) !

I have never seen the RES2CV360.. you could put it on some standoffs on your chassis bottom or sidewall.

Looks like a neat idea.

but for $300, I'd get the Chenbro 36port SAS2 with some cheapo monoprice sas cables and save a few bucks.

 

As far as the RES2CV260

Yes, it comes with $100 worth of cables.

I think I got mine for $180ish I have seen them for about $210 on sale. (see http://lime-technology.com/forum/index.php?topic=16769.msg153296#msg153296)

 

Be warned, there is also a RES2CV260NC that has no cables...

 

Bottom line though. for unraid. you only need to expand to 20 ports plus a parity/cache. the cheaper Intel should be everything you need.

 

36ports sounds nice... but until tom expands unraid drive limit.... do we need to go so big?

 

Not to mention. if we are running Virtual unRAID on ESXi with DAS boxes, I'd rather run 2x 20 drive arrays then a single 40 drive array.

something about putting all of your eggs in one basket sort of thing.

The more drives i add to my server. the more sluggish it is getting (not due to expanders, but just file searches).

[Johnm]

The Chenbro Expandershave their pluses and minuses.. the nice feature is the external port for a DAS box.

the down side is there are in and out plugs on the card. the one you linked has a misleading description, it can only connect 16 internal drives..

you need the 36 port version.

 

Just a correction to what I said. It looks like... you could use the external connectors to use all 5 internal ports as "outs" giving you 20 drives.

while it is not supported by Chenbro. it is supported by the LSI chipset. all ports can be in or out according to the guys at [H]ard. It also looks like dual linking is supported.

you would still be short a port for a 24Drive DAS.

 

Chenbro also sells a EUK kit to make a chassis (like the RPC-4224) into a DAS. it mounts their card into a tray with 1 or 2 expanders. that bolts to the motherboard area. it has a power led header and power switch header for the front of the case. this would allow you to power off the das without building a custom cable.

 

perhaps the Chenbro might be the way to go?

 

It looks like you cant really go wrong with either the intel or the chenbro. it is just about price vs number of ports vs your final build.

[jbrodriguez]

It looks like you cant really go wrong with either the intel or the chenbro. it is just about price vs number of ports vs your final build.

thanks Johnm.

 

i think i might get the chenbro ck23601 ... it has the external port missing in the intel card .. and, well .. here's hoping tom will expand drive support to 24 drives ... so both the RES2CV240 and chenbro ck22803 wouldn't work as they would only support 20 drives on the das.

[Johnm]

It looks like you cant really go wrong with either the intel or the chenbro. it is just about price vs number of ports vs your final build.

thanks Johnm.

 

i think i might get the chenbro ck23601 ... it has the external port missing in the intel card .. and, well .. here's hoping tom will expand drive support to 24 drives ... so both the RES2CV240 and chenbro ck22803 wouldn't work as they would only support 20 drives on the das.

you're overlooking you get 20+4 with the RES2CV240 or chenbro ck22803 and a 2 port HBA like an M1015. one port gets expanded to 5 ports and you still have the fast port.

 

on the other side of the coin, if the 36port chenbro truly supports dual linking... you could possibly get a faster rebuild time feeding both of your sas ports into it from the m1015,.it is not tested though. you might ask on [H] for a "tested" answer.

[Johnm]

I think it is time i finally did some cable management.

 

I decided to try my hand and making a custom power cable for Atlas..

 

Test cable..

 

 

I think Ill make 6 sets. 2 for each of my norcos.

[elkay14]

Great job on that cable -- nice and tidy!

[rhinoman]

Hi Johnm

 

Back in post 453 you said "the older Kingston SSDnow V+'s had this feature but are hard to find" referring to the "trim style" support.

 

Is the Kinston SSDNow V +200 with the second gen Sata3 Sandforce the correct one as I've found a deal on some 120gb ones?

 

thanks

[Johnm]

Hi Johnm

 

Back in post 453 you said "the older Kingston SSDnow V+'s had this feature but are hard to find" referring to the "trim style" support.

 

Is the Kinston SSDNow V +200 with the second gen Sata3 Sandforce the correct one as I've found a deal on some 120gb ones?

 

thanks

I don't believe any of the sandforce based SSD's have that feature. As far as I know, it was only on the older SATAII SSDnow V+ models. Those seem to be impossible to find now.

[rhinoman]

Ok thanks, glad I checked.

[dikkiedirk]

Hi Johnm

 

Back in post 453 you said "the older Kingston SSDnow V+'s had this feature but are hard to find" referring to the "trim style" support.

 

Is the Kinston SSDNow V +200 with the second gen Sata3 Sandforce the correct one as I've found a deal on some 120gb ones?

 

thanks

I don't believe any of the sandforce based SSD's have that feature. As far as I know, it was only on the older SATAII SSDnow V+ models. Those seem to be impossible to find now.

 

Would this one be ok?

Kingston 64GB SSDNow V+180 SATA2 128MB cache 1.8"  SVP180S2?

 

 

[bobmagnuson]

I think it is time i finally did some cable management.

 

I decided to try my hand and making a custom power cable for Atlas..

 

Test cable..

 

 

I think Ill make 6 sets. 2 for each of my norcos.

 

John, can you provide a little more info regarding your custom cable?  Wire gauge, connectors, etc.?

[dikkiedirk]

Sorry for hijacking this thread but did any of you try this method of flashing a M1015 on X9SCM?

 

http://lime-technology.com/forum/index.php?topic=20045.0

[Johnm]

 

John, can you provide a little more info regarding your custom cable?  Wire gauge, connectors, etc.?

 

can do. give me a few days. I am a little crazy busy.

 

Sorry for hijacking this thread but did any of you try this method of flashing a M1015 on X9SCM?

 

http://lime-technology.com/forum/index.php?topic=20045.0

not looked into it.

 

 

 

So My ESXi datastore spinners are getting to the end of life. they are starting to go bad.

I have been pounding on them 24x7 since day one. especially the usenet download target drive.

They were already old drives and close to death when i put them in the server.

no loss there. I got a year or so from what would have been sitting in a desk drawer.

 

I had always told myself i was going to build a "head" and put a hardware raid in the server for the spinner part.

With the raise of drive prices, i just sort of gave up on this.

 

Well.. now it is time to bite the bullet.. but i have an odd idea.. I am thinking about putting a Solaris guest on the server. Have it be the first VM to boot.

Put a 4 or 6 drive ZFS array on the guest and share it out NFS...

then put the datastores for my other guests on that VM's array..

• This should give me snapshot support from ZFS.
  
• protect against data corruption.
  
• save me the price of a raid card.
  
• it would also be a workaround for the painfully slow free ESXi datastore speed. (putting my datastore on NFS means i can back up much faster. and migrate VMs to another server quickly.)
   
  

 

I could then also pull my WHS drive, my cache drive and possibly put my unraid parity drive all on the ZFS array if the performance is good.

.. this should be quite a challenge... the question now, is it worth it? I could probably get a used HP 400i 512MB for $50... take an easy way out.

 

(There is something strange about a storage server hosted on a virtual storage server... )

[jbrodriguez]

(There is something strange about a storage server hosted on a virtual storage server... )

 

doesn't seem so strange, as per what some _other_ people are planning to do (look for the storage wars post)

[Johnm]

(There is something strange about a storage server hosted on a virtual storage server... )

 

doesn't seem so strange, as per what some _other_ people are planning to do (look for the storage wars post)

 

I assume that is your site by the fact it all links back to our other conversations.. 

nice write up.

You need to post pictures of your builds!!!

 

 

PS. You are my wallets nemesis! Every time I see your posts, I want to run out and buy a head for my ESXi server and make 2 of my Norco's into DAS boxes. with 3 storage arrays (fast RAID5 or RAID6 and 2 unRAID servers) in one big effen' server.

 

If you keep me on this path, it will be time to close this thread and make "ATLAS 2" the the ultimate storage server.. lol

[BetaQuasi]

I'm thinking of going the FreeNAS + ZFS + iSCSI target for my datastores.  Best of all worlds, and I don't have to bust my butt learning Solaris (already enough flavours of unix/linux in my head )  The FreeNAS VM will live on a Corsair Performance Pro SSD, and I'll probably just do a 4 disk ZFS array.

[Johnm]

I have thought about it.

 

I do have both an HP microserver and and a SuperMicro micro server I could use for external NFS or iSCSItargets. I have looked at freeNAS and napp-it along with win2008 as possible options.

I have also thought about getting a set of used fiberchannel cards from the junk pile at work and building a true SANs ..

 

I am undecided at the route to go right now. I do have, Solaris, Open Indiana/Napp-it and a freeNAS guest all installed as guests right now.

not to mention a win2008r2 with NFS and iSCSI target installed. 

 

I got a case of 10 drives delivered yesterday. I could build a few nas boxes and try them all at the same time on physical hardware first.

[c3]

 

Well.. now it is time to bite the bullet.. but i have an odd idea.. I am thinking about putting a Solaris guest on the server. Have it be the first VM to boot.

Put a 4 or 6 drive ZFS array on the guest and share it out NFS...

then put the datastores for my other guests on that VM's array..

 

(There is something strange about a storage server hosted on a virtual storage server... )

 

Yes.

 

I use 10 drive raidz2, SSD (10GB ZIL, rest L2ARC), compression, and replication too. Dedup seems too memory intensive for VM.

[dikkiedirk]

Is it possible to add an existing datastore disk which holds 2 VMs, as datastore to a newly installed ESXi server?

[brian89gp]

Is it possible to add an existing datastore disk which holds 2 VMs, as datastore to a newly installed ESXi server?

 

Yes, as long as you didn't somehow reformat it inbetween.  Go through the "Add Storage" wizard just like you would if you were creating a new datastore, somewhere along the line you will have an option to add existing VMFS datastores.

 

ESX 5.0 uses VMFS5 by default but can still use VMFS3.  ESX4.x uses VMFS3 and cannot read VMFS5.  Might come into play depending on which versions you were on and which one you are going to.

[dikkiedirk]

Is it possible to add an existing datastore disk which holds 2 VMs, as datastore to a newly installed ESXi server?

 

Yes, as long as you didn't somehow reformat it inbetween.  Go through the "Add Storage" wizard just like you would if you were creating a new datastore, somewhere along the line you will have an option to add existing VMFS datastores.

 

ESX 5.0 uses VMFS5 by default but can still use VMFS3.  ESX4.x uses VMFS3 and cannot read VMFS5.  Might come into play depending on which versions you were on and which one you are going to.

This is safe for my existing VMs? The disk doesn't get formatted?

 

 

[dikkiedirk]

Is it possible to add an existing datastore disk which holds 2 VMs, as datastore to a newly installed ESXi server?

 

Yes, as long as you didn't somehow reformat it inbetween.  Go through the "Add Storage" wizard just like you would if you were creating a new datastore, somewhere along the line you will have an option to add existing VMFS datastores.

 

ESX 5.0 uses VMFS5 by default but can still use VMFS3.  ESX4.x uses VMFS3 and cannot read VMFS5.  Might come into play depending on which versions you were on and which one you are going to.

 

How do I add the VMs to the inventory. In datastore browser I can see the vm folders but add to inventory is greyed out.

 

Both VMs are back! Thank to everyone wh helped and put me on the right track.

[Johnm]

in general, ESXi is very light. It is very quick and easy to reinstall.

 

It is also very easy to move datastores around or re-import them.

 

just be aware of the limits when moving from one version to another.

[dikkiedirk]

in general, ESXi is very light. It is very quick and easy to reinstall.

 

It is also very easy to move datastores around or re-import them.

 

just be aware of the limits when moving from one version to another.

 

Yes, you're right. I reinstalled and am on 5.0 Update 1 now. I also installed Chilly's vib for the additional NIC, which now shows up in Vsphere Client. After enabling passthrough on the 2 LSI 2008's the now show up in my unraid and Windows 7 installed drivers for the (something with Falcon). I had to ad 2 new PCI devices for that.