ATLAS My Virtualized unRAID server

[Johnm]

 

Ok i decided to add another BR10i card and moved everything off the onboard sata ports and used those for ESXi datastores then i followed your instructions and I now have it all working great! I just finished setting up UPS backup scripts for shutdown per your instructions i just need to test a power outage in the morning to see if its working right and shuts down right.  I also setup a Win 7 Pro vm and have unRaid working fine thru passthru.

 

Thanks Johnm for all your help!!  Your detailed instructions worked great!!

 

Not a problem. Glad it helped.. I personally have used this thread to remember what I forgot...

 

 

[Johnm]

For those that might remember, I mentioned that my SAS Expander was acting up..

 

well I had yet another power failure today. this one lasted longer then my "ups uptime" so the server shutdown cleanly. the power was out so long that the battery ran out of juice. removing standby power to the board..

 

on power resume. the expander refused to power up.

the two events had nothing to do with each other. I think the next time i did full power off, I would have lost the expander anyways..

It should be under warranty. we'll find out tomorrow,

 

 

In my panic, i grabbed all my spare m1015's and started to reflash them into IT mode to fill the case up with HBA's to get buy until i got my expander replaced. bad news...

 

After flashing the 4th m1015 in a row, I must have accidentally hit the rest button instead of the power button, I saw the light on the mobo go off so i pulled the card, as i pulled it.. yup power came back on.. fried card. moral of the story.. unplug your server before swapping hardware.

I'm pretty sure this is not under warranty... this card is a server pull that i have forever. i would be under the servers warranty it came out of... i'll call it in tomorrow anyways just to see what IBM says. This mistake was my fault anyways..

 

anyways. Atlas is back up and running with 3x M1015's and a SASLP-MV8 until I can properly rebuild it.

[dheg]

Could I get some advice on what RAM to buy, I'm not an expert and getting lost with all the different specs (CL, ECC, unbuffered, etc). I am in Spain so I can't buy at newegg of monoprix.

 

The mb is a supermicro X9SCM-F-O. I have found these modules readily available:

 

KVR1333D3N9/8G

KVR1333D3D4R9S/8GHB

KVR1333D3E9SK2/16G

KVR13E9K2/16I

 

Any other I should look for? Any Spanish (European) forum members that could chip in?

 

[BetaQuasi]

KVR1333D3E9SK2/16G is the correct RAM for the X9SCM.  Hmm it's come down to a reasonable price too (I have 2 of the KVR1333D3E9SK2/8G kits in mine.. maybe it's time to upgrade!)

 

[dheg]

KVR1333D3E9SK2/16G is the correct RAM for the X9SCM.  Hmm it's come down to a reasonable price too (I have 2 of the KVR1333D3E9SK2/8G kits in mine.. maybe it's time to upgrade!)

thanks!

 

Sent from my GT-I9100 using Tapatalk 2

[Johnm]

KVR1333D3E9SK2/16G is the correct RAM for the X9SCM.  Hmm it's come down to a reasonable price too (I have 2 of the KVR1333D3E9SK2/8G kits in mine.. maybe it's time to upgrade!)

 

It is down to $154.99 for 16Gigs on superbiz http://www.superbiiz.com/detail.php?name=W16GE1333K

Thats about 1/2 the cost it was just 2 months ago.

 

I knew it was going to drop in price soon. but it happened quicker then I expected. I might have waited myself.

[shlomiassaf]

Hi guys.

 

This post ROCKS! it really does.

 

I am migrating my unraid sitting on a Intel celleron system to my new server build...

Was afraid to do so but this post really helped.

 

I am currently using an E3-1240v2 on a SuperMicro X9SCM-F.

 

I would like to buy the IBM M1015 but i`m facing a dillema, could you help?

 

Amazon sales this card at 188$

http://www.amazon.com/Serveraid-M1015-Sas-sata-Controller/dp/B0034DMSO6/ref=sr_1_1?ie=UTF8&qid=1342557485&sr=8-1&keywords=IBM+M1015

 

While ServerSupply.com sales it for 85$ (FREE SHIPPING)

http://www.serversupply.com/products/part_search/pid_lookup.asp?pid=134940

 

I`m not from the US but I can send it through service provided for this purposes.

This is a hugh difference... Add to that the import tax i`ll have to pay, which is a % of the cost.

 

Since i`m not from the US this price gap is scary, do you recommand ServerSupply?

 

Thanks, Shlomi

[dheg]

Maybe too late for me, but what are the pro/cons of the M1015 vs the AOC-SAS2LP-MV8?

[Johnm]

Hi guys.

 

This post ROCKS! it really does.

 

I am migrating my unraid sitting on a Intel celleron system to my new server build...

Was afraid to do so but this post really helped.

 

I am currently using an E3-1240v2 on a SuperMicro X9SCM-F.

 

I would like to buy the IBM M1015 but i`m facing a dillema, could you help?

 

Amazon sales this card at 188$

http://www.amazon.com/Serveraid-M1015-Sas-sata-Controller/dp/B0034DMSO6/ref=sr_1_1?ie=UTF8&qid=1342557485&sr=8-1&keywords=IBM+M1015

 

While ServerSupply.com sales it for 85$ (FREE SHIPPING)

http://www.serversupply.com/products/part_search/pid_lookup.asp?pid=134940

 

I`m not from the US but I can send it through service provided for this purposes.

This is a hugh difference... Add to that the import tax i`ll have to pay, which is a % of the cost.

 

Since i`m not from the US this price gap is scary, do you recommand ServerSupply?

 

Thanks, Shlomi

 

No problem. thats why it was posted..

 

I am not sure why server supply is selling so cheap. thats about what it costs on eBay.

they might be selling server pulls.

 

 

I would check this guy out http://lime-technology.com/forum/index.php?topic=20111.0

He might be a better deal. He might even flash it for you at that price. I know flashing it on the X9SCM is a bit tricky.

I have bought from him in the past myself.

 

 

 

[Johnm]

Maybe too late for me, but what are the pro/cons of the M1015 vs the AOC-SAS2LP-MV8?

 

Good question.. I dont have a definitive answer for you... BUT..

they are competing cards. they are pretty much the same on paper.

 

The biggest difference is one uses a marvel chipset and one uses an LSI.

 

In general the SAS2LP-MV8 should cost less for the consumer. But because so many M1015's have flooded the market from server pulls, you can get a better deal on ebay for an M1015 (it was even better a year or so back).

The SAS2LP-MV8 are still hard to get for some reason. many places they are special order.

 

The LSI chipset has large company support (Dell, IBM, Intel, ETC.)

Because of that, the LSI based cards are more recognized in the industry and has more support globally.

for example, the LSI cards work in ESXi natively.  (I honestly do not know of the SAS2LP-MV8 works in ESXi even with a hack. anyone?).

I prefer to stick to the LSI myself in case I want to re-purpose the card for say freenas or solaris/napp-it (and i do have both with M1015's).

The LSI might also have better Expander support since many expanders use the LSI chipset.

 

as long as the bugs are all worked in Version 5. there is no reason to use one over the other for baremetal unraid.

For other uses, a little google might help.

 

A side note, the SASLP-MV8, while did not have lots of support outside of unRAID, is a little workhorse. it will take all of the abuse i can toss at them and keep on working. I would not hesitate to buy another one if it was on sale for a good price.

Hopefully the SAS2LP-MV8 grows up to be the same way.

 

 

EDIT:

One big difference that i ran into. Warranty!

IBM pretty much told me yesterday, they they will not replace an M1015 without the serial number of the server it was pulled from or a copy of an invoice from IBM or a Certified retailer like CDW stating it was bought as an upgrade part for an IBM server. and they still wanted the server serial number.

 

The supermicro card is a retail card so it has an easy to replace warranty.

 

 

[dheg]

Maybe too late for me, but what are the pro/cons of the M1015 vs the AOC-SAS2LP-MV8?

 

Good question.. I dont have a definitive answer for you... BUT..

they are competing cards. they are pretty much the same on paper.

 

The biggest difference is one uses a marvel chipset and one uses an LSI.

 

In general the SAS2LP-MV8 should cost less for the consumer. But because so many M1015's have flooded the market from server pulls, you can get a better deal on ebay for an M1015 (it was even better a year or so back).

The SAS2LP-MV8 are still hard to get for some reason. many places they are special order.

 

The LSI chipset has large company support (Dell, IBM, Intel, ETC.)

Because of that, the LSI based cards are more recognized in the industry and has more support globally.

for example, the LSI cards work in ESXi natively.  (I honestly do not know of the SAS2LP-MV8 works in ESXi even with a hack. anyone?).

I prefer to stick to the LSI myself in case I want to re-purpose the card for say freenas or solaris/napp-it (and i do have both with M1015's).

The LSI might also have better Expander support since many expanders use the LSI chipset.

 

as long as the bugs are all worked in Version 5. there is no reason to use one over the other for baremetal unraid.

For other uses, a little google might help.

 

A side note, the SASLP-MV8, while did not have lots of support outside of unRAID, is a little workhorse. it will take all of the abuse i can toss at them and keep on working. I would not hesitate to buy another one if it was on sale for a good price.

Hopefully the SAS2LP-MV8 grows up to be the same way.

 

 

EDIT:

One big difference that i ran into. Warranty!

IBM pretty much told me yesterday, they they will not replace an M1015 without the serial number of the server it was pulled from or a copy of an invoice from IBM or a Certified retailer like CDW stating it was bought as an upgrade part for an IBM server. and they still wanted the server serial number.

 

The supermicro card is a retail card so it has an easy to replace warranty.

Thanks a lot johnm. I'll keep you posted with the performance of the SAS2LP, hopefully it works fine in esxi/unraid. Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

 

Sent from my GT-P7500 using Tapatalk 2

[Johnm]

Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

 

Sent from my GT-P7500 using Tapatalk 2

 

That all depends on what you put on it. I would put no more then 3 30GB disk images. you want to leave a good amount of overhead. that would be more then enough from normal 2008 or win7 images. (plus a small disk for the unraid boot if you go that route)

 

you could use thin provisioned drive images to save space but you would have some fun if you restored the drive images from backup or moved them around between datastores since it tends to expand them normally. also when you backup VM's, many  backup programs/scripts for ESXi start a temporary snapshot and that can be pretty big if the backup is slow.

[dheg]

Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

 

Sent from my GT-P7500 using Tapatalk 2

 

That all depends on what you put on it. I would put no more then 3 30GB disk images. you want to leave a good amount of overhead. that would be more then enough from normal 2008 or win7 images. (plus a small disk for the unraid boot if you go that route)

 

you could use thin provisioned drive images to save space but you would have some fun if you restored the drive images from backup or moved them around between datastores since it tends to expand them normally. also when you backup VM's, many  backup programs/scripts for ESXi start a temporary snapshot and that can be pretty big if the backup is slow.

Right now I'm thinking of a couple of win machines (win7 or win7 + xp),  unraid, and a linux distro. Mid term I'm thinking of pfsense and maybe mac (is this even posible?)

The reason for two win VMs is that my intention is to build a photoshop workstation on win7 64 (if a get a successful gpu passthrough).  The second win VM would be for common tasks (office,  ms money, etc)  and xbmc so my library is automatically updated from CP and SB

 

Sent from my GT-P7500 using Tapatalk 2

[Johnm]

I have gotten PS to work along with 3D max 2010 without a GPU. unfortunately the hotkeys dont seem work correctly. I mostely used the PS for dropplets and the MAX for rendering. not day to day production work.

 

XBMC (v12) does not like to run in a RDP window. I tried to do the same thing myself in the past. I cant remember if i gave up or got it working that was when atlas was first built. I lost that VM a long time ago.

 

as far as drive size. you can always use a spinner for non-priority VM's.

[BetaQuasi]

I was able to get XBMC to run in a window within a RDP session by doing the following.  Bit of mucking around, but I wanted it that way for the reasons mentioned:

 

http://forum.xbmc.org/showthread.php?tid=78068&pid=1115420#pid1115420

[dheg]

Thanks guys, at least i know that what I want to do is feasible!

 

Sent from my GT-I9100 using Tapatalk 2

[gbdesai]

Thanks a lot johnm. I'll keep you posted with the performance of the SAS2LP, hopefully it works fine in esxi/unraid. Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

Very interested in your findings, I ordered 3 x SAS2LP for a ESXi build before I saw that no one has tested it in such a config.  I'm hoping it works with the same hack as the SASLP...  Thanks!

[dheg]

Thanks a lot johnm. I'll keep you posted with the performance of the SAS2LP, hopefully it works fine in esxi/unraid. Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

Very interested in your findings, I ordered 3 x SAS2LP for a ESXi build before I saw that no one has tested it in such a config.  I'm hoping it works with the same hack as the SASLP...  Thanks!

I don't think I'll start before next week, pretty busy at work at the moment, but I'll keep you posted

 

Sent from my GT-I9100 using Tapatalk 2

[gbdesai]

Thanks a lot johnm. I'll keep you posted with the performance of the SAS2LP, hopefully it works fine in esxi/unraid. Still awaiting to receive all the parts though and to decide on the ssd datastore.  Would 128gb be enough?

 

Very interested in your findings, I ordered 3 x SAS2LP for a ESXi build before I saw that no one has tested it in such a config.  I'm hoping it works with the same hack as the SASLP...  Thanks!

I don't think I'll start before next week, pretty busy at work at the moment, but I'll keep you posted

 

Sent from my GT-I9100 using Tapatalk 2

 

I may get to it before you... perhaps this weekend.  I'll let you know if I do...

[shlomiassaf]

Hi,

 

Again, I cant help it, I must thank you guys!!!

I know you say you do it for the people, but I`m the 'people' and I must say THANK YOU!

 

If possiable, I would like to get your input about my setup.

 

My System:

MotherboardSupermicro MBD-X9SCM-F-O

CPUIntel XEON E3-1240v2

Memory16GB KVR1333D3E9SK2/8G (4x4GB)

ControllerIBM M1015

CaseFractal Design Define XL Full-Tower

PSUSeaSonic 760W Gold X-Series

Storage3x2TB WD Green (P+D), 2x1TB WD Green (Data), 1x1TB WD Black (Cache)

Storage1xOCZ Agile 3 120GB SSD

 

All spinners will be the slaves of Unraid...

The SSD will be the slave of the vSphare which is a horror movie

 

The M1015 will passthrough to Unraid while the MB`s controller will host the SSD.

 

The system will upgrade with time.

My first upgrade goals are:

[*]Upgrade to 32GB (4x8GB) of memorey.

[*]Add 3 2TB 7200 spinners for a datastore/s using ZFS RAIDz.

 

For now, the current stock will have to do...

This is my planned VM structure:

 

OSUsageMemory (GB)HD Capacity (GB)Physical HD

Linux

UnRaid Server

2

6

SSD 1

Windows Server 2008 R2 64BitDC, SqlServer,Misc460SSD 1

Windows 7 64BitDeveopment (Visual Studio)560SSD 1

LinuxOpenVPN12SSD1

 

This is what I think to do...

This leaves me with 4GB free ram for other Test systems which will run on demend.

 

Currently, all 24/7 OS`s will run on the SSD using Thin provisioning.

In the future I will move the 2008 R2 to the ZFS datastore and use the SSD for the development environment only...

I will not fill up the drive, the 2008R2 and Win7 will never reach 60GB Each.... not before i`ll move to ZFS anyway.

 

This is a work in progress, for now I`ll have to do with the current inventory.

 

I`ll appreciate any suggestion about this setup.

 

Thank you, again

 

Shlomi

[BetaQuasi]

Shlomi, that looks great - just note that your x9scm probably won't come with v2.0 BIOS, which is required for the v2 Xeons.  If you have an i3 lying around, you could use that - otherwise you'd need a cheap G630 (I think thats the chip) just to get the board to boot in order to flash it.

[BetaQuasi]

You must have altered your post as I was posting - with that many VMs, I'd be looking at a 240Gb SSD if you can swing it - Specifically a Corsair Performance Pro so you get TRIM-like performance.  I note you don't think you will fill it up, but keep in mind you want a bit of free space on the SSD in order to maintain good performance levels.  Up to you in the end I suppose!

[BetaQuasi]

Post spam! Turns out the x9scm-II's are showing up with the v2 BIOS so might want to go with one of those to be sure

[Johnm]

 

If possiable, I would like to get your input about my setup.

 

What Beta said..

I would look into possibly getting a Supermicro X9SCM-IIF (Updated X9SCM-F with 2 ESXI compatible NICs and V2 Bios fr IVY Bridge CPU's). otherwise you might have problems if you do not have a compatible sandy bridge CPU about.

 

I would also consider buying 2x 8GB stick instead of 4 4GB sticks. unless you can reuse that ram, it seems silly to throw it away.

 

EDIT and he sniped in before me..

I updated teh front page yesterday to show that I am recommending the X9SCM-iiF now

[BobPhoenix]

Shlomi, that looks great - just note that your x9scm probably won't come with v2.0 BIOS

If his doesn't I would be willing to trade.  I bought a replacement X9SCM 2 weeks ago from NewEgg and got a 2.0 bios.  Was hoping to get 1.0C (what I had on my original bricked board).  1.0C is compatible with an AVerMedia Duet tuner in passthrough 2.0 is not.  The card completely disappears from the ESXi passthrough screen.  Put any other PCIe card in and it shows up.  I currently have two HVR-2250s but the Duet is better with multi path signals so want one non-HVR-2250 in VM.