ATLAS My Virtualized unRAID server

[kman321]

Would this memory work with the X9SCM? (since the other memory is hard to come by)

 

http://www.superbiiz.com/detail.php?name=W1333EB8GS

 

[Johnm]

Would this memory work with the X9SCM? (since the other memory is hard to come by)

 

http://www.superbiiz.com/detail.php?name=W1333EB8GS

 

the W1333EB8GS and the W1333EB8GH are the same memory with the exception that the chips are from

samsung instead of Hynix.

 

It should work fine.

 

those are 2 of 3 companies that can actually maunfacture those specific chips at the moment. (the 3rd escapes me)

 

I ordered the hynix chip flavor myself. They are currently on backorder. I' should see if I can switch my order to those.

 

 

 

[kman321]

yeah, i "requested a quote" on the Hynix stuff and they came back with the following response:

 

"I just checked in with  my purchasing manager and unfortunately there is no ETA. You may be interested in W1333EB8GS, which is the same specs, but just made with a Samsung chip"

[Johnm]

For those that want an SSD for ESXi or baremetal unRAID. I am suggesting you take a look at the Plextor M3 Series or the Corsair Performance Pro Series!

 

They have 'advanced garbage collection' modes similar to "auto trim" and cleaning features to prolong disk life and 'trim like' write speeds when other SSD's in ESXi would be "dirty".

 

the older Kingston SSDnow V+'s had this ffeature but are hard to find, overpriced and are pretty slow by todays SDD standards..

Now we have powerhouse SSD'd that can perform this. Enter the Marvell 88SS9174 based SSD's.

 

This what is happening to our SSD's:

The "30min idle cleaning bar" is what we need for ESXi, unRAID, *nix, and OSX.

 

Take a look at this article > http://www.xbitlabs.com/articles/storage/display/marvell-ssd_7.html

Ilya Gavrichenkov did an excelent writeup of what the new Marvel based SSD's are doing and why they would be better for our use.

 

 

 

These features also make them good for Linux desktops or macs. They also work in raid arrays.

 

I am now deciding on the corsair or the plextor myself... I think I'm leaning to the corsair..

the plextors 5year warranty is the big decision..

 

Corsair Performance Pro Series CSSD-P256GBP-BK 2.5" 256GB SATA III MLC Internal Solid State Drive (SSD)

 

Plextor M3 Series PX-256M3 2.5" 256GB SATA III Internal Solid State Drive (SSD)

 

 

Both are priced at $339 for a 256GBSSD. That's a better deal then 2 OCX 128's in the price per GB that I built Atlas on originally.

(the Plextor also has a "Pro Model" for about $40 more... that is slightly better suited for servers and has slightly higher IOPS. )

 

 

Edit:

I Edited the first post to reflect my updated recommendations and post links.

I think I'll be ordering 2 of the Corsair for my server, maybe one for my desktop.

[ill just put the 3 SSDs that I'm taking out of Atlas into my Desktop/laptop's. those are windows and have trim.]

[BetaQuasi]

Thanks for the tip John - I'd purchased a 256Gb Corsair Force 3 for my build (still waiting on Supermicro parts) as it claimed to have Background Garbage Collection, but that article certainly illustrates the advantage of the Performance Pro model clearly!  I've just ordered one myself for use as an ESXi datastore, and will use the Force 3 in my laptop, which I've been meaning to stick a SSD in for some time!

 

 

[Johnm]

Glad it helped...

 

It not only shows a major speed improvement and even blowing away my existing OCZ's, it shows that it will improve the lifespan of the SSD. (or so i hope)

 

The first thing I did when I read this article was curse up a storm thinking about how much I spent on my OCZ's (that have preformed like champions so far) and knowing that I'd be silly not to upgrade those to newer SSD's...

 

Then I realized that I'm not really wasting any money, the new SSD'd were not available at the time of build for me. I knew the OCZ's were "throw away" when i bought them. I was thinking I would burn them out and then pitch them in the trash when Bigger, Better SSD's were available. Seeing as I was about to buy new SSD's for my laptop and desktop. i can just reuse the OCZ's for those PC's since they have TRIM support (although they will still be much slower then the corsair SSDs LoL). I'll actually be saving money this way because i'll be able to reuse the OCZ's if you look at it that way. ( I can also report how badly I burned them out in the last 8 months of 24x7 use  for those people that are using SSD's for datastore use.)

 

I bit the bullet and ordered 2 of the Performance Pro models for datastore drives. I was thinking about a 3rd one for my cache drive, but that price point is a bit crazy for a cache drive. maybe I can test one for a cache drive for a bit and report the results.

[Witchdoc]

John,

 

Do you plan on running your new ssd's as raid1 for your datastore or are you going to raid0 them?

[Johnm]

good question,

 

I had not really thought about that yet. They perform so fast I do not need to raid0 them, but that is tempting with this particular drive. it would nice to see a if they can hit 900+MB/s (not that I need that!)

 

Most likely I wont RAID1 them. it would be much cheaper to back them up to a spinner or iSCSI target instead of a mirroring such an expensive drive. 

 

3-4 in raid 5 would be something i would try if i had unlimited budget. but that is just a dream...

 

I do plan to update my build specs and photos when I do the next upgrade.  I'll post what and how i did.

 

After I get my RAM upgrade, the last thing this server will need for he next year or so are data storage drives. I need 1 more for the WHS2011 build and 6? more 3TB drives to fill in all 20? I'll also need a mod or 2 to hold the datastore drives.

[savestheday]

John I kind of called you out on the Beta 14 thread someone posted a beta 14 compile with fixed LSI support and knowing that you had some test boxes, I said I was waiting to see what you said before trying it!

[Johnm]

John I kind of called you out on the Beta 14 thread someone posted a beta 14 compile with fixed LSI support and knowing that you had some test boxes, I said I was waiting to see what you said before trying it!

 

Found it and looked.... my response..

 

I will wait for reports from JohnM because I'm sure he's testing this out on his LSI's!

 

Actually i have been absent from the boards for the last week. We had a major disaster at work and a runaway SCCM script from corporate formatted about 100 of the systems at our location overnight. I have been doing damage control.. ugh.

 

User created updates... It looks very promising...

 

I do not think I'll have time to test this for a while. but it sounds like a like a needed upgrade!

I do have a spare m1015 and 4 or 5 120gb SSD's... maybe i'll build RAJ's dream ssd server next week..

the problem is, those dont sleep to trip the bug.. I might have some older 160GB-500GB drives in a box and build a test unRAID vm.

 

Honestly,

I don't think I want to put my production box on the line right now. I have not backed it up in a few weeks(bad bad bad). i need to get more drives for my backup server first. I totally filled it up the backup server. I am trying to bargain hunt some 3TB or 4TB drives. I am done with 2TB drives (Yes i keep saying that, i need to hold my stance).

 

The next problem is, I don't have a testing license. I''ll have to build it on the free version. IMO, that should be fine. I just cant test a cache drive.

 

Thanks Peter_sm for the update

[savestheday]

I know you're usually first to test out LSI stuff, was hoping you were subscribed to the thread. 

 

Off topic

I've had those issues working in IT.  Working 18 hour days when crap goes wrong.  Sucks!  How the heck did SCCM run that?  Were they in a collection that had an OSD assigned to it?

[Johnm]

I know you're usually first to test out LSI stuff, was hoping you were subscribed to the thread. 

 

Off topic

I've had those issues working in IT.  Working 18 hour days when crap goes wrong.  Sucks!  How the heck did SCCM run that?  Were they in a collection that had an OSD assigned to it?

 

Off Topic:

It was all the old XP stations that were ghosted back about 3-5 years ago. they all had the same GUID in the SCCM Client and the SCCM console did not show any of them as dupes.

 

We have new hardware on order to replace those boxes.. I guess someone used that image on a conference room PC  that was running on hardware that was not being replaced. so corporate pushed a windows 7 refresh image to it and it took all the workstations that were on vintage hardware along with it. stuff that was to old run windows 7.. oh well..

 

On Topic: If I can find 3 POS drives laying about, I "might" have time in the next few days to test the custom beta. I just need to find a HDD sale ASAP.. I am running low.

[znelbok]

Qucik question if I may on passthrough.

 

Will any card work (lets say a generic off the shelf NIC) and be able to be passed through to the guest, or does it have to be a PCIe card that is supported by ESXi before it can be passed through.

 

Thanks

 

Mick

[c3]

Qucik question if I may on passthrough.

 

Will any card work (lets say a generic off the shelf NIC) and be able to be passed through to the guest, or does it have to be a PCIe card that is supported by ESXi before it can be passed through.

 

Thanks

 

Mick

 

While most cards work with passthrough, not all do. The ones I have seen people have trouble with are video and TV cards. Most storage and NICs work.

[c3]

I should have posted an update!  I've got 16GB of the Hynix stuff running alongside the Kingston 8GB. All has been running perfect for 2-3 weeks now. You should be fine John. Tempted to pick up another 16GB at that price.

 

Good to know..

 

then I got this today..

 

We are sorry to inform you that the item you ordered, ( Super Talent Server Memory, Part # W1333EB8GH), is currently out of stock, but the good news is that we are expecting to receive new shipments in 10 days. I apologize for the delay.

Oh well.

Maybe it will be cheaper in 2 weeks and I get a price reduction.

 

Kingston (2)x8GB is in stock at Buy.com for $224.

[Johnm]

I should have posted an update!  I've got 16GB of the Hynix stuff running alongside the Kingston 8GB. All has been running perfect for 2-3 weeks now. You should be fine John. Tempted to pick up another 16GB at that price.

 

Good to know..

 

then I got this today..

 

We are sorry to inform you that the item you ordered, ( Super Talent Server Memory, Part # W1333EB8GH), is currently out of stock, but the good news is that we are expecting to receive new shipments in 10 days. I apologize for the delay.

Oh well.

Maybe it will be cheaper in 2 weeks and I get a price reduction.

 

Kingston (2)x8GB is in stock at Buy.com for $224.

 

A day late and a dollar short...

That's great info.. for everyone else..

I am glad it is "somewhat" affordable... (yeah right.. thats still expensive)

Unfortunately for me, I already picked up the Samsung memory on Monday for about the same price.

Personally Id rather have the Kingston.

Thanks for the Tip.

 

This will be a busy weekend for me.

Atlas will get an overhaul.

32 GIGs of ram,

2 new Corsair SSD's

I have to do some cable management.

Mount some drives internally.

Add a 2TB drive to the WHS guest for backup.

I decided to test out some of the new Seagate ST3000DM001 3TB 7200 RPM drives in the unraid guest.

[The Hitachi 3TB 7200 RPM Parity drive I put in a few weeks ago has redballed (or was it orangeballed?) on me twice now... it says it is 52TB (I Wish) in size. It is self healing after a reboot.  it just had to recreate parity.]

It will probably end up being a clean ESXi reinstall and then reimporting all of the existing guests.

 

I am also considering an unRAID server swap. (Swapping Goliath and Spartacus)

Basicly downgrading the unRAID guest to all 2TB drives and move all of the 3TB drives to the "Physical unRAID back-up server". To me, it makes more sense that the larger of the 2 servers is the backup...

[babaloui]

Thank you for this thread Johnme and all others who contributed. 

 

Followed your guide on ESXI and all went smooth. To make it even easier you could include the guide for making a vmdk of the usb stick and a link to vmtools, first thing I did and made everything very easy.

 

Im currently rocking this hardware:

Supermicro motherboard X9SCM-F

Intel Xeon Quad-Core E3-1240 3.3GHz 8MB, 80W, LGA1155

SM 8GB DDR3 1333MHz ECC/Unbuffered (MEM-DR380L-SL01-EU13)

IBM ServeRAID M1015 LSI9220-8i 46M0861 (IT Mode)

 

So Im running 3 VMs, UNRaid, WinXP and a 2008 Server.

How would you or anyone else recommend the resource allocation?

Sorry Im new to ESXI.

 

/Robert

[Johnm]

New toys for Atlas.

 

SUSHI!!

 

 

 

OK.. Not Sushi.

32GB of Genuine Samsung Server RAM

This stuff is re-branded as Super-Talent, I peeled off the (still soft) super-talent sticker to find the Samsung Sticker under it.

This is the same stuff that came in our Intel ESXi servers at work (That we paid about 3x this price for). It is probably a better quality then the Kingston Value RAM ..

It is also on the certified RAM list on Supermico's X9SCM Page.

[supermicro is selling the same RAM as MEM-DR380L-SL01-EU13 for 3 times the price. So is HP as a different Part#]

This box of RAM still has the new PCB ozone killer smell to it.

I got this stuff for about $115 a stick. Don't pay much more then that. the price is about to fall on this stuff now that it is in mass production. I expect it to be like like $70-$90 in a few months. (The problem is only 3 companies can make this stuff. Samsung, Hynix and Micron. even companies like kingston has to buy the chips from them.)

 

 

 

---

 

NEW DATASTORES:

Two new Corsair Performance Pro 256GB SSD's.

See here for my reason to switch Datastore drives.

 

 

Nice Brushed Aluminum Shell.

 

 

New Storage Drives?:

I also picked up 3x "Seagate Barracuda 7200 3 TB 7200RPM SATA 6 Gb/s NCQ 64MB Cache 3.5-Inch Internal Bare Drive ST3000DM001" drives

My first non-Hitachi drives in my pool.

these should be very fast with 3x 1TB platters in them.

I will probably assign one as Parity and 2 as data drives. downgrading my Hitachi 3TB 7200 Parity drive to data duty.

I'll have to double check, but i think that gives me 16x 3TB drives in the unraid guest totaling 45 TB usable space..

 

 

This should be the last of the purchases for Atlas for a while.

I'll have to take some time out and update the parts list of the final build and post photos of it.

 

Enough typing.. time to tear down the server and start the rebuild..

 

 

 

[Johnm]

Thank you for this thread Johnme and all others who contributed. 

 

You are very welcome. This was created not just for people here to use as a base, but for others to post better ideas or ways to do this.

 

Followed your guide on ESXI and all went smooth. To make it even easier you could include the guide for making a vmdk of the usb stick and a link to vmtools, first thing I did and made everything very easy.

 

Both are in the thread, lost in the clutter. I swore they were linked in the "basic instructions". Thanks for pointing that out, I'll update it.

 

Im currently rocking this hardware:

Supermicro motherboard X9SCM-F

Intel Xeon Quad-Core E3-1240 3.3GHz 8MB, 80W, LGA1155

SM 8GB DDR3 1333MHz ECC/Unbuffered (MEM-DR380L-SL01-EU13)

IBM ServeRAID M1015 LSI9220-8i 46M0861 (IT Mode)

 

Nice Build. I hope you did not pay to much for that RAM. I just looked up that part number to see it is Samsung ram.

 

So Im running 3 VMs, UNRaid, WinXP and a 2008 Server.

How would you or anyone else recommend the resource allocation?

Sorry Im new to ESXI.

 

Thats OK, I was new to ESXi when I created this... now I am just 1 step up from noob..

Honestly, It depends on your needs...

This will vary for every user/server

I assigned 1 core and 4 GIGs of RAM to unRAID (I think I need more RAM now that i have millions of files on my server in "cachedirs" although this might not help. (one reason for my new RAM upgrade today)

I would assign 2cores and 2-gigs RAM to basic Windows servers and workstations. maybe less for simple Servers/Workstations.

If you have heavy use workstations for say media encoding, I would play around with 2-4 cores and 4 or more gigs of RAM.

Then see how those VM runs and see how the rest of your server reacts.

adjust as needed.

 

This is just a basic suggestion and not a rule. this is also assuming you have an 8 thread CPU and lots of RAM.

If you only have 8 gigs total, I would start with less RAM per VM and adjust.

the two biggest bottlenecks in your build will be RAM and disk IO.

/Robert

[babaloui]

Thanx.

 

Paid around 270 USD for the RAM which i think is not half bad considering it's Norway.

Will be getting SSD for datastore and 8GB more RAM and some of the new 3TB Seagates for data, when back from 6 weeks work trip in Asia.

 

Also have SAB and SickBeard setup in a XP VM, i have set the complete folder to the unc path of my tv shows directory on the UNRAID box, wondering if this is the best way to do it?

 

/Robert

[dikkiedirk]

John,

 

Is this the right memory:

 

http://www.ebay.nl/itm/SAMSUNG-8GB-2RX4-PC3-10600R-REGISTERED-ECC-DDR3-1333-M393B1K70CH0-CH9Q4-/190669021781?pt=LH_DefaultDomain_0&hash=item2c64c26255

 

 

[Johnm]

John,

 

Is this the right memory:

 

http://www.ebay.nl/itm/SAMSUNG-8GB-2RX4-PC3-10600R-REGISTERED-ECC-DDR3-1333-M393B1K70CH0-CH9Q4-/190669021781?pt=LH_DefaultDomain_0&hash=item2c64c26255

NO.

 

That is registered memory. that will not work.

[dikkiedirk]

John,

 

Is this the right memory:

 

http://www.ebay.nl/itm/SAMSUNG-8GB-2RX4-PC3-10600R-REGISTERED-ECC-DDR3-1333-M393B1K70CH0-CH9Q4-/190669021781?pt=LH_DefaultDomain_0&hash=item2c64c26255

 

Thanks,

 

Glad I asked .

NO.

 

That is registered memory. that will not work.

[Johnm]

@dikkiedirk

Yeah, you will need UDIMMS or unregistered ECC.

I think those would work in an X8 series (X8SIL for example). not in an X9 series.

 

 

@rest

Still doing the upgrade...

 

I was hoping to have been done in an hour or two.. unfortunately i ran into a few issues migrating the data stores..

 

I had to power the server back up. I could not leave it down for to long. I had people trying to get onto the webserver..

 

My two biggest problems was i somehow managed to fill one SSD with hidden files.. i had to  shove it into a windows box and clean it. then try the migration again.

 

The other issues was that all of my old VDMK's were thin provisioned. when I moved them, they converted to thick and filled up my SSD's. Now i have to do some moving to get it all balanced.

 

The RAM however was transparent.  I just stuck it in and it saw 32GiB.

I also noticed that after i did that, my VM consoles got faster. they used to lag bad under 7 or 2008.

I hardly ever use that feature due to lag. I tend to use RDP.

 

I really should have just formatted my ESXi flashdrive and started all over. deleting and migrating a few  VM's if fine.  but 25+ VM's is a pain. At least all of these SSD's are fast.

 

Edit...

I just noticed I need a SAS 36pin (SFF-8087) Male to SAS 32pin (SFF-8484) Female Cable and a SAS 36pin (SFF-8087) Male Forward Breakout Cable...

I guess Ill finish the upgrade next weekend..

[c3]

Thank you for this thread Johnme and all others who contributed. 

 

Followed your guide on ESXI and all went smooth. To make it even easier you could include the guide for making a vmdk of the usb stick and a link to vmtools, first thing I did and made everything very easy.

 

Im currently rocking this hardware:

Supermicro motherboard X9SCM-F

Intel Xeon Quad-Core E3-1240 3.3GHz 8MB, 80W, LGA1155

SM 8GB DDR3 1333MHz ECC/Unbuffered (MEM-DR380L-SL01-EU13)

IBM ServeRAID M1015 LSI9220-8i 46M0861 (IT Mode)

 

So Im running 3 VMs, UNRaid, WinXP and a 2008 Server.

How would you or anyone else recommend the resource allocation?

Sorry Im new to ESXI.

 

/Robert

 

As John said, it depends on what will be running on those guests. unRAID with no addons is easy in 1G, but could be much more. You'll have to decide how to divide up the RAM, 1+3+3, 2+2+3, remembering to save about 800M for ESXi. Also keep the number of vCPU as low as possible, 1 is best. Be sure to install vmware tools on the guest, it will help with memory management.