WeeboTech

I've had good results with SANS DIGITAL TR4UTBPN 4Bay USB 3.0 / eSATA Hardware RAID 5 Tower RAID Enclosure (no eSATA card bundled) http://www.newegg.com/Product/Product.aspx?Item=N82E16816111149 I choose this over the simpler model because of the built in PSU and built in RAID. If I choose to use it elsewhere I can change raid modes with an adjustment. I also like the slide in design.

Port Multiplier Technology is neat. It's good for unRAID as a media server. When you access a drive at a time, you will get top speed. For a very busy server, it may have issues with competition for bandwidth on certain drives. I.E. when you access multiple drives from multiple apps on the same box/cable. If you lay your drives out well, the technology works. Your parity drive must be on your fastest internal port. Your busiest drive(s) and cache need to be on your internal port. Other then that you can probably have 4 external 4/5 drive units with current eSATA technology. Since the machine only accepts low profile cards, I've only found x1 cards that provide 2 eSATA ports. I bet with a more advanced raid controller and other external drive connection technology you could get more. For me, I could not get the other eSATA port on the back panel to be reliable with Port Multipliers. It could be because I was using ESX 5.1. I choose to use the PCIe x1 Startech controllers with the ASM1061 chipset since they supported 6GB/s. In my tests, it made a small difference. But for Port Multiplier technology, I found the Silicon Image chipsets to be the best for PMP management. I.E. When using the ASM Chipset, drive writes very chunky, so simultaneous drive access really hampered the performance of the other drives. With Silicon Image chipsets, I found you had a top speed of about 120MB/s, each parallel access halved that. So two drives together dropped to 60MB/s, 4 Drives dropped to 30MB/s. With the other chipsets it wasn't so smooth. 4 drive access dropped really low to around 10MB/s. So far in all these years of playing with this technology, and my limited testing, I've found the silicon Image chipsets best for PMP usage. I've tested with Silicon Image, Marvell and the ASMedia chipsets. With as many drives as you plan to have, I'm not sure all these external are the best way to go. 1 large machine with the right technology could actually save you power considering that each box will have a PSU and Fan running all the time.

No Yes Yes with JBOD and RAID0. The box I selected can connect via USB 3.0 or eSATA. I used eSATA.

I'm loving the idea of a community sponsered build slideshow!

In order to speed up the boot, the BIOS needs to support it. if it's under ESX, I do not think the virtual bios supports the USB 3.0 speed. I could be wrong, but I remember reading that.

Good point. Although as long as there's an available SATA port, an SSD can be mounted "creatively" in a lot of spare spots in most cases An excellent idea -- much better than using a dedicated drive. HOWEVER ... for many of us that would mean a new motherboard & CPU to get USB v3 ports (On some boards you could use an add-in card) I don't think ESX uses USB 3.0. I believe it can pass it through to a capable OS, but it will not use it directly. I would surmise that the virtual BIOS may not achieve the usb 3.0 speeds either. I also believe unRAID is capable of using USB 3.0 yet either. on my Supermicro X9SCM-F-O, with the Patriot Supersonic Rage XT 64GB USB 3.0 key, ESX boots up very fast. In fact so much faster then from my proliant micro server. However, I don't think unRAID would boot as fast as a virtual .vmdk on an SSD. I don't see much of a benefit getting an extra controller card to have USB 3.0 for unRAID booting vs an SSD datastore.

You can use the PCIe if it has eSATA ports. I am doing it with both PCIe slots. Must be Low Profile. So before you purchase a card, with eSATA make sure it has a low profile adapter. I've used these with success. PCI-Express SATA II 3.0Gb/s Controller Card SD-PEX40031 http://www.ebay.com/itm/PCI-Express-SATA-II-3-0Gb-s-Controller-Card-SD-PEX40031-/330558178838?pt=US_Computer_Disk_Controllers_RAID_Cards&hash=item4cf6cd8616 Addonics ADSA3GPX1-2E PCI Express eSATA and SATA II 2 Port eSATA II RAID Controller http://www.newegg.com/Product/Product.aspx?Item=N82E16816318005 http://www.newegg.com/Product/Product.aspx?Item=N82E16816129101 StarTech PEXESAT322I PCI-Express x1 Low Profile Ready SATA III (6.0Gb/s) 2 Int/2 Ext SATA Controller Card The Startech is SATA III. It worked really well for the Samsung PM840 I put in place.

I think I used the ROM from the skydrive account. I cannot remember. There were so many references to a modded bios. I also read the forum post by thebay regarding the mods.

I have installed unRAID 5.Xrc under ESX 5.1 on a microserver with 16GB of ram. I allocated 1GB to unRAID. Using internal SATA, External SATA, ASmedia and SIL3134, SIL3132 cards I get 'raw' read rates from 95MB/s up to 180MB/s on the ASmedia 6GB/s SATA card. These are raw DD reads of the hard drive itself, bypassing any form of network or filesystem reads. When writing to a drive without parity, the speeds varied anywhere from 90MB/s up to 195MB/s depending on how much data I was writing and whatever filesystem allocation there was. i.e. journaling absorbs some of the raw speed for writing. I used port multipliers also and they were in the 100-120MB/s range for a single drive on the Silicon Image chipset and up to 190MB/s on the ASmedia SATA III card. Tests were done using the Seagte 3TB 7200 RPM drive. I used vmkfstools with the -z and -a pvscsi option to RDM the entire disk. My tests without RDM proved to be quite slow and almost pointless to have resierfs on top of a virtual .vmdk container file on top of a VMFS data store. Even with fast RAID0 using 2 3TB drives, it was painfully slow and lengthy just to format. 1. Format the data store (fast). 2. Create the .vmdk (about hour and half). 3. Format the reiserfs filesystem (40 minutes or so). RDM of the disk was almost as fast as native bare metal unraid. Very close in speed. As far as transfer over the network. I did not test it. I knew if unRAID could read/write to the disk under ESX as fast, or nearly as fast, as bare metal, the rest would work at acceptable levels. What network adapter did you choose? E1000 or VMXNET? What SCSI adapter did you use for the RDM disk (if you configured it that way). I found the LSI SAS adapter to be fast, but it bottle necked at around 120 MB/s. I immediately reconfigured with the PVSCSI adapter and the speed went up to 180MB/s. as a test do this to see your raw hard drive read capability. dd if=/dev/sd? of=/dev/null bs=4096 count=1024000 Where ? is the device's last character. This will read 4GB raw from the hard drive. It will show you the maximum speed you could possibly get from the hard drive for sequential data on the outer tracks. There are all sorts of file system references that occur, so file reads on a formatted file system will be slower. you can do the same to write a file and read it if the hard drive is mounted echo 3 > /proc/sys/vm/drop_caches # drop pagecache, dentries and inodes dd if=/dev/zero of=/mnt/disk1/test.dd bs=4096 count=1024000 echo 3 > /proc/sys/vm/drop_caches # drop pagecache, dentries and inodes dd if=/mnt/disk1/test.dd of=/dev/null bs=4096 count=1024000

I successfully updated my N54L's BIOS last night using TheBay's version. Afterwards, I went into the BIOS, loaded optimal defaults, and then tweaked a few settings. Among the adjustments, I set the "SATA 5/6 IDE Mode" (I don't remember the exact label, but it will be close to this) option to "Disabled" under the "SouthBridge Configuration" options. Upon next boot, all my drives populated in AHCI mode and unRAID started up without a hitch. Thanks. I'll attempt it this weekend.

There is a standby mode built into it, if you are running VirtualCenter there is an option to manually put a host into standby and exit standby either by WOL or iLO. They tie it into the DPM (Dyanamic Power Management) feature in clusters to shut down and boot up hosts on demand to meet/match resource demands to save on power. You could probably find some method to trigger it through the remote CLI since I would assume they built it into the CLI API. You would also need to suspend/shut down the guests manually by script since this is normally a task VirtualCenter does before the host standby (the function being to VMotion all guests off). I've never used it so have no idea how similar it is to normal sleep mode. This is good and interesting news. If anyone finds out how to do it via cli, please post!

Thanks guys!

I would do a little more research to see if your board supports VT-D and if ESX will handle the Pass through. I don't know enough to guide you, but I'm not sure the i3 will support it either. What we know works is the popular Supermicro X9SCM boards and the Sandy Bridge XEON CPU along with the IBM 1015 controller. I believe there is a hack for the super micro controller. Again. I'm not advanced on the topic. I would suggest reading JohnM's atlas thread. Supermicro works, and its not really a "hack", its just a specific piece of configuration you need to do for a lot of passthru devices.. Is there a source of this information for 'other' devices? Link for the SASLP-MV8, http://lime-technology.com/forum/index.php?topic=7914.msg128847#msg128847 however I'm interested in where to obtain information on other controllers that may support pass through with this sort of information.

Highly unlikely. unRAID may go into suspend or sleep mode, but it will not tell ESX to do the same. ESX is designed to be running and available all the time. Although it may be possible, I've not seen anything to support ESX going into low power sleep/suspend mode. We asked Tom for this, we'll see how it goes.

Sell the i3 on ebay and buy another used processor on eBay. I've been doing this for years. I thought I was going to lower my machine count after Sandy. Turns out I ended up just buying more laptops until I did all the research for buying new towers. At least with ESX I will be minimizing my box count somewhat since I can boot slackware, centos, RHEL, Solaris, XP and other Dev environments on demand without using separate boxes. I can't say how much VMWare workstation to ESX deployment saves me. I got a massive laptop with 32GB of ram, dual 512GB SSD's, 256msata and 17" monitor for building.. then deploy it to ESX. Really nice. BetaQuasi's assistance with this VMDK saved me allot of work. Thanks B!!

pjneder, I just read a post about someone with an Intel DQ77KB motherboard with an I7-3770s saying VT-D pass through worked for him. Do some more checks. Looks like you may be able to pass through a controller.

I would do a little more research to see if your board supports VT-D and if ESX will handle the Pass through. I don't know enough to guide you, but I'm not sure the i3 will support it either. What we know works is the popular Supermicro X9SCM boards and the Sandy Bridge XEON CPU along with the IBM 1015 controller. I believe there is a hack for the super micro controller. Again. I'm not advanced on the topic. I would suggest reading JohnM's atlas thread. As far as 24x7 drive spinning. I will be tolerating it for my small ESX Micro servers. At least one of them is an ESX host, a download file server, my main repository for driver/ISO downloads and my home folder. Along with my music. My other workstation used to have 4 10,000 RPM drives spinning 24x7x365. Those drives lasted 6 years until hurricane sandy. I'm sure they would have lasted longer. For my current workstations, I went with laptops and swapped almost all of the drives out for 256gb SSD's. The only ones I did not swap out were my dj and download laptops which need massive storage. A few spinning drives all the time is not going to kill them, you just have to manage heat and monitor the smart stats.

Why?

What are you using for the ZFS Array? I'm contemplating an experiment.

I bet it works for pass through. Although I wonder how it would work with an Areca controller. They have a bios setting to spin down the drive during idle time. I know unRAID could not spin down my RAID0/RAID1 array, but the controller would do it for me.

With pvscsi, and the lsisas controller you can get smartctl information. Although it wont spin down. I'm going to try a test where I set the spindown time into the drive with hdparm -S241 /dev/sd? on bare metal unRAID. Then boot into vmware and see if does actually spin down. Since I cannot tell directly, I can try to see if there is a delay when I access the drive. From what I saw, ESX probes the drives every now and then, so the drives may never spin down.

I've used a command like this. vmkfstools -z /vmfs/devices/disk/vml.xxxx diskname.vmdk -a lsilogic I ended up changing it to pvscsi, in my tests it was faster, but only if you had the hardware to support it. Blinking green balls and no spin down. Yes, that's my case too. Actually they are probably spinning, but the status cannot be determined. Therefore they cannot be spun down correctly either. You should be able to get smart status though.

What controller did you choose? I think I remember fiddling around to pick a controller type.

Has anyone upgraded their N54L with the modified Bios for the improved SATA options?

I played with ESXi on N36L shortly but because (1) you can not passthrough a hardware controller to a VM and (2) CPU is incredibly weak for any practical application, it makes little sense to run ESXi on a microserver other than for testing and experimenting. How it can make sense to run ESXi on a microserver when you say CPU is incredibly weak for any practical application? I am considering selling N40L so I can build my own mini server (same server case size) with much powerful CPU and better motherboard that support passthrough. Tempted to do the same, but the cost of new server will be higher than the N40L I assume? I have the N54L with 16GB of ram. ESX runs OK on it. In comparison, at work we have a dual processor 2.5 ghz opteron we've been running RHEL 4 with vmware server 1.0 for years. It gets about 5184 Bogo mips. We hit this hard with about 3-9 RHEL linux VM's at a time. The N54L is 2.2ghz dual core and gets about 4400 bogo mips. Since I do not hit the server so hard as we do at work, Plus with the use of ESX rather then RHEL/VMware Server, ESX is pretty do-able with light loads and/or use of an SSD for the datastore.