ESXI 5.1 and UnRAID 5.0.5 reboot / PSOD issue (mSATA related) <-- SOLVED

[phreshjive]

My build at present is:

- Intel DQ77KB Motherboard with an i5-3470T processor

- 2 x 8GB  CT102464BF160B Crucial RAM

- Crucial M500 120GB mSATA drive

- 8 x Seagate 2TB 5900 RPM ST320005N4A1AS drives

- SuperMicro AOC-SASLP-MV8 SAS card

- U-NAS NSC-800 case

 

I initially wanted the system to serve 4 purposes: (1) pfSense router (2) unRAID server (3) Plex Server (4) Plex Player.  Through my research, I found out that having it as a Plex Player might be more trouble than it's worth.  The motherboard (selected for being a mini-itx, having 2 Intel NICs and Vt-d support) only has one 4x PCI-e slot being used by the SAS card and from what I read doing passthrough on the onboard Intel graphics card is a major pain.  Plus I have a ChromeCast in its box since launch. If there was a viable alternative to ESXI that would allow all 4 and setup would be fairly painless, what would that be?

 

Bit of background: had system working running ESXI 5.1 (with Intel LAN drivers slipstreamed) and UnRAID Pro 5.0.4.  I messed around with a setting in the BIOS related to the graphics card and lost video.  All known methods of resetting the BIOS to factory defaults failed (including removing the CMOS battery). After lengthy discussion with Intel, ended up RMAing the board.  Sent a brand new motherboard.  With the new one, updated to latest BIOS, installed ESXI 5.1, and it was rebooting when trying to start my array (which luckily had no data transferred on it).  Thought the SAS card might have been a dud and ordered a new one. Same issue. 

 

I am now pre-clearing 4 of my drives (then the remainder) with the hopes that it will solve my issue.  This build has taken the life out of me as I have never experienced so many issues with any build.  Granted this is my first venture down the virtual road.  I specifically chose the MB and CPU for Vt-d and extremely low power consumption.  Hard drives were chosen as I bought them new for a fraction of their MSRP (otherwise it would have been WD Reds).

 

I should also note that I followed the unRAID VMDK guide (updating to 5.0.5 files), the unRAID VMDirectPath Hardware Passthrough and applied the MV8 hack. I also disabled INT 13h on the SAS card (running BIOS 21 with RAID disabled / JBOD enabled).

 

Update: Attached are the pre-clear reports of first 4 drives. All seemed well and fine.

 

Help would very much be appreciated.

log.txt

preclear_reports.zip

[pjneder]

I have a few questions/comments.

 

How are you powering the drives?

 

I realized that there are a few differences from what I did. I started with a DQ77KB and then I couldn't figure out how to get the HDD's to spin up when they were connected to the power supply and SMC HBA. I also had started with the wrong i3 CPU to do VT-d. I knew I could easily re-purpose the 77KB and i3 to something else, so I kind of started over a bit. I went to the DQ77KM (u-ATX) and i5-3470S, all with VT-d capabilities. Eventually I figured out that my SASLP had a bad port 5 on the board. Well, now that I had a PCIEx8 slot I jumped up to a M108 card.

 

That being said, what are you trying to do *seems* like it should work. I never did any of that 13h disabling stuff, so not sure about that.

 

What are you booting/running ESXi from? Is the mSATA disk your boot + datastore for ESXi?

 

I have a buddy also running ESXi but not doing unRAID so he skipped the VT-d parts. He had terrible problems with an mSATA drive, but I forget the details. I'll point him to this thread and see if he will send me some comments. You could easily be having a mPCI issue. In my setup I boot ESXi from a USB stick and have an old laptop 2.5" HDD as my datastore. You could consider taking the mSATA out of the picture for a while and see if that helps. There are plenty of guides on how to put ESXi on a USB stick then any old HDD connected to the mobo ICH sata port for your DS.

 

Your almost there, don't give up yet...

[phreshjive]

POWERING THE DRIVES: Right now I am temporarily using a Corsair CX430 externally that is jumpered using a paperclip / electrical tape that is connected to the (EDIT: SAS backplanes) that are connected to the hard drives and the Noctua NF-S12A fans.  The motherboard is being powered by the Dell PA-10 (19.5 VDC - well within the 19 ±10% VDC spec - pg 53/63) --> http://downloadmirror.intel.com/21339/eng/dq77kb_TechProdSpec07.pdf

 

LONG TERM POWER STRATEGY: Once this puppy is ready to be sealed up, I have on hand a SeaSonic SS-350M1U with a Flex ATX to 1U bracket. I also purchased a DROK DC Converter Voltage Regulator 8-32V to 9-46V 12/24V 150W Boost Step Up Power Supply Module and a Molex 5566-2 connector.  The goal is to take a 12V from the semi-modular PSU cable, wire it into the Step Up converter then wire that to the Molex connector stepping up VDC to 19.  That PSU will then be able to power all 8 drives (9 including the mSATA) at boot without being staggered and motherboard/CPU avoiding having two cables powering everything.

 

VT-D: I believe it was one of your posts that made me realize that I needed a CPU and MB that supported VT-d as I originally was going to go the exact same route you did.  I decided on the i5-3570T for power savings and it does support Vt-d --> http://ark.intel.com/products/65521/intel-core-i5-3570t-processor-6m-cache-up-to-3_30-ghz

 

INT 13H DISABLE: I disabled that INT 13h based on the ATLAS thread --> http://lime-technology.com/forum/index.php?topic=14695.msg138465#msg138465

 

BOOTING ESXI: Right now I am booting+datastore using the mSATA.  Your post did make me realize a difference in how I had the system setup prior to the RMA -- I also purchased a Samsung 840 SSD and was planning on using either the mSATA SSD or Samsung as a cache drive but ran into PSODs under 5.5.  Tried under 5.1 with lots of frustration and gave up.  Decided on using the mSATA.  Will revert back to the Samsung once the pre-clear is done and try again and report back.

[pjneder]

A quote from my friend who had problems and read this thread:

 

I did have problems with my mSATA initiially. I was a bad flash. Returned it for a replacement. That said, I did not get to test it when I got the replacement due to the motherboard having issues too. I *just* got that back from repair but have not had a chance to to test the MB nor flash. And I think I need to a CPU too since I bought a replacement MB due to the long wait for the repair. Longer story than you wanted, eh?

 

Now, on to this guy. I'd recommend he boot from a Live CD and try some basic fdisk/parted/gparted stuff on the mSATA. Create a partiton. Create a filesystem. If he gets that far, use dd to stream the whole partition with stuff. If that works, then he has other problems besides the disk/mSATA.

 

Bottom line: mSATA's and BIOSes can have problems. D'uh.

[phreshjive]

Wanted to truly thank you.  It never dawned on me that my issue was with the mSATA drive.  Spent the day going through the motions of setting everything up and everything so far is working under 5.1 with the 3 updates that are out there using the Samsung 120GB SSD and a Sandisk 30 GB ReadyCache drive converted to regular SSD as the boot drive. 

 

Going to create a step by step guide including pictures of my build like the ATLAS thread as I have to build a second one for a friend.  I believe it will be beneficial for anyone who wants to setup an unRAID / PlexServer ESXI build in a compact form factor using similar hardware.