ATLAS My Virtualized unRAID server

[RockDawg]

No problem.  Johnm did us all a huge favor with this thread.  It was invaluable for me and many others I'm sure.  Kudos to him!

 

Not sure what USB problems you are having, but make sure to add both the USB controller and then the USB device to your VM.  In the very beginning, I didn't realize I needed to add the controller first for the thumb drive to show.

[BLKMGK]

No problem.  Johnm did us all a huge favor with this thread.  It was invaluable for me and many others I'm sure.  Kudos to him!

 

Not sure what USB problems you are having, but make sure to add both the USB controller and then the USB device to your VM.  In the very beginning, I didn't realize I needed to add the controller first for the thumb drive to show.

 

Right now the USB controller won't stay past a boot so getting to the device isn't happening. I add the controller, boot, it's gone like magic. This is on 5.1. About to post in my build thread about it - very frustrating lol.

[RockDawg]

Sorry to hear that. Hope you get it figured out.

[BLKMGK]

Sorry to hear that. Hope you get it figured out.

 

Bob is running one of these boards so I know it's possible. I just hope it's not something that forces me to move to an older ESX version. I see there's an updated BIOS for this, minor rev, just out as well. I might try that if I run out of patience completely...

[sam_roberts30]

Can I just ask what sort of power you pulling with you esxi build?

 

I think I am going to virtualize Unraid along with whs 2011 and just wanted to see what the power trade off would be like.

 

Sorry if this has been covered I skimmed all the posts but didn't see it anywhere.

[BLKMGK]

My issue - solved! Was passing thru the controller the boot stick was on.

 

Which of us are you asking about for power usage? I have not yet placed mine on a UPS, which with a storm coming is not cool. I may buy one today! I have a Kill-A-Watt here but it's not been hooked up. My ESXi build has a 750watt Gold rated PSU in it and I honestly do not think it will be terribly thirsty. Presently it's only got a few disks in it too so power usage won't be really high I expect. I admit I'm curious too so I'll hook this meter up when next the system is down. I'm copying data right now so it could be a good little while I'm afraid. So far performance is darn nice but with so few drives that's to be expected. No other VM running yet but I think OSX is next up and some Ubuntu as well.

[joelones]

Ok, so I've entered into a somewhat neurotic stage of panic after booting my unRAID vm and seeing one disk missing...

 

Oct 28 15:21:20 Clara-Belle kernel: sas: ata1: end_device-0:0: dev error handler

Oct 28 15:21:20 Clara-Belle kernel: sas: ata2: end_device-0:1: dev error handler

Oct 28 15:21:20 Clara-Belle kernel: sas: ata3: end_device-0:2: dev error handler

Oct 28 15:21:20 Clara-Belle kernel: sas: ata4: end_device-0:3: dev error handler

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: ATA-8: ST31000528AS, CC38, max UDMA/133

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 31/32)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: qc timeout (cmd 0xef)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: failed to set xfermode (err_mask=0x4)

Oct 28 15:21:20 Clara-Belle kernel: drivers/scsi/mvsas/mv_sas.c 1522:mvs_I_T_nexus_reset for device[3]:rc= 0

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: failed to IDENTIFY (INIT_DEV_PARAMS failed, err_mask=0x80)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: revalidation failed (errno=-5)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: qc timeout (cmd 0xec)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: failed to IDENTIFY (I/O error, err_mask=0x4)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: revalidation failed (errno=-5)

Oct 28 15:21:20 Clara-Belle kernel: ata4.00: disabled

Oct 28 15:21:20 Clara-Belle kernel: ata4: hard resetting link

Oct 28 15:21:20 Clara-Belle kernel: mvsas 0000:0b:00.0: Phy3 : No sig fis

Oct 28 15:21:20 Clara-Belle kernel: drivers/scsi/mvsas/mv_sas.c 1522:mvs_I_T_nexus_reset for device[3]:rc= 0

Oct 28 15:21:20 Clara-Belle kernel: ata4: EH complete

Oct 28 15:21:20 Clara-Belle kernel: sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0

 

I created a new thread for my issue.

syslog.txt.zip

[sam_roberts30]

I don't mind who answers, just looking for a ball park figure really. I'm going to running whs2011 with 1 2tb drive and Unraid with 8 4tb drives. Just looking at whether its going to make my levy bill go through the roof.

[BLKMGK]

Levy meaning power? Shouldn't be any more power than any other loaded machine with say a 700watt psu. I actually expect to lower my usage moving from two servers that had 600watt psu to one server while increasing the utility of the device. Instead of plugins to unRAID making me nervous about my data I'll run additional VMs to support the other things I need.

[ldasilva]

im at around 100 watts 9 unraid drives(mixed sizes) and 3 datastore drives xeon running at 90%+ at all times

 

I don't mind who answers, just looking for a ball park figure really. I'm going to running whs2011 with 1 2tb drive and Unraid with 8 4tb drives. Just looking at whether its going to make my levy bill go through the roof.

[BLKMGK]

Just checked mine, 115watts jumping as high as 125 - about 12+ drives spinning and it's clearing 3 drives. Almost none of my drives are "green" FWIW. Surprised at how low it is honestly.

[dheg]

Apologies in advance for the noob question

 

Is there any difference between the m1015 and lsi 9220-8i? I just received a card i bought in ebay and don't see any sort of reference to IBM or m1015 on it. Would it be any difference with the flashing?

 

How can i check the firmware status?

 

Sent from my GT-P7500 using Tapatalk 2

 

[tranm5]

The only difference is the firmware

 

You'll find some useful informations about cross flashing this controller here: http://www.servethehome.com/ibm-m1015-part-1-started-lsi-92208i/

[dheg]

I had Atlas out of the rack today.

 

I thought I might get a current photo of it's innards for those that like server pr0n.

 

 

I still need to replace the back fans. only one is hooked up right now.

 

I would like to still mod the chassis to hold my Supermicro 4in1 and relocate the SSD's to that.

 

Hey Johnm,

 

As many in this forum I am using Atlas as a reference for my build. By looking at this picture I saw you are using custom cables to power up the backplanes. Can I ask where did you buy those connectors?

 

What do you think of these http://www.jab-tech.com/ConnectorZ-90-Degree-Pass-thru-Molex-Connector-Black-pr-3155.html?

 

I'm also using a modular PSU (Corsair HX 850). Where can I find the connector going into the PSU?

 

Also, and I may be wrong, it looks like you are powering up both plugs in each backplane from a single PSU. Is this so? Why are you doing it?

 

thanks,

 

[prtsmgr]

I'm also using a modular PSU (Corsair HX 850). Where can I find the connector going into the PSU?

 

You can find connectors and custom sleeved cables at http://www.moddiy.com or http://www.frozencpu.com

 

http://www.moddiy.com/products/Corsair-AX%7B47%7DHX%7B47%7DTX-Series-Custom-PSU-Modular-Cables.html#.UJO_RoaO5lM

http://www.moddiy.com/products/Corsair-HX-Series-PSU-8%252dPin-%286%252b2-PCI%252de-%26-4%252b4-CPU%29-Modular-Connector-w%7B47%7D-Pins-%252d-Black.html#.UJPCKoaO5lM

[sam_roberts30]

Thanks for the replies. There was a typo, Levy was meant to say leccy as in electric. Dam ipad autocorrect.

 

That's not too bad at all. going on an average of 100w is it will cost roughly £120 a year...ish.

 

It solves all my problems. Unraid for 1:1 dvd and bluray rips for the cinema room and remote access via whs for my music and divx movies.

 

A little off topic but does esxi/Unraid allow the disc to spin down when not in use?

[BobPhoenix]

Thanks for the replies. There was a typo, Levy was meant to say leccy as in electric. Dam ipad autocorrect.

 

That's not too bad at all. going on an average of 100w is it will cost roughly £120 a year...ish.

 

It solves all my problems. Unraid for 1:1 dvd and bluray rips for the cinema room and remote access via whs for my music and divx movies.

 

A little off topic but does esxi/Unraid allow the disc to spin down when not in use?

If you are passing through a whole controller then Yes it does allow the drives to spin down.

[BLKMGK]

Yeah, my experience is that they spin down. Honestly my problem is figuring out what to do with 8 slots dedicated to ESX. I'd love to dedicate those to unRAID and just use a few onboard connectors for ESX if I could stuff drives inside. I'd also like to cache the ESX drives with an SSD but I've not got that solved yet. Any other uses for those slots? Need to research more about what others have done for sure.

 

I do expect I'll be making custom power cables now - thanks!!

[BobPhoenix]

What I've done is dedicate the 6 MB Sata ports to ESXi datastore and RDM drives and pass through the HDD controller cards to VMs for use.  That is one possibility you might consider but there are many options.

[BLKMGK]

RDM I'll have to figure out but I'd agree that sounds like a much better way to leverage the hardware. I'll have to figure out where to sneak more drives in the case. The SSD was easy! Cleaner cables and routing will help for sure...

[sam_roberts30]

Bobpheonix: you have a lsi controller to intel expander in your sig. How many sata ports do you have/what is the maximum with this setup?

[BobPhoenix]

Bobpheonix: you have a lsi controller to intel expander in your sig. How many sata ports do you have/what is the maximum with this setup?

I have one cable going to the expander.  So I can then have 20 drives off the expander and 4 off the 2nd port of the M1015 for 24 total drives.  With a second SAS expander connected to the 2nd port on the same M1015 I could get 40 drives connected to the same controller.  By connecting from one SAS expander to another I believe you can get to the M1015 card maxium of 128 drives.  Note I read that somewhere but not sure where and I'm going from memory so that could be wrong.  Another way to connect to the expander would be to use both M1015 ports and 2 on the expander.  Then you would get a max of 16 drives off the expander.  I tried it that way with my first expander but it didn't work for me.  unRAID would appear to work on parity checks but I was getting crashes when copying data to the array.  Not sure if it was a cable problem or a port problem but disconnecting the second connection to the expander solved my problems.  The actual drive count on my 2 unRAID servers is 19 on one server (all on expander) and 15 on the other server (all on expander).

[sam_roberts30]

And you don't notice any performance sacrifices?

 

Last off topic question I promise.

[BobPhoenix]

And you don't notice any performance sacrifices?

 

Last off topic question I promise.

I don't notice any no.  I'm sure it isn't as fast as connections straight to the controller but for me it is fine.  My usage is 99% read 1% write over a month.  I usually write to my disk shares all at once about 1 day a month the rest of the time it is reads.  Note my drives are spun down 95% of the time as well.  The 15 drive server has parity checks that are almost 9 hours with all 2TB drives and a mix of 7200 and 5900.  So I get about 60MB/s+ on a parity check.  I'm not listing the speeds from the other server because it has problems unrelated to being on an expander.  I fried 12 of the 19 drives when they were in a different server that had a fan go bad either before or during a parity check.  Hope to have all of the drives replaced by some time next year.  Meanwhile they still work but I don't write anything to them just reads until they get replaced.

[dheg]

I'm also using a modular PSU (Corsair HX 850). Where can I find the connector going into the PSU?

 

You can find connectors and custom sleeved cables at http://www.moddiy.com or http://www.frozencpu.com

 

http://www.moddiy.com/products/Corsair-AX%7B47%7DHX%7B47%7DTX-Series-Custom-PSU-Modular-Cables.html#.UJO_RoaO5lM

http://www.moddiy.com/products/Corsair-HX-Series-PSU-8%252dPin-%286%252b2-PCI%252de-%26-4%252b4-CPU%29-Modular-Connector-w%7B47%7D-Pins-%252d-Black.html#.UJPCKoaO5lM

 

thanks, I didn't know modDIY, very large range, already ordered some parts