Solved... do NOT try and run 4 HDDs off of SYBA PCI Express x1 cards (http://www.newegg.com/Product/Product.aspx?Item=N82E16816124064) Those cards gave me nothing but headaches and in the end, they had to be replaced. Not sure if they'd work for less than four drives but having 2 of them in my system with 4 drives on each was a bad idea. Stay away!!!!

 

Solved... Combination of bad HDDs, bad (underpowered?) PSU, and crappy drive cages was causing these problems. Replacing those three seem to have stabilized the environment. I've now run 6 parity syncs and gotten a lot of usage without any issues. Thanks!!

 

Hi all... I just built my first 12 drive unRAID Pro box and I REALLY like it. I'm having an issue though where some drives seem to just drop out at random. From the wiki, it looks like this is likely a problem with crappy SATA cables. I'm about to go and find a new set of higher quality cables since the ones I'm using are the ones that came with my drive enclosures.

 

However, before I do that, I thought I'd ping you all to make sure that you don't think this could be a power supply issue. I've got a NZXT 750W PS which has the ratings:

+3.3V@30A, +5V@30A, +12V@60A, [email protected], [email protected] From what I've read a single 12V rail at 60A should be sufficient for these drives (most of which are "Green" drives). Do any of you disagree?

 

A little data: Running 5.0-rc12a, 9 of the drives are in these Thermaltake MAX-2533 HDD canisters:  http://www.newegg.com/Product/Product.aspx?Item=N82E16817998145 The drive failures do not seem to be limited to the drives that are in those bays though. I just had one fail that wasn't in there.

 

When I stop the array the drive doesn't show up as available. Once I reboot, then it's detected again. The drive failures don't happen after I've moved the server or opened it up or anything, they just drop for no apparent reason... I guess all signs point to crappy SATA cables but I'm hoping for a second opinion. Thanks!

-Dan

 

PS. I'll post my syslog the next time it happens. Unfortunately I rebooted to get the drive/array back online and so don't have one showing the failure. If/when it happens again, I'll get that up here.

 

PPS. SUPER annoying when this happens on my parity drive. That parity re-sync takes forever! TYIA!

I have the same power supply and it is more than capable of running 12 drives.  Does each of your cages have its own power cable.  As the power supply has 4 moduler plugs, give each of the cages their on cable.  If you need additional cables, you can call Seasonic and get their recommended supplier for cables.  This power supply is an OEM Seasonic supply.  I purchased extra cables for mine and the vendor was very fast.  They will even customize them if you want.  I had mine limited to one connector.

I have the same power supply and it is more than capable of running 12 drives.  Does each of your cages have its own power cable.  As the power supply has 4 moduler plugs, give each of the cages their on cable.  If you need additional cables, you can call Seasonic and get their recommended supplier for cables.  This power supply is an OEM Seasonic supply.  I purchased extra cables for mine and the vendor was very fast.  They will even customize them if you want.  I had mine limited to one connector.

 

Excellent... glad to know the PS is solid. I'll definitely check to see if I've got individual cables for each of the cages, that's a great idea. Looks like Antron Electronics is the vendor to use. I'll reach out to them about some custom cables. Appreciate the heads up/suggestions!

 

I guess though, this reaffirms my suspicion that it's the SATA cables. I'll order some of those as well, swap them all out, and hopefully my problems will go away. Thx again.

 

EDIT: It seems that my PS is NOT modular... So that's a no go for custom cables unfortunately. I'll still check to make sure that I'm as distributed across those cables as I can be.

Well crap. Parity drive (sdj) just dropped out again. System doesn't even recognize it as a device at this point. That's three (different) HDDs dropped in 6 hours. I  grabbed the syslog (cut out some stuff) and it's attached.

 

Are all of these SATA link bounces to blame?

 

Apr 25 13:11:41 hoarder kernel: ata10.00: SATA link up 3.0 Gbps (SStatus 123 SControl 330)

Apr 25 13:11:41 hoarder kernel: ata10.01: SATA link down (SStatus 0 SControl 0)

 

I guess have no idea at this point what's causing this. I just can't imagine three bad cables. All disks were pre-cleared with no errors.

 

Other system details:

Proc - Intel Celeron G530 2.4GHz LGA 1155

Mobo - GIGABYTE GA-H61M-HD2 Micro ATX

Memory - 4GB Patriot DDR3

SATA Interface Cards: 2 x SYBA SI-PEX40064 PCI-Express 1x SATA III Controller (Marvell 88SE9215 Chipset) with 4 ports each.

 

Drives that failed today are (surprisingly) across three different controllers, one on each PCI card and one on the on board controller. Ugh. Getting frustrating. Seems too widespread to be a SATA cable issue. Thoughts? Thanks in advance.

 

syslog-2013-04-25.txt

Alright alright. I just rebooted and my parity drive wasn't detected on boot so here I sit unprotected. I'm going back to the cable theory now. I'll go pick up a whole new set at MicroCenter on my way home from work and hopefully this will fix the problem for good. Should have more details tomorrow. Yaaay.

Check the BIOS versions of the MB and any SATA cards.

Check the BIOS versions of the MB and any SATA cards.

 

Thanks! Just to clarify, are you suggesting that I look at updating the BIOS?

 

(BTW, I appreciate the links in your sig. I read through them and determined that I'm not worried about the link resets in my syslog so that's good!)

Based on your specs, it appears you have the HALE82N 750W.  It should be more than adequate for your system.  Sorry for my earlier assumption that it is the same unit I am running. 

 

Are all of the drives you are having problems with plugged into the SYBA SI-PEX40064 controllers.  The reason I ask is that when I search for usage of that controller or chipset with unraid, the only link that comes up is yours.

 

Are any of the drives plugged into your motherboard having issues.

 

I would not update the firmware of the mother board yet.

Based on your specs, it appears you have the HALE82N 750W.  It should be more than adequate for your system.  Sorry for my earlier assumption that it is the same unit I am running. 

 

Are all of the drives you are having problems with plugged into the SYBA SI-PEX40064 controllers.  The reason I ask is that when I search for usage of that controller or chipset with unraid, the only link that comes up is yours.

 

Are any of the drives plugged into your motherboard having issues.

 

I would not update the firmware of the mother board yet.

 

Gosh, no apology necessary! I appreciate any and all help I can get at this point!!

 

It doesn't seem to be the SYBA controller that's causing the issue. The drive I have the most problems with is the parity drive and it's plugged into the motherboard controller. In fact, it wasn't until today that I started having issues on drives connected to the SYBA controller...

How long has this system been configured.  I have no other thoughts as to what it is.

How long has this system been configured.  I have no other thoughts as to what it is.

 

Couple weeks. I'm going to go swap out the SATA cables and re-wire the power cables to make sure they're as distributed as possible and we'll see if that makes a difference. Thx for your help.

The power supply should have plenty of power for only 12 drives -- no absolute guarantee, but it's very unlikely that it's the problem.  ... seems far more likely that it's either cabling, or possibly the design of the drive cages (discussed below).

 

A couple of cabling questions ...

 

(1)  Are all of the data cables locking SATA cables?

 

(2)  Are all of the power cables firmly seated?  [Two on each of the cages; plus the cables to the drives that aren't in cages]

 

(3)  3 cages (x2 power connections each) + 3 single drives = 9 SATA power connections.    How many of these are "native" (from the PSU) and how many are from adapters [either Molex -> SATA or a SATA Y adapter].    And are these all firmly connected?

 

 

An operational question:  Do you have the system set to spin-down the drives?  Reading through a bunch of on-line reviews for the Thermaltake cage, there are two criticisms that may have some bearing here:  (a)  powering a drive in the cage when the others are already powered can cause a momentary drop-out of one of the other drives;  and (b)  spinning down all the drives in a cage will cause it to power off the cage (fans stop, lights off).    It sounds like these functions MAY be causing your issue.  IF that's the case, then disabling spin-down may eliminate the problem => although I'd consider that an unacceptable solution, as it means you'd lose one of the nice features of UnRAID.    But I'd at least disable spindown and see if that fixes this => if so, then you'll know that the issue is the behavior of the drive cages.    Then you can decide whether or not to replace them.

 

I'd proceed as follows:  Reseat ALL of the SATA cables (power and data) ... perhaps replacing the data cables with latching cables if you aren't already using them.    Then boot to the Web GUI and disable spindown.    If the array works okay at that point, just use it for a while and see if the issue goes away.  If so, it's almost certainly the drive cages (as I noted above).    If not, then I'd try another power supply.

 

The power supply should have plenty of power for only 12 drives -- no absolute guarantee, but it's very unlikely that it's the problem.  ... seems far more likely that it's either cabling, or possibly the design of the drive cages (discussed below).

 

A couple of cabling questions ...

 

(1)  Are all of the data cables locking SATA cables?

 

(2)  Are all of the power cables firmly seated?  [Two on each of the cages; plus the cables to the drives that aren't in cages]

 

(3)  3 cages (x2 power connections each) + 3 single drives = 9 SATA power connections.    How many of these are "native" (from the PSU) and how many are from adapters [either Molex -> SATA or a SATA Y adapter].    And are these all firmly connected?

 

 

An operational question:  Do you have the system set to spin-down the drives?  Reading through a bunch of on-line reviews for the Thermaltake cage, there are two criticisms that may have some bearing here:  (a)  powering a drive in the cage when the others are already powered can cause a momentary drop-out of one of the other drives;  and (b)  spinning down all the drives in a cage will cause it to power off the cage (fans stop, lights off).    It sounds like these functions MAY be causing your issue.  IF that's the case, then disabling spin-down may eliminate the problem => although I'd consider that an unacceptable solution, as it means you'd lose one of the nice features of UnRAID.    But I'd at least disable spindown and see if that fixes this => if so, then you'll know that the issue is the behavior of the drive cages.    Then you can decide whether or not to replace them.

 

I'd proceed as follows:  Reseat ALL of the SATA cables (power and data) ... perhaps replacing the data cables with latching cables if you aren't already using them.    Then boot to the Web GUI and disable spindown.    If the array works okay at that point, just use it for a while and see if the issue goes away.  If so, it's almost certainly the drive cages (as I noted above).    If not, then I'd try another power supply.

 

Thank you SO much for your time and assistance with this issue. I really appreciate it! To answer your questions:

 

1) They are now! Just got back from MicroCenter with brand new latching SATA cables for all connections. They seem to sit a bit better on the enclosure side. Here's hoping this was the problem!

2) I definitely went through and re-seated them completely and made sure they were all super secure/snug and flush.

3) Almost all of these are "native." Unfortunately because each enclosure requires (?) 2x Molex AND 2x SATA power I had to hook up two splitters. One Molex Y and one SATA -> 2x Molex. These are all pretty evenly distributed. I gave the enclosure that has the problem parity drive it's own dedicated "line" with no splitters. The Molex Y and SATA -> 2x Molex are brand new as of this evening.

 

As for spin-down. I had actually disabled this pretty much right after install. I'm using an unRAID provided NFS share as a datastore for an ESXi server with 10 VMs and thought this might have detrimental effects on the guests so I turned it off.

 

So right now, it's back powered on and running the Parity-Sync. All power and SATA cables have been re-seated (and in the case of the splitters and SATA data cables, completely replaced). Hopefully this has eliminated my problems! Should know in like, oh, 48 hours. 

 

As an aside, what is the benefit of drive spin-down? Is it solely for power saving measures? I thought I'd read that spinning a drive up and down has a more detrimental effect than letting it stay spun up. I've definitely not looked into it much and am certainly no expert in the field!!

 

Thanks again for your (and everyone else's!) help with this issue. I'm REALLY looking forward to getting this sucker locked down!!

 

P.S. If anyone is in the market for 18 questionably reliable SATA data cables you can have them for the low low price of FREE. Heck, I'll even pay for shipping.

Aaaaarrrrgh. I JUST had another disk fail while I was rebuilding the parity. FML. This sucks. No ideas at this point. I guess I'm going to scrap these drive enclosures and see if they're the issue. I have no idea what else it could be. So so so frustrating.  :'(

This shouldn't be an issue (i.e. it shouldn't hurt anything, so is unlikely to impact your problem);  but you're not supposed to connect BOTH the Molex and SATA connections to the drive enclosures.    They provide both so you can use whichever you have available -- but you only need to connect EITHER the two Molex OR the two SATA connections.

 

It's VERY unlikely (as I noted before) that your issue is due to the power supply;  but one thing you could do is reduce the load on the PSU to see if that makes a difference.    Try building the array with only 6 disks connected and see if it's reliable.    You can either do this with or without the cages -- obviously if you do it without you're taking them out of the picture.

 

I'd probably do this if I was in your shoes right now:

 

(a)  Build a 6 drive array WITHOUT using any of the 3-2 cages.  Run it for a day or so (long enough that you're confident in it).

 

(b)  If that works well, add the other 6 drives -- still WITHOUT using any of the cages.  As with (a), run it long enough to be confident you know whether or not it works.

 

If (b) doesn't work, replace the PSU and repeat the test.  If that works, you've isolated the issue.  If not, the issue is more complex and you'll need to expand what you're testing !!

 

If (b) works, then move 3 drives at a time to a 3-2 cage and see if the issue repeats.    This will take some time (since you need to run it long enough to have confidence in each step) ... but you should be able to isolate WHICH drive cage is causing the issue (assuming it's only one).

 

 

This shouldn't be an issue (i.e. it shouldn't hurt anything, so is unlikely to impact your problem);  but you're not supposed to connect BOTH the Molex and SATA connections to the drive enclosures.    They provide both so you can use whichever you have available -- but you only need to connect EITHER the two Molex OR the two SATA connections.

 

It's VERY unlikely (as I noted before) that your issue is due to the power supply;  but one thing you could do is reduce the load on the PSU to see if that makes a difference.    Try building the array with only 6 disks connected and see if it's reliable.    You can either do this with or without the cages -- obviously if you do it without you're taking them out of the picture.

 

I'd probably do this if I was in your shoes right now:

 

(a)  Build a 6 drive array WITHOUT using any of the 3-2 cages.  Run it for a day or so (long enough that you're confident in it).

 

(b)  If that works well, add the other 6 drives -- still WITHOUT using any of the cages.  As with (a), run it long enough to be confident you know whether or not it works.

 

If (b) doesn't work, replace the PSU and repeat the test.  If that works, you've isolated the issue.  If not, the issue is more complex and you'll need to expand what you're testing !!

 

If (b) works, then move 3 drives at a time to a 3-2 cage and see if the issue repeats.    This will take some time (since you need to run it long enough to have confidence in each step) ... but you should be able to isolate WHICH drive cage is causing the issue (assuming it's only one).

 

Thanks... I will try this.

Well, drives have been running outside of the enclosure for 48 hours now with no problems so it clearly is something with the enclosures. I guess I wonder how likely it is that its the fact that I had both the Molex and SATA power connectors hooked up rather than just SATA power. I guess this is promising though. I'll try to hook them up one enclosure at a time to see if that fixes it. Thanks again everyone for your help!

It could be either the data connections in the enclosures -- the drives may not have been firmly seated, or the connections may simply not be tight -- or it could be a power issue.

 

I think it's unlikely that having both the SATA and Molex connectors attached made any difference -- it's a single-rail supply, so they're both power from the same PS rail.    I'd think it's more likely the electronics on the cages that cause the behavior folks have complained about whereby power can "hiccup" when one drive powers up -- causing momentary loss of power to another drive.

 

There's really no way to isolate which of those two is causing the problem.    Your best bet is to hope it's not a generic design issue with the enclosures, but that one of them is causing the problems.  Unfortunately, the only way to confirm that is to use one cage at-a-time and populate it with 3 drives.    Run long enough to be confident ... then repeat the process.

 

FWIW, I've stopped using 5-in-3 drive cages in the last couple of UnRAID systems I built.  Not for any particular issues like you've experienced, but for two reasons:

 

(1)  I've seen several cases where the drives would drop out of the UnRAID array, but they were still okay -- they simply needed to be re-seated in the cage (thus re-seating the connections).

 

(2)  I've found that drive temps are controlled MUCH better with these 4-in-3 cases:  http://www.newegg.com/Product/Product.aspx?Item=N82E16817993002&Tpk=coolermaster%204-in-3&IsVirtualParent=1    The 120mm fan blowing directly over the drives does a much better job than the 60-80mm fans on the rear of the 5-in-3 cages.    In addition, since the drives have dedicated cables to them, you can use locking SATA cables, and I've never seen a drop-out in these cases.

 

... so in my recent builds, I've switched to the Cooler-Master cages.  [With one exception:  I'm a BIG fan of the Lian-Li PC-Q25B case for small, low-power systems; and the 5 hot-swap cages in this case have been completely reliable.]

 

It could be either the data connections in the enclosures -- the drives may not have been firmly seated, or the connections may simply not be tight -- or it could be a power issue.

 

I think it's unlikely that having both the SATA and Molex connectors attached made any difference -- it's a single-rail supply, so they're both power from the same PS rail.    I'd think it's more likely the electronics on the cages that cause the behavior folks have complained about whereby power can "hiccup" when one drive powers up -- causing momentary loss of power to another drive.

 

There's really no way to isolate which of those two is causing the problem.    Your best bet is to hope it's not a generic design issue with the enclosures, but that one of them is causing the problems.  Unfortunately, the only way to confirm that is to use one cage at-a-time and populate it with 3 drives.    Run long enough to be confident ... then repeat the process.

 

FWIW, I've stopped using 5-in-3 drive cages in the last couple of UnRAID systems I built.  Not for any particular issues like you've experienced, but for two reasons:

 

(1)  I've seen several cases where the drives would drop out of the UnRAID array, but they were still okay -- they simply needed to be re-seated in the cage (thus re-seating the connections).

 

(2)  I've found that drive temps are controlled MUCH better with these 4-in-3 cases:  http://www.newegg.com/Product/Product.aspx?Item=N82E16817993002&Tpk=coolermaster%204-in-3&IsVirtualParent=1    The 120mm fan blowing directly over the drives does a much better job than the 60-80mm fans on the rear of the 5-in-3 cages.    In addition, since the drives have dedicated cables to them, you can use locking SATA cables, and I've never seen a drop-out in these cases.

 

... so in my recent builds, I've switched to the Cooler-Master cages.  [With one exception:  I'm a BIG fan of the Lian-Li PC-Q25B case for small, low-power systems; and the 5 hot-swap cages in this case have been completely reliable.]

 

Nice on those CoolerMaster cages. Seems like that's the way I'll go if I continue to have problems with this setup. I tossed the boxes on these Thermaltake cages so hopefully I can still return them to Newegg (eek)!

 

FWIW, right now, we're stable at about 12 hours with all drives back in the cages and JUST the SATA connectors hooked up. I'll try and push the system and see if I can get one to fail again and then just scrap them altogether.

 

I agree with you 100% that it seems really unlikely that having both power connectors hooked up would make any difference and that it's more likely a design issue or flaw in one of the cages themselves. However, for the sake of time and effort, I'm going to try this setup one more time with just the SATA power connectors just because. I will definitely report back! Thanks!!

-Dan

12 hrs & counting is a good sign ... although 12 DAYS or so will be a much better sign after all the problems you had before 

 

... It's strange that it would make a difference, but it does seem like perhaps having both power connections connected had some impact (negatively).    It could also be that you simply have the drives better-seated this time, but that also seems unlikely.

 

Unfortunately (based on my own experience) I don't think Newegg will take the cages back without their boxes ... but you could probably sell them on e-bay without too much of a loss.  However, hopefully they'll just work, and we'll never really KNOW why they were causing problems.    In any event, if the problems recur, I'd just bite the bullet and buy some Cooler-Master units 

The only issue with the CoolerMaster cages is that they are just that.  If you need/want to swap the drive, you have to unplug the power/data cables and potentially disturb other drives cables..  This could cause issues.  They do not have a back-plane where the drive can just slide in.

 

If I were purchasing a cage now, I would go with the iSTAR USA BPN-DE350SS in Black.  It looks like they are available in 1, 3, 4 and 5 drive versions.

 

http://www.istarusa.com/istarusa/products.php?model=BPN-DE350SS

Yes, they're just cages 

(a rather obvious point)

 

I've used 5-in-3 cages by iStar, IcyDock, and SuperMicro => they all work well, and yes, they make it very simple to swap drives.    Of the 3, the SuperMicro tends to have slightly better cooling -- probably because it uses a 90mm fan, whereas the iStar and IcyDock both use 80mm fans.

 

But none of them do as good a job of keeping the drive temps down as the CoolerMaster -- which has a 120mm fan blowing air directly over the drives instead of a much smaller, off-center fan drawing air in from behind the drives.

 

For my recent builds, I'm definitely willing to accept the slightly more complex task for replacing a drive to get lower temps [which may reduce the number of times I need to replace them  ];  and to coincidentally save a fair number of $$, since the Cooler-Master costs less than 1/4th the price of the hot-swap cages 

 

By the way, for a small (6-drive) system, there's a nice alternative that provides BOTH better cooling and hot-swap trays:  the Lian-Li PC-Q25B case.  This is a GREAT case for a 6-drive UnRAID setup (I've built 3 with it).    There's a well-ventilated bottom tray (good place to mount the parity drive);  and 5 hot-swap drive slots that are side-oriented, with a 140mm front fan that blows directly over them.

If I was buying now, I'd definitely go with the iStarUSA BPN-DE350 - the all metal construction should be a little more solid than the previous model which I have.  Also, I'm guessing that ventilation holes on the doors are a little less obstructive, allowing better airflow.

 

But none of them do as good a job of keeping the drive temps down as the CoolerMaster -- which has a 120mm fan blowing air directly over the drives instead of a much smaller, off-center fan drawing air in from behind the drives.

 

I wouldn't worry about the standard fans on a drive cage - I removed mine and rely on the case ventilation to draw air in over the drives.  This certainly works because, even during parity check, my drive temperatures remain at 7-8C above ambient.  That is, with an ambient temperature of 33C the other day, my drives stayed at 40C.  With temperatures that high I like to take it easy - but my unRAID server keeps working away!

The drives in my IcyDock's stay reasonably cool as well -- typically mid-30's, hitting 40-41 during parity checks.    But the same model drives in the CoolerMaster cages run in the low 30's, and never exceed 35 during parity checks.    It's simply hard to beat a 120mm fan blowing directly on the drives 

 

By the way, the WD Reds in my Lian-Li PC-Q25B also stay VERY cool -- even during a parity check they've never been above 35.    As I noted before, this is a VERY well designed drive cage, with a 140mm fan blowing directly over the drives from their side.

The drives in my IcyDock's stay reasonably cool as well -- typically mid-30's, hitting 40-41 during parity checks.    But the same model drives in the CoolerMaster cages run in the low 30's, and never exceed 35 during parity checks.    It's simply hard to beat a 120mm fan blowing directly on the drives 

 

However, all of your figures are rather worthless if you don't say what the temperature of the air is which is being blown by the fan.  If you notice, I was quoting drive temperatures when the ambient air temperature was 33C.  In that circumstance I would be unable to maintain my drives 'in the low 30s' without some form of refrigeration.