May 1, 201313 yr The system has 2 IcyDock enclosure and 2 CoolerMaster enclosures, with the same 2TB WD greens in all enclosures. So while I don't recall just hot warm the room was when I last noted those temps; I do know that last week when I ran a parity check the room was at 77F (25C), and the drives in the IcyDocks hit 40, while the parity drive never got above 32 (it's in a CoolerMaster). Regardless of the temps, however, the drives in the CoolerMasters are always 5-6 degrees C cooler than those in the IcyDocks. In fact, if I cool the room down to 72 or so (22C), the drives in the CoolerMasters never even hit 30. Same system; same drives -- in fact, I have in the past experimented with changing the relative locations of the enclosures (IcyDocks on the top part of the case vs. on the bottom part. Doesn't matter -- the CoolerMasters do a better job of cooling. For a build with up to 12 drives, a single Cooler-Master in an NZXT-H2 case would be good option. There's a description of that case here: http://lime-technology.com/forum/index.php?topic=26154.0 ... the front fans can be removed in ~ 1 second to get access to the tool-less drive trays; and you could mount a CoolerMaster in the top bays to add 4 more drives. Too bad they don't make a version with 3 of the removeable fan units instead of the 5.25" bays on top The build detailed in that case shows temps in the low-mid 30's during a parity test ... about the same you get with CoolerMasters. It's not, of course, surprising -- better airflow is ALWAYS going to do a better job. That's also why Lian-Li does such a nice job -- the hot swaps are mounted sideways, so the 140mm fan blows directly through the side of the compartment and directly over the drives. Don't misread this -- I'm not against the hot-swap cages at all ... I have 5 of them, and have installed a couple dozen more; and have no intention of changing them out. I simply don't use them anymore, since the CoolerMasters keep the temps so much lower.
May 1, 201313 yr By the way, I'm surprised you keep your equipment in 90+ degree rooms!! (33C ambient)
May 1, 201313 yr By the way, I'm surprised you keep your equipment in 90+ degree rooms!! (33C ambient) My house is open plan and running air-con to cool it would be horrendously expensive. In any case, I'm not particularly fond of air-con. Now, just after 9pm, the temperature is 29.7C.
May 1, 201313 yr By the way, I'm surprised you keep your equipment in 90+ degree rooms!! (33C ambient) My house is open plan and running air-con to cool it would be horrendously expensive. In any case, I'm not particularly fond of air-con. Now, just after 9pm, the temperature is 29.7C. He probably does not know where you live.
May 1, 201313 yr Author Oh man, I was JUST about to post in here that this was SOLVED but lo and behold a disk (not the parity disk) just dropped out of the array. It had been running since Sunday with no problems and was just running a parity check when I performed a transfer (that did not include the newly failed drive) and it dropped out. I'm SUPER bummed. Ugh. I guess I'll track down some different drive cages and go with that. Also, just to add to the drive temp conversation... ...the joys of having a nice cool basement!
May 1, 201313 yr By the way, I'm surprised you keep your equipment in 90+ degree rooms!! (33C ambient) My house is open plan and running air-con to cool it would be horrendously expensive. In any case, I'm not particularly fond of air-con. Now, just after 9pm, the temperature is 29.7C. He probably does not know where you live. Yes, I do ... I've even been there (although it was a L..O...N...G time ago
May 1, 201313 yr Also, just to add to the drive temp conversation... ...the joys of having a nice cool basement! Now THOSE are some good temps !! I do indeed wish we had a nice cool basement. Sorry to hear about the drive issue. I think it's time to abandon those cages !! Needless to say, you know what I'd do at this point
May 1, 201313 yr Author What was the error in the syslog that preceded the most recent drop-out? Argh. I'm a idiot. I didn't download the syslog before rebooting the host. I did see a lot of these (which seem to be occurring very frequently... like a couple of times every minute). May 1 07:50:48 hoarder kernel: ata7.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen (Errors) May 1 07:50:48 hoarder kernel: ata7.00: failed command: IDENTIFY DEVICE (Minor Issues) May 1 07:50:48 hoarder kernel: ata7.00: cmd ec/00:00:00:00:00/00:00:00:00:00/00 tag 0 pio 512 in (Drive related) May 1 07:50:48 hoarder kernel: res 40/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout) (Errors) May 1 07:50:48 hoarder kernel: ata7.00: status: { DRDY } (Drive related) May 1 07:50:48 hoarder kernel: ata7: hard resetting link (Minor Issues) May 1 07:50:49 hoarder kernel: ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300) (Drive related) May 1 07:50:49 hoarder kernel: ata7.00: configured for UDMA/133 (Drive related) May 1 07:50:49 hoarder kernel: ata7: EH complete (Drive related) If it happens again, I'll TRY to remember to grab the syslog before rebooting. Off to look for new cages in the meantime I guess.
May 1, 201313 yr Author Went ahead and pulled the trigger on 3x CoolerMaster cages as per the recommendation. Super bummed though. I really liked the aesthetics of these cages. Oh well, if I can get 1/2 of what I paid for these Thermaltakes on eBay I'll still have saved some scratch. Here's hoping this really is the issue!
May 1, 201313 yr From all your testing, it sure seems like that was indeed the issue; but in this business nothing's ever "absolute." However, your tests without using the cages were consistently perfect --right? ... and using the CoolerMaster's is essentially just that -- the cages just hold the drives and have no interaction with the connecting cables, so there shouldn't be any problem.
May 1, 201313 yr Author From all your testing, it sure seems like that was indeed the issue; but in this business nothing's ever "absolute." However, your tests without using the cages were consistently perfect --right? ... and using the CoolerMaster's is essentially just that -- the cages just hold the drives and have no interaction with the connecting cables, so there shouldn't be any problem. Yup... sure seems that way. Luckily my house is only one story so if I choose to jump out the window in the event that this doesn't solve my problem I'll likely survive and be able to come back and continue troubleshooting. Here's hoping!!!
May 7, 201313 yr Author Well... I'm back! :'( Ran on the old Thermaltakes for a while because they seemed fairly "stable". Most recent failure made me decide to go ahead and move over to these cooler master cages and what do you know... more drive failures within 12 hours of getting those installed. I haven't gone through it yet but syslog is here: http://squirrel.danmackin.info/syslog.1 I'm pretty bummed. I guess I'll look at a different PS/SATA cards now? Ugh. -Dan EDIT: I mean, it's got to be the power supply now right? So far this morning I've gotten errors on (4) drives that are connected directly to both the mainboard and the PCI-E controllers. I guess it could also be the motherboard/CPU itself but that seems really unlikely. Thanks for your help.
May 7, 201313 yr Bummer. Well at least you know for certain it's not the drive cages, since they have NO electronics involved in the process. I'd reseat all of the cables (data and power) and try one more time before buying another power supply. Also double-check that all of the SATA connections are tight (unplug/replug them). I don't recall if it's been mentioned earlier in this thread (didn't see it with a quick re-scan), but did you pre-clear these disks for a couple cycles? If not, is it always the same drive(s) that are dropping out?
May 7, 201313 yr Author Bummer. Well at least you know for certain it's not the drive cages, since they have NO electronics involved in the process. I'd reseat all of the cables (data and power) and try one more time before buying another power supply. Also double-check that all of the SATA connections are tight (unplug/replug them). I don't recall if it's been mentioned earlier in this thread (didn't see it with a quick re-scan), but did you pre-clear these disks for a couple cycles? If not, is it always the same drive(s) that are dropping out? I definitely re-seated all cables (data and power) when I switched over to the CoolerMaster cages. I did pre-clear every disk that's in there but only once. I guess I thought that because all of the drives were in use without fail in their previous roles that there wouldn't be a need to pre-clear multiple times... and no, it's seemingly random which drive drops out. All SATA connections are tight with locking data cables so I feel pretty good about that. I guess I'm just trying to figure out what else it could be. Something in the BIOS that's powering down these drives (doubtful)? Power supply issues (maybe more likely)? Controller card issues (I see as many problems with the mobo controllers as I do the PCI-E controllers)? I guess I would just like opinions on what the next piece to rule out with a replacement would be... seems like previous threads point to PSU. With that in mind, either of these should be fine (if not overkill) correct? http://www.newegg.com/Product/Product.aspx?Item=N82E16817139011 http://www.newegg.com/Product/Product.aspx?Item=N82E16817151102 Looks like I can get that SeaSonic for $120 if I order before tomorrow... would love opinions here!! THANKS ALL! (I guess, like you said, the silver lining is that it's (mostly) definitive that it's not the cages. I like(d) the Thermaltake cages!)
May 7, 201313 yr I certainly had issue on my system with drives randomly dropping out that have completely disappeared since I upgraded the power supply. Therefore there is a good chance that might be the problem area as it can be hard to diagnose with relatively random failures.
May 7, 201313 yr Author I certainly had issue on my system with drives randomly dropping out that have completely disappeared since I upgraded the power supply. Therefore there is a good chance that might be the problem area as it can be hard to diagnose with relatively random failures. Good enough for me. Upgrading PSU... now. Thx for chiming in!!
May 7, 201313 yr I had a nightmare last week with random disks disconnecting and red balling. I almost drove myself insane changing cables, but after ruling everything else out I bought a proper psu (corsair gold 750w) and now everything is perfect. 3 par checks with 5 other vms in esxi running and no problems.
May 7, 201313 yr Author I had a nightmare last week with random disks disconnecting and red balling. I almost drove myself insane changing cables, but after ruling everything else out I bought a proper psu (corsair gold 750w) and now everything is perfect. 3 par checks with 5 other vms in esxi running and no problems. Another vote for a new PSU! Glad to hear it. I overnighted the SeaSonic X-850 so hopefully I'll tag this one as solved within a week or so. I'm kinda butthurt that my existing PSU is outside the 30 day return period at Newegg. Blargh. I guess that's what eBay is for.
May 7, 201313 yr High quality, stable power is ALWAYS a good idea FWIW, the only power supplies I buy are Seasonic, PC Power & Cooling, and the Corsair HX & AX units
May 7, 201313 yr By the way, while you're upgrading to ensure stable power, is your system on a UPS? I've seen more failures (not just on UnRAID systems, but PCs in general) due to "unplanned" power outages -- i.e. instantaneous power loss rather than an orderly shutdown -- than any other cause. And of course with UnRAID there's the additional inconvenience of a full parity check automatically being run after any improper shutdown. In any event, ALL UnRAID systems should have a UPS A good choice: http://www.newegg.com/Product/Product.aspx?Item=N82E16842101381 ... but regardless of what model you get, be sure it has AVR
May 7, 201313 yr Author This is SOOO awesome... Just for giggles, while I'm waiting for my new PSU, I decided to try and bring the array back online and run a parity check. No problems there... ran through fine and everything was up and running. As soon as I tried to access some data off of the NFS share... bam! Parity drive and two other disks drop out. One was on the MoBo controller and one on each of the PCI-e controllers. Just amazing. Smells more and more like a PSU issue with every passing error.
Archived
This topic is now archived and is closed to further replies.