Lots of fails

[Dal1980]

Hi

 

I had some issues last week which got resolved by the help of some folks on here (thanks again guys). It was initially down to a data corruption of the USB boot drive of unRaid which failed again last night so I've replaced it with a new USB stick. I also had a HDD in the data array fall off completely and not even BIOS could read it (checked cables and SATA ports). I bought a brand new 4 TB NAS IronWolf drive to replace this very old 2TB (Green) WD (which didn't surprise me that it gave up after only a year). Everything was fine until I noticed last night that my other much newer 4TB WD (black, I think) seems to be having issues. I have a red cross next to the drive on the dash.

 

That's 3 drives failing all at the same time pretty much. I always safely power down the device if I need to turn the electric off (which isn't very often, maybe 3 times a year) and I don't have power cuts very often (maybe once or twice a year).

 

It was originally the USB drive that caused an issue after I tried to restart unRAID as I was having issues with PLEX (turned out PLEX external server was the issue and nothing to do with my box). I wanted to mention this as it wasn't as if there was any power issue that somehow surged everything (even though I do have surge protection).

 

Is this just bad luck and extreme coincidence?

 

 

[testdasi]

Multiple drives don't usually drop dead (completely dead, not just a write error).

If that happens, the first culprit to suspect is power.

 

Your PSU could be fine but it could be other factors e.g. bad connection arcing etc.

Recently there's another user on here who lost 5 drives because of a bad 5-bay rack.

 

And then your PSU could also be the problem.

Surge protection only protects against surge into the PSU.

A broken PSU can lose control of the voltage / current (out of it) and sensitive electronics needs less than 1V surge to die.

 

If I were you, I would double check the fail drives on a 2nd computer to see if they show up.

If they do then use smartmontools to check SMART to see if there's any clue.

Would be even better if the other computer can read xfs to have a look at the data.

 

Edited February 13, 2020 by testdasi

[Dal1980]

Thanks testdasi

 

I do have access to a PC I could use as a test bench. I think it has windows on it but could easily dual purpose and install debian or ubuntu on there to read XFS (something I don't think I've ever done). I'll at least report back with what SmartMonTools says about the drives.

 

Luckily, I'm replacing the kit with a updated mobo and Ryzon 5 chip (https://uk.pcpartpicker.com/user/Dal1980/saved/4fCz7P), I was trying to keep cost down as its already cost me quite a bit for 2 parity drives, cases and my replacement drive. The PSU is a good quality one but it is quite old so maybe its that. 

 

 

[itimpi]

6 hours ago, Dal1980 said:

Thanks testdasi

 

I do have access to a PC I could use as a test bench. I think it has windows on it but could easily dual purpose and install debian or ubuntu on there to read XFS (something I don't think I've ever done). I'll at least report back with what SmartMonTools says about the drives.

 

You could always boot one of the Linux ‘Live’ distributions that lets you run Linux without the need to install it anywhere.  That would give you the ability to read XFS format (and potentially copy files to a Windows NTFS formatted drive).

[Dal1980]

I don't know if it was me being exhausted last night, the drink I had or if things are just different today but the result changed. I've revisited the situation again today and found the following:

 

I'm working on just the 4TB to start with:

4TB drive (showing with a red cross next to it in unRAID), I've put this into a USB caddy and its showing up on Ubuntu on the "disks" utility.

 

I've installed SmartMonTools for Ubuntu and skipped the Postfix settings as I thought I'd apt installed the wrong tool as postfix is an email thing.

 

I've since ran

 

sudo smartctl - i /dev/sdc

Which provided me with info about my WD drive and it supporting SMART (and it being enabled)

 

sudo smartctl -c /dev/sdc

Which showed me that the extended test would take 506min (which is fine with me)

 

sudo smartctl -t long /dev/sdc

Which then started the test and told me to check back in 5hrs 26 minutes.

 

I'll run the following commands to gather further information after this period has passed

sudo smartctl -l selftest /dev/sdc
sudo smartctl -a -d ata /dev/sdc

 

Is everything I'm doing correct so far?

 

 

Edited February 14, 2020 by Dal1980
Readdress the problem with clear head

[Dal1980]

Is there anything I can do to test PSU?

 

It's a Cooler Master 1000W Silent Pro (80 Plus Bronze) Semi Modular (http://us.coolermaster.com/product/Detail/powersupply/silent-pro-m/silent-pro-m1000.html)

My Mobo is P6X58D-E (https://www.asus.com/Motherboards/P6X58DE/specifications/)

 

Array was rebuilt last May (2019)

2x Seagate 6 TB IronWolf 3.5 Inch 7200 RPM Internal Hard Drive for 1-8 Bay NAS Systems (256 MB Cache, 180 TB/Year Workload Rate, Up to 210 MB/s) for parity drives

2x Crucial MX500 CT250MX500SSD1(Z) 250 GB Internal SSD (3D NAND, SATA, 2.5 Inch) for cache

 

My data drives are the two I'm testing (2TB WD green and 4TB WD blue) of which the 2TB has now been replaced with a Seagate IronWolf 4 TB 3.5 inch Internal Hard Drive for 1-8 Bay NAS Systems (5900 RPM, 64 MB Cache up to 6.0Gb/s, 180 TB/Year Workload Rate)

 

 

Edited February 14, 2020 by Dal1980
typo

[JonathanM]

1 hour ago, Dal1980 said:

Is there anything I can do to test PSU?

Depends on your skill level and test equipment at your disposal.

[Dal1980]

I have a multimeter but not sure what I'm looking for so not sure if that answers anything lol

[itimpi]

1 minute ago, Dal1980 said:

I have a multimeter but not sure what I'm looking for so not sure if that answers anything lol

You can pick up PC Power supply testers quite cheaply on Amazon.     Not sure if they test what happens under load but they do check that all the voltages on the different connectors (e.g. motherboard, molex. SATA etc) are correct.

[Dal1980]

Report is attached from SMART

 

 

[Dal1980]

Tried my 2TB drive this morning, it doesn't even show up in the Disk utility on Ubuntu. Think this drive is beyond broke.

 

Both drives are out of warranty 

WCC4E0ESURC2Out of Limited WarrantyWD40EZRZWD Blue 01/04/2018

WCAZA4495628Out of Limited WarrantyWD20EARSWD Green / GP 03/19/2014

 

I knew my 2TB wouldn't last but surprised my 4TB died at pretty much the same time. I just hope that I haven't lost any data (can 2 parity have 2 drives fail?)

 

I'm getting another 4TB soon and then I'll be upgrading the kit.

 

As far as PSU fail, how would any of us know what to do to detect that? I'm really not sure if I should spend £200 to replace this since I've got no evidence that it is failing.

 

Thoughts anyone?