I am running 5.0-rc4 and have been for a year or more now with zero problems.  However, I have just had disk 5 of my array become "red balled".  I replaced it with a brand new drive and started a rebuild, however the rebuild causes the webpage to become unresponsive after some time and the rebuild will not complete.  Initially the rebuild does start and has a speed of 50-60MB/s with estimated time to completion of ~800 minutes.  Once the web GUI went down I went ahead and let the server run thinking it would go ahead and complete the rebuild.  Two days later and everything is still unresponsive.  The array appears to still be online because I'm able to telnet in, however I am not able to access any shares over the network and the server doesn't seem to function correctly to commands.  For example, I'm not even able to force a shutdown using "powerdown".  When typing this command I get no response.  I have rebooted the system and attempted the rebuild 3 times now with the same outcome each time.  Attached is a syslog after I rebooted the system, started a data rebuild and let it run for 24 hours. 

 

Motherboard is a Biostar A880G+

SATA Expansion Card is Supermicro AOC-SASLP-MV8 8-Port SAS/SATA Add-on Card

syslog-11-6-2014.txt

Did you preclear your new drive?

 

Did you double (and triple) check that all cable (SATA and power) connections are secure on ALL drives.  (while replacing one drive, it not uncommon for the connector on another drive to be loosened!) 

 

Also, get a smart report on the new drive.

No, I did not pre-clear the new drive.  I have pre-cleared every time I've added a new disk...except this time.

 

Yes, I double checked all cables.

 

I am unable to get a smart report on the new drive.  My server is acting up more and more.  Most of the time it will not boot to completion, hangs in the middle, and when it does appear to boot it doesn't mount the new drive.  Also, when it does boot "powerdown" and "reboot" commands don't work.  Usually, when trying

"powerdown" the console shows a message of

 

ls:  cannot access /dev/sd[a-z]

 

Only way to turn off the server is to pull the plug. 

 

The web gui is no longer accessible at all, ever.

 

Below is a syslog when the server boots but is unresponsive to any commands.

syslog-11-8-2014.txt

Recently, I had a parity drive on my Media Server start throwing up a lot of errors.  I shut the array down, purchased a new hard drive, preclear it twice and used this drive to replace the one with errors.  I then attempted to preclear the old drive in my Test Bed Server hoping it had had some transient problem.  It locked up that server with much the way as your server did.  Removing the drive from the server restored  it to operation.  I decide it was not worth the effort  testing any further, so I destroyed it and tossed it in the trash!

 

By the way, it not unknown to have a drive be DOA...

Well, that was my next thinking.  Remove the new drive and see if I can boot up.  I don't know the first thing about syslog interpretation and was hoping someone here would look at it and say "ah-ha", there's your problem.  Maybe not so simple to find the problem.

 

The data on the failed drive I'm trying to rebuild is not that important to me.  It's just some movies I had backed up.  My main concern is to get the server back up and running and if that means losing the data on the failed disk so be it. 

If you only have one failed drive, you should be able to remove it, then start and use the server pretty much normally, the missing drive will be emulated by the rest of your data drives + parity. You could then copy the data from the emulated drive to another location if it would be easier than rebuilding from backups.

 

Hopefully you only have one failed drive. Do you still have the drive that red balled? Are you sure it's dead? Can you get a smart report on it? I haven't looked at your syslog, but your symptoms seem to indicate more than just a single failed drive.

I tried removing the failed drive from the server completely, as jonathanm suggested, but still no luck.  The server still appears to boot normally, but web gui is not reachable, and when using "powerdown" command the machine hangs and never powers down.  Attached is a syslog after a boot up and nothing else. 

 

What would be the best way for me to start fresh while maintaining the data I have on all of my presumed good drives?  Can I just take my flash drive and upgrade to the 5.0 stable release and then rebuild parity?  Seems like doing this would eliminate the software as a possibility of being the culprit.  Then if I still have problems it would mean it's most likely hardware related, right?

syslog-11-11-14.txt

Drive with # 9YN1 is having problems. Did you move this drive between the last 2 system logs?

 

 

Nov  7 08:21:58 Tower kernel: scsi 0:0:3:0: Direct-Access    ATA      ST3000DM001-9YN1 CC95 PQ: 0 ANSI: 5

Nov  7 08:21:58 Tower kernel: sd 0:0:3:0: [sdi] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB)