After a restart of my server, the system came back to life stating that no disks were assigned 

 

Fortunately, I had a screenshot of the disk assignments so it did not take long to reestablish this.

 

However, it stated that parity was invalid. When started in maintenance mode, it says (when initiating parity sync) that the disk array is 4 TB (where it should be 13 TB). So I cancelled the parity sync for now.

 

All disks mark as passed on SMART test, so there doesn't - on the surface - seem to be any HW related trouble.

 

Do I just proceed to do parity sync or is there something else I should do first?

 

Thanks in advance for any advice!

 

/Thomas

First, tell us what version you are running.

 

Second, you need to attach a syslog.  See here for instructions on how to do that:

 

      http://lime-technology.com/forum/index.php?topic=9880.0

Hi Frank,

 

thanks for replying. It was late last night where I live, so I missed basic info 

 

System is 5.05 - will come back with log when I return home from work.

 

/Thomas

Running unRAID version 5.05.

 

System: SuperMicro X7SPA-HF, 4GB RAM

 

Add-ons: OpenSSH, CrashPlan, APC UPS and Dynamix WebGUI.

 

 

Server started, array started in maintenance modem parity sync cancelled.

 

The attached log was then copied.

Towerlog.zip

I am NOT real expert in these matters but I see that Disk 2 has a lot of errors. 

 

I would open up the case and check both power and SATA cables going to the disk and make sure that they are secure.  Since you will be messing with the cables check to make sure that you haven't disturb any of the others.  (The cable connectors that were designed for SATA drives leave much to be desired from the standpoint of providing a rugged secure connection!)   

I have checked now, and nothing seems out of the ordinary. I have lockable SATA connectors, and everything was in place.

 

Ran a short SMART selftest which passed.

 

As long as I have the disk order right, all that can happen by running the parity sync in Maintenance mode is that parity is updated.

 

Actual data on the data disks, won't be touched, right?

Right, went out on a limb and started the parity sync - which turned out was very slow and revealed countless read errors on parity drive.

 

Cancelled that, powered down the server and replaced the SATA cable to the parity drive. Powered up, entered maintenance mode and started parity sync again.

 

This time, things went better - however, after 30 minutes I took a look at the log which again showed many read errors - and I can see that the sync speed occasionally goes through the floor, only to revert to good speed again. And apparently some trouble talking to one of the other disks (sdd). I've attached the log - near the end the read errors can be seen - excerpt below:

 

Nov  7 22:09:54 Tower kernel: md: disk2 read error, sector=3221320
Nov  7 22:09:54 Tower kernel: md: disk2 read error, sector=3221328
Nov  7 22:23:19 Tower emhttp: shcmd (255): /usr/sbin/hdparm -y /dev/sde &> /dev/null
Nov  7 22:37:14 Tower kernel: ata3.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x0
Nov  7 22:37:14 Tower kernel: ata3.00: irq_stat 0x40000001
Nov  7 22:37:14 Tower kernel: ata3.00: failed command: READ DMA EXT
Nov  7 22:37:14 Tower kernel: ata3.00: cmd 25/00:00:b8:d9:2b/00:04:07:00:00/e0 tag 0 dma 524288 in
Nov  7 22:37:14 Tower kernel:          res 51/40:4f:60:da:2b/00:03:07:00:00/e0 Emask 0x9 (media error)
Nov  7 22:37:14 Tower kernel: ata3.00: status: { DRDY ERR }
Nov  7 22:37:14 Tower kernel: ata3.00: error: { UNC }
Nov  7 22:37:14 Tower kernel: ata3.00: configured for UDMA/133
Nov  7 22:37:14 Tower kernel: sd 3:0:0:0: [sdd] Unhandled sense code
Nov  7 22:37:14 Tower kernel: sd 3:0:0:0: [sdd]  
Nov  7 22:37:14 Tower kernel: Result: hostbyte=0x00 driverbyte=0x08
Nov  7 22:37:14 Tower kernel: sd 3:0:0:0: [sdd]  
Nov  7 22:37:14 Tower kernel: Sense Key : 0x3 [current] [descriptor]
Nov  7 22:37:14 Tower kernel: Descriptor sense data with sense descriptors (in hex):
Nov  7 22:37:14 Tower kernel:         72 03 11 04 00 00 00 0c 00 0a 80 00 00 00 00 00 
Nov  7 22:37:14 Tower kernel:         07 2b da 60 
Nov  7 22:37:14 Tower kernel: sd 3:0:0:0: [sdd]  
Nov  7 22:37:14 Tower kernel: ASC=0x11 ASCQ=0x4
Nov  7 22:37:14 Tower kernel: sd 3:0:0:0: [sdd] CDB: 
Nov  7 22:37:14 Tower kernel: cdb[0]=0x88: 88 00 00 00 00 00 07 2b d9 b8 00 00 04 00 00 00
Nov  7 22:37:14 Tower kernel: end_request: I/O error, dev sdd, sector 120314464
Nov  7 22:37:14 Tower kernel: md: disk2 read error, sector=120314400
Nov  7 22:37:14 Tower kernel: md: disk2 read error, sector=120314408
Nov  7 22:37:14 Tower kernel: md: disk2 read error, sector=120314416

 

Could it be that the errors are actually being produced by the other drive?

Towerlog2.zip

I don't know if this might be a part of the problem but read this thread....

 

    http://lime-technology.com/forum/index.php?topic=36065.0

I've changed both the SATA cable for the parity drive (sdb) and the drive that seems to act up (sdd) with non-lock SATA cables. No change, still getting many read errors on parity drive and the beforementioned log entry stating some kind of problem with the data drive (sdd).

 

Starting to wonder about how to recover from this.

 

If the data drive (sdd) is the culprit but it has screwed up the parity (sdb), how do I get to a situation where I can replace the data drive without data loss?

 

I'm thinking along the lines of this:

 

1. Run Reiserfsck to save as much from the data drive as possible first

2. Then remove it, mount it on another computer and migrate the data to an external disk

3. Put a new drive in the array to replace it

4. Move data to new drive from external disk

5. Rebuild parity

 

Or is there a better way? I've toyed with the idea that the onboard SATA controller could be at fault. This could possibly be tested by either

 

A. Moving to another platform; or

B. Moving the disks onto an M1015 I have available, still in the same server.

 

However, I am a bit leery of "migrating" something that is not synced and with an unresolved problem.

 

I'd rather have a lengthy process than suffer too much data loss. I should note that I have backup of essential stuff, but there's still a lot media files I'd hate having to rebuild.

You are right in assuming that if you have a drive playing up then you have probably messed ip your parity.

 

However past experience has shown that in many cases when unRAID 'red balls' a drive it is because a write has failed for an intermittent reason.  Therefore as long as the 'problem' drive is still working at the physical level then the data is probably intact (or at least the data put there before the red-ball happened).

 

I think your basic approach looks OK.  However I have a few comments

• Reiserfsck may not required.  You could try mounting the drive manually with the array stopped to see if that works.  If that works then the reiserfsck is probably not needed.
  
• If the array is stopped then you could plug in another drive to copy data to
  
• Another approach is to create a USB stick using the 'free version of unRAID and put the problem drive plus another one on a different system.  With that you could create a 2-drive array and copy the data from the problem drive to the other drive.  Once completed that drive could be put into back into the main unRAID server and you could do a 'new config' to define the new disk layout.  Since the new disk in this case would already be in unRAID format it would be added with its data left intact.  You could therefore start immediately doing the initial parity sync.
  

 

itimpi,

 

thank you for the reply. The status right now is that as a replacement for the iffy 3TB data drive (sdd), I have acquired a new 4TB drive which is currently undergoing preclearing on another computer. On the main system, parity sync is about 5 hours away from completing - in Maintenance mode with no drives mounted.

 

1. Is it safe to assume that if the parity sync completes, I can attempt starting the array and if it mounts the data drive (sdd), I don't need to run Reiserfsck. Or will starting the array have bad consequences if the data drive (sdd) doesn't mount?

 

2. Assuming bullet 1 is positive, I will put the data drive (sdd) in the other system with the new 4TB drive once it preclears. Do I then "just" copy every folder and file (using MC) to the new drive?

 

3. Once I re-introduce the now populated 4TB drive into the array, I am to do a "New Config" - will everything regarding users, user shares etc be intact or will I have to set that up again? Is it really so simple that it is just a matter of moving the files and folder structure to a new drive, and then slide that into the place of the old one, do "new config" and assign drives, and re-run parity sync? :-)

 

 

New drive just about precleared now. If it passes, I am set to move on.

 

Parity sync completed as well on main system.

 

Rebooted the main system to see if anything came up, and when it didn't spring to life, I checked the console. It was stuck at this

 

  "AHCI Port 2 device error. Press F1 to resume." - port 2 corresponds to the 3TB drive.

 

So clearly, something is going on with it.

 

Update: I removed the iffy drive and took a look at it from another system - while I haven't scrutinized every file, it seems that everything is intact. So I've taken the plunge and put the 4TB in and started a Data Rebuild, based on the assumption that while the drive was acting up when accessing it, there doesn't seem to be any data corruption.

After a lot of concern, things actually worked out pretty well.

 

Even though the data drive was the culprit and was acting up and had messed up parity, I managed to complete a new parity sync.

With this in place, I could replace the data drive with a new one (after pre-clearing) and start re-building.

 

This has now completed and I am happy to say that I have not yet found problems. Everything looks to be intact.

 

All is right with the world again. And an RMA has be filed with WD :-)

 

Frank1940 and itimpi: Thank you for the support!