2 Disk Issues

[thefly]

Hey team.  Just returned form a vacation to an extended power outage.  Upon restarting I was presented with a disk outage (Disk 8 - see image).  I shut down and inserted an new drive.  When restarting and preparing to initialize,  Disk 10 is now showing and error (Disk 10 - see image). Am I in major trouble?

[JorgeB]

4 minutes ago, thefly said:

Am I in major trouble?

Possibly, post your diagnostics, ideally from when the 2nd disk was disabled and before rebooting.

[thefly]

Here are the diagnostics

tower-diagnostics-20180314-1243.zip

[JorgeB]

Unfortunately the diags are after rebooting, so can't see what happened to disk10, but it's not in the best health:
 

ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  5 Reallocated_Sector_Ct   0x0033   001   001   036    Pre-fail  Always   FAILING_NOW 4095

You can try re-enabling disk10 to rebuild disk8 but it might fail again, if it does grab the diagnostics before rebooting, also keep old disk8 intact in case it's needed, you're using a very old version, this procedure should work but can't be sure, to re-enable disk10:

 

-Tools -> New Config -> Apply (IIRC there are no retain assignments options on v6.1.9, so you'll need to re-assign all disks)

-Main -> Re-assign all disks in the same order they were, double check assignments.

-Important - After assigning all disk leave the browser on that page, the "Main" page.

-Open an SSH session and type:

 

mdcmd set invalidslot 8

-Back on the GUI do not check the "parity is already valid" box and start the array, disk8 will start rebuilding.

[thefly]

Please bear with my ignorance.  I have not ssh'd in over 3 years.  I have a monitor connected to the box.  Login is root and takes me to root@Tower:~#. I then enter mdcmd set invalidslvot 8 [single space between entries, then return]. Good?

[JorgeB]

Yes, you can copy/paste from the code line

[trurl]

35 minutes ago, johnnie.black said:

you're using a very old version,

Possibly newer version could have saved you from getting to this state, where you were trying to rebuild one disk with another already failing. Does that old version have Notifications? 

[thefly]

It has archived notifications in the tools section

[trurl]

25 minutes ago, thefly said:

It has archived notifications in the tools section

In Settings, does it have Notifications? Have you configured it to send you emails? That is a lot of reallocations to appear all at once so I'm thinking it should have told you about them before the other disk had problems.

 

[thefly]

I have completed the prior instructions. Is this the process I should be achieving to repair?

[trurl]

Assuming you got there by following johnnie's instructions I think that must be OK. Whatever you do, don't format anything. Looks like there is probably going to be some filesystem repair after the rebuild.

[JorgeB]

Not a good sign disk10 being unmountable, disk8 is likely a consequence of that, but let the rebuild finish and then run a filesystem check on both, it's reiser so there's a good change it's fixable, unless something catastrophic happened with disk10 when it was disable, pity there are no diags.

[JorgeB]

Strike that, looking more carefully at the screenshot you're doing a parity sync instead of rebuilding disk8, so rebuilding disk8 is not an option anymore, old disk8, if not completely dead, is your best option to recover that data.

[trurl]

1 hour ago, johnnie.black said:

Strike that, looking more carefully at the screenshot you're doing a parity sync instead of rebuilding disk8, so rebuilding disk8 is not an option anymore, old disk8, if not completely dead, is your best option to recover that data.

I saw that but thought maybe the webUI was just wrong about the invalidslot command since it wasn't initiated there.

[JorgeB]

6 hours ago, trurl said:

I saw that but thought maybe the webUI was just wrong about the invalidslot command since it wasn't initiated there.

No, either the OP missed a step or invalid slot is not working correctly on v6.1.9, I know for sure it works on v6.2 and newer but it's been a long time for v6.1.

 

OP, you should still let the parity sync finish, this will at least result in a protected array if it's successful, then run reiserfsck on disk10, and finally try to mount old disk and/or post a SMART report.

[thefly]

Based on some differing posts, I stopped the process last night and performed a clean shut down.  I restarted this morning and before bringing the array online could I please get direction of the steps I should take this time:

 

1.        Repeat johnnie.blacks attempt to re-enable disk 10 or

2.       do not check the "parity is already valid" box and start the array or

3.       another option

 

In notice my option to bring the array online only allows: Start will bring the array on-line and start Parity-Sync.

 

Thanks.

[JorgeB]

Option to rebuild disk is no more, since parity was overwritten, best option now is what I posted above:

 

6 hours ago, johnnie.black said:

OP, you should still let the parity sync finish, this will at least result in a protected array if it's successful, then run reiserfsck on disk10, and finally try to mount old disk and/or post a SMART report.

 

[thefly]

Bigger issues now. Created a new config:

 

1. disk 2 has now disappeared from the selection list;

2. disk 8 is showing the new disk that I previously entered

3. disk 10 is unassigned

 

[trurl]

4 minutes ago, thefly said:

Bigger issues now. Created a new config:

 

1. disk 2 has now disappeared from the selection list;

2. disk 8 is showing the new disk that I previously entered

3. disk 10 is unassigned

 

1. You need to figure out why it isn't detecting disk2

2. Don't understand why this is an issue. Do you think it should be showing something else?

3. Is there some issue with reassigning it?

[thefly]

Disk 2 was not a problem previously. It is now gone for assignment.  Is there a terminal command to try and re-mount it?

[trurl]

1 minute ago, thefly said:

Disk 2 was not a problem previously. It is now gone for assignment.  Is there a terminal command to try and re-mount it?

It's not going to be mounted until the array is started. Mounting is not what you need. You need unRAID to see the drive is there available for assignment. Do you see it in Unassigned Devices? My guess is it has dropped offline for some reason.

[thefly]

The drive is not available for assignment anymore.

[JorgeB]

18 minutes ago, thefly said:

Bigger issues now. Created a new config:

There was no need for doing a new config, but if disk2 is missing there's a problem,  post new diags.

[thefly]

Attached

tower-diagnostics-20180315-1034.zip

[JorgeB]

Both disks 2 and 10 dropped offline, rebooting should bring them online, but there are problems there, either connection/cable, power supply, etc.