I am out of space in my array and empty hard drive trays.  My only option now is to start replacing disks with larger disks.  To this end I bought a new 8TB drive to replace my 3TB parity drive.  During the swap when the array was coming back online with the 8TB array something went wrong.  I don't know what happened but now one of the member disks won't start back up.  I have swapped its location in the enclosure and updated the firmware on my LSI 9211-8i adapter (thinking it might be too old to support 8TB drives).  Now I am missing one member disk and I have a disk configuration that expects a blank 8TB parity drive.  Anyone know how I can force the configuration to take the old (and still working fine) 3TB parity drive so I can recover my dead member disk?

 

TLDR: Tried to swap in larger parity drive, rebuild failed when a member disk died, now I have no parity and a dead disk.  Need to reconfigure disks to take back the old smaller parity drive.

 

DIAG: bane-diagnostics-20190723-2140.zip

Edited July 24, 20196 yr by ngurganus

Go to Tools - Diagnostics and attach the complete diagnostics zip file to your next post.

This will only work if old parity is still valid:

 

-Tools -> New Config -> Retain current configuration: All -> Apply
-Assign any missing disk(s) including old parity and a new disk1
-Important - After checking the assignments leave the browser on that page, the "Main" page.

-Open an SSH session/use the console and type (don't copy/paste directly from the forum, as sometimes it can insert extra characters):

mdcmd set invalidslot 1 29

-Back on the GUI and without refreshing the page, just start the array, do not check the "parity is already valid" box (GUI will still show that data on parity disk(s) will be overwritten, this is normal as it doesn't account for the invalid slot command, but they won't be as long as the procedure was correctly done), disk1 will start rebuilding, disk should mount immediately but if it's unmountable don't format, wait for the rebuild to finish and then run a filesystem check

Edited July 24, 20196 yr by johnnie.black

@ngurganus

 

Since you edited your first post to attach your diagnostics instead of attaching them to your next post as I asked, this thread didn't appear to have anything new in it. So I didn't come back to it until johnnie.black made his post. If nobody had made a new post your thread might have been ignored. Just something to keep in mind when using the forum.

 

Disk1 is indeed BAD but the others look OK. I have to wonder though. Why didn't you know disk1 was bad before you decided to replace parity instead? Do you have Notifications setup?

 

 

Disk 1 did have some SMART warnings but the real issue was that I wanted to upgrade the size of my disks so purchasing another 3TB (my current max size) wasn't ideal.  I do have several other 8TB drives and it was going to be replaced after the parity was upgraded.

The real issue was your disk1 wasn't good enough to allow you to replace any other disks.

 

If you had only asked for advice first.

 

There is a special procedure (parity swap) that would have allowed you to copy parity to a new larger disk then replace disk1 with the old parity, but I think it may be too late now.

 

@johnnie.black

 

What about instead of invalidslot to rebuild to a new disk1, New Config with valid parity and then proceed with the parity swap?

@johnnie.black

Thanks for the info I will try this out after I hit the store for another 3TB drive.

If you wait for johnnie.black to reply to my proposal

2 minutes ago, trurl said:

What about instead of invalidslot to rebuild to a new disk1, New Config with valid parity and then proceed with the parity swap?

then perhaps you won't need to buy another disk.

@trurl

Ok I am in no real rush.

 

Thanks

And some additional advice about using the forum. When you want to ping another user as you attempted above

5 minutes ago, ngurganus said:

@johnnie.black

1 minute ago, ngurganus said:

@trurl

you must not type it out completely. You actually have to make a selection from the list of matching users that appears. Like this:

 

@ngurganus

3 minutes ago, trurl said:

What about instead of invalidslot to rebuild to a new disk1, New Config with valid parity and then proceed with the parity swap?

Should work, OP could use the old disk (though it would invalidate other recovery options using it if needed), after the invalid slot command disk1 would start rebuilding at array start, cancel rebuild, stop array, unassign disk1, start array, stop array and could go on with the swap procedure from there.

@johnnie.black

Ok so just to clarify what @trurl is suggesting is:

1. Stop Array 
2. Reinstall the old parity drive
3. Tools -> New Config -> Retain current configuration: All ->Apply
4. Start the array (still have the bad disk1, but i should have a functioning array running off parity)
5. Stop Array
6. Unassign bad disk1 
7. Start Array
8. Stop Array
9. Insert New parity drive (might need to powerdown)
10. Stop Array
11. Unassign old parity drive
12. Assign new parity drive
13. Assign old parity drive into missing disk1 position
14. Main -> Array Operation -> Copy (check the "Yes I want to do this" checkbox) -> Start
15. Wait

I was using the instructions from https://wiki.unraid.net/The_parity_swap_procedure for the swap procedue.  Did I miss anything?

 

Thanks for all the help with this!

1 hour ago, ngurganus said:

Ok so just to clarify what @trurl is suggesting is:

You can do that, but on #4 start the array in maintenance mode only, or there will be a few writes to parity.

 

At step #8 follow the parity swap procedure (continue from step 5).

38 minutes ago, johnnie.black said:

You can do that, but on #4 start the array in maintenance mode only, or there will be a few writes to parity.

He didn't specifically mention checking the parity valid box. Does starting in maintenance mode prevent the parity rebuild you normally get from New Config if you don't check the box?

1 minute ago, trurl said:

He didn't specifically mention checking the parity valid box. Does starting in maintenance mode prevent the parity rebuild you normally get from New Config if you don't check the box?

I missed that, "parity is already valid" need to be checked, together with start in maintenance mode.

Ok so far so good!  Creating a new config and starting up worked great.  Right now my array is copying the old parity to the new parity drive.

 

Thanks for all the help!

Everything recovered fine.  I have a working array with my 8TB parity drive!

 

Thanks again!