Hey all,

 

Last night I had a sudden drive failure (Seagate ST2000DM001) so I shut down the machine to put a new drive in.. on booting the system, ALL drives assignments where GONE. Every slot said 'unassigned'. Thankfully I had taken a screenshot of my drive assignments a few weeks ago, so I set all the other drives back as usual, ticked 'Parity is valid' box and started the array (still with a missing disk).

 

I now have a replacement disk for it, and want to plug it in an assign it to the slot.. my question is, will a parity rebuild automatically start, or is there something special I have to do? I know that NORMALLY when replacing a disk it will start a sync/rebuild, but in this case, its sort of a 'new config' because all drive assignments were lost for some reason.

 

Any advice appreciated. At this moment, the machine is powered off, but all drives are assigned (except the failed/replaced disk, which is still on my desk).

 

1 hour ago, wishie said:

ticked 'Parity is valid' box and started the array (still with a missing disk).

That's no good, did you write to array after starting it?

 

 

No, no writing at all.

 

Nothing writes directly to the array (its all via cache) and the cache disk has no pending data to write.

 

12 minutes ago, johnnie.black said:

That's no good, did you write to array after starting it?

 

 

Correct me if I am wrong, but if I HADNT ticked that, it would have wiped out my parity and I would have lost all that data, no?

 

Still not good since some filesystem housekeeping writes would happen, but better than the alternative.

 

There's a change to rebuild the missing disk, but would like to see the diagnostics before.

Just now, wishie said:

Correct me if I am wrong, but if I HADNT ticked that, it would have wiped out my parity and I would have lost all that data, no?

 

That would be much worse, but what would did was still wrong since parity isn't valid with a missing disk.

1 minute ago, johnnie.black said:

That would be much worse, but what would did was still wrong since parity isn't valid with a missing disk.

I didn't have a disk to replace it with at the time, and I was hoping that rebooting the system would bring that failing disk back online (its happened before with other disks, ages ago).. but on reboot, ALL disks being unassigned wasn't a good sign.

 

So you want me to boot and get diagnostics now?

 

1 minute ago, wishie said:

So you want me to boot and get diagnostics now?

Yes

Ok, I've attached the new replacement disk, but won't assign it or start the array. Give me a few minutes.

FWIW, the 'failed' drive is now only being recognised (in an external dock on another machine) as an 'uninitialised 4.14Gb' disk

 

Here are the diagnostics.

wishie-diagnostics-20180315-2018.zip

Assuming disk5 is the missing disk:

 

-Tools -> New Config -> Retain current configuration: All -> Apply
-Assign the new disk5

-Important - After checking the assignments leave the browser on that page, the "Main" page.

-Open an SSH session and type:

mdcmd set invalidslot 5 29

-Back on the GUI and without refreshing the page, just start the array, do not check the "parity is already valid" box, disk5 will start rebuilding, if the disk is unmountable don't format, wait for the rebuild to finish and then run a filesystem check.

Ok, done that, and it appears to be rebuilding..

Can you explain the:

mdcmd set invalidslot 5 29

for me please

And did it mount OK?

Well I assume so, since its rebuilding..

'mount' shows:

/dev/md5 on /mnt/disk5 type xfs (rw,noatime,nodiratime)

It would still rebuild if unmountable, but it's a good sign it mounted OK, still run xfs_repair -v when it finishes rebuilding.

Ideally, I should have a spare disk on hand for these moments, so I can replace straight away.. any idea why I lost my configuration when I rebooted the server?

 

There a bug that would cause that on a much earlier release, assuming you were already on v6.4.1 when it happened, likely a flash drive issue/corruption.

Yeah, been on 6.4.1 for ages.. never had that issue before. Will check flash drive.

At this stage, the data of disk 5 is being 'emulated' by the parity, right? So I should be able to use the system?

My kids are bugging me to turn on Plex.

 

You can use it, but performance will be reduced and rebuild will take longer.

So back to this: 

mdcmd set invalidslot 5 29

I get that 'invalidslot 5' likely tells unRAID that the disk is 'invalid' and perhaps should be rebuilt.. but what does the '29' mean?

Parity2 is invalid since you don't have one.