Parity Drive Failing & Replace Procedure

[someoneotherthanyou]

My current array consists of a total of 9 drives (5 X 8TB, 2 X 3TB, 1 X 4TB and 60GB SSD cache). Obviously my parity drive is one of the 8TB models. I started getting notices the other day that the 4TB and the 8TB parity drives were throwing errors. Specifically that the parity had 198 errors with current pending sectors at 8 and offline correctable at 1. The  4TB has 768 errors. The 4TB was marked as being offline and is no longer able to be mounted. Furthermore, I performed a read check on the 8TB parity drive and it finished with 128 errors after some 18 hours running.

 

 

My question is this: one of my 8TB drives shows as only having a few (under 10) gigabytes written to it. Can I have that drive safely take over for the 8TB parity drive without losing any information or do I need to go purchase a new 8TB drive and add it to take over the parity. In either case, what is the procedure to do this safely? I looked at two articles from the wiki (the parity swap procedure & replacing a data drive) but am unsure which applies here.

 

TIA for any and all input..

[trurl]

You must concentrate on rebuilding disk6 or you will lose its data.

[someoneotherthanyou]

Thanks for your reply

 

OK...Makes sense. 2 thoughts come to mind though. a) I don't know how to mount it and the GUI will not let me and b) to the best of my knowledge no data has been written to it as of yet. Doesn't my config write up to 50% of each drive sequentially through to the end of the array? For instance disk4, 5 & 7 have no data written to them yet since disk3 has not reached 50% full yet. Picture below to illustrate.

 

 

 

[trurl]

Tools - Diagnostics, attach complete zip to your next post.

[someoneotherthanyou]

As requested zip file is attached. 

 

Thank you

nasnashurray-diagnostics-20181209-1218.zip

[trurl]

The disabled but emulated disk6 does seem to be mostly empty but it is still part of parity. I don't see the disk in your diagnostics though. Do you still have it? If you could attach it we could take a look and see if there is anything wrong with it.

 

The only way to maintain the parity you already have would be to rebuild disk6. And parity swap doesn't apply in your current situation

 

You could copy the data from that 8TB you mentioned (and also the data from the emulated disk6) to another disk then set a New Config without disk6 and with that other 8TB assigned as parity.

[someoneotherthanyou]

Is there anyway to rebuild disk6 within Unraid if I cannot mount it? I'm thinking about rebooting the box as it hasn't been powercycled in 43 days. Would that maybe encourage disk6 to mount again

[JorgeB]

Since parity is failing and there's no data on disk6 IMO you should just do a new config with the remaining disks, or use a new disk for disk6, and resync using a new parity drive.

[trurl]

I don't understand what you mean about mounting the disk. The disk is emulated and the emulated disk is showing the xfs filesystem so you should be able to read the emulated disk6 even if you dropped the actual physical disk6 in a well.

 

Are you saying you have tried to manually mount the disk some way, or are you just confused about the meaning of some of these terms?

[trurl]

Just now, johnnie.black said:

Since parity is failing and there's no data on disk6 IMO you should just do a new config with the remaining disks, or use a new disk for disk6, and resync using a new parity drive.

 

He wanted to use one of his data disks to replace parity so

 

1 hour ago, trurl said:

You could copy the data from that 8TB you mentioned (and also the data from the emulated disk6) to another disk then set a New Config without disk6 and with that other 8TB assigned as parity.

 

[JorgeB]

1 minute ago, trurl said:

You could copy the data from that 8TB you mentioned (and also the data from the emulated disk6) to another disk then set a New Config without disk6 and with that other 8TB assigned as parity.

That's a good plan, and since disk6 is empty no need to copy the data from the emulated disk.

[trurl]

Just now, johnnie.black said:

That's a good plan, and since disk6 is empty no need to copy the data from the emulated disk.

I wasn't sure if it was empty from looking at diags. Could be his disk7 is empty too. I assume that is the one he wants to use as parity.

 

OP, you can click on the folder icon at the far right of disk6 and disk7 to see if there are any contents.

[JorgeB]

Disks 4, 5, 6 and 7 are all empty, or at least no media files, at most some very small files like TXTs.

[someoneotherthanyou]

2 hours ago, trurl said:

I don't understand what you mean about mounting the disk. The disk is emulated and the emulated disk is showing the xfs filesystem so you should be able to read the emulated disk6 even if you dropped the actual physical disk6 in a well.

 

Are you saying you have tried to manually mount the disk some way,

I cannot figure out how to actually access it via the GUI as shown here. 

2 hours ago, trurl said:

 

or are you just confused about the meaning of some of these terms?

That is extremely likely TBH. But from what I am understanding now, is since disk6 is emulated that when I click on folder contents it is an actual representation of that drive. Is that correct? And that by doing the same for disk7 that it is also void of any files?

 

Hopefully I am making enough sense to help you help me. I very much appreciate all the questions everyone is asking to clarify the situation. Thank you for not getting frustrated by my lack of understanding. 

[someoneotherthanyou]

2 hours ago, trurl said:

I wasn't sure if it was empty from looking at diags. Could be his disk7 is empty too. I assume that is the one he wants to use as parity.

 

OP, you can click on the folder icon at the far right of disk6 and disk7 to see if there are any contents.

[someoneotherthanyou]

2 hours ago, johnnie.black said:

Disks 4, 5, 6 and 7 are all empty, or at least no media files, at most some very small files like TXTs.

👍

[trurl]

We should try to remember to check the physical disk6 later after everything else is square just so we can decide if the disk is worth keeping for some possible future use.

 

But for now, make careful note of each of the disk serial numbers. The screenshot you already posted is a good record, or you can print out that page. And we can also see them in the diagnostics you already posted. So they aren't in any danger of getting lost. But you will need to know them yourself when you complete the next steps.

 

This is somewhat from memory and somewhat from how I know it needs to work, so if you have any questions as you go ask.

 

Go to Tools in the webUI then New Config - Retain All. You will get a chance to change your disk assignments before starting the array.

 

Set disk6 and disk7 to Not Assigned.

 

Set the parity slot to your new parity disk, the disk that was the former disk7.

 

Do NOT check the box saying parity is valid. If it also presents you with a box to Format anything, do NOT check it but come back here with a screenshot.

 

Start the array to begin parity build.

 

[trurl]

The winter storm has taken out my power but I should be able to check back from my mobile if necessary. Others such as @johnnie.black can finish getting you going again if I don't report back soon.

[someoneotherthanyou]

Sorry it has taken me through today to address this. I have the array stopped and am in the new config menu with the retain all setting checked. I am presented with this rather ominous warning however.

 

I'm a little hesitant to move forward. However, in the interim the amount of errors on the parity drive has exploded to this

 

I guess my ultimate question is how much danger am I in of losing everything on this array?

 

[someoneotherthanyou]

Addendum to above: I mistakenly thought that the array had stopped. It has been attempting to stop the array unsuccessfully for over 45 minutes now but keeps having issues unmounting disk shares. 

[JorgeB]

46 minutes ago, someoneotherthanyou said:

I am presented with this rather ominous warning however.

Yes, but you're no trying to replace a drive, you're trying to remove a drive from the array, and a new config is needed for that.

 

40 minutes ago, someoneotherthanyou said:

Addendum to above: I mistakenly thought that the array had stopped. It has been attempting to stop the array unsuccessfully for over 45 minutes now but keeps having issues unmounting disk shares. 

Make sure nothing else is using the disks and no terminal/ssh session is idle on one of the mounted devices, close them all.

[trurl]

1 hour ago, someoneotherthanyou said:

I guess my ultimate question is how much danger am I in of losing everything on this array?

Very little danger.

 

Just in case you may wind up needing to do an unclean shutdown, go to Settings, Disk Settings, and turn off Autostart.

[someoneotherthanyou]

 

 

I was able to stop the array, and did as instructed. Here is screenshot from new config...

and screenshot prior to starting array

 

I'll wait for further instructions prior to firing the array back up. Or should I power down to pull the drives from the array?

 

 

[trurl]

Looks good. Start array to sync parity. Check occasionally to see if it is making reasonable progress and see if it is throwing any errors. Parity sync for 8TB will take most of a day.

[someoneotherthanyou]

41 minutes ago, trurl said:

Looks good. Start array to sync parity. Check occasionally to see if it is making reasonable progress and see if it is throwing any errors. Parity sync for 8TB will take most of a day.

 

When should I pull the bad 8TB and 4TB drives out of the case so that I can RMA them? The 8TB is less than 90 days old SMH