Old disk "redballed" while pre-clearing new disk

[maddog808]

I have a somewhat tricky situation here (tricky for me at least). My current Unraid config is as follows:

 

 

I bought a WD Red 6TB drive with the intention of replacing my 4tb parity drive with it, then re-using the 4tb parity drive to replace disk 6, which has had errors lately. While pre-clearing the new WD Red 6tb drive, I got read errors on disk 5. Can I move forward with my plan of replacing the parity drive with the new 6tb, then just re-use the 4tb to replace disk 5 instead of disk 6? Then I can just move the contents of the failing disk 6 onto the new 4tb disk 5?

 

I'm open to suggestions, as I'm sure someone here has a better idea. I've just never personally dealt with this scenario before of a failing drive and a red ball drive, and my new drive is larger than my existing parity.

 

Any help is greatly appreciated

 

Thanks,

Matt

[JorgeB]

Post your diagnostics: Tools -> Diagnostics

[Frank1940]

If you opened up that case to install that new drive for preclearing, you might have loosened a SATA connector on Disk 5.  But still post up that diagnostics file. 

[maddog808]

Here's the diagnostics file. Thanks for helping out.

unraid-diagnostics-20171220-2306.zip

[JorgeB]

Both disks 5 and 6 have some health issues, but with some luck you should be able to rebuild disk5 without errors, keep old disk intact until done, you'll need to use the parity swap procedure.

 

 

[maddog808]

19 hours ago, johnnie.black said:

use the parity swap procedure.

 

 

 

Hey Johnnie, just so I'm clear, the instructions I linked to below will work for my 6.3.5 server, even though the instructions say first release, tested on v6.1 only?

 

https://wiki.lime-technology.com/The_parity_swap_procedure

[JorgeB]

Yes, they work on v6.3.5

[maddog808]

I've tried twice to follow the step by step instructions for parity swap. It gets to about 35% on the copy old parity drive to new parity drive(I'm not sure exactly, because I'm not staring at it the whole time). Then it displays errors for disk 5 (which is the old parity disk). At this point, should I just assign the old disk 5 back in its original spot, then rebuild parity on the new 6tb drive? Then when that completes, use the 4tb old parity drive to replace disk 5? Let me know if I should post another log or diagnostics file. Below is a screenshot of the error:

 

 

 

[JorgeB]

22 minutes ago, maddog808 said:

Then it displays errors for disk 5 (which is the old parity disk).

Looks like your current parity is also failing, you have limited options with multiple disk errors and single parity, and possibly some data loss.

 

23 minutes ago, maddog808 said:

At this point, should I just assign the old disk 5 back in its original spot, then rebuild parity on the new 6tb drive?

You can try, disks 5 and 6 have issues, but not completely failed yet, you'll need some luck though.

 

25 minutes ago, maddog808 said:

Then when that completes, use the 4tb old parity drive to replace disk 5?

You know now that old parity is failing, so maybe not a good idea to keep using it?

 

[maddog808]

Dang it!! My server is falling apart! 

 

I'll order another new disk (I should've bought 2 of the WD Red 6tb when they were on sale for $171 2 weeks ago!).

 

So what steps should I take first? When I try to reassign the old disk 5 back in its spot, it isn't showing as available at all in the drop down choices.

 

 

[JorgeB]

Try again after power cycling the server, you'll need to do a new config before that and reassign all disks except old parity, and don't forget you'll lose any added/modified data on the emulated disk5, i.e., once it got disabled.

 

 

[JorgeB]

Forgot to mention, before doing the new config make sure old disk5 is being detected, don't proceed if it's isn't, and even if it is you might want to try and copy anything irreplaceable from the emulated disk5 before doing it.

[maddog808]

I power cycled the server, now my old 1tb disk 5 is available. When I choose that disk for disk 5, it turns blue, and my option under "Array Operation" is: "Start will start Parity-Sync and/or Data-Rebuild"

 

Should I do this for now?

 

[JorgeB]

55 minutes ago, johnnie.black said:

you'll need to do a new config before that and reassign all disks except old parity, and don't forget you'll lose any added/modified data on the emulated disk5, i.e., once it got disabled.

 

Tools -> New config -> Retain all -> Apply

[maddog808]

Shouldn't I go ahead and leave the old parity in place, and rebuild disk 5 before replacing the old parity with the new parity?

[JorgeB]

7 hours ago, maddog808 said:

Shouldn't I go ahead and leave the old parity in place, and rebuild disk 5 before replacing the old parity with the new parity?

 

Not sure what you want to do:

 

9 hours ago, maddog808 said:

At this point, should I just assign the old disk 5 back in its original spot, then rebuild parity on the new 6tb drive?

 

You have 2 options, none of which is perfect:

 

option 1: rebuild disk5

your current parity is failing, there will likely be read errors doing a rebuild, you can try if you want but I'd only do it on a new spare disk, or you can make current disk5 worse than it is now.

 

option 2: new config a try to re-sync a new parity disk with current disks

it may or may not work depending if both disks 5 and 6 hold up, you'll lose any modified/new data on the emulated disk5, it can fail and lose the ability to try and rebuild disk5 so if there's any irreplaceable data there try to copy it to another disk/PC before doing it.

 

With multiple disk issues and single parity your options are limited, if you have a spare you can do option 1 then 2 to increase your chances.

[maddog808]

I went ahead and rebuilt the contents of disk 5 on the existing 1tb disk. It finished without any errors. Then I unassigned the failing 4tb parity disk, and assigned the new 6tb disk to the parity slot. I'm currently waiting for the parity sync to finish. Once that is done, I'll be replacing the failing 1tb disk 5 with a new - either 4tb or 6tb disk (depending on prices). Then I'll move all the contents from the failing disk 6 over to the new, larger disk 5. Then I'll remove disk 6 from the array, and do a new config. Then once the budget permits, I'll add a 2nd parity disk.

 

What do you think about the HGST NAS disks? Newegg has a 6tb on sale for $175 at the moment. Some reviews say they run hot due to the 7200 RPM speeds?

 

https://www.newegg.com/Product/Product.aspx?Item=N82E16822146118#close

 

Does this sound like a good plan?

Edited December 23, 2017 by maddog808

[JorgeB]

7 minutes ago, maddog808 said:

I went ahead and rebuilt the contents of disk 5 on the existing 1tb disk. It finished without any errors.

No read errors on the parity disk? It would still finish but surprised there were no errors when it failed twice during the parity copy, but if it did lucky you.

 

Rest of the plan looks good, HGST drives are considered some of the most reliable, though if you are worried about heat WD Reds are good also.

[maddog808]

Yeah I lucked out with the parity disk. Now I'm doing a parity-sync on the new 6tb (83% finished), and 5 read errors popped up on disk 5 again. I pulled the trigger on the HGST 6tb NAS drive @ Newegg for $175. When that arrives next week, I'll be retiring the old failing disks in my array!

 

Is it possible that loose cables caused the errors on the 4tb parity disk? Would running it through a preclear help to make the decision on whether to trash that drive or not? I'd hate to throw it out prematurely. It's barely out of warranty, purchased 26 months ago.

[pwm]

Bad power could potentially give reported uncorrect etc, but not a bad data cable.

 

If you have a data cable issue then a transfer error will either be caught by the drive, in which case you'll get a 199 UDMA CRC Error Count. Or it will not be caught by the drive, in which case the drive will write down the bad data but assuming it was correct.

 

Mechanical chock while writing could also result in bad sector content - that's why some drives also have added 191 G-Sense Error Rate or 189 High Fly Writes.

[maddog808]

OK so now this is really getting weird. While I was pre-clearing the new HGST 6TB drive, that I had planned on using to replace the failing disks 5 & 6, my BRAND NEW WD Red 6TB parity drive starts throwing errors. It got so bad, that Unraid wouldn't show the GUI or share the array. The only thing I could see on the network was the flash drive. I did some searching on this forum, and ended up disconnecting the drive's SATA cable, which got me into the GUI. Then I ended up doing a new config, with the HGST 6TB as the new parity, and the failing disk 5 as trusted (I know that no new data had been written to it). The parity sync finished successfully, and I have no errors now (all green ball drives).

 

When I pre-cleared the WD Red 6TB drive a couple weeks ago, it passed. Maybe I misinterpreted the preclear results? I just find it odd that it would fail so quickly after a preclear pass. I've attached the preclear results here, if anyone wants to help confirm whether it was good or not. I've also attached a fresh diagnostics report here if that helps.

 

I currently have the drive connected as an unassigned device. I would attach a SMART report, but it won't complete a SMART test. I've tried both the short & long test, and both get results stating "Interrupted (host reset)". Here's a screenshot of the disk attributes:

 

 

 

preclear_report_WD-WX41D17PDC2K_2017.12.16_09.59.54.txt

unraid-diagnostics-20180103-2035.zip

Edited January 4, 2018 by maddog808
Added diags

[pwm]

SMART looks fine. And no UDMA CRC errors from problems with the data cable.

 

But were you able to capture any system log data?

[maddog808]

15 minutes ago, pwm said:

SMART looks fine. And no UDMA CRC errors from problems with the data cable.

 

But were you able to capture any system log data?

 

 

No I wasn't able to capture the log data before the array was unreachable. The only way to get the array back online and access the GUI was to disconnect that drive. And why would Unriad not be able to complete a SMART test?  I was able to download a SMART report just now. I've attached it here. You can see the UNC errors at the bottom of the report. Should I try to preclear the disk again again and see what happens?

 

unraid-smart-20180103-2038.zip

Edited January 4, 2018 by maddog808

[Frank1940]

You might want to have a good look at your PS.  unRAID puts some unusual load patterns on the 12V buss as it will spin up all of the drives at one time and if the PS can't gracefully handle the load, wonky things can happen...

[maddog808]

2 minutes ago, Frank1940 said:

You might want to have a good look at your PS.  unRAID puts some unusual load patterns on the 12V buss as it will spin up all of the drives at one time and if the PS can't gracefully handle the load, wonky things can happen...

 

Thanks for the suggestion Frank. Would you be able to expand on how I should go about having a good look at it? It's a Corsair CX500 if that helps.