Replacing disk I think it has gone bad


Recommended Posts

I decided to replace my oldest disk a 3tb wd green.  I had a new 8tb red ready to go all pre-cleared so I stopped the array assigned the new drive to the slot I wanted to use and started array and all looks good rebuild underway.  Then this morning I wake up to see the array is now rebuilding at 345kb/sec I also noted that mover started and started to move some files.  The first file it moved seemed to go as per normal in terms of speed but after that things went bad.  What I did notice is that mover was moving data to the new currently being rebuilt disk I though the free space on a larger disk was not available until it was rebuilt and then even after a stop and start of the array??  Am I in trouble here??

 

It rebuilt 2.5tb in 11 hours now sayd it will finish in 256 days.

 

 

tower-diagnostics-20170326-0754.zip

Edited by mejutty
spelling
Link to comment

Maybe a problem, or a reporting bug in the preclear script

 

Mar 25 20:04:42 Tower emhttp: WDC_WD80EFZX-68UW8N0_VKHZ2DPX (sdi) 7814026532
Mar 25 20:04:42 Tower emhttp: WDC_WD80EFZX-68UW8N0_VKHYKGVX (sdn) 7814026532
Mar 25 20:04:42 Tower kernel: md: import disk1: (sdn) WDC_WD80EFZX-68UW8N0_VKHYKGVX size: 7814026532 erased
.
.
.
Mar 25 20:05:12 Tower preclear.disk: Pausing preclear of disk 'sdn'
Mar 25 20:05:12 Tower preclear.disk: Pausing preclear of disk 'sdi'
.
.
.
Mar 25 20:12:52 Tower kernel: md: recovery thread: recon D1 ...
.
.
.
Mar 25 20:12:54 Tower preclear.disk: Resuming preclear of disk 'sdn'
Mar 25 20:12:54 Tower preclear.disk: Resuming preclear of disk 'sdi'

Personally, if I saw on my system that it was resuming a preclear of a disk that has already been precleared, and is in the process of being rebuilt I'd be very worried....  If this is indeed what is happening, then it could quite possibly explain the very slow speed.

Edited by Squid
Link to comment

is dd still running though?  Depending upon how the preclear script was made, any processes spawned by it may not themselves be killed alongside the parent

 

ps -aux | grep dd

 

 

Beyond that, and if the stats are correct, then there is nothing particularly wrong in the syslog to indicate anything wrong.

Edited by Squid
Link to comment

nope no dd running.  

root@Tower:~# ps aux | grep dd
root         2  0.0  0.0      0     0 ?        S    Mar23   0:00 [kthreadd]
root        11  0.0  0.0      0     0 ?        S<   Mar23   0:00 [lru-add-drain]
root      3683  0.0  0.0 273020  3948 ?        S    00:35   0:00 /usr/sbin/winbindd -D
root     11800  0.0  0.0 273020  7168 ?        Ss   Mar25   0:00 /usr/sbin/winbindd -D
root     11802  0.0  0.0 272588  6804 ?        S    Mar25   0:00 /usr/sbin/winbindd -D
root     13732 11.2  0.0      0     0 ?        S    Mar25 133:58 [unraidd]
root     17046  0.0  0.0   9640  1832 pts/2    S+   15:54   0:00 grep dd
 

so what about it writing to the disk before it's finished a rebuild ???

Edited by mejutty
Link to comment

Disabling mover & ca appdata backup will stop most large writes to the array.  You can try to avoid incidental writes to the array.  I've never particularly worried about them.  So long as my checks / rebuilds happen in what I consider to be a reasonable time, I don't care about the actual raw values.

 

But should it drop down to kb/s then I'd be ripping everything apart and just start playing to see what fixes it.

 

Unfortunately, I don't see anything particularly worrisome outside of the preclear within the syslog, so I'm not quite sure where to go from here beyond stopping everything and restarting it all again.  Maybe wait for someone else to pipe in here...

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.