UnRAID removed two drives from my array because of performance problems.

I replaced both with "refurbs".  I restarted the array, and one of the drives dropped out again during the rebuild.  I let the rebuild finish, expecting to restore one of the drives.

Now, I'm seeing both of those drives listed as "Unmountable", even though all but one of the drives appears to be back in operation.

Did I just rebuild my parity and lose the data on my drives?  One of the drives was dumping so many "power on reset" errors that my log doesn't go back to start of the array rebuild.

Edited August 27, 20241 yr by bithoarder

Please post the diagnostics.

7 hours ago, JorgeB said:

Please post the diagnostics.

Of course... thanks!

I've got a really bad feeling that I did this the wrong way, but I swear the disc activity showed writing to the new drive(s).

 

ood-diagnostics-20240822-1418.zip

Replace cables for disk4 and post new diags after array start.

3 hours ago, JorgeB said:

Replace cables for disk4 and post new diags after array start.

Will do. 

I opened the machine up and realized that I don't have a spare.  Hopefully I'll have a replacement "SFF-8643 to (4) SATA" tomorrow.

 

Thank you!

On 8/22/2024 at 6:16 AM, bithoarder said:

UnRAID removed two drives from my array because of performance problems.

Unraid disables a drive when a write to it fails, and for no other reason.

On 8/23/2024 at 3:59 AM, JorgeB said:

Replace cables for disk4 and post new diags after array start.

The cable was delayed.  New diagnostics are attached.

I'm pretty sure I did this wrong, and rebuilt the parity instead of the drives.

Thanks for looking!

EDIT:  I might actually be okay, data wise.  It's showing that Drive 4 is emulated, but it's no longer showing the two drives as unformatted.  Unfortunately, I need to head out for a couple of hours, but I'm more optimistic now.
 

ood-diagnostics-20240825-1619.zip

Edited August 25, 20241 yr by bithoarder

Check filesystem on disks 4 and 13, run it without -n.

2 hours ago, JorgeB said:

Check filesystem on disks 4 and 13, run it without -n.

Thanks Again!

I see that it needs to be started in "maintenance mode" to do that. 

Do you have any idea how long I can expect that to take with a 14tb drive?

It usually takes a few seconds/minutes.

 

 

2 hours ago, JorgeB said:

It usually takes a few seconds/minutes.

 

 

It took me a little while to figure out how to do this... I almost did it from a prompt, but when I saw the UI element in the Device Settings, I understood.

Disk 4:

Phase 1 - find and verify superblock... writing modified primary superblock Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this.

Disk 13:

Phase 1 - find and verify superblock... writing modified primary superblock Phase 2 - using internal log - zero log... ERROR: The filesystem has valuable metadata changes in a log which needs to be replayed. Mount the filesystem to replay the log, and unmount it before re-running xfs_repair. If you are unable to mount the filesystem, then use the -L option to destroy the log and attempt a repair. Note that destroying the log may cause corruption -- please attempt a mount of the filesystem before doing this.

You now need to repeat those runs without -n but adding -L.

 

39 minutes ago, itimpi said:

You now need to repeat those runs without -n but adding -L.

 

Attached... disc 13 looks pretty scary.



 

Discs_4_13.txt

Disk should mount now, look for a lost+find folder.

25 minutes ago, JorgeB said:

Disk should mount now, look for a lost+find folder.

I'd like to be sure that I'm following this.  The disk hasn't been reporting as "unmountable or unformatted" for a while.

Disk 13 appears to be active in the array, and Disk 4 is emulated.  The array is still in maintenance mode.

What should I do next?

Edited August 26, 20241 yr by bithoarder

@bithoarder Do you have either of the original drives still intact?     If do they may provide another option for data recovery.

6 minutes ago, itimpi said:

@bithoarder Do you have either of the original drives still intact?     If do they may provide another option for data recovery.

I do... I haven't checked them yet. 

 

I've been kind of afraid to know, and wanted to get everything else protected, first.

Thank you for mentioning it.

I'm not really sure I've lost anything at the moment.  At least some missing files reappeared after the restart.

 

Edited August 26, 20241 yr by bithoarder

14 minutes ago, bithoarder said:

What should I do next?

Start the array in normal mode.

44 minutes ago, JorgeB said:

Start the array in normal mode.

Started.

Disc 4 is currently emulated - I really don't know at the moment if I'm missing any data.
 

ood-diagnostics-20240826-1231.zip

I don't see a lost+found share, so the data should be OK for both repaired filesystems, but if you still have old disk4, you can mount it with UD to compare contents.

7 hours ago, JorgeB said:

I don't see a lost+found share, so the data should be OK for both repaired filesystems, but if you still have old disk4, you can mount it with UD to compare contents.

Thank you again for your help... I'm not really sure where to go from here, and I want to tread lightly - especially since I don't fully understand what the repairs actually did.

 

My guess is that I'm ready to try to bring Drive 4 back online?  Another 14TB drive arrived today, but I'm thinking the first one may be okay, after the recabling(?)

 

 

Edited August 27, 20241 yr by bithoarder

9 hours ago, bithoarder said:

My guess is that I'm ready to try to bring Drive 4 back online?

Yes, but it would still be a good idea to compare the data with the old disk.

2 hours ago, JorgeB said:

Yes, but it would still be a good idea to compare the data with the old disk.

It looks like everything is there... thanks again!