Need help with adding to my array

[PapaThiele]

I installed a new 8TB drive to replace a 4TB drive.  After pre-clearing, I added it to the array.  It rebuilt the data but then would not recognize it.  Now I see it in Uassigned Devices.  Its showing as xfs & I can see the data that was rebuilt onto it, but it won't add to the array & the Array Operations tab it has the option to format that drive.  Please help!

 

 

wideglide-diagnostics-20220106-0808.zip

[itimpi]

Was the drive showing as unmountable while it was being emulated before rebuilding it?   If so that would not get cleared by a rebuild.  
 

the standard process for handling an unmountable disk is covered here in the online documentation accessible via the ‘Manual’ link at the bottom of the GUI.

[PapaThiele]

There was no uncountable errors at any point, that I saw.  I was replacing a good drive with a new larger drive.

[trurl]

"Unmountable", not "uncountable".😉

[trurl]

Connection problems with multiple disks, parity, disk7, disk8, at least. This is probably why the emulated disk is unmountable. Maybe disk10 disconnected and reconnected also so it showed up unassigned though it didn't change from sdl. Which controller are these disks on?

 

SMART for all 4 disks looks OK, didn't check the others.

 

Emulated disk10 was mounted when rebuild began but then the connection problems showed up while rebuilding.

 

In any case, mounting that disk outside the array has made it out-of-sync with parity. If you are sure its contents look good, and you fix your connection problems, it might be best to New Config and rebuild parity instead of rebuilding that disk again. Don't do anything with the original (4TB) disk10 until you are satisfied your array is stable and all your files are there.

 

Not related, but looks like you have installed every package from Nerd Tools plugin. Do you even know what most of that is? I recommend only installing those that you use regularly. atop in particular is likely to use up all your log space.

[PapaThiele]

Oh geez, that went right past me...

 

Anyhow, the only time I saw the unmountable error was after the rebuild.  It's as if it rebuilt it, then forgot it did so.

 

 

 

[PapaThiele]

Thanks @trurl.  I'll check out the cables to start with

[trurl]

11 minutes ago, PapaThiele said:

the only time I saw the unmountable error was after the rebuild.  It's as if it rebuilt it, then forgot it did so.

If the rebuild was in fact successful, then the disk was disabled again due to connection issues.

 

disk10 replaced for rebuild:

Jan  6 07:41:43 wideglide kernel: md: import disk10: (sdl) WDC_WD80EMZZ-11B4FB0_WD-CA05K2KG size: 7814026532 
Jan  6 07:41:43 wideglide kernel: md: import_slot: 10 replaced
Jan  6 07:42:00 wideglide kernel: mdcmd (35): start RECON_DISK

 

emulated disk10 mounts:

Jan  6 07:42:05 wideglide kernel: XFS (md10): Mounting V5 Filesystem
Jan  6 07:42:05 wideglide kernel: XFS (md10): Ending clean mount

 

rebuild begins but is cancelled due to connection issues, lots more entries about ata6 & ata3 before exit status -4

Jan  6 07:42:17 wideglide kernel: md: recovery thread: recon D10 ...
Jan  6 07:42:28 wideglide kernel: ata6: COMRESET failed (errno=-16)
Jan  6 07:43:10 wideglide kernel: ata3: COMRESET failed (errno=-16)
Jan  6 07:44:24 wideglide kernel: md: recovery thread: exit status: -4

So I don't think the disk was successfully rebuilt. Since all that happened in less than a minute it couldn't have had time to rebuild the disk.

 

Then the array starts but disk10 is unmountable:

Jan  6 07:50:20 wideglide emhttpd: shcmd (1558): mount -t xfs -o noatime /dev/md10 /mnt/disk10
Jan  6 07:50:20 wideglide kernel: XFS (md10): Filesystem has duplicate UUID 19822d8e-c509-49d2-8917-0574f0cbf162 - can't mount
Jan  6 07:50:20 wideglide root: mount: /mnt/disk10: wrong fs type, bad option, bad superblock on /dev/md10, missing codepage or helper program, or other error.
Jan  6 07:50:20 wideglide emhttpd: shcmd (1558): exit status: 32
Jan  6 07:50:20 wideglide emhttpd: /mnt/disk10 mount error: not mounted

The duplicate UUID is because the disk is also an Unassigned Device for some reason. Maybe that will clear itself up after reboot.

 

The main thing I don't understand is why the Unassigned Device, which is the same disk assigned as disk10, even mounted, since no rebuild completed on it. It does appear to be mostly empty though, so I doubt it has all your files.

 

Might be better to resync parity with the original (4TB) disk10, then try all this again.

[trurl]

3 minutes ago, trurl said:

Might be better to resync parity with the original (4TB) disk10, then try all this again.

And parity resync would be a good test of your connections before attempting rebuild again.

[PapaThiele]

Replacing cables...started by removing everything but the power supply.  Hope this solves my woes.  I've got my fingers & eyes crossed.  I'll let it sit idle overnight, this old dog is whooped.
 

 

 

If it doesn't work, is it wrong to blame my 9 year old daughter since she was helping? 

 

Again, I feel like a dolt for the 'uncountable' confusion.  I should have caught that, but was juggling chainsaws with Unraid, work & family.

[PapaThiele]

Everything was working fine after the re-cabling effort.  I'd been using Plex all morning.

 

All of a sudden my dockers stopped working (but were still showing as started in the Dockers tab).  When I tried to restart them, I got an error "Execution error.  Server error" and it looks like /mnt/user has no owner & no perms.

 

(Edited for relevance)

root@unraid:~# ll  /mnt
d?????????  ? ?      ?       ?            ? user

 

Is this normal during a rebuild or parity sync?  Also, how can I tell which one it is doing since the Array Operations &  Dashboard tabs both show "Parity-Sync/Data-Rebuild".  I thought i had seen it somewhere before, but I cannot find it now.

Edited January 7, 2022 by PapaThiele

[PapaThiele]

I found this thread, but haven't tried the suggested step "Tools, New Permission, User shares" since the rebuild is ongoing

 

[trurl]

20 hours ago, trurl said:

Might be better to resync parity with the original (4TB) disk10, then try all this again

Is this what you decided to do?

 

Post new diagnostics.

[PapaThiele]

No, I just started the array with the new drive & it doesn't say exactly what it is doing (rebuild or sync).

wideglide-diagnostics-20220107-0923.zip

Edited January 7, 2022 by PapaThiele

[trurl]

4 minutes ago, trurl said:

Post new diagnostics.

 

[PapaThiele]

I edited my last comment to add the diags

[JorgeB]

https://forums.unraid.net/bug-reports/stable-releases/683-shfs-error-results-in-lost-mntuser-r939/

Some workarounds discussed there, mostly disable NFS if not needed or you can change everything to SMB, can also be caused by Tdarr if you use that.

[trurl]

3 hours ago, PapaThiele said:

doesn't say exactly what it is doing (rebuild or sync)

Rebuilding disk10 since it was still invalid due to rebuild not completing

Jan  6 19:52:50 wideglide kernel: md: recovery thread: recon D10 ...

 

[trurl]

54 minutes ago, trurl said:

Rebuilding disk10 since it was still invalid due to rebuild not completing

Jan  6 19:52:50 wideglide kernel: md: recovery thread: recon D10 ...

Looks like emulated disk10 has the same amount of data (not much if any) that disk had on it when you mounted it Unassigned, so it was probably empty anyway and it rebuilt enough of it to mount it as an empty filesystem.

 

In any case, you must let rebuild complete so the whole disk is in sync with parity.

 

If you think there should be more data on it then we can take a look at original (4TB) disk10 as an Unassigned Device.