Hello reader,

Thank you for taking the time to read this. Here's my problem.

I was in the process of moving 2 4tb to 1 8tb drive. I followed the steps shown here and picked the faster method cuz my old drives are slow: https://docs.unraid.net/legacy/FAQ/replacing-multiple-data-drives/

After it was done with 1 drive I turned it off so I could place it safely on a cabinet. When I turned the server back on it showed unmountable: wrong or no file system on disk 3.

There was an error message when I tried fixing the file system in maintenance mode.
I have a part of it here:

Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... Invalid block length (0x0) for buffer Log inconsistent (didn't find previous header) empty log check failed zero_log: cannot find log head/tail (xlog_find_tail=5) - scan filesystem freespace and inode maps... clearing needsrepair flag and regenerating metadata - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 4 - agno = 7 - agno = 5 - agno = 6 - agno = 2 - agno = 3 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (2:504) is ahead of log (1:2). Format log to cycle 5.

 

There was more but it would've been too much.
I figured if I rebuild it onto itself I'd get my data back. I stopped at step 4 since it didn't show the emulated drive.
It told me to ask around here on the forum, and thus here I am.
I'd like to know how to continue. As of now I don't have any spare 8tb drives. My preferred end result is being able to use the 8tb drive that I added last.

I know the drive has like 32 reallocated sector count iirc.
Again, thanks in advance.

Kind regards,

Bram

tower-diagnostics-20250313-2217.zip

Nothing else I can recommend except try again.

 

Check filesystem on emulated disk3 from the webUI. Capture the output and post all of it. You can zip it if needed.

Thank you @trurl for your reply. Since I followed the steps in https://docs.unraid.net/unraid-os/manual/storage-management/#rebuilding-a-drive-onto-itself

The array is now stopped. If I add the same drive back to it's original place, it tells me All existing data on this device will be OVERWRITTEN when array is Started.

Is there a way to for the array to accept the drive back to how it was? Or should I let it rebuild from the parity drive?
The system is now doing the "Check filesystem" through the option of Unassigned Disk Devices. I will upload when it is ready

Edited March 14, 20251 yr by Bramovitz

Make sure you don't rebuild on top of the old disk before checking if the emulated disk is mounting and the contents look OK.

 

The filesystem check should be don on the emulated disk3, not the unassigned disk, do that and post new diags after array start, with the old disk still unassigned.

Thank you for clarifying @JorgeB
I added the outcome of the XFS_repair status AND the new diags.

tower-diagnostics-20250314-1507.zip 20250314 XFS repair status disk 3.zip

Just to make sure we are all together now.

 

The XFS repair status you posted was for the emulated disk3, ran from the Unraid webUI. Correct?

 

That doesn't look very good.

 

Is the Unassigned Disk (that was disk3) mountable from Unassigned Devices?

The XFS repair status was indeed from the emulated disk3, ran from the Unraid webUI. YES

I'm in the process of buying a new 8tb drive. I was hoping it wasn't necessary but it does seem that way.

There are 2 Unassigned Devices, one of 4TB. That's an old WD I changed for the 8TB. The other is 8TB which is the one that was disk3 in the array.
Both of those have the Mount button, but I was afraid of mounting them, not knowing what would happen.

6 minutes ago, Bramovitz said:

in the process of buying a new 8tb drive

Having a spare might be good. But we aren't even thinking about rebuilding anything yet until we have a mountable filesystem to rebuild.

 

9 minutes ago, Bramovitz said:

Both of those have the Mount button

Go ahead and try both.

Alright, gotcha.
The picture shows the response to trying to mount the 8TB drive. Which failed.
The 4TB is successfully mounted.

I don't know if this is a crazy idea but, since I rebuild the 8tb drive from the 4tb drive. Does that mean I can start a New config and use the 4tb drive as disk3 again? Or am I getting ahead of things.

New Config is exactly what we are considering. Do you think that 4TB has all of the data? Even if it doesn't we could New Config it back into the array, rebuild parity, and then see if the unassigned 8TB could have its filesystem repaired and maybe get anything that's missing.

There was a short while where the 8tb took over from the 4tb. I feel not much has changed in that while.

Just to get things straight for me. I'll New config the 4tb drive back into the array as disk3. Let it rebuild parity. Then try to repair the filesystem of the 8tb. When that's done I can get the files that weren't on the 4tb back from the 8tb and leave the 4tb in the array. Correct?

If you really want to go the extra mile, we could go ahead and repair the emulated disk3, even though it looks bad. But sorting through all of that just in case it might have something the other options don't seems like too much trouble to me.

 

Do you have backups of anything important and irreplaceable?

I don't really have backups of it. Then again I doubt much was done in the short time that my 8tb took over from the 4tb.

If I New config with the 4tb and notice too much is missing. Can I then revert back to the 8tb when it's fixed?
If that's the case I feel like trying the 4tb and see how things are.

Several options.

 

You could just copy files from the repaired 8TB to the array. That would save you from having to rebuild parity with another New Config.

 

Or if you really want the larger disk in the array you could repair, New Config with it and rebuild parity, and copy anything that was missing from the repair from the 4TB.

Thank you for the options.

I went ahead and kept trying to repair the file system corruption. It now tells me: No file system corruption detected.

Since the 8tb data won't fit to the rest of the array I shall New config with it and rebuild parity.

Edited March 14, 20251 yr by Bramovitz

1 hour ago, Bramovitz said:

I went ahead and kept trying to repair the file system corruption. It now tells me: No file system corruption detected.

Was that on the emulated disk, or the unassigned 8TB disk?

It was on the emulated disk.
I tried New config and it went back to Unmountable...

The Fix button I've tried for multiple times, but after fixing, I'll check, and then it'll say that it needs a fix again. Feels like I've tried fixing it this way for 5 or 6 times if not more.
I'm not really sure what to try now.
Should I try New config and put in the 4tb as disk3? Considering it is mountable.

2 minutes ago, Bramovitz said:

It was on the emulated disk.
I tried New config and it went back to Unmountable...

Of course, because when you New Config you are taking the physical disk and not the repaired emulated disk. Nothing is emulated after New Config.

 

After repairing the emulated disk, you have to rebuild it.

Frick, I feel like I f'd up..

It started to rebuild parity. So I figured it wasn't rebuilding the drive. Should I have clicked that the parity was valid?

If you wanted to New Config the physical disk back into the array, you have to let it rebuild parity.

 

If I recall correctly, the 8TB physical disk was unmountable, but the 4TB physical disk was mountable.

 

Which physical disk did you assign?

I assigned the 8tb drive. I confused fixing the emulated drive with the unassigned one.

You are right, the 8tb is unmountable. The 4tb is mountable.

There is no longer any emulated drive to repair and rebuild.

 

If you want your array going again with valid parity, it would be quickest to New Config the mountable 4TB disk back into that slot and rebuild parity, then try to repair the 8TB unassigned disk to see if anything needs to be (or can be) recovered that the 4TB doesn't have.

 

You seemed to think not much was written to the 8TB after it replaced the 4TB, so there shouldn't be much to copy to the array from it after repair. Looks like disks 1 & 2 had nearly 2TB free for that.

 

Then after you have as much data back as we can get, you can rebuild the 4TB to the 8TB if you want more capacity.

I have now New config the 4tb back as disk3. The disks are showing as normal and parity is rebuilding.
I'll start docker again when parity has been rebuild. Hopefully that'll speed up the process.
Thank you for you help, I feel it might not be as easy with noob like me. It is very much appreciated!

Do you think it is still possible to repair the 8tb drive? How would I go at it?

You should be able to check filesystem on the 8TB unassigned disk. Since UD may not know what filesystem it is supposed to be we might have to try it from the command line. It works basically the same, you just have to get the command right.

14 minutes ago, Bramovitz said:

The disks are showing as normal and parity is rebuilding.

Post new diagnostics