Something went terribly wrong!

[trurl]

4 minutes ago, jcarre said:

I meant the sizes of the "files" are a lot smaller, a lot less data in there.

You mean the sizes of individual files in lost+found? Doesn't your screenshot show more than 18TB in there?

[jcarre]

1 minute ago, trurl said:

You mean the sizes of individual files in lost+found? Doesn't your screenshot show more than 18TB in there?

 

The sizes of files in disks 6,7 and 10 is relatively small, compared to the ones that I repaired today.
6 has 3.6GB, 7 1.8GB and just 6B on 10.

[trurl]

1 hour ago, trurl said:

It often is a lot of work to actually recover any files from it, and you may have a lot of data there.

Filesystem repair puts things in lost+found that it doesn't know what to do with. It usually contains files without their correct filenames or any idea what folder they belonged to.

 

Usually simpler to just try to recover from backup or just forget about them.

[jcarre]

What should I do then?
I do have backups of important files, but those 18tb are probably not part of them. There is really no way to salvage them? Is there any way to know which files are in there?
 

I do have 1 disk ready (which was old disk4) and another one on the way. Should I try to put the old disk back into the array and start parity. Do I wait until Monday with the new one and do the two at the same time?

[JorgeB]

So main problem with lost files are disks 2 and 4, disk2 is healthy so data on the actual disk should be in a much better shape than the emulated disk2, what about disk4? You mentioned it's disconnected, do you know if it failed or it was just disconnected and assumed OK?

[trurl]

15 hours ago, jcarre said:

Is there any way to know which files are in there?

Linux 'file' command might give some clue to what sort of data a file might contain.

[trurl]

4 hours ago, JorgeB said:

disk2 is healthy so data on the actual disk should be in a much better shape than the emulated disk2, what about disk4?

This is probably the best bet, and a reason we sometimes recommend rebuilding to a spare instead of on top of the same disk.

 

17 hours ago, trurl said:

Until you get your array stable, you might consider not rebuilding on top of any existing disk and only rebuilding onto spares so the original disks aren't changed and might be used to recover files that repairs and rebuilds don't

 

Unraid emulates disabled or missing disks by reading all the other disks and getting the emulated result from the parity calculation. If your system is having lots of problems reliably reading all other disks for the emulation may not work well.

[jcarre]

5 hours ago, JorgeB said:

So main problem with lost files are disks 2 and 4, disk2 is healthy so data on the actual disk should be in a much better shape than the emulated disk2, what about disk4? You mentioned it's disconnected, do you know if it failed or it was just disconnected and assumed OK?

 

Disk 2 is the one that is now disabled in the array. Do you mean that the contents in the disk might be in better shape than the ones being emulated and now showing in Unraid?

Disk 4 is the one that is not currently in the array. I installed unassigned plugins, mounted it and I can see some data in it, I assume that it is correct.
This disk is the one that started giving me problems previously on my last post, so went it started getting errors again I took it out and ordered a replacement (assuming it was bad). I will probably rma it after the new one arrives.
Knowing now that the errors continued to show after taking it, it would make sense to assume that it's not necessarly bad, and another issue may be the cause of all of the errors I am now seeing.

This is a screenshot of the current state of the arra (All spin down disk are ok)

 

 

[trurl]

4 minutes ago, jcarre said:

Do you mean that the contents in the disk might be in better shape than the ones being emulated and now showing in Unraid?

Exactly, for both emulated disks.

 

5 minutes ago, jcarre said:

Disk 4 is the one that is not currently in the array. I installed unassigned plugins, mounted it and I can see some data in it, I assume that it is correct.

Is is currently attached?

5 minutes ago, jcarre said:

it's not necessarly bad, and another issue may be the cause of all of the errors I am now seeing.

Diagnostics (or just SMART report) from that disk might tell.

[jcarre]

The new disk should be arriving today, should I put it into the array as disk 4? And then try to recover data from the other disks?
How do I go about this?

[trurl]

Attach original disk 4 if it isn't, don't assign, start array, post new diagnostics 

[JorgeB]

First you should post diags or SMART report with/for disk4.

[jcarre]

Disk 4 has always been attached, just out of the array. Here is a new diagnostics file.

 

What I mean is that I'll have a brand-new disk today. Should I try rebuilding the array? Right now I have no parity whatsoever.

ivpiter-diagnostics-20220617-1541.zip

[JorgeB]

I assume this was disk4?

 

Model Family:     Seagate IronWolf
Device Model:     ST10000VN0008-2PJ103
Serial Number:    ZS5033S2

 

If yes also looks healthy, a few CRC errors, likely due to a bad cable.

 

You can rebuild using new disks, but IMHO not the best option, data on disks 2 and 4 will be mostly on the lost+found folder, you'll then soend a lot of time copying the data from the original disks, I would first confirm actual disks 2 and 4 are mounting and contents look correct, you can use the UD plugin for that, if yes do a new config with the original disks and rebuild parity.

[jcarre]

25 minutes ago, JorgeB said:

I assume this was disk4?

 

Model Family:     Seagate IronWolf
Device Model:     ST10000VN0008-2PJ103
Serial Number:    ZS5033S2

 

If yes also looks healthy, a few CRC errors, likely due to a bad cable.

 

You can rebuild using new disks, but IMHO not the best option, data on disks 2 and 4 will be mostly on the lost+found folder, you'll then soend a lot of time copying the data from the original disks, I would first confirm actual disks 2 and 4 are mounting and contents look correct, you can use the UD plugin for that, if yes do a new config with the original disks and rebuild parity.

 

Yes this is disk 4.

Looking at the contents of this disk with UD it looks like everything is in lost + found.

 

 

That means that if I create a new config I'll lose parity and then I'll have to copy everything from the lost+found folder from the disk itself.
Right now I have emulated lost+found with exactly the same data right?

[trurl]

59 minutes ago, jcarre said:

Should I try rebuilding the array? Right now I have no parity whatsoever.

That is what we are trying to decide based on your diagnostics and the contents of the physical disks. Of course you do have parity (dual), just no redundancy since 2 data disks are disabled.

 

6 minutes ago, jcarre said:

Right now I have emulated lost+found with exactly the same data right?

Could be the same but no reason to assume that. That is the whole point of comparing the data on the physical disk with the data on the emulated disk. If the emulation isn't working well due to whatever problems you have been having, then the physical disk might be better than the emulated disk.

 

If you unassign disk2 then you can also mount it as Unassigned Device and compare its contents to emulated disk2.

[trurl]

15 minutes ago, jcarre said:

Looking at the contents of this disk with UD it looks like everything is in lost + found.

What does ncdu give you for /mnt/disk4?

 

What does ncdu give you for /mnt/disk2?

 

[jcarre]

On 6/17/2022 at 4:52 PM, trurl said:

What does ncdu give you for /mnt/disk4?

 

What does ncdu give you for /mnt/disk2?

 

 

Sorry I had a busy weekend.
 

I know have both disks out of the array + a brand new 14tb drive (also in unassigned).

The ncdu command of disk 2 is:

 

 

and disk 4 is:

 

 

Looks like disk 2 is good to go, but disk 4 has nearly everything on lost + found.

[JorgeB]

Disk 4 must have been repaired before.