Hi, I'm making a new thread (hope this is okay) after suffering metadata loss in my other thread here:

 

What I've been able to do so far:

Figure out what degree of data was orphaned per drive.

 

//TOWER/lost+found/

Disk 1 - WDC_WD40EFRX-68N32N0_WD-WCC7K5CCCED9 - 4 TB (sde)
1772 objects: 1014 directories, 758 files (64.8 GB total)

Disk 2 - WDC_WD40EFRX-68N32N0_WD-WCC7K5CCCED9 - 4 TB (sde)
6419 objects: 550 directories, 5869 files (17.4 GB total)

Disk 3 - TOSHIBA_DT01ACA300_14SRJ24GS - 3 TB (sdc)
4163 objects: 3170 directories, 993 files (84.6 GB total)

 

root@Tower:/mnt/user/lost+found# files *

cryptobin.co/r1k6s5y5
password: unraid

 

The majority of damaged data seems to be JPG, PNG, HTML and FLAC files as shown in the crypobin link above.

 

I found this post: https://techcult.com/how-to-restore-files-from-lostfound/

 

Which contains a script at the bottom which I will put here (had to edit slightly to fix bad syntax):

#!/bin/bash
fsck -y /dev/sdc1
mkdir /tmp/recover
mount /dev/sdc1 /tmp/recover -o rw
cd /tmp/recover/lost+found
(
echo 'set -v'
file * | grep directory 2>/dev/null | perl -pe 's/^(\#[0-9]+)\:.*$/ls -l '"'"'$1'"'"'/'
) | sh > /tmp/listing

 

I was wondering does this have some chance of working for me? What exactly does it do? It looks to append file extensions to the files using their headers via the 'file' command? Or am I wrong about this?

 

EDIT: Couldn't get the script to work. I don't think it's written properly.

Edited April 17, 20224 yr by plantsandbinary

Thanks to how I structured my file/folder tree I found that using

 

find | less

 

and scrolling through the results, I was able to find out the following based on the fact that the highest level folder names were missing but enough information was left in their subfolders by way of folder names and filenames and their extensions.

 

This left me with the following:

 

./669603920
^^^ This is the //TOWER/tank/Files/Backup folder
There should be Documents, Media, Downloads, Videos, Music, eBooks etc. in there
>> Move everything from this location back to the surviving folder on the machine

./2147483779
^^^ This is the //TOWER/tank/torrents folder
>> Move everything from this location back to the surviving folder on the machine

./2169315211/
^^^ More photos not sure what (possibly some unsorted backup) but there's a lot

/656130169/
^^^ Not sure what this is? Check it out. It has interesting stuff in it.

./2147483875
^^^ This is possibly //TOWER/tank/work/employername folder

./2155167598/
^^^ Some previous unsorted phone backup.

./2222606136
^^^ Girlfriend's artworks

./2224369673
^^^ More important stuff

./2530965706
^^^ GIrlfriend's design stuff

./2673591577
^^^ Another unsorted phone Backup

./4511549153
^^^ Former Windows 10 & 11 Backup

 

The other hundred thousand-something folders seem to be 90% git repos which I cloned for work or hobby purposes. As some of these repos contained tens of thousands of files. This is most likely what makes up the bulk of lost+found.

 

I'm now going to mount the drives. Pay most attention to moving everything out of the above folders.

Then dump the drives in the cupboard after I've had enough and revisit this issue in days, months, decades when I feel like going through them again.

 

One final thing that also helped was using the flat view in 7zip (free) or even better "Branch View" in XYPlorer. This let me find all the PNG, JPG, etc. files in ALL of the folders which were spread thin, so I could then view them as a list, sort them by "type" and grab them all and move them into one folder. I then re-organized these by "date created" which is how they were done before.

 

Lesson learned. Always have a backup... of the backup.

Edited April 17, 20224 yr by plantsandbinary

Some other odd things I noticed.

• The initial amount of damaged files was over 120,000 with more than 62,000 folders.
• However after rebooting. I only had about 12,000 files and 600 folders in the lost+found directory.
• After stopping the array and starting it again, this number dropped down to about 8000 and 515. So presumably every time the array is mounted, it does some sort of check to see if it can restore orphaned files and folders to their original locations?

That was about as far as I got though with that. However, it did make things much easier to recover from.

 

I used XYPlorer to show the folder sizes, culled any that were empty or under a certain size. Also I used it to cull any files which were under at least 100KB. This left me with about 2000 files and about 250 folders to go through.

 

Then it was just a process of using XYPlorer to show only eg. "Image Files" or "Music Files" or "Documents" and move them accordingly their proper locations.

 

It helped to have my folder structured like this and keep things mostly totally organized over the years:

 

//TOWER/tank/Backups/<OS_type>

//TOWER/tank/Files/

//TOWER/tank/Files/Media/ (Documents, Downloads, Photos, Music etc)  > Photos/2003, 2004, 2005, et cetera.

//TOWER/tank/Files/torrents/<tracker_name>

//TOWER/tank/Files/Work/>company_name>/repos

 

Because everything was tightly organized, I managed to restore a lot of it without too much trouble. At least the stuff I cared about. Maybe I lost about 5% in the end.

I can say that UNRAID probably had something to do with removing a lot of the files out of lost+found and putting them back too upon mounting.

Edited April 17, 20224 yr by plantsandbinary