Did something stupid and deleted a partition - Help needed

[DrBobke]

Hi everyone,

 

I did something very stupid yesterday, when following Spaceinvader Ones video 

 

 

I followed what he was doing, but I first emptied drive 3 and put everything on drive 1. After that, I followed his video and also took disk 1 and deleted the partition (I know, VERY stupid!!).  However, I realized my mistake before formatting the drive to put it into the encrypted format.

I did the parity check and after a VERY long time (22 hours and 35 mins), I got the notification that partity was good.  Mostly containing movies, as well as some other files, I can still see the movies on my array and just clicking on a couple of them, it seems they are still there and playing.

 

I have 3* 12TB drives in the array, protected by a 14TB parity drive. There is also a 2TB cache drive.

The 12 TB HDD that I formatted after seeing the parity was good, was done and the encrypted xfs is good.  The HDD is empty though (well, it says 83.8 GB, which is false).  Drive 3 is also formatted to the encrypted file system (encrypted xfs too), also showing 83.8 GB. 

My question now is - where are the files (probably on the parity) and what can I do to get the files back on the HDD's/array?

 

I know, I am very stupid in making the mistake, baby was crying, girlfriend shouting, dog barking, NOT the best time to do that kind of work...

Any help will be appreciated.

I don't think dumping a log will help, but if needed, let me know.

[itimpi]

The files are still on the disk1, but the question is whether the actions you have taken have damaged the file system to the point where they are no longer retrievable.   From your description you have already updated parity so that it agrees with what is currently on disk1 so the unRaid built in recovery options are not going to work.

 

You are going to have to run some sort of recovery software against disk1 to see what you can get off it.    My personal experience is with UFS Explorer running on Windows.   It has a trial version which can be used to see what data looks like being retrievable, but to actually do a recovery you need the paid version.

 

There may be other tools that can be suggested - I would wait to see if anyone chimes in.  Whatever you do make sure that nothing further gets written to disk1 as each write would reduce the chance of successful data recovery.

 

 

[DrBobke]

32 minutes ago, itimpi said:

The files are still on the disk1, but the question is whether the actions you have taken have damaged the file system to the point where they are no longer retrievable.   From your description you have already updated parity so that it agrees with what is currently on disk1 so the unRaid built in recovery options are not going to work.

 

You are going to have to run some sort of recovery software against disk1 to see what you can get off it.    My personal experience is with UFS Explorer running on Windows.   It has a trial version which can be used to see what data looks like being retrievable, but to actually do a recovery you need the paid version.

 

There may be other tools that can be suggested - I would wait to see if anyone chimes in.  Whatever you do make sure that nothing further gets written to disk1 as each write would reduce the chance of successful data recovery.

 

 

 

Thanks a lot for your reply. If the files are on disk1 and I can find them via 'network locations' ànd play the files, how can they need to be recovered? 

I know all about not writing to the disk anymore, as I am a professional photographer and have helped many recover their files from flash disks (luckily I am always very careful not to do anything stupid, except for yesterday evening).

 

Any idea if Sandisk recovery software will work on drives of 12 Tb in size? I have many codes to use, as you get one for each SD/CF card you buy and I have never had to use them.  Hmm - apparently, RescuePro Deluxe can only support drives up to 1 TB.

 

So, to do the recovery on the WIN machine, do you take out the HDD from the array and put it into the WIN machine? Or how does this work best? I assume I can't do it via network location?

Edited February 9, 2021 by DrBobke

[itimpi]

1 hour ago, DrBobke said:

 

Thanks a lot for your reply. If the files are on disk1 and I can find them via 'network locations' ànd play the files, how can they need to be recovered? 

I know all about not writing to the disk anymore, as I am a professional photographer and have helped many recover their files from flash disks (luckily I am always very careful not to do anything stupid, except for yesterday evening).

 

Any idea if Sandisk recovery software will work on drives of 12 Tb in size? I have many codes to use, as you get one for each SD/CF card you buy and I have never had to use them.  Hmm - apparently, RescuePro Deluxe can only support drives up to 1 TB.

 

So, to do the recovery on the WIN machine, do you take out the HDD from the array and put it into the WIN machine? Or how does this work best? I assume I can't do it via network location?

that statement is not consistent with you saying you had formatted the disk and rebuilt parity!    If you simply removed the disk so it is being ‘emulated’ then that would be a reason the files could still be seen.

 

perhaps you should provide your system’s diagnostics zip file (obtained via Tools - > Diagnostics so we can see the current state of play

 

[trurl]

9 minutes ago, itimpi said:

diagnostics zip file (obtained via Tools - > Diagnostics so we can see the current state of play

Frankly I can't make sense of what you said you did but maybe the diagnostics would clarify.

[DrBobke]

Hi guys,

 

Thanks for chiming in...

 

So what I did, was (as Spaceinvader One suggested), went to tools, new config and preserved the cache drive.

Then I went to unassign HDD1 (which should have been disk 3) and removed the partition and added it again.  When I then started the array and had to format the disk, I suddenly realized my mistake. I did the parity check and the files would be okay.

After that, I thought it was safe to format the drive (as I could still see my movies files and play them). I now have the drive as encrypted xfs.  As HDD3 was also empty, I encrypted that drive as well, leaving only HDD2 ams xfs (I am currently moving files with unbalance to HDD3).

 

I know HDD1 had most of my movie files on there (I had set it to put movies primarily on HDD1), together with some other files (don't really remember which ones).

When going through network locations, I can still see the movie files and play some of them too - but obviously didn't test all of them.

HDD1 is still shown as 83.8GB now, so I thought Parity had seen the drive "go down" and repair it during the 22h35 which it took to check the parity.

leonore-diagnostics-20210209-2235.zip

[trurl]

Disks 1 and 3 are encrypted and look mostly empty. Disk2 is about half full. Your Movies share is on all three drives.

 

I am still unclear what you did and why. Looks like you must have rebooted after whatever you did so can't see any of that in syslog.

 

Did you format a disk in the parity array and then attempt to rebuild its contents? When you format a disk in the parity array, parity is updated and so agrees that the disk is now empty, and rebuilding it will just result in an empty disk.

[trurl]

4 hours ago, DrBobke said:

where are the files (probably on the parity)

Parity contains no files. It is just an extra disk that contains parity bits that allows the contents of a data disk to be calculated from all the other disks.

 

And as noted

2 minutes ago, trurl said:

When you format a disk in the parity array, parity is updated and so agrees that the disk is now empty

 

[trurl]

4 hours ago, DrBobke said:

The HDD is empty though (well, it says 83.8 GB, which is false).  Drive 3 is also formatted to the encrypted file system (encrypted xfs too), also showing 83.8 GB. 

Disk1 says 1% full, which is just going to be filesystem overhead. Disk3 says 5% full, 500G+ so it probably has some movies on it. Disk2 as mentioned is over half full.

[trurl]

1 hour ago, DrBobke said:

the 22h35 which it took to check the parity.

Sounds like you just did a parity check, which doesn't do anything to the contents of any data disks. You formatted the disks and parity agrees they have been formatted.

[DrBobke]

I did do a reboot, as I thought that might trigger the Parity to transfer the contents (which I thought it would have), over to HDD1, that was in the array again (encrypted this time).  The reason for thinking that, is that during Parity check, in the "main" tab, I could see mostly reads from Disk1 and writes on the Parity. The Parity now also says writes '34.453.604' and the reads on HDD1 states '30.366.991'. Parity also read some of Disk 2 and Disk 3, but as you can see almost everything from disk 1.

 

I'm still a little confused why I can still see the movies (however, I did notice that I'm missing a few movies that I added over the weekend from my desktop).

Any ideas what would be the best course of action at this point to get the data back? (Or at least be able to see in detail what was on the drive?).  As this is a new build, I probably have about 99% of the files on my old NAS and desktop.

 

HDD3 is getting fuller, as I am using unbalance to move everything off drive 2 into drive 3 to encrypt that one too.

 

As mentioned before, I followed Spaceinvader One's video and deleted the partition, readded the drive in the array and when I then clicked on format the disk (and before hitting "next"), I realized my mistake, so I didn't format the drive until after the Parity check was done.

 

Best to figure out which HDD is disk 1, attach it to my current desktop and run UFS Explorer to see what is on HDD1, or how can I best do that?

Unbalance is running now, moving data from drive 2 to drive 3, I assume I can stop the move, stop the array, shut down unraid and figure out which drive is which? (Unbalance is moving at @47 MB/s, did 2.55TB and is only at 41%, so would have to wait another 20-ish hours for it to complete, I'd rather protect Disk1 as much as possible from writes in the meantime).

 

Thanks for helping!

 

Screenshot below is an old Chrome window that was still open, where you can see the writes on the Parity and the message parity/the system came up with.

 

This is the screen right now : - not sure what the warning messages are though.

Edited February 10, 2021 by DrBobke

[DrBobke]

In the meantime, I have taken out HDD1 from the unraid (first stopped array to determine which drive it was) and shut down unraid.  I removed HDD1 and started unraid again and started the array (without Disk1 in).  I noticed all my dockers and vm's were missing - is this normal?

 

I have plugged in disk1 on my desktop and am currently running UFS Explorer. I do see that the drive is LUKS. I even installed paragon software to try to unencrypt it, but the software said it couldn't find an HDD (although it is showing up in Disk management, but I cannot assign a letter to it). Also UFS finds it okay. I put in that it should scan for file systems : (except for the NTFS, but couldn't deselect that on the drive for the printscreen).

 

I have managed to stop the recovery and found a good article/video here : https://www.ufsexplorer.com/articles/how-to/recover-data-luks-encryption.php

So I got the drive unencrypted (or at least entered my correct set password).  This opened the drive and I selected to search for EXT2,.. and SGI XFS.

 

I'll let you know once it finishes (probably another 13-14 hours...) *sigh*.

Edited February 10, 2021 by DrBobke

[trurl]

7 hours ago, DrBobke said:

I probably have about 99% of the files on my old NAS and desktop.

This is probably going to be the easiest and cheapest way to get those files.

 

7 hours ago, DrBobke said:

thought that might trigger the Parity to transfer the contents (which I thought it would have), over to HDD1

Can't imagine why you would have thought anything remotely like that.

 

The rest of this post is going to be a little long, but I encourage you to study it.

 

Everything you have said in this thread reveals profound misunderstandings about how parity in general works (on any system) and how Unraid uses parity. This is one reason I had trouble understanding anything you said, because it didn't really make any sense based on how you should use Unraid.

 

As mentioned, parity contains no files and is in no way to be considered a backup. There is no magic that could possibly allow parity to have the capacity to contain the data for all your data disks.

 

Parity is a common concept in computers and communications, and it is basically the same idea wherever it is used. Parity is just an extra bit that allows a missing bit to be calculated from all the other bits. Here is the Unraid wiki on parity:

 

https://wiki.unraid.net/UnRAID_6/Overview#Parity-Protected_Array

 

Understanding parity can help make more sense of how Unraid works to manage and protect your data, and can help make more sense of how you work with Unraid. And how certain ideas about how you might work with Unraid can't possibly make sense.

 

There is also a misunderstanding, apparently common, about format. Format and pretty much everything else you do with a disk except a read is a write operation. Write and copy are obvious, but delete also writes to the disk the files are deleted from, and move between disks is just a copy followed by a delete (move within a disk updates the folders so it is also a write of course).

 

Format is also a write operation. It writes an empty filesystem to the disk. That is what format has always done in every operating system you have ever used. Many people seem to have some vague idea that format just means "get a disk ready to use", whatever that might mean.

 

Unraid parity is realtime. Any time a write operation takes place on a data disk in the array, parity is updated so that the array stays in sync. After formatting a disk in the parity array, parity is updated just like with any write operation, and the array agrees the disk has been formatted. Rebuilding a formatted disk can only result in a formatted disk.

 

Why do you want to use encryption? It is just added complexity that can make recovery difficult even when you don't make serious mistakes.

 

In the future, please ask on the forum before you attempt to make any changes to your disk formats or assignments, or if you have any other indication of a problem such as disabled or unmountable disks. Don't make a problem become multiple problems and data loss just because of user error.

 

Be sure to setup Notifications to alert you by email or other agent as soon as a problem is detected. Don't let a single problem go until it becomes multiple problems and data loss.

 

And remember, parity is not a backup. You absolutely must always keep another copy of everything important and irreplaceable, preferably on another system. You get to decide what qualifies as important and irreplaceable.

 

[trurl]

2 hours ago, DrBobke said:

I noticed all my dockers and vm's were missing - is this normal?

According to those diagnostics you posted earlier your dockers and VMs are on cache as they should be. I did notice, though, that cache was a bit full. Possibly some other things have happened since you posted those diagnostics that have broken other things.

 

Post new diagnostics.

[DrBobke]

Thanks a lot for the explanation, that really helps in my understanding.

I've always had a NAS, configured in RAID5. When a drive failed, I just had to put in another drive, and it would rebuild everything - I thought Parity drive was a same sort of thing. Apparently not, or is that because I formatted the disk (i.e. wrote over all data)? If a drive were to fail, the Parity would be able to rebuild that, no?

 

Yes, I made sure Appdata, iso, domains and system were on the cache drive.  The cache is pretty full, as I was testing the movie, music and another folder to use cache and see what that did.  I meant it for when I would want to watch a movie, cache would step in, making it play smoother, than getting it off an HDD, which resulted in the cache being filled up.  As HDD1 then formatted, I put use of cache drive off for all but appdata, domains, iso and system, to keep it from moving the movie files back onto HDD1, thus making writes to the HDD and making the attempt to restore moot.

 

Unraid is switched off for the moment, I will power it back on tomorrow, when the restore files on UFS Explorer has finished, to not stress the system even more.

Finally when I had more or less everything set up as I wanted.  Encrypting the drives : I have some data on it that I'd rather keep private, other files are movies, so don't really care too much about encrypting, but am going down that road anyway.  If there would be a nice beginner guide to set up unraid, that would be really helpful, as I would've done that from the beginning, before copying all my files over to unraid.

[trurl]

31 minutes ago, DrBobke said:

I've always had a NAS, configured in RAID5. When a drive failed, I just had to put in another drive, and it would rebuild everything - I thought Parity drive was a same sort of thing. Apparently not, or is that because I formatted the disk (i.e. wrote over all data)? If a drive were to fail, the Parity would be able to rebuild that, no?

Even RAID5 parity is the same concept, just an extra bit that allows a missing bit to be calculated from all the other bits. But RAID5 doesn't have a single disk dedicated to parity, it stripes data and parity across all disks. This makes things faster, but it also means none of the disks can be read or written by themselves, and different sized disks can't be used.

 

Unraid IS NOT RAID, but just like RAID, Unraid parity allows you to calculate the data for a missing disk from all the other disks, so a failed disk can be rebuilt. Unlike RAID, an Unraid disk can also be replaced with a larger disk (up to size of parity) and be rebuilt to that larger disk.

 

31 minutes ago, DrBobke said:

I formatted the disk (i.e. wrote over all data)

Format doesn't write over all data (which would take hours). It just writes an empty filesystem to the disk. This empty filesystem only changes the bits needed to represent a filesystem (of the chosen type) with an empty top level folder and whatever else the filesystem needs (metadata) to allow you to work with its folders and files.

 

Most of the bits that represented the previous contents are unchanged until reused (delete works the same way), but they are no longer part of the filesystem so can't be accessed in the normal ways. Even tools that are designed to recover this sort of data often can't make complete sense of them, and you will wind up with files or partial files with no indication of what folders they were in or even what the filename was.

 

31 minutes ago, DrBobke said:

Yes, I made sure Appdata, iso, domains and system were on the cache drive.  The cache is pretty full, as I was testing the movie, music and another folder to use cache and see what that did.  I meant it for when I would want to watch a movie, cache would step in, making it play smoother, than getting it off an HDD, which resulted in the cache being filled up. 

appdata, domains, and system files on cache allows dockers and VMs to perform better since parity isn't involved, and it helps keep array disks spun down since these shares always have open files. Putting other things on cache for speed doesn't usually make much difference since each file is read from the single disk it resides on, parity isn't involved for simple reads (assuming the disk isn't disabled and being emulated), and single disk read speed is sufficient for most purposes, even 4K video.

 

But more importantly, if you fill cache you can definitely break things.

52 minutes ago, trurl said:

cache was a bit full. Possibly some other things have happened since you posted those diagnostics that have broken other things.

 

Post new diagnostics.

 

[DrBobke]

Thanks for clarifying further, trurl!

That still doesn't explain why I was still able to see "all" movies and the ones that I cherrypicked to see if they'd work, played without any issue (obviously I didn't play the entire movies, but I clicked through them and they were fine).

 

The UFS now, is just to see what files were actually on the drive, as I was also changing the shares, so all movies would be Disk1, music would be disk2, together with tutorials, etc, so when watching a movie, disk 2 and 3 could spin down and conserve some power and disk-wear.

 

What would you suggest I do? Abort the UFS Recovery and get the server up and running with the disk (HDD1) inside, or wait for it to finish and see which files to transfer accross? Ideally, I wouldn't want to have to copy over 10 TB of vids from the old NAS, which only averages around 40 MB/s...

 

The cache drive hasn't filled up more than what it currently is. Understood that it shouldn't be used that way, but again, a starter guide would be great. I wouldn't mind reading for hours on end, with some vids from Spaceinvader one in between.  Now, I'd had to find a way to make sense of all the movies, some outdated, programs missing/no longer there/renamed/... One could easily skip over parts that are not interesting for their use-case..?

[trurl]

Is this still a problem?

5 hours ago, DrBobke said:

all my dockers and vm's were missing

That is why I have asked twice thrice now for new diagnostics.

[DrBobke]

As I wrote, I have unraid off for now. Should I restart the server and the array, without disk1 in there? Hdd1 is now in my WIN10 desktop, for the Recovery (to at least see which files I have to transfer over again).

[trurl]

Probably just as well to leave it down since without that disk you have no redundancy.

[DrBobke]

Okay, So just to give an update on the current state of things :

This morning, UFS Explorer finally finished checking and I could see some files (although all had a different name, etc).  As that wasn't really helpful, I decided in putting back HDD1 into the server and noticed that the SATA connector of the SSD cache drive had actually popped out.  When I readded that into the array, I could see my dockers and VM's again - happy that was all it was and stupid that I missed the loose SATA connector earlier.

 

I am now using Teracopy to transfer over movies again - good that it is actually checking every file in the subsequent folders, so it's a lot easier to copy.  I noticed "all" folders being there, but some or all files were missing from several folders.  Also, my tutorials had all been wiped (incl folders), but that is easy to copy over.  I am now also using unbalance to transfer files off Disk2, so I can encrypt that drive as well (yes, I put the movie and tutorial shares on exclude Disk2, so I don't need to move everything over again).

 

I'll keep posting more information when going forward...

[DrBobke]

I have copied over everything in the end again and am up and running again since some time. All data drives are now encrypted, but not the parity drive, nor the flash drive.  Is that needed? I think I read that parity is not needed, as there will only be data on there when a drive would fail?

[JorgeB]

1 hour ago, DrBobke said:

Is that needed?

Parity doesn't have data, so not needed, and the flash drive can't be encrypted.

[DrBobke]

Thanks a lot, JorgeB! 🙂