Have I lost data?

[cdixon]

I have an Unraid server that I have had up and running for two years. Over the past two years I have been uploading my DVD collection to the unraid server. I have 7 drives in the server. All of the drives are reformatted with XFS. Five of the drives are used to store the video and image files and the two other drives are used as parity drives. One day I checked the Unraid dashboard  and noticed that the parity drive 2 and drive 1 were disabled after a parity sync. I then rebooted the server and drive 2 showed up as unmountable and parity drive 2 was shown as invalid. I decided to reformat drive 1 thinking that I could rebuild the disc with the original data using the parity sync / data rebuild option. After I went through with this option, I noticed that none of the data I originally had on drive 1 did not come back. Is there a way of getting the drive 1 data back through the parity drive?

[trurl]

15 minutes ago, cdixon said:

I decided to reformat drive 1

Format is a write operation. It writes an empty filesystem to the disk. When you perform any write operation in the parity array, Unraid updates parity at the same time. So after a format of a disk in the parity array, parity agrees that the disk has an empty filesystem.

17 minutes ago, cdixon said:

Is there a way of getting the drive 1 data back through the parity drive?

So no, there is not.

 

There are prominent warnings in the webUI to try to prevent you from making this mistake. We would have given you different advice about how to repair an unmountable disk if you had asked us before doing anything.

 

If the data was very important, you might be able to recover some of it with some third party software, such as UFS Explorer. Obviously you should not write anything to the disk if you want to have any chance.

 

If you want additional advice about how to proceed with your existing setup, let us know what you want to do, and go to Tools - Diagnostics so you can include the complete diagnostics zip file attached to your next post.

[cdixon]

What third party recovery program would you recommend?

[JorgeB]

such as UFS Explorer

It has been used successfully before by other Unraid users.

 

 

[cdixon]

So should I use drive 1 or parity drive 1 with UFS explorer to recover the data?

[JorgeB]

Use it on the drive that was formatted.

[trurl]

Just now, cdixon said:

So should I use drive 1 or parity drive 1 with UFS explorer to recover the data?

Parity has no data. It only allows data to be reconstructed from the parity calculation using the bits on parity together with all the bits on all the other disks.

[cdixon]

So drive 1 it is then.

 

[cdixon]

How do I set up UFS file explorer in unraid?

[trurl]

3 minutes ago, cdixon said:

How do I set up UFS file explorer in unraid?

You don't. You would have to use it in another system. Have you been to their website to see about that software?

[cdixon]

Yes. Would it be a good idea to take drive one out of the server plug it in my computer and then use UFS file explorer to recover the data?

[trurl]

That would be a first step. I've not used that software myself. I don't know if it writes to the disk or if it tries to extract the data to some other location. If it alters the disk contents in any way then you won't be able to put it back in your server without rebuilding parity.

 

Do you have backups of anything important and irreplaceable? Many of us don't have the capacity anywhere to try to backup our whole server, but you should have another copy of anything important and irreplaceable. You get to decide what qualifies. Parity is no substitute for backups, as you have seen.

[cdixon]

I could re-upload the dvds to the drive but I just want to make sure that there isn't a better option of reloading the data since re-uploading the dvd files would be very time-consuming. (Drive 1 can hold 8 terrabytes)

[John_M]

4 minutes ago, trurl said:

it tries to extract the data to some other location

^This. In fact you are warned a number of times not to write to the disk you're trying to recover. I haven't used it on XFS disks but I've successfully used it to recover photos from a corrupt SD card. You need first to choose which OS you're going to run it on. My MacBook Pro has an SD card reader so I chose the macOS version. Other versions are available to run under Windows and Linux. They can all read the same file systems. You need to choose which edition of the software you want to download. The Standard Recovery edition is likely to be the one you want. It's free to install and test but unless you buy a licence (€49.95 for personal use) you won't we able to recover any but the smallest of files. So install it on your PC and let it scan the disk. This will take a long time but you get an indication of how long it's going to take and a progress bar. It will show you what it finds as a reconstructed virtual file system and then you can decide whether it's worth paying the money for the licence. I decided it was. You simply select the files you want to recover and choose where to save them.

[cdixon]

I was going through my server files and noticed that all the files that I lost are now back but drive 1 is still empty. How can I get my my previously lost files to fill up drive 1 again?

[John_M]

2 hours ago, cdixon said:

I was going through my server files and noticed that all the files that I lost are now back but drive 1 is still empty.

In the context of what has gone before what you describe is impossible. It looks as though you are saying that after formatting disk 1 the files have suddenly reappeared elsewhere. That can't happen unless you somehow had duplicates on other disks. There's a step missing. Do you perhaps mean that you have successfully recovered your files, copied them back to your server and would now like to move them back onto disk 1? I think that's more likely, though I would expect the recovery process to take more than a day. Just a bit of clarification needed!

[leeknight1981]

Hiya had a weird issue the other day Disk 7 smart health (198) offline uncorrectable 8, then later that day returned nor Normal as telegram says. Then i got Disk 7 Unmountable disk present after a reboot, took the drive out checked it reseated  booted back up and did the same Disk 7 Unmountable disk present. So i ran the Check Filesystem Status checked the smart report etc so i stopped the array again rebooted same Disk 7 Unmountable disk present and it said i had to format the drive which i did and started a re build. i appear to have LOST everything on that disk 7.85TB worth of Emby media i have dual Parity 2 x 12TB WD Disk's so why have i lost so much data? I was under the assumption i wont loose data if a disk dies and i re build as parity was Valid and i have dual parity. Have i missed something here? the share is set as High-water and i have 2 x 1TB SSD's as cache-pool for app data. can someone please explain to a thick person Why iv lost all that media buy loosing a disk 


TIA 

 

Lee 

[John_M]

53 minutes ago, leeknight1981 said:

why have i lost so much data?

 

56 minutes ago, leeknight1981 said:

i had to format the drive

 

If you have an unmountable disk, formatting it is not going to fix it. Formatting actually destroys data. It always has, whether it's your Windows computer's C : drive, your digital camera's flash card or your file server's storage. When you format a disk in Unraid you're shown a big warning that you have to acknowledge before you can proceed. If you can suggest a way of making the wording any clearer, then please do so.

 

[leeknight1981]

1 hour ago, John_M said:

 

 

If you have an unmountable disk, formatting it is not going to fix it. Formatting actually destroys data. It always has, whether it's your Windows computer's C : drive, your digital camera's flash card or your file server's storage. When you format a disk in Unraid you're shown a big warning that you have to acknowledge before you can proceed. If you can suggest a way of making the wording any clearer, then please do so.

 

Sorry i don't mean to be frustrated here But I Understand what formatting A disk does, But isn't some of the point to UnRaid that you can loose a disk without loosing data? i also have dual parity so if that 8TB totally died DEAD DEAD and i had to put a new drive in it should re build with NO Data loss! 

Unraid said the Drive Had to be formatted to be added to the array, Unmountable disk so i clicked on it and it said it had to be formatted! Yes i got a warning but i assumed i was Parity Protected so whats the difference in the disk just completely failing and me formatting it? Surely with Dual Parity and Parity Being VALID i shouldn't loose data  
 

Regards 

 

Lee 

 

[John_M]

2 minutes ago, leeknight1981 said:

I Understand what formatting A disk does

 

Then you understand that formatting it erases the data structures that allow stored files to be retrieved, in order to prepare it to accept new ones.

 

7 minutes ago, leeknight1981 said:

isn't some of the point to UnRaid that you can loose a disk without loosing data?

 

Absolutely, but formatting is never part of the rebuilding process.

 

8 minutes ago, leeknight1981 said:

Unraid said the Drive Had to be formatted to be added to the array,

 

To add a new disk to the existing array, it would need to be formatted. But you were not adding it. You were replacing a failed disk. If you add a disk the total number increases. If you replace a disk the total number remains the same. It's a completely different process - like adding a second disk to your PC would be different from replacing one that had failed.

 

12 minutes ago, leeknight1981 said:

Yes i got a warning but i assumed i was Parity Protected

 

You were parity protected, but you need to understand how parity works, not only in Unraid, but in all other applications. Parity protects the data integrity of the array. If you add a file, then parity is updated in real time to reflect that change. If you format a disk, then parity is updated in real time to reflect that change too. When a disk fails its contents are emulated - that is, reconstructed on the fly by a calculation that involves parity and the remaining working data disks. Having dual parity allows two disks to fail and be emulated, using more calculations and all the remaining disks. The emulated disk behaves just like the real one did before it failed - you can read from it and write to it and the data is preserved by updating the parity so that when you replace the disk it can be rebuilt from the emulated disk. Because you can write to an emulated disk you can format it and if you format it, when you rebuild it the result is a freshly formatted disk.

 

None of this is any different from how any parity protected array works, except that with Unraid you only formatted one disk and not the entire array.

 

I'm sorry you're frustrated. It's a hard lesson, but that's how it is. The warning message is very clear. I don't know how it could be made clearer. I'll repeat the mantra: "Parity is no substitute for backups."

 

[leeknight1981]

To add a new disk to the existing array, it would need to be formatted. But you were not adding it. You were replacing a failed disk. If you add a disk the total number increases. If you replace a disk the total number remains the same. It's a completely different process - like adding a second disk to your PC would be different from replacing one that had failed.

 

OK i put the formated drive in and it said ReBuild so thats what i did, Unraid also said Unmountable disk present so i clicked on it and it Said it had to be formatted so thats what i did. So if UnRaid see's an 8TB formatted disk and asks to rebuild why has the data that was on the failed disk not gone back. Why did unraid tell me that the disk had to be formated if thats Not what had to be done? I may have been naive here but i just assumed as i have dual parity it would Not loose any data.

[John_M]

If a disk is unmountable there are two ways to get it to mount. Either attempt to repair any file system that's already on the disk (and thereby preserve its contents) or format it and start over. That applies to all systems and has nothing to do with parity at all. You chose to format it and start over.

 

[leeknight1981]

1 minute ago, John_M said:

If a disk is unmountable there are two ways to get it to mount. Either attempt to repair any file system that's already on the disk (and thereby preserve its contents) or format it and start over. That applies to all systems and has nothing to do with parity at all. You chose to format it and start over.

 

I did that i tried repair file system in the disk 7 dropdown. Sorry i am just so Pissed off iv lost that data i cant run two 64TB servers its not cost effective i was under the dilution if a disk failed id be kinda protected against data loss Obviously not as i have lost 8TB of Media and don't even know whats missing. SO is that it my Valid Parity is now missing the failed 8TB Gone Forever. Sorry i cant even think straight let alone express what i mean properly all these years of using unraid never lost a disk and never lost data not once   

[John_M]

2 minutes ago, leeknight1981 said:

protected against data loss

 

You were protected against data loss, but not against operator error. The big scary warning is really trying to tell you that this is probably not the option you want to choose. I'm sorry for your loss but arguing about it won't bring it back. Some people have reported success in recovering deleted files with a utility called UFS Explorer.

 

[leeknight1981]

19 minutes ago, John_M said:

 

You were protected against data loss, but not against operator error. The big scary warning is really trying to tell you that this is probably not the option you want to choose. I'm sorry for your loss but arguing about it won't bring it back. Some people have reported success in recovering deleted files with a utility called UFS Explorer.

 

Im Not trying to argue i am telling you What UnRaid Told me to Do, It told me that the disk needed to be formatted so i did, it said start and rebuild So i did. Yes it warned me about formatting My bad for thinking i could loose a disk and re build and preserve.