I think I just lost a bunch of data?

[itimpi]

1 hour ago, u0126 said:

 

How would that work? As of right now /mnt/disk9 is already putting in fresh data. Is that both being rebuilt and able to function at the same time, or is there some version of disk9 that's being rebuilt? Struggling to see how it can rebuild something at the same time it's adding to it.

The rebuild process knows what sector it has reached (it works serially through the sectors on the disk).    If you write data that goes to a sector earlier than the point reached by the rebuild then it is written to both the physical drive and parity updated accordingly.    If it is after that point then just parity is updated so that when that point in the rebuild is reached then the correct data can be written.

 

FYI:   Although you can write to the drive during the rebuild process that will badly degrade performance of both write and rebuild while they are running in parallel due to disk contention.

[u0126]

3 hours ago, itimpi said:

The rebuild process knows what sector it has reached (it works serially through the sectors on the disk).    If you write data that goes to a sector earlier than the point reached by the rebuild then it is written to both the physical drive and parity updated accordingly.    If it is after that point then just parity is updated so that when that point in the rebuild is reached then the correct data can be written.

 

FYI:   Although you can write to the drive during the rebuild process that will badly degrade performance of both write and rebuild while they are running in parallel due to disk contention.

It is rebuilding while the array is active, out of curiosity I looked at /mnt/disk9 and seeing new downloads are hitting that disk.

 

I'm not really concerned about performance (it's still performing well enough) and I'm on vacation not in any hurry for that to finish... it's not at top speed but seems at least 50% last I checked.

 

Previous post seemed shocked that new stuff is being sent to that disk, but I'm just letting unraid do its thing. I'd expect if that was "crazy" it'd leave the disk out of the array while it did that. I'm just confused what it might be emulating (or maybe it's because I'm conceptually thinking it's emulating a "disk", but really it's just emulating "missing sectors" overall?) - when it says a disk is being emulated does it not mean the disk but rather simply the "missing data" (sectors) are being filled in?

[itimpi]

1 minute ago, u0126 said:

I'm just confused what it might be emulating (or maybe it's because I'm conceptually thinking it's emulating a "disk", but really it's just emulating "missing sectors" overall?) - when it says a disk is being emulated does it not mean the disk but rather simply the "missing data" (sectors) are being filled in?

I guess emulating sectors is probably the easiest way to think about it in this scenario.    At the beginning of the rebuild all sectors are being emulated, but as the rebuild progresses the number of sectors being emulated is decreasing.    For simplicity sake Unraid will only remove the emulated text in the GUI when the rebuild completes and no sectors are now being emulated.

[Kilrah]

2 hours ago, u0126 said:

Previous post seemed shocked that new stuff is being sent to that disk, but I'm just letting unraid do its thing.

You were still talking of possibly doing file recovery, and running xfs_repair - considering those implied disk9 was still unmountable, and as such no data could have been written to it.

 

If you're writing data to it then it means the disk was formatted and there is a good fresh filesystem on it that's starting to get filled up, and that there is absolutely no option for recovery at this point. 

 

 

Edited March 21, 2023 by Kilrah

[u0126]

33 minutes ago, Kilrah said:

You were still talking of possibly doing file recovery, and running xfs_repair - considering those implied disk9 was still unmountable, and as such no data could have been written to it.

 

If you're writing data to it then it means the disk was formatted and there is a good fresh filesystem on it that's starting to get filled up, and that there is absolutely no option for recovery at this point. 

 

 

Yeah, I had set it as ready to restore to.

 

what I'm still curious about is if parity is restoring things back as they were isn't it restoring/emulating a corrupted xfs filesystem? If it's sector-based?

 

[Kilrah]

As you know formatting a drive doesn't set all the bits of a disk to a given state, it just writes a few things to create the filesystem "skeleton". 

When the filesystem was borked parity was consistent with said borked state. Starting a rebuild started rewriting that state to the drive. Then formatting wrote the fresh filesystem skeleton, both to the drive and the parity. The rebuild continues, to ensure any bits not touched by either the format or any new data writes since then are consistent with parity as itimpi explained. Since the drive was formatted all of them are "don't cares" from that filesystem's point of view, but they still have to be consistent with parity to be able to allow recreating another drive it one failed.

Edited March 21, 2023 by Kilrah

[SC8198]

Just jumping on this old thread. I did the exact same thing. I mistakenly formatted when I shouldn't have. I have a backup so I am not as worried. One question though. After formatting the drive and I am writing new data to it. Will this be corrupting the Parity?? I.e causing more damage? Or will that "NEW" data just be apart of the parity now and I can restore from my backups???

 

[Hoopster]

4 hours ago, SC8198 said:

Or will that "NEW" data just be apart of the parity now and I can restore from my backups???

Parity drive is just a bucket of bits based on the calculation of a 0 or 1 depending on the data in the same location on all the data drives.  If you write new data to a disk, parity just gets recalculated.  Rebuilding a disk from parity is a process of using the calculated values on the parity disk in conjunction with all the other data drives to determine what is missing and write that to the new disk.  Parity is not a backup of actual data.

 

If you restore data from a backup, parity is recalculated based on the new data.

Edited January 21 by Hoopster

[SC8198]

Okay cool, I think I understand. Sounds good to me. I ran a parity calculation right after I messed up. I knew that data was lost. Not terribly worried. I was in the middle of a migration to a new server and that is what caused the issues.