Replacement drive shows less data than before

[gitbox]

I just replaced a failing drive in my 6.2.4 unraid system. The preclear and rebuild went fine but now the new 2T drive only shows 115G used whereas before if was around 1.5T. I can still access everything I have tried but this puzzles me. Did I lose data? Why the change if not?

 

Thanks

[RobJ]

If the drive was failing, it's possible that a failed write corrupted the file system of the original drive.  You may have to fix the file system, see Check Disk File systems.

 

For more help, please see Need help? Read me first!, and attach the diagnostics zip.

[gitbox]

I shouldn't have said failed. I was actually getting errors on the drive even though it passed that short smart test. It was six years old so I decided to be preemptive and replace it. I thought after the rebuild all the former data would be there but it isn't as evidenced by looking at the folders and files just on that drive. I can still access the old data on the original drive.

 

My understanding is that if a drive is replaced and a rebuild is completed, all the original data should be back on the new drive.

[trurl]

Rebuilding is a bit-for-bit copy and so cannot fix filesystem corruption. Did you read the link RobJ gave you?

[gitbox]

Maybe formatting the new drive as ZFS and then doing the rebuild may be to blame? From what it looks like now, the new drive has just what I put on it since it was installed. The rest is empty. Now I am copying the files on the old drive back to the NAS using a Windows PC and Diskinternals Linux Reader. (I checked and everything that was on the old drive is missing on the NAS now)

 

Here is what I think: replacing a drive with a different format and rebuilding will not recover the data that was on it. Therefore it is not a bit for bit rebuild. If it was, the file system would revert to Reiser. It is showing ZFS just as I formatted it. If I am right, then I am rather disappointed. At least I still have my data.

[itimpi]

Maybe formatting the new drive as ZFS and then doing the rebuild may be to blame? From what it looks like now, the new drive has just what I put on it since it was installed. The rest is empty. Now I am copying the files on the old drive back to the NAS using a Windows PC and Diskinternals Linux Reader. (I checked and everything that was on the old drive is missing on the NAS now)

 

Here is what I think: replacing a drive with a different format and rebuilding will not recover the data that was on it. Therefore it is not a bit for bit rebuild. If it was, the file system would revert to Reiser. It is showing ZFS just as I formatted it. If I am right, then I am rather disappointed. At least I still have my data.

you cannot change a file system type as part of a rebuild as the rebuild process always puts back the file system type that was present before the rebuild.  I am also a bit puzzled by the reference to ZFS as unRAID does not support that format - did you mean XFS?  Note that a format is never part of a rebuild process so if you did one it would wipe any data on the drive (and update parity to reflect the format) which would explain what you are seeing.

[gitbox]

Yes I meant XFS. I should read my posts more carefully. 

 

Memory is getting fuzzy now, but I'm sure I had to format the drive before I could mount it. That is when I chose XFS, since I heard it was the preferred file system. If I had chosen ReiserFS, all would have been fine?

 

I guess I could take this experience has a how-to for changing file systems. 

 

Once I get all the data back on the NAS, I should be okay so no harm done.

 

Thanks much for the responses. It always good to have experts around.

[gubbgnutten]

It does not matter which filesystem you pick, formatting a drive always results in an empty one.

 

Did you read the link RobJ gave you? Keep that one in mind in the future.

[gitbox]

Gub, I formatted the drive BEFORE the rebuild so I could mount it. The rebuild went fine but I wound up with no data on the replacement drive. That still leaves me with the question about the rebuild - if it is bit by bit, as others have said, what difference does the format make, other than allowing me to mount the drive. According to past posts it should have rebuilt everything back to the original drive state and file system. It did not. To clarify my steps:

 

1. Stopped array

2. Powered down

3. Removed old drive and installed replacement drive (same size)

4. Powered up

5. Ran preclear on new drive

6. Formatted new drive as XFS (format on old drive was Reiserfs)

7. Mounted drive

8. Started rebuild

 

All completed without error with the exception of no data being restored on the new drive. This still puzzles me.

[JorgeB]

Gub, I formatted the drive BEFORE the rebuild so I could mount it.

 

You can't do that, the rebuild itself will format the disk, same filesystem as it was before.

[Squid]

6. Formatted new drive as XFS (format on old drive was Reiserfs)

By formatting the disk through the UI (you changed the file system type, so it offered to format it for you), you wound up formatting the emulated drive which effectively wiped the filesystem for you.  Formatting on unRaid is no different than any other OS.  It clears the contents of the drive  And that is why you have data missing.

[trurl]

Gub, I formatted the drive BEFORE the rebuild so I could mount it. The rebuild went fine but I wound up with no data on the replacement drive. That still leaves me with the question about the rebuild - if it is bit by bit, as others have said, what difference does the format make, other than allowing me to mount the drive. According to past posts it should have rebuilt everything back to the original drive state and file system. It did not. To clarify my steps:

 

1. Stopped array

2. Powered down

3. Removed old drive and installed replacement drive (same size)

4. Powered up

5. Ran preclear on new drive

6. Formatted new drive as XFS (format on old drive was Reiserfs)

7. Mounted drive

8. Started rebuild

 

All completed without error with the exception of no data being restored on the new drive. This still puzzles me.

There is one detail missing from these steps that might be instructive.

 

At what point exactly did you assign the new drive to the slot of the old drive?

[gitbox]

Squid, I formatted the drive BEFORE the rebuild.

 

trurl, that would be step 7.

 

I don't recall having the option to rebuild without assigning the drive nor do I recall being able to assign the drive without formatting it, hence the format.

 

As an aside, I wanted to convert to XFS anyway so in the end, no data lost and new drive is XFS formatted.

[trurl]

trurl, that would be step 7.

Doesn't really clear up the mystery. So when you said "Mounted" did you actually mean Assigned?

 

How exactly did you format the drive without assigning it? From the command line? From Unassigned Devices? On a different system?

[gitbox]

Gosh, I really wish I had paid more attention to the details. 

 

I'm not really sure at this point the exact sequence of steps. What I am positive about is that I had an assigned and formatted (as XFS) drive in place BEFORE the rebuild and the rebuild ran without any errors at all so I assumed I had my data back in place. Regardless of what I did before the rebuild, shouldn't the rebuild process have notified me that something was amiss?

 

Once again, if the rebuild is bit by bit, it should not have cared what I had in that slot as long as it was a good drive with the correct size. Is that not correct?

 

I do appreciate your patience with these questions. 

[Squid]

Gosh, I really wish I had paid more attention to the details. 

 

I'm not really sure at this point the exact sequence of steps. What I am positive about is that I had an assigned and formatted (as XFS) drive in place BEFORE the rebuild and the rebuild ran without any errors at all so I assumed I had my data back in place. Regardless of what I did before the rebuild, shouldn't the rebuild process have notified me that something was amiss?

 

Once again, if the rebuild is bit by bit, it should not have cared what I had in that slot as long as it was a good drive with the correct size. Is that not correct?

 

I do appreciate your patience with these questions. 

It is bit by bit.  But because you assigned it then formatted you wound up formatting the emulated drive

 

Sent from my LG-D852 using Tapatalk

 

 

[JorgeB]

To change that disk's filesystem during a rebuild there are only three options:

 

a) you changed the filesystem of that disk before assigning the replacement disk, started the array, emulated disk appeared as unmountable and you pressed the format button.

 

b) you changed the filesystem of that disk before assigning the replacement disk, assigned the replacement disk, started the array, rebuild begun but the disk was unmountable, you pressed the format button.

 

c) you assigned the replacement disk, changed filesystem, started the array, rebuild begun but the disk was unmountable, you pressed the format button.

 

unRAID rebuilt the disk bit by bit but reflecting the format you did, because parity was updated when you formatted the disk, there's no way to change a disk filesystem on a parity protected array without updating parity.

 

[trurl]

Formatting is a write operation. It writes an empty filesystem to the disk. If the format is to an array disk, unRAID treats this write operation just like any other, by updating parity. So after the format, parity agrees the disk has an empty filesystem, and rebuilding the disk results in an empty filesystem.

[Squid]

Why don't we make a feature req to prevent unraid from ever formatting an emulated drive (or prevent allowing fs changes to a drive that's currently emulated)

 

I can't think of any legit reason beyond an edge case where this is something you would want to do.

 

Sent from my LG-D852 using Tapatalk

 

 

[gitbox]

I think I get it now thanks to Johnnie. When I formatted the drive, unraid thought I wanted to dispense with the data that was on the old drive and wrote the parity to reflect it, hence my data loss.

 

I agree with squid. There should have been some warning when I formatted the drive letting me know that I was effectively deleting everything I had on my old drive. Data protection is the primary purpose of unraid, is it not? I know all stupid actions can't be prevented but I really had no idea that what I was doing was destructive. Destroying data shouldn't be that easy for a user to do.

[gitbox]

Why don't we make a feature req to prevent unraid from ever formatting an emulated drive

 

Squid, will you be doing that? I really think it's needed. I can't be the only who's ever done this.

[RobJ]

I wanted to agree with this feature/safety request, but no matter how I think through it, I still don't understand how it could have happened.  I don't think there is any way to format an emulated drive, as once you assign a drive to the emulated slot, the ONLY option is to rebuild.  Your steps list left out several crucial steps, and I was about to ask for them, and then trurl asked exactly what I wanted to know.  But your answer was essentially that you formatted in step 6, but didn't assign it until step 7, so I'm still confused.  How exactly did you format the drive, and at what point did you change the file system type?  Changing the file system type would not even matter once the rebuild begins, as it's going back to the previous type.  And I think there has to be a good drive already assigned, before you can change the FS type.  Come to think of it, the only way it all makes sense is to have done the rebuild first, then changed the file system type and formatted the drive.

[Squid]

I wanted to agree with this feature/safety request, but no matter how I think through it, I still don't understand how it could have happened.  I don't think there is any way to format an emulated drive, as once you assign a drive to the emulated slot, the ONLY option is to rebuild.  Your steps list left out several crucial steps, and I was about to ask for them, and then trurl asked exactly what I wanted to know.  But your answer was essentially that you formatted in step 6, but didn't assign it until step 7, so I'm still confused.  How exactly did you format the drive, and at what point did you change the file system type?  Changing the file system type would not even matter once the rebuild begins, as it's going back to the previous type.  And I think there has to be a good drive already assigned, before you can change the FS type.  Come to think of it, the only way it all makes sense is to have done the rebuild first, then changed the file system type and formatted the drive.

I'm not about to try it, but yes when you assign a disk its going to automatically begin rebuilding.  But, I'm sure that you can also change the fs on an emulated drive in which case regardless of what's going on with the rebuild you're going to get the option to format also.  The only thing you can't do is change the fs while its rebuilding as that requires the array to be stopped.  But on a restart it will commence the rebuild and prompt you to format

 

 

[gitbox]

Robj, I didn't CHANGE the file system, it became an XFS drive only after I formatted it. I popped the replacement drive in right from the new WD box it came in. I ran the preclear script and when it passed I tried to assign it and here is where it gets fuzzy. I seem to recall that it would not allow me to assign the drive without formatting it first. That's when I chose XFS. The format completed and I assigned the drive and started the rebuild. At no point did any message or warning come up saying that what I was doing might be destructive. That's the thing that disappoints me. I had no clue that I was deleting a whole drive full of data.

 

I'm in the tech biz myself and the steps I did seemed rather logical: take out a drive, pop in a new one, test it, format it, and start a rebuild. I suppose the format part is how I screwed up but in my experience with every OS I've worked with, a drive has to be formatted to be of any use.

[JonathanM]

I suppose the format part is how I screwed up but in my experience with every OS I've worked with, a drive has to be formatted to be of any use.

Unless the drive in question is a replacement for a failed RAID array member.