Drive failed, data lost, parity present

[darber]

I had a drive fail recently and I seem to have lost data even though my parity disk is ok?

 

I hope someone can help.

 

Thanks,

tower-diagnostics-20231204-1546.zip

[itimpi]

At the moment it looks like disk1 is the problem drive and is being emulated but is showing as unmountable (due to file system corruption)?   The correct handling of unmountable disks is described here in the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page. 
 

You should run the check/repair on the emulated drive before attempting a rebuild as the rebuild process would just replicate the corruption onto the replacement drive.   Keep the ‘failed’ drive intact as it may offer additional recovery options if the repair fails.

 

Are you sure the drive in question has actually failed?   Often a drive gets disabled due to external factors rather than the drive being faulty?   If you are not sure you can try running an extended SMART test on the drive.   If it passes error free the drive is probably OK.

[darber]

5 minutes ago, itimpi said:

At the moment it looks like disk1 is the problem drive and is being emulated but is showing as unmountable (due to file system corruption)?   The correct handling of unmountable disks is described here in the online documentation accessible via the ‘Manual’ link at the bottom of the GUI or the DOCS link at the top of each forum page. 
 

You should run the check/repair on the emulated drive before attempting a rebuild as the rebuild process would just replicate the corruption onto the replacement drive.   Keep the ‘failed’ drive intact as it may offer additional recovery options if the repair fails.

 

Are you sure the drive in question has actually failed?   Often a drive gets disabled due to external factors rather than the drive being faulty?   If you are not sure you can try running an extended SMART test on the drive.   If it passes error free the drive is probably OK.

The disk seems to be making a "dink" noise which doesnt sound good. I currently have disk one set as not installed as I was going to RMA that drive (it is the newest one I have bought) but usually when doing this my array just uses the parity disk and continues but I currently have a lot of data missing this time.

[itimpi]

4 minutes ago, darber said:

The disk seems to be making a "dink" noise which doesnt sound good. I currently have disk one set as not installed as I was going to RMA that drive (it is the newest one I have bought) but usually when doing this my array just uses the parity disk and continues but I currently have a lot of data missing this time.

If the check/repair I pointed you at works without issue then the data on disk1 will ‘reappear’.

[darber]

15 minutes ago, itimpi said:

If the check/repair I pointed you at works without issue then the data on disk1 will ‘reappear’.

disk 1 shows up as a new device now is that ok?

[itimpi]

I did suggest you ran the repair on the emulated drive (before assigning a new disk1) 🙂    Does Unraid say it is going to rebuild disk1 (which is what I would expect)?   If so starting the array will start the rebuild.
 

However all the rebuild process does is make the rebuilt disk match the emulated one so it does not fix file corruption.   You can attempt to fix the corruption later if you want.

[darber]

1 minute ago, itimpi said:

I did suggest you ran the repair on the emulated drive (before assigning a new disk1) 🙂    Does Unraid say it is going to rebuild disk1 (which is what I would expect)?
 

However all the rebuild process does is make the rebuilt disk match the emulated one so it does not fix file corruption.   You can attempt to fix the corruption later if you want.

I had disk 1 disconnected so I just shutdown the server and plugged it back in so I could try and repair the drive.

 

Thats when I noticed its showing as a new device, now I dont know if I should start the array disk 1 in it.

[JorgeB]

For now start with that disk unassigned.

[darber]

24 minutes ago, JorgeB said:

For now start with that disk unassigned.

 

24 minutes ago, JorgeB said:

For now start with that disk unassigned.

the array is now rebuilding very slowly should I stop it and unassign?

[JorgeB]

Do you mean that you started with the disk assigned? Post new diags.

[darber]

1 minute ago, JorgeB said:

Do you mean that you started with the disk assigned? Post new diags.

yeah

tower-diagnostics-20231204-1754.zip

[JorgeB]

There are what look like power/connection issues with disk1, but since it's unmountable, first cancel the rebuild, stop array, unassign disk1, check filesystem on the emulated disk, run it without -n, if successful post new diags after array start, with disk1 still unassigned.

[darber]

4 minutes ago, JorgeB said:

There are what look like power/connection issues with disk1, but since it's unmountable, first cancel the rebuild, stop array, unassign disk1, check filesystem on the emulated disk, run it without -n, if successful post new diags after array start, with disk1 still unassigned.

I dont see an emulated disk, do you mean check the filesystem on disk 1? (which is now dev1 as its unassigned)

Edited December 4, 2023 by darber

[itimpi]

5 minutes ago, darber said:

I dont see an emulated disk, do you mean check the filesystem on disk 1? (which is now dev1 as its unassigned)

Disk1 in the main array (it should be showing unmountable).

[darber]

It didnt seem to let me run the file system repair without -n

tower-diagnostics-20231204-1929.zip

[itimpi]

1 hour ago, darber said:

It didnt seem to let me run the file system repair without -n

tower-diagnostics-20231204-1929.zip 170.27 kB · 0 downloads

It will prefill the option field with -n but you can remove that or nothing gets done.   If it subsequently prompts for -L you need to add that,

[darber]

ok all done, had to use -L

shall I take the array out of maintenance mode now and start in normal mode with the disk unassigned and post new logs?

[itimpi]

4 minutes ago, darber said:

ok all done, had to use -L

shall I take the array out of maintenance mode now and start in normal mode with the disk unassigned and post new logs?

Restart in normal mode and hopefully the disk will now mount OK.  
 

You should check to see if there is a lost+found folder on the drive as that is where the repair process puts files for which it could not find the directory entry to give it the correct name.    Not having that folder is a good sign that the repair process recovered everything cleanly.

[darber]

12 minutes ago, itimpi said:

Restart in normal mode and hopefully the disk will now mount OK.  
 

You should check to see if there is a lost+found folder on the drive as that is where the repair process puts files for which it could not find the directory entry to give it the correct name.    Not having that folder is a good sign that the repair process recovered everything cleanly.

the disk now mounts ok and most of my data is back, some stuff in the lost and found.

the disk is still making that weird noise though but no errors are being reported, maybe I need to run a SMART test?

[itimpi]

You should post new diagnostics so we can check the state of things.

[darber]

21 minutes ago, itimpi said:

You should post new diagnostics so we can check the state of things.

 

tower-diagnostics-20231204-2159.zip

[darber]

rebuild is very slow 0.4% in 45 minutes

[itimpi]

You are getting continual resets on ata4 which I assume is the disk1 you are intending to rebuild.   You can see this if you look at the syslog (accessible via one of the icons at the top right of the GUI.    This could be a cabling issue to the drive (power or SATA) or it could be the drive itself going bad.   Do not see much point in continuing the rebuild until you have resolved this issue.

[darber]

2 minutes ago, itimpi said:

You are getting continual resets on ata4 which I assume is the disk1 you are intending to rebuild.   You can see this if you look at the syslog (accessible via one of the icons at the top right of the GUI.    This could be a cabling issue to the drive (power or SATA) or it could be the drive itself going bad.   Do not see much point in continuing the rebuild until you have resolved this issue.

can I swap this drive with a new one without losing the data?

[JorgeB]

10 hours ago, darber said:

rebuild is very slow 0.4% in 45 minutes

I had already mentioned before that there's an apparent power/connection issue with disk4, replace the cables and try again, if the same replace the disk.