Disk errors

[lennygman]

Quick question:

 

I started getting disk errors on one of the drives. I had issues before and this drive was getting disabled which turned to be a cable issue. This time i replaced with new cables twice but errors are still showing up.

 

Should I replace the drive?

 

 

 

 

tower-smart-20180212-1340.zip

[lennygman]

I re-run smart after reboot

tower-smart-20180212-1537.zip

[trurl]

The disk needs to be replaced ASAP

197 Current_Pending_Sector  0x0032   196   196   000    Old_age   Always       -       1465

Do you have Notifications setup?

[lennygman]

Yes.. I got email that is how i knew there was an error.

 

So just take out.. put new.. assign to same disk number and rebuild array?

[trurl]

1 minute ago, lennygman said:

Yes.. I got email that is how i knew there was an error.

 

So just take out.. put new.. assign to same disk number and rebuild array?

yes

[lennygman]

ok  Thank you.. will try.. I was thinking it is a cable issue again but no such luck  

[lennygman]

Hi.. i followed this process

1. Stop the array
2. Unassign the old drive if still assigned (to unassign, set it to No Device)
3. Power down
4. [ Optional ] Pull the old drive (you may want to leave it installed for Preclearing or testing)
5. Install the new drive
6. Power on
7. Assign the new drive in the slot of the old drive
8. Go to the Main -> Array Operation section
9. Put a check in the Yes, I'm sure checkbox (next to the information indicating the drive will be rebuilt), and click the Start button
10. The rebuild will begin, with hefty disk activity on all drives, lots of writes on the new drive and lots of reads on all other drives

Disk is now showing green and no errors but it is empty.. Has no data. This disk had my VMs so VMs are not starting.  

[trurl]

That process should work OK. Are you sure you didn't check a box telling it to format?

[lennygman]

I mounted bad disk as unassigned and moving data into new disk 4 manually. I restarted array so I guess logs are lost  .  Attached are current diagnostics (ignore disk 5 missing, i unplugged it for now)  

 

One thing i recall i did in the above process was to acknowledge an error in "Fix Common Problems" after i moved disk 4 into No Device state and before i shut down server to replace drive. I dont recall the message but I like to think it was just a warning that disk 4 was missing. Would this app identify that a disk was removed from the array and ask me to confirm that data is not needed?  I dont recall the message. Click to quick. 

tower-diagnostics-20180213-1003.zip

tower-syslog-20180213-1013.zip

[trurl]

If you had filesystem corruption, then it's possible that unRAID couldn't mount the emulated disk and so it might offer to format it. There is a checkbox for that when you start the array also.

 

So are you saying you can read the original disk in Unassigned Devices?

 

Restarting the array does not lose logs, but rebooting does. And it looks like you rebooted instead of restarted.

 

It is never necessary to also post the syslog with your diagnostics. Diagnostics already includes the syslog.

[lennygman]

@Trurl  thank you for your support. I was just playing again trying to replicate my steps and i think you might be right.. I think i did checked of the "format" Ugh..  so that did that do?  cleared data within array?

 

 

[trurl]

Format is a write operation. In every operating system you have ever used, "format" means "write an empty filesystem to this disk".

 

unRAID treats this just like any other write operation, by updating parity. So parity agrees the disk is empty. It is not clear though, it is just an empty filesytem. And it only affects the disks that were formatted. Are all your other disks OK?

 

It is sometimes possible to recover files from a formatted disk, but if you can still read them from the original disk that is probably the simplest.

 

 

[lennygman]

I understand format, but was clicking to quick without reading (my fault). I assumed that drive needed to be formatted before data gets rebuilt on it (typical step in windows) 

That is what I am doing right now.. i mounted old drive and copying data. So once it is done, data will be "seen" as part of the array like before?  

[lennygman]

Trurl.. things are never easy  so as i am copying the files, one vdisk for one of the VMs i had does not read on old drive (30G file). You mentioned there is a way to get data after format. If I want to just get one file, how complicated is that process?

[trurl]

Might be easier to try filesystem repair on the UD. What filesystem?

[trurl]

2 hours ago, lennygman said:

I assumed that drive needed to be formatted before data gets rebuilt on it (typical step in windows) 

 

I don't really remember since I have only needed to do it a couple of times, but I don't think I have ever formatted a drive in Windows before restoring an image to it. What would be the point of writing an empty filesystem when you are going to restore a non-empty filesystem?

 

Of course people often format a disk when they are getting ready to use it, but that just blurs the distinction I was trying to make, and that is what often gets people in trouble here.

[JonathanM]

53 minutes ago, trurl said:

Of course people often format a disk when they are getting ready to use it

And by use it, you mean filling it with files.

 

Which is what people ASSume parity does, recreate their files. Problem is, as you well know, parity doesn't have any files in it. None. Zero.

 

Obviously there is something broken in our education or communication about unraid, because we keep seeing the SAME issue, time after time.

 

Is there a way to fix the way we present and communicate? We already got automatic formatting turned into a checkbox, and people STILL shoot themselves in the foot.

 

I proposed a few months ago that disk operations like this should be on their own page, because formatting happens so rarely that it shouldn't have a place near the start button. The warnings about setting a new config not restoring data seems to have worked, we haven't had a case like that in a few years. Maybe if we moved the format option to the disk properties page, and surrounded the button with a clear explanation of what format is and does, and when it's appropriate and when it's not, then maybe?

[lennygman]

I totally say it is my fault as I was not reading descriptions as i was going thru rebuild and it is my first failure/rebuild (plus i am not an expert on UnRAID), but to jonathan's point,  i do think Formatting a drive would not make sense to have anywhere near array operations. I think it belongs to disk operations section or something like a utility like "preclear" 

[JorgeB]

6 minutes ago, jonathanm said:

Is there a way to fix the way we present and communicate? We already got automatic formatting turned into a checkbox, and people STILL shoot themselves in the foot.

v6.4 already has an additional warning in red that formatting a drive will make it impossible to recover with parity, OP is still on v6.3.5 so not useful to him.

[lennygman]

3 minutes ago, johnnie.black said:

v6.4 already has an additional warning in red that formatting a drive will make it impossible to recover with parity, OP is still on v6.3.5 so not useful to him.

Was thinking to upgrade but could not find time..  good to know.. thank you

[lennygman]

I recovered all by 1x30Gb file (VM vdisk).  Should i try running reiserfsck on failed disk? any other options within unRAID to get low level access to the disk?  I am willing to try anything drastic. VM was not important but a pain to reinstall OS. 

[JorgeB]

15 minutes ago, lennygman said:

I recovered all by 1x30Gb file (VM vdisk)

I assume you can't copy that file from the old disk because you get a I/O error due to the bad sectors, correct?

 

17 minutes ago, lennygman said:

Should i try running reiserfsck on failed disk?

Reiserfsck might be able to help on the formatted disk if it was using the reiser filesystem, all your disks are xfs, unless it was reiser before the format?

 

If the answers are yes and no respectively you have to options, running a file recover utility on the formatted disk, like ufs explorer, to try and recover the image or clone the old disk with ddrescue to skip the bad sectors, but the vdisk will be corrupt, you'd need some luck for the corruption to be in a place that woudln't affect it.

 

 

[lennygman]

Yes .. I/O error.. ok.. you might be right.. going thru this whole effort and then VM crashing all the time because it is missing some files might not be worth it.  Thank you for your support.  

[JorgeB]

The clone method takes some time and you need and extra disk as destination (or you could also clone to an array disk, but it wood need to be in maintenance mode during the clone, i.e., server wouldn't be accessible) but it's not much work, you can find the procedure here:

 

https://lime-technology.com/forums/topic/46802-faq-for-unraid-v6/?do=findComment&comment=525075

 

Still work in progress but the first part, the clone part (except cloning to an array disk), is complete and tested.