February 10, 201214 yr I'm having a problem similar to this post: http://lime-technology.com/forum/index.php?topic=18261.0 Basically I came home to a physically dead drive in my array and a drive marked "disabled" that seemed fine after running my "diagnostic steps" below. Based on my research, I came up with my "repair steps" and everything seemed to be going great until step 8... After starting my array, the parity disk was marked with an orange ball and was receiving a ton of writes, quickly blowing up my hopes of recovering the actual dead disk. I stopped the array ASAP, knowing that damage was being done right before my eyes! I'm *very* unhappy with the "mdcmd set invalidslot 3" command* at this point, my array is offline, and I really don't know what steps to take to convince unraid that the parity drive is good. (*in my case, 3 was the bad egg) Should I run something other than mdcmd? Is there another way to identify the "bad" disk? ======== Diagnostic steps (unscientific/risky at best, but good enough for me): 1) Stop the array 2) Reconfirm drive serial numbers/slots to identify the "disabled" and dead drives 3) Mount the disabled drive by hand, read-only Using telnet/ssh/console, log in as root Create a temporary mount point: mkdir /mnt/testdrive Identify the sd_ (a,b,c,d...) letter of the disabled drive: /usr/sbin/hdparam -I /dev/sda (change sda to sdb, sdc, etc. until you locate the serial number of the disabled drive) Mount the drive: # mount -t reiserfs -o ro /dev/ad4s5 /mnt 4) "Poke around" on the filesystem. I did this by adding a public samba share to /etc/samba/smb-shares.conf and starting the samba service Using a text editor, added the following lines to /etc/samba/smb-shares.conf (note: the smb-shares.conf file is rebuilt when the server / array are restarted, so any changes are only stored temporarily) [public] path = /mnt/testdrive guest ok = yes read only = yes Start the samba service: /etc/rc.d/rc.samba start Connected using a windows client, browsed the filesystem, watched some old TV episodes... (Of course, the full impact of the failed drives happened right before I had company! The show must go on!) ======== Repair steps: 0) Come to terms with the fact that the physically-dead drive might be unrecoverable. 1) Stop array 2) Pull dead drive 3) Preclear new drive 4) Reconfirm drive serial numbers/slots *again* to avoid losing additional data through user error 5) Run "initconfig" to blow away the previous array configuration 6) Refresh the web interface and assign the drives 7) Run "mdcmd set invalidslot <n>" to set the "bad" drive. Note: Typically, the bad drive in a new array is the parity drive since it hasn't been calculated yet. By setting a different "invalidslot" you're basically saying "trust the parity, this *other* drive is wrong" Note: Based on input from another thread, do not refresh the web interface after running mdcmd 8 ) Cross Fingers, Start array, and let it rebuild...
February 10, 201214 yr I don't have an answer on forcing the md driver to trust your parity. but if you cannot get it fixed, see these threads on riplinux and ddrescue. I was able to recover data off a failed drive. My problem was a bad sector. So if your drive is at least spinning maybe you can access your data. http://lime-technology.com/forum/index.php?topic=16734.msg153098#msg153098 http://lime-technology.com/forum/index.php?topic=17165.15
February 13, 201214 yr Author @WeeboTech: Thanks for the suggestions. Sadly, this drive spins up, hiccups, then tries again... classic failed drive problems that I've experienced in the past. I haven't tried the freezer trick yet, but without being able to trust the parity, I'm pretty much lost. I think that the most unfortunate part of this is how poorly documented/understood the mdcmd command is. Anyone else out there have some insight? Thanks in advance.
February 13, 201214 yr Freezer trick never worked for me. I do remember having the click of death on a WD drive. I walked away ticked off, The machine was still running. It must have gotten warmed up or something, because about an hour later I rebooted and was able to get the data off the drive. So in that case, the drive needed to be hot. Hope you left a note for Tom and he answers you back.
March 3, 201214 yr Author Bump. I sent a request through the support form on the lime technology website and still have heard nothing. In the meantime, I've packed up my entire house, moved, unpacked, and am finally getting back to this project. Any thoughts out there?
March 3, 201214 yr I assume you're using a beta? I hear mixed things about the mdcmd command working on the beta. I've read it works if you run it and then start without refreshing the main web page but I can't confirm. I believe you should not have used the replacement disk directly after preclearing. It needs to have a valid partition without the cleared signature. The easiest and safest way is to make another USB key and assign the replacement drive as the only data drive, start the array and then format it. You could have also assigned it as the only drive and done an initconfig. At this point, you could try dropping to 4.7 and do the same thing. I helped someone recover from the same problem and he got the drive to come back completely successfully. But, you've lost data on the start of the disk by writing to the parity so you'll likely have to rebuild and then run reiserfsck with --scan-whole-partion to try and recover the files. The mdcmd command is supposed to be an internal command set that is used to interface the user interface with the underlying disk system so it's never been documented because you're not supposed to use it. Peter
Archived
This topic is now archived and is closed to further replies.