Hypothetical


ashman70

Recommended Posts

Lets say I'm replacing a drive in my array with a larger one and its doing the rebuild, during the rebuild another drive fails. Under the current single drive parity model, I've now lost the data on that second drive (this assumes XFX file system). How do I figure out what data was on that second drive so I can replace it? What are the steps?

Link to comment

Well, assuming the initial smaller drive you removed is still intact, (Ie you haven't wiped it or it hasn't failed), could you put that back in its original position and used your new larger drive to replaced the 2nd failed drive?

 

I'm assuming that the act of rebuilding the replaced drive is non-destructive on parity so your should be able to reseat the original smaller drive and your back to a normal failed drive situation.

 

You could also retrieve the data on the second failed drive from your backups  ;)

 

Link to comment

If you'd used rsync or other similar utility to back up your data initially, you could use the same tool in restore mode to figure it out for you.

 

Others with more In-depth knowledge of unraid might be able to help you with figuring out missing files in the dual drive failure scenario - I don't know how you'd go about doing that.

Link to comment

I guess what I am asking, never mind my example then, is if you suffer a dual drive failure, how do you know what data was on the second drive so you can restore it? Is it just a question of manually going through the shares to see what is missing and restoring it?

 

You are basically correct. You can also look at on a disk by disk basis to see what data is on the disks that are still on your array.

Link to comment

There is also the question of whether the second drive has really failed or has it simply had a write fail so that it was disabled!  The vast majority of the time a drive has not physically failed so that the data on it is retrievable.  It is worth posting the diagnostics for your system (Tools->Diagnostics) which will include (amongst other things) the SMART reports for all your drives.

Link to comment

I guess what I am asking, never mind my example then, is if you suffer a dual drive failure, how do you know what data was on the second drive so you can restore it? Is it just a question of manually going through the shares to see what is missing and restoring it?

Also worth noting that in the event you do have a dual drive failure, and both of those are data disks, you have lost both the first and the second drive since all other drives are required to rebuild a single drive.
Link to comment

... Also worth noting that in the event you do have a dual drive failure, and both of those are data disks, you have lost both the first and the second drive since all other drives are required to rebuild a single drive.

 

While this is true, the original question assumes that the first drive has NOT failed, but is simply being replaced by a larger drive.    IF that is the case, then a failure of another drive during the rebuild could be resolved as follows:

 

(a)  Shut down the array.

(b)  Replace the original (smaller) drive.

©  Do a New Config, assigning everything as it originally was, and check the "Parity is Already Valid" box.

(d)  Start the array;  then Stop it and unassign the drive that had failed (Note that it will NOT be marked as failed at this point, since you've just created a new config)

(d)  Replace the failed drive;  assign it to the array in the same slot; and Start the array.

 

The failed drive will now be rebuilt.

 

 

However ... I think the real question is how to identify what files are on specific disks when you've had two failures, and thus neither disk can be emulated.

 

There are several approaches to this ...

 

(1)  If you have a full set of backups (which you should, of course), you can simply do a sync from your backup server to the primary server, and only those files that are missing will be copied [These will, of course, be the files that were on the failed disks].

 

(2)  If you don't maintain backups, but do have a list of your complete file collection [This could be a saved directory of the entire array;  a movie collection database (e.g. DVD Profiler or equivalent);  or some other cataloguing utility], you can simply manually confirm what files are missing.    Some cataloguing utilities have a function to check the paths for all the files that are catalogued => this would let you automate this function.

 

(3)  If you don't have either backups or a catalog, you're simply out of luck.    One thing you can do to avoid this in the future is to periodically save a Directory of each of the disks on your server.    Then you'll at least have a list that's current as of the last time it was updated [Weekly; Monthly; etc.].    If you're depending on this technique, you may want to use the "Fill Up" allocation method, so you only need to save a directory of the currently in-use disk as you fill them up; and once they're full the content is essentially static.

 

By FAR the best approach is to simply maintain a complete set of backups  :)

Link to comment

Thanks Gary for the detailed reply. In my case my monster UnRaid server is a backup server to two NAS's which are primary storage, so I could indeed do a synchronization from the primary source to reconstitute whatever was on the failed drives. I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head.

Link to comment

While my method is by no means the best, it works for me quite well. 99.9% of what is on my unRAID box is media, so I can afford to lose it in the worst case scenario, but I'd rather not. And I can replace most of it. Anything that is personal data like pictures, documents, etc i keep backed up in google docs, or dropbox, etc. I generally fill my disks on the server to the point where they have 100GB or less available before I expand (10 data drives currently so this still lives quite a bit of free space across the array). About once a month or so, I dump the directory contents of each disk to a .txt file and save it to google docs. My logic is if I ever did have a scenario where I lost multiple disks or some form of severe data corruption were to occur, I would have a record of what I lost and be able to replace it. It would be a pain in the butt yes, and time consuming to re-download all the media, but at least I would be able to see what was there. It would be cool if there was a plug in or docker method for something like this actually, that would be awesome. I have never automated it myself, even though I have thought about it, probably wouldn't be too hard to do.

Link to comment

... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head.

 

Not sure what's available in Linux to do this.  What I do is simply use Directory Printer (in Windows)  [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ]  => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. 

 

Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here:  http://www.majorgeeks.com/files/details/karens_directory_printer.html   

 

... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".]

 

Link to comment

One more important point - when people talk about drives failing, they are usually referring to the standard RAID systems, where a drive kicked out is considered failed and all data lost.  In unRAID, there is no striping, drives are independent file systems, and complete drive failure is very rare.  To be completely failed, the drive would have to be unable to spin up or be electrically fried, and maybe something else I can't remember but is even rarer.  That is very uncommon!  Drives are considered failed (red balled) in unRAID for write failures, and 99% to 100% of the data is still accessible!

 

So perhaps we should use a different term for 'failed', although we know what is meant.  The important point is that the data is almost always still accessible, although it may not be easy or quick.

Link to comment

... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head.

 

Not sure what's available in Linux to do this.  What I do is simply use Directory Printer (in Windows)  [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ]  => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. 

 

Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here:  http://www.majorgeeks.com/files/details/karens_directory_printer.html   

 

... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".]

 

How do you print a directory of each disk though? I can print the contents of a network share, but thats not what I want, I want what is on each disk.

Link to comment

... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head.

 

Not sure what's available in Linux to do this.  What I do is simply use Directory Printer (in Windows)  [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ]  => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. 

 

Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here:  http://www.majorgeeks.com/files/details/karens_directory_printer.html   

 

... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".]

 

How do you print a directory of each disk though? I can print the contents of a network share, but thats not what I want, I want what is on each disk.

fixed your quoting

 

You can temporarily turn on Disk sharing in Global Share Settings and give Windows access to each disk that way. Then when you're done, I suggest you turn it off since you don't want to accidentally mix disk and user shares when moving or copying files.

Link to comment

Thanks, my next problem is that I have an AD domain setup at home, for whatever reason, on the particular windows 7 machine I am trying to use either Kathy's directory printer or the other directory printer, neither programs can see my UnRaid server when I go to the network option. Sigh. Leave me with the only option of mapping a network drive to each disk, one at time.

Link to comment

Yes, its not a permission issue its more of a network browsing issue, I can access \\tower just fine on the win 7 virtual machine too, but if I go into Network and wait for it to display the machines on the network it can see, I don't even get a WORKGROUP option, only my domain and of course it only sees domain joined machines.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.