ashman70 Posted February 28, 2016 Share Posted February 28, 2016 Lets say I'm replacing a drive in my array with a larger one and its doing the rebuild, during the rebuild another drive fails. Under the current single drive parity model, I've now lost the data on that second drive (this assumes XFX file system). How do I figure out what data was on that second drive so I can replace it? What are the steps? Quote Link to comment
meep Posted February 28, 2016 Share Posted February 28, 2016 Well, assuming the initial smaller drive you removed is still intact, (Ie you haven't wiped it or it hasn't failed), could you put that back in its original position and used your new larger drive to replaced the 2nd failed drive? I'm assuming that the act of rebuilding the replaced drive is non-destructive on parity so your should be able to reseat the original smaller drive and your back to a normal failed drive situation. You could also retrieve the data on the second failed drive from your backups Quote Link to comment
ashman70 Posted February 28, 2016 Author Share Posted February 28, 2016 I guess what I am asking, never mind my example then, is if you suffer a dual drive failure, how do you know what data was on the second drive so you can restore it? Is it just a question of manually going through the shares to see what is missing and restoring it? Quote Link to comment
meep Posted February 28, 2016 Share Posted February 28, 2016 If you'd used rsync or other similar utility to back up your data initially, you could use the same tool in restore mode to figure it out for you. Others with more In-depth knowledge of unraid might be able to help you with figuring out missing files in the dual drive failure scenario - I don't know how you'd go about doing that. Quote Link to comment
gundamguy Posted February 28, 2016 Share Posted February 28, 2016 I guess what I am asking, never mind my example then, is if you suffer a dual drive failure, how do you know what data was on the second drive so you can restore it? Is it just a question of manually going through the shares to see what is missing and restoring it? You are basically correct. You can also look at on a disk by disk basis to see what data is on the disks that are still on your array. Quote Link to comment
itimpi Posted February 29, 2016 Share Posted February 29, 2016 There is also the question of whether the second drive has really failed or has it simply had a write fail so that it was disabled! The vast majority of the time a drive has not physically failed so that the data on it is retrievable. It is worth posting the diagnostics for your system (Tools->Diagnostics) which will include (amongst other things) the SMART reports for all your drives. Quote Link to comment
trurl Posted February 29, 2016 Share Posted February 29, 2016 I guess what I am asking, never mind my example then, is if you suffer a dual drive failure, how do you know what data was on the second drive so you can restore it? Is it just a question of manually going through the shares to see what is missing and restoring it? Also worth noting that in the event you do have a dual drive failure, and both of those are data disks, you have lost both the first and the second drive since all other drives are required to rebuild a single drive. Quote Link to comment
garycase Posted February 29, 2016 Share Posted February 29, 2016 ... Also worth noting that in the event you do have a dual drive failure, and both of those are data disks, you have lost both the first and the second drive since all other drives are required to rebuild a single drive. While this is true, the original question assumes that the first drive has NOT failed, but is simply being replaced by a larger drive. IF that is the case, then a failure of another drive during the rebuild could be resolved as follows: (a) Shut down the array. (b) Replace the original (smaller) drive. © Do a New Config, assigning everything as it originally was, and check the "Parity is Already Valid" box. (d) Start the array; then Stop it and unassign the drive that had failed (Note that it will NOT be marked as failed at this point, since you've just created a new config) (d) Replace the failed drive; assign it to the array in the same slot; and Start the array. The failed drive will now be rebuilt. However ... I think the real question is how to identify what files are on specific disks when you've had two failures, and thus neither disk can be emulated. There are several approaches to this ... (1) If you have a full set of backups (which you should, of course), you can simply do a sync from your backup server to the primary server, and only those files that are missing will be copied [These will, of course, be the files that were on the failed disks]. (2) If you don't maintain backups, but do have a list of your complete file collection [This could be a saved directory of the entire array; a movie collection database (e.g. DVD Profiler or equivalent); or some other cataloguing utility], you can simply manually confirm what files are missing. Some cataloguing utilities have a function to check the paths for all the files that are catalogued => this would let you automate this function. (3) If you don't have either backups or a catalog, you're simply out of luck. One thing you can do to avoid this in the future is to periodically save a Directory of each of the disks on your server. Then you'll at least have a list that's current as of the last time it was updated [Weekly; Monthly; etc.]. If you're depending on this technique, you may want to use the "Fill Up" allocation method, so you only need to save a directory of the currently in-use disk as you fill them up; and once they're full the content is essentially static. By FAR the best approach is to simply maintain a complete set of backups Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Thanks Gary for the detailed reply. In my case my monster UnRaid server is a backup server to two NAS's which are primary storage, so I could indeed do a synchronization from the primary source to reconstitute whatever was on the failed drives. I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head. Quote Link to comment
swells Posted February 29, 2016 Share Posted February 29, 2016 While my method is by no means the best, it works for me quite well. 99.9% of what is on my unRAID box is media, so I can afford to lose it in the worst case scenario, but I'd rather not. And I can replace most of it. Anything that is personal data like pictures, documents, etc i keep backed up in google docs, or dropbox, etc. I generally fill my disks on the server to the point where they have 100GB or less available before I expand (10 data drives currently so this still lives quite a bit of free space across the array). About once a month or so, I dump the directory contents of each disk to a .txt file and save it to google docs. My logic is if I ever did have a scenario where I lost multiple disks or some form of severe data corruption were to occur, I would have a record of what I lost and be able to replace it. It would be a pain in the butt yes, and time consuming to re-download all the media, but at least I would be able to see what was there. It would be cool if there was a plug in or docker method for something like this actually, that would be awesome. I have never automated it myself, even though I have thought about it, probably wouldn't be too hard to do. Quote Link to comment
garycase Posted February 29, 2016 Share Posted February 29, 2016 ... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head. Not sure what's available in Linux to do this. What I do is simply use Directory Printer (in Windows) [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ] => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here: http://www.majorgeeks.com/files/details/karens_directory_printer.html ... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".] Quote Link to comment
JorgeB Posted February 29, 2016 Share Posted February 29, 2016 Two more options Dynamix file integrity plugin – exports a list of all files by disk to your flash drive, as a bonus it also detects corrupted files. Cathy - catalog software, never used it but it was mentioned before on the forum. Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Thanks guys, just grabbed Karens directory printer, looks like exactly what I am looking for, cheers. AM Quote Link to comment
RobJ Posted February 29, 2016 Share Posted February 29, 2016 One more important point - when people talk about drives failing, they are usually referring to the standard RAID systems, where a drive kicked out is considered failed and all data lost. In unRAID, there is no striping, drives are independent file systems, and complete drive failure is very rare. To be completely failed, the drive would have to be unable to spin up or be electrically fried, and maybe something else I can't remember but is even rarer. That is very uncommon! Drives are considered failed (red balled) in unRAID for write failures, and 99% to 100% of the data is still accessible! So perhaps we should use a different term for 'failed', although we know what is meant. The important point is that the data is almost always still accessible, although it may not be easy or quick. Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Very true Rob, but bad sectors on a drive can make a drive unusable, and it can lead to corruption as well, so yes, perhaps the term 'failed' is not appropriate in this example. Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 ... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head. Not sure what's available in Linux to do this. What I do is simply use Directory Printer (in Windows) [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ] => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here: http://www.majorgeeks.com/files/details/karens_directory_printer.html ... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".] How do you print a directory of each disk though? I can print the contents of a network share, but thats not what I want, I want what is on each disk. Quote Link to comment
trurl Posted February 29, 2016 Share Posted February 29, 2016 ... I like the idea of keeping a record of the directories on the drives though, perhaps someone could come up with a docker for this, not sure what it would do but perhaps index the drives and output them in text or something, just a thought off the top of my head. Not sure what's available in Linux to do this. What I do is simply use Directory Printer (in Windows) [ http://www.galcott.com/dp.htm ] to "Print" a directory of each disk, with the "printer" being CutePDF [ http://www.cutepdf.com/Products/CutePDF/writer.asp ] => so I have a PDF file for each disk in the array showing the contents the last time I "printed" it. Glenn Alcott's Directory Printer isn't free, but there's a free equivalent available here: http://www.majorgeeks.com/files/details/karens_directory_printer.html ... and CutePDF is free and a VERY useful thing to have installed on your Windows systems [if you can print something, you can make it a PDF file instead by simply selecting "CutePDF Writer" as the "printer".] How do you print a directory of each disk though? I can print the contents of a network share, but thats not what I want, I want what is on each disk. fixed your quoting You can temporarily turn on Disk sharing in Global Share Settings and give Windows access to each disk that way. Then when you're done, I suggest you turn it off since you don't want to accidentally mix disk and user shares when moving or copying files. Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Thanks, my next problem is that I have an AD domain setup at home, for whatever reason, on the particular windows 7 machine I am trying to use either Kathy's directory printer or the other directory printer, neither programs can see my UnRaid server when I go to the network option. Sigh. Leave me with the only option of mapping a network drive to each disk, one at time. Quote Link to comment
garycase Posted February 29, 2016 Share Posted February 29, 2016 Not sure what the issue might be, but clearly it's a permissions issue. I have no problem accessing \\Tower [actually a different name on my server] ... and all of the disk shares. You DO, I presume, have disk shares enabled in the UnRAID settings -- right? Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Yes, its not a permission issue its more of a network browsing issue, I can access \\tower just fine on the win 7 virtual machine too, but if I go into Network and wait for it to display the machines on the network it can see, I don't even get a WORKGROUP option, only my domain and of course it only sees domain joined machines. Quote Link to comment
garycase Posted February 29, 2016 Share Posted February 29, 2016 Ahh ... I understand. This may help: https://lime-technology.com/forum/index.php?topic=41647.0 Quote Link to comment
ashman70 Posted February 29, 2016 Author Share Posted February 29, 2016 Thanks, but I'm not really interested in joining my UnRaid server to my domain, I only have it for work purposes, I'll just map a drive to each individual drive and print its contents, thanks though for the link. Quote Link to comment
John_M Posted February 29, 2016 Share Posted February 29, 2016 How about using the ls command with the -R option to list the root of each disk recursively, like this: ls -lR /mnt/disk1 ls -lR /mnt/disk2 ... as a quick and dirty way of getting a full contents listing of each disk? The output could be redirected to a text file, as required. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.