SlowNIC Posted January 13, 2018 Share Posted January 13, 2018 (edited) I've searched the forums for about an hour now and I give up, sorry for bothering everyone. I had a handle on it in V5. Can't find "ddrescue" tool anywhere, closest I came was Nerd tools plugin. If I don't get any responses, I'll pull the drives and plug them into another computer that runs linux. Thanks in advance, and sorry for my poor searching skills (this must have been asked before). Edited January 14, 2018 by SlowNIC Solved Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 You can request for it in the Nerd Tools thread, don't know if it can be easily added but it would be a good addition. Quote Link to comment
SlowNIC Posted January 13, 2018 Author Share Posted January 13, 2018 (edited) Thank you for the suggestion. Follow up question to the group: Is this the method that people use to recover data from a failed disk from a broken array? The drive is readable, but the rebuild parity operation on the array was taking 75 days and making thousands of recoverable errors. Prior experience tells me that rebuild would fail prior to the 75 day window completing. Prior experience also tells me that ddrescue would recover most of my data quickly and efficiently. Dynamix File Integrity plugin that I run on all my XFS drives would then allow me to identify any corrupted files. How are people recovering from data loss in version 6 without ddrescue? Or am I asking the question wrong? Sorry my mind is stuck back in version 5 when this was easy to my old way of thinking. Edited January 13, 2018 by SlowNIC added type of rebuild Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 Is that 75 days rebuilding to a new disk? If yes you have other issues, if you're trying to rebuild to a known bad disk don't. Quote Link to comment
SlowNIC Posted January 13, 2018 Author Share Posted January 13, 2018 Parity rebuild (reading the disk I am trying to recover) is taking 75 days and produces thousands of recoverable errors on the disk. I have 6 arrays of 24 disks each, so this does happen to me from time to time. This is the first time with version 6 and I find myself without one of my most basic tools that I have relied on for years. I feel like one arm is tied behind my back. So, I figure I must be doing something different from everyone else to have this problem. I'm not the first one that needs to recover a disk, right? So, my version 6 solution now seems to be to remove the drives from the array and find a linux system that has ddrescue. It is not as nice as my version 5 solution, which was to type ddrescue at the command prompt, but it still can work. What do other people do? Quote Link to comment
bonienl Posted January 13, 2018 Share Posted January 13, 2018 Do you have notifications enabled? And set up email and/or mobile agents to receive them? A rebuild of 75 days means there is something seriously wrong with the disk(s). Notifications are used to warn you in an early stage and prevent lingering problems. unRAID6 has disk recovery tools built-in the GUI. Start the array in maintenance mode and from the Main page click on the disk which needs to be verified/fixed. Turn on HELP to get more info on the possibilities. Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 1 hour ago, SlowNIC said: Parity rebuild (reading the disk I am trying to recover) is taking 75 days and produces thousands of recoverable errors on the disk. I'm still not clear on what you are doing and what disk is having errors, whatever it is there's something very wrong, you could post your diagnostics. 1 hour ago, SlowNIC said: What do other people do? They replace the failing disk with a new one and let it rebuild, even for large disks it should take less than a day, e.g., my server with 8TB disks takes around 15 hours to do a rebuild. Quote Link to comment
SlowNIC Posted January 13, 2018 Author Share Posted January 13, 2018 I do appreciate the responses. The array is broken. What do I mean by broken? The parity drive is not valid. How did it become invalid? The question is not relevant for this thread. I may start a new thread to address that. So I have a known data drive with issues and an invalid parity drive. I've already done the hardware troubleshooting and confirmed that 3 different ways. The 4th way to confirm it is the smart report that I attached, below... So my concise question is, with a broken array, and a data drive with errors, what do people do to recover data? Up to this point, I thought ddrescue was the only option, yet ddrescue is missing from unraid V6, something that I view as a basic tool for maintaining storage subsystems when other measures fail. I must be missing something, because I know that people do have the combination of "broken array" plus "drive with errors" happen. Nobody is denying that this happens, right? So, do people fire up some virtual machine solution that runs ddrescue? Do they boot another flash drive that runs ddrescue? Is there a different utility that efficiently reads and handles errors like ddrescue does? Some other solution that I am missing? WDC_WD20EARS-00MVWB0_WD-WCAZA5082257-20180113-1433.txt Quote Link to comment
bonienl Posted January 13, 2018 Share Posted January 13, 2018 It seems your array is in a highly degraded state. I don't know what would be the best approach at this moment. The key lesson for the future is to be ahead and resolve issues before they turn major. The built-in notifications and repair tools of unRAID6 should cover most (all?) cases and when a disk is beyond repair it needs to be replaced. The earlier this is recognized the greater the chance of success. Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 If no backups are available most just copy what they can mounting the disk normally, any file that won't copy because of a read error would be corrupt if copied with ddrescue, others use dd with conv=noerror,sync, still ddrescue is optimized for reading a bad disk and can be a valid solution for some cases, hence why I said it would be a good addition to the nerd tools. Quote Link to comment
bonienl Posted January 13, 2018 Share Posted January 13, 2018 6 minutes ago, johnnie.black said: and can be a valid solution for some cases, hence why I said it would be a good addition to the nerd tools. But better to act earlier and avoid to come in a situation like this, IMHO. Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 1 minute ago, bonienl said: But better to act earlier and avoid to come in a situation like this, IMHO. Definitely, ddrescue is a last resort attempt only, since some/many files will be corrupt when it skips the read errors. Quote Link to comment
SlowNIC Posted January 13, 2018 Author Share Posted January 13, 2018 Thanks again for the replies. I was aware of the dd with the options. It is a good suggestion, but it is extremely time consuming as it does not track which blocks it was unable to read. You have to keep track manually with a script and retry them. That was why they wrote ddrescue. I've had this happen before, and using ddrescue, I've been able to recover 99% of my data from a drive in this condition. Perhaps I lose 3 files. The file integrity plug in tells me exactly which ones were affected. I've been doing this for 12 years, but feel free to remind me about lessons anytime if you think next time is the time it will "sink in" :-) Quote Link to comment
bonienl Posted January 13, 2018 Share Posted January 13, 2018 1 minute ago, SlowNIC said: I've been doing this for 12 years The most difficult things to change are "habits". Sounds familiar to me Quote Link to comment
CHBMB Posted January 13, 2018 Share Posted January 13, 2018 (edited) If you do need ddrescue, although I would encourage you to take the advice of @johnnie.black and @bonienl, then you could install it.... wget http://slackware.cs.utah.edu/pub/slackware/slackware64-14.2/slackware64/ap/ddrescue-1.21-x86_64-1.txz installpkg ddrescue-1.21-x86_64-1.txz This installs the version compiled on Slackware 14.2, although Unraid is based on Slackware Current, but I would imagine it will be ok. Edited January 13, 2018 by CHBMB 1 Quote Link to comment
SlowNIC Posted January 13, 2018 Author Share Posted January 13, 2018 Thank you so much! Copy is underway so far with a 26 hour ETA. I will make a request in Nerd plugin thread. Woo Hoo! Quote Link to comment
JorgeB Posted January 13, 2018 Share Posted January 13, 2018 49 minutes ago, SlowNIC said: Copy is underway so far with a 26 hour ETA. If you don't mind sharing post how many files were corrupt when you finish checking them, just out of curiosity, since that disk looks to be in really bad shape. Good luck! Quote Link to comment
SlowNIC Posted January 14, 2018 Author Share Posted January 14, 2018 (edited) ddrescue has been added to "Nerd Tools". Thank you to all who made this happen. Edited January 14, 2018 by SlowNIC Quote Link to comment
SlowNIC Posted January 15, 2018 Author Share Posted January 15, 2018 First pass completed, now I will identify the files that were affected... root@Tower1:~# ddrescue --force /dev/disk/by-id/ata-WDC_WD20EARS-00MVWB0_WD-WCAZA5082257-part1 /dev/disk/by-id/ata-ST33000651NS_Z292V792-part1 /boot/WCAZA5082257.ddrescue GNU ddrescue 1.21 Press Ctrl-C to interrupt ipos: 143929 MB, non-trimmed: 0 B, current rate: 14848 B/s opos: 143929 MB, non-scraped: 0 B, average rate: 11819 kB/s non-tried: 0 B, errsize: 6385 kB, run time: 1d 23h 39s rescued: 2000 GB, errors: 1016, remaining time: 0s percent rescued: 99.99% time since last successful read: 0s Finished root@Tower1:~# WCAZA5082257.ddrescue Quote Link to comment
SlowNIC Posted January 16, 2018 Author Share Posted January 16, 2018 It appears that only 13 files were affected out of 705. There is no way I could recover this data without a ddrescue or a similar tool. Given the amount of time that unraid V6 has been released, I don't understand why I am the first person to run into this, but I am grateful to the community for their generous support in getting it resolved. root@Tower1:~# echo -n 'UnRaid 6 with ddrescue ' >~/fill.txt root@Tower1:~# ddrescue --force --fill=- ~/fill.txt /dev/disk/by-id/ata-ST33000651NS_Z292V792-part1 /boot/WCAZA5082257.ddrescue GNU ddrescue 1.21 Press Ctrl-C to interrupt Initial status (read from mapfile) filled size: 0 B, filled areas: 0 remaining size: 6385 kB, remaining areas: 1016 Current status filled size: 6385 kB, filled areas: 1016, current rate: 0 B/s remain size: 0 B, remain areas: 0, average rate: 0 B/s current pos: 143929 MB, run time: 0s Finished root@Tower1:~# xfs_repair /dev/disk/by-id/ata-ST33000651NS_Z292V792-part1 xfs_repair /dev/disk/by-id/ata-ST33000651NS_Z292V792-part1 Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 2 - agno = 3 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... done root@Tower1:~# mkdir /mnt/test root@Tower1:~# mount /dev/disk/by-id/ata-ST33000651NS_Z292V792-part1 /mnt/test -o ro root@Tower1:~# find /mnt/test -type f -exec grep --binary --quiet 'UnRaid 6 with ddrescue ' {} \; -print /mnt/test/media/myth/1378_20150519113400.mpg /mnt/test/media/myth/10997_20140807035900.mpg /mnt/test/media/myth/11039_20130619035900.mpg /mnt/test/media/myth/11012_20140528125900.mpg /mnt/test/media/myth/11095_20140616040300.mpg /mnt/test/media/myth/1386_20140401144400.mpg /mnt/test/media/myth/10992_20151026052900.mpg /mnt/test/media/myth/3441_20160603005900.mpg /mnt/test/media/myth/11028_20170629053200.mpg /mnt/test/media/myth/3041_20141124022900.mpg /mnt/test/media/myth/11040_20131012065900.mpg /mnt/test/media/myth/11079_20151114050100.mpg /mnt/test/media/myth/1195_20170319160000.mpg 1378_20150519113400.mpg INDIEHD*378*dish2 May 19 09:34 'Water' 10997_20140807035900.mpg SHOWHD*9460*dish6 Aug 07 01:29 'Blood Out' 11039_20130619035900.mpg TLCHD*9488*dish4 Jun 19 01:01 'My Teen Is Pregnant and So Am I @ For the Love of Family' 11012_20140528125900.mpg TNTHD*9420*dish5 May 28 10:01 'Supernatural @ Plucky Pennywhistle's Magical Menagerie' 11095_20140616040300.mpg CCHD*9485*dish7 Jun 16 00:36 'South Park @ Cartman Finds Love' 1386_20140401144400.mpg SONYHD*9529*dish1 Apr 01 12:39 'The Body' 10992_20151026052900.mpg WCVBDT*6318*dish5 Oct 26 02:31 'Elementary @ M.' 3441_20160603005900.mpg WGBXDT*43*UHF Jun 02 22:01 'Miss Fisher's Murder Mysteries @ Blood and Money' 11028_20170629053200.mpg AETVHD*9419*dish2 Jun 29 02:05 'Storage Wars: Northern Treasures @ The Seven Habits of Highly Effective Instigators' 3041_20141124022900.mpg WBZDT*30*UHF Nov 23 22:31 'The Good Wife @ The Trial' 11040_20131012065900.mpg APLHD*184*dish2 Oct 12 04:01 'Monsters Inside Me @ It Came From a Tick ...' 11079_20151114050100.mpg LIFEHD*9470*dish7 Nov 14 01:03 'Step It Up @ Fan Chat: Beware of Bat Woman' 1195_20170319160000.mpg AHC*195*dish6 Mar 19 13:01 'Dawn of the Apocalypse @ Dark Age of Egypt' Quote Link to comment
JorgeB Posted January 16, 2018 Share Posted January 16, 2018 15 hours ago, SlowNIC said: It appears that only 13 files were affected out of 705. Not bad, I was expecting more after seeing that SMART report, good result. Quote Link to comment
Frank1940 Posted January 16, 2018 Share Posted January 16, 2018 Any chance of someone writing a reasonably user-friendly guide to using ddrescue with unRAID? I realize this might be difficult as ddrescue appears to have a rather complex set of options but it might provide a good start for folks with some reasonable OS system background and command line familiarity. I would envision something with the required steps with typical command line examples and an explanation of where the augments came from and what the switches are telling the program to do. Quote Link to comment
joshbgosh10592 Posted January 21, 2021 Share Posted January 21, 2021 On 1/16/2018 at 5:04 PM, Frank1940 said: Any chance of someone writing a reasonably user-friendly guide to using ddrescue with unRAID? I realize this might be difficult as ddrescue appears to have a rather complex set of options but it might provide a good start for folks with some reasonable OS system background and command line familiarity. I would envision something with the required steps with typical command line examples and an explanation of where the augments came from and what the switches are telling the program to do. I know this is a very old post, but I'm just now having an issue where I need to use ddrestore myself. While looking around, I found THIS PAGE that seems to be helping me. Hopefully this will help future Googlers. Quote Link to comment
JorgeB Posted January 21, 2021 Share Posted January 21, 2021 7 hours ago, joshbgosh10592 said: I found THIS PAGE There's also a FAQ entry about that. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.