m1a8x2 Posted December 28, 2020 Share Posted December 28, 2020 Hi everyone, I'm having some issues with my array. My dog knocked over my server while I was working on it. When I started the server back up, it said disk 4 was not installed. I checked all cables, etc. and it still wasn't showing up. I removed the drive and tried seeing if another computer would recognize it and it made some terrible sounds. When I started the server back up with this drive removed, I now see that disk 5 says "Unmountable: No file system". Disk 4 now also says this as well. I've ordered two 8tb drives, so I'll need to do a parity swap as well I believe? Can anyone help me figure out how to fix this file system error and what order I should be doing everything? Thanks in advance! krieger-diagnostics-20201228-1233.zip Quote Link to comment
JorgeB Posted December 28, 2020 Share Posted December 28, 2020 22 minutes ago, m1a8x2 said: I removed the drive and tried seeing if another computer would recognize it and it made some terrible sounds. Then you have a problem, because disk5 is also failing, single parity can only tolerate one failure. Quote Link to comment
m1a8x2 Posted December 28, 2020 Author Share Posted December 28, 2020 I was still able to access everything a couple days ago and disk5 seemed fine. Does it look like it's gone too? At that point there's no recourse? Quote Link to comment
JorgeB Posted December 28, 2020 Share Posted December 28, 2020 For disk5 you can try ddrescue, depending on how bad the damage is it might recover a lot or most of the data, you could also then try rebuilding disk4 with the clone in place, but there always be some corruption. Quote Link to comment
m1a8x2 Posted December 28, 2020 Author Share Posted December 28, 2020 3 minutes ago, JorgeB said: For disk5 you can try ddrescue, depending on how bad the damage is it might recover a lot or most of the data Would I need to do a parity swap first? My parity is 5tb and my data drives are all 4tb. I have two new 8tb drives arriving tomorrow. Quote Link to comment
JorgeB Posted December 28, 2020 Share Posted December 28, 2020 To use ddrescue and then re-use the clone in the array you'd need to use a disk of the same size, or Unraid won't accept it. Quote Link to comment
m1a8x2 Posted December 28, 2020 Author Share Posted December 28, 2020 Well crap. So I should order another 4tb and use ddrescue... then parity swap and then swap disk4 and rebuild? Quote Link to comment
JorgeB Posted December 28, 2020 Share Posted December 28, 2020 Yes, with the clone you can then do a new config, you still need a temp disk4, but you can use the old disk5, then disable it and do the parity swap, when the clone is ready ask and I can post detailed instructions, but like mentioned rebuild will be as corrupt as there are errors in the clone operation. Quote Link to comment
m1a8x2 Posted December 28, 2020 Author Share Posted December 28, 2020 Okay, thank you. I'm going to go buy a 4tb locally and run ddrescue. I'll report back when that's done. Quote Link to comment
m1a8x2 Posted December 28, 2020 Author Share Posted December 28, 2020 3 hours ago, JorgeB said: Yes, with the clone you can then do a new config, you still need a temp disk4, but you can use the old disk5, then disable it and do the parity swap, when the clone is ready ask and I can post detailed instructions, but like mentioned rebuild will be as corrupt as there are errors in the clone operation. I have the new drive in the server and am ready to use ddrescue... I'm not entirely sure I understand the command yet so I haven't started it until I read the instructions again. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 @JorgeB I've had ddrescue running for 11 hours now. pct rescued: 36.09%, read errors:1150916909, remaining time: n/a time since last successful read: 5h 40s Quote Link to comment
JorgeB Posted December 29, 2020 Share Posted December 29, 2020 Bad news, that's a lot of errors, and it will also affect the other disk rebuild. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 4 minutes ago, JorgeB said: Bad news, that's a lot of errors, and it will also affect the other disk rebuild. Is it possible to know what data is still intact and copy it manually to the new drive? Would I be unable to rebuild disk4 if I did that? Quote Link to comment
JorgeB Posted December 29, 2020 Share Posted December 29, 2020 8 minutes ago, m1a8x2 said: Is it possible to know what data is still intact and copy it manually to the new drive? After the clone you can do that, assuming the disk mounts. 9 minutes ago, m1a8x2 said: Would I be unable to rebuild disk4 if I did that? Unrelated, problem with rebuilding disk4 is all the errors on that disk will result in the same corrupt sectors on the rebuilt disk, so more errors, more corruption. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 18 minutes ago, JorgeB said: Unrelated, problem with rebuilding disk4 is all the errors on that disk will result in the same corrupt sectors on the rebuilt disk, so more errors, more corruption. Disk4 is the one that's completely dead, so wouldn't it be able to be emulated/rebuilt from parity? I don't know what I'm talking about, I'm sure... but just want to make sure I'm on the same page. Quote Link to comment
JorgeB Posted December 29, 2020 Share Posted December 29, 2020 25 minutes ago, m1a8x2 said: Disk4 is the one that's completely dead, so wouldn't it be able to be emulated/rebuilt from parity? Parity plus all the other disks. Quote Link to comment
JorgeB Posted December 29, 2020 Share Posted December 29, 2020 Just to expand, parity disk itself don't have any data, it needs all the other data disks to be read successfully for a rebuild. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 So what's my best approach here? What's my endgame - will I be able to restore my array in any way or am I completely screwed here? Quote Link to comment
JorgeB Posted December 29, 2020 Share Posted December 29, 2020 Best chance to recover any data from disks 4 and 5 is to let ddrescue finish and then rebuild, though there will always be some data loss on both, possibly a lot due to all those errors, if you just want to bring the array online and forget about those disks you can do a new config. Also in the future consider adding a second parity, small price to pay for the added redundancy, even for small arrays, but still not a substitute for backups. Quote Link to comment
trurl Posted December 29, 2020 Share Posted December 29, 2020 Do you have Notifications setup to alert you immediately by email or other agent as soon as a problem is detected? If you let things go until you have multiple problems then it can be difficult or impossible to fully recover. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 1 hour ago, trurl said: Do you have Notifications setup to alert you immediately by email or other agent as soon as a problem is detected? If you let things go until you have multiple problems then it can be difficult or impossible to fully recover. No I don't have any email notifications unfortunately. I need to get those setup once I'm out of the woods here. 2 hours ago, JorgeB said: Best chance to recover any data from disks 4 and 5 is to let ddrescue finish and then rebuild, though there will always be some data loss on both, possibly a lot due to all those errors I can live with some data loss I suppose. I'm okay with losing media files, but it's pictures and files from school/work I don't want to lose. So far, when I connect to my shares on another PC it appears most all of my important files are safe. 2 hours ago, JorgeB said: Also in the future consider adding a second parity, small price to pay for the added redundancy, even for small arrays, but still not a substitute for backups. I'm assuming this parity would need to be the same size as the other parity drive? Right now I have a 5tb parity, and five 4tb data drives. I just added a 4tb drive for this recovery and I have two new 8tb drives to swap into the array/parity. Quote Link to comment
itimpi Posted December 29, 2020 Share Posted December 29, 2020 Parity drives do not need to be the same size - but they must be at least as large as the largest data drive. Quote Link to comment
JonathanM Posted December 29, 2020 Share Posted December 29, 2020 1 hour ago, m1a8x2 said: So far, when I connect to my shares on another PC it appears most all of my important files are safe. Since you have access, NOW is the time to back them up to another location, before you start changing things around and possibly getting in a worse situation. Anything important should always be backed up elsewhere, preferably 2 other places if it's truly irreplaceable and valuable. Unraid or any RAID is not backup, it's high availability so you can still access your files when a drive goes down, and rebuild to a new drive. Quote Link to comment
m1a8x2 Posted December 29, 2020 Author Share Posted December 29, 2020 15 minutes ago, jonathanm said: Since you have access, NOW is the time to back them up to another location, before you start changing things around and possibly getting in a worse situation. Good call. I'm going to dump as much as I can onto another external drive. Quote Link to comment
m1a8x2 Posted December 30, 2020 Author Share Posted December 30, 2020 ddrescue has finished. rescued: 1444 GB, tried: 2556 GB, bad-sector: 1175 GB, bad areas: 1765 Current status ipos: 4000 GB, non-trimmed: 0 B, current rate: 0 B/s opos: 4000 GB, non-scraped: 0 B, average rate: 0 B/s non-tried: 0 B, bad-sector: 2556 GB, error rate: 27596 kB/s rescued: 1444 GB, bad areas: 1764, run time: 11h 43m 30s pct rescued: 36.09%, read errors:2697716168, remaining time: n/a time since last successful read: n/a Finished What's next? I now have SMART warnings for disk 2 about current pending sectors. Is that drive failing now too or is that read errors from running ddrescue or something else? Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.