June 22, 201610 yr Hi everyone, Unfortunately the worst nightmare has happened to me not long after setting up UnRaid for the first time, i have had 2 of my drives fail at exactly the same time with the same amount of errors, around the 300 mark, in my wisdom i re-booted the system before capturing the system log so i cannot show you that, the issue now is the drives wont even show up when running lsblk from the command line. I have lost about 4TB worth of data and would love to be able to get some of it back, or atleast try and get one drive back working so i can do a re-build, any tips or suggestions on how i can proceed?
June 22, 201610 yr Author Some further information, I have plugged one of the drives into another PC running ubuntu and run the xfs_repair command, that has shown me that the file system has a "bad primary superblock - bad magic number !!!" Im guessing this is what has happened to both drives at the same time, these drives were both plugged into an IBM m1015 so maybe this has caused the file system corruption?
June 22, 201610 yr Community Expert Did you actually do anything to repair the filesystem on the other machine, or did you just check it for filesystem corruption? You really would be better off just trying to get them working again on your unRAID system. If it wasn't able to see the drives then it wasn't due to filesystem corruption but something else such as a connection issue. Messing with the drives on another system is going to make it impossible for unRAID parity to have any chance of helping recover the data. If you are sure the drives haven't been written to by the other machine then I would put them back in unRAID and see if you can get it to see them. Then we can go from there to see what needs to be done.
June 22, 201610 yr Author Did you actually do anything to repair the filesystem on the other machine, or did you just check it for filesystem corruption? You really would be better off just trying to get them working again on your unRAID system. If it wasn't able to see the drives then it wasn't due to filesystem corruption but something else such as a connection issue. Messing with the drives on another system is going to make it impossible for unRAID parity to have any chance of helping recover the data. If you are sure the drives haven't been written to by the other machine then I would put them back in unRAID and see if you can get it to see them. Then we can go from there to see what needs to be done. I did run xfs_repair on another system and that seemed to fix the drive as i could access all the data, but yes later after some more diagnosis it seems it was/is a connection issue in my icybox 5 in 3 bay, not sure what is going on but seems like the backplane is causing issues on 3 of the 5 drive bays.
June 22, 201610 yr Community Expert By repairing on another system, you have invalidated parity on your unRAID. If you had repaired it on unRAID you could have maintained parity. And even though rebuild from parity is not the usual method of recovering from filesystem corruption, it is possible the filesystem corruption was actually caused by the drive dropping in the middle of a write and the completion of the write would have been in the parity array and could have been recovered with a rebuild. If you only have one parity drive then you could only have rebuilt one data disk, but we don't know whether unRAID had actually disabled either or both disks. It would have been better if you had asked for advice before doing anything. So even if you get everything back you will have to rebuild parity now.
June 22, 201610 yr Author By repairing on another system, you have invalidated parity on your unRAID. What if the repair actually did nothing and said that the file system was intact?
June 22, 201610 yr Community Expert By repairing on another system, you have invalidated parity on your unRAID. What if the repair actually did nothing and said that the file system was intact? I assume this is a hypothetical question since you already said it had a bad superblock. In the case of the hypothetical scenario I would at least do a parity check to make sure.
June 22, 201610 yr Community Expert Sorry only one drive had a bad superblock the other was fine If any drive was altered by the other system in any way you have invalidated parity.
June 22, 201610 yr Author Sorry only one drive had a bad superblock the other was fine If any drive was altered by the other system in any way you have invalidated parity. Ok, is there a special why i need to follow to re-create parity or will the system recognize the problem and fix it?
June 22, 201610 yr Community Expert Just to elaborate on how this could have been handled by unRAID without invalidating parity. There are 2 different ways to access a disk when repairing the filesystem, by specifying the sdx1 partition, and by specifying the mdn device. Repairs to the mdn device include parity updates when making changes to the disk, and repairs to the sdx1 partition do not.
June 22, 201610 yr Author Just to elaborate on how this could have been handled by unRAID without invalidating parity. There are 2 different ways to access a disk when repairing the filesystem, by specifying the sdx1 partition, and by specifying the mdn device. Repairs to the mdn device include parity updates when making changes to the disk, and repairs to the sdx1 partition do not. Yep, this has been a good lesson, i suppose i just never would assume that something would randomly go wrong with my icybox backplane that would cause 2 drives to go offline.
June 22, 201610 yr Community Expert Sorry only one drive had a bad superblock the other was fine If any drive was altered by the other system in any way you have invalidated parity. Ok, is there a special why i need to follow to re-create parity or will the system recognize the problem and fix it? unRaid has no idea that you did anything to the drives on another system. If the changes made by the other system were not very extensive, then a correcting parity check will take care of getting parity back in sync. If you find it is making a very large number of parity corrections then the check will probably take longer than a parity rebuild since with a rebuild it doesn't bother reading parity to check it before it writes parity.
June 22, 201610 yr Author ok yep i'm following now, another question, would it be possible to take one drive offline and re-build the data that was on it into the rest of the array? So in a way downsizing the array
June 22, 201610 yr Community Expert ok yep i'm following now, another question, would it be possible to take one drive offline and re-build the data that was on it into the rest of the array? So in a way downsizing the array No. You will have to copy the data from it onto the rest of the array, and then when you remove it you will have to rebuild parity.
June 22, 201610 yr Community Expert If you haven't already, see the unRAID wiki for an explanation of how parity works. It is really not very complicated and if you understand that the answer to many questions become obvious and unRAID procedures make a lot more sense.
June 24, 201610 yr Author Ok, so have managed to get all my disks back in the array but now one of the disks, i'm guessing it was the one i performed a repair on, it now will not let me start the array and instead says "The replacement disk must be as big or bigger than the original." What steps can i follow to get my array back online?
June 24, 201610 yr Community Expert Tools - New Config. Reassign all drives. Be sure you don't assign a data disk to the parity slot.
June 25, 201610 yr Author Tools - New Config. Reassign all drives. Be sure you don't assign a data disk to the parity slot. Ok thanks for all your help Trurl but i ended up putting a new disk in the place of the one that was reporting as wrong and UNraid seemed to rebuild the data on that disk but when the process was complete about 8 hours later it now says that disk is unmountable and i need to format it? What could be the issue here?
June 25, 201610 yr Community Expert Tools - New Config. Reassign all drives. Be sure you don't assign a data disk to the parity slot. Ok thanks for all your help Trurl but i ended up putting a new disk in the place of the one that was reporting as wrong and UNraid seemed to rebuild the data on that disk but when the process was complete about 8 hours later it now says that disk is unmountable and i need to format it? What could be the issue here? you could have file system corruption on the drive as that causes a drive to be unmountable. To fix this you need to put the array into Maintenance mode and do a file system check/repair on the drive. Do not format the drive as doing so will mean you lose all the data on the drive.
Archived
This topic is now archived and is closed to further replies.