April 7, 201412 yr A drive larger than 2T is required to image a 2T drive. You should be able to copy the disk, however.
April 7, 201412 yr Author A drive larger than 2T is required to image a 2T drive. You should be able to copy the disk, however. Thanks for the heads up! I wasn't sure if that was the case, but I'm glad you mentioned it so I know what size drive to get. Are there any advantages to making an image as opposed to just copying the disk? I know you mentioned running recovery on the image using SystemRescueCD, so I'm thinking you might not be able to do that if just copying the disk to another disk. Thank you again so much for the advice. I'll be writing all of this down so if it happens again that I won't have to be such a pain!
April 7, 201412 yr An 'image' is an exact replica of a disk, warts and all. So if you make an image on a backup disk, then try to fix the file system on another, if the fix fails, the 'image' is like the perfect backup of the broken file system. You can try again to fix things, starting again at the beginning. Depending upon what you actually do, A 'copy' sounds like the OS is reading the files from the bad disk and copying them to a good. This is done file by file. The OS doesn't realize that the filesystem may be corrupt...it just does its best based upon the file directory information that it has available. Its likely to be successful for those files which aren't broken. Its likely that 'broken' files will either be ignored, or worse, copied onto the new drive, but without any warning that they aren't accurate or complete.
April 7, 201412 yr A block level copy should be identical to an image. SystemRescueCD should be able to make either.
April 14, 201412 yr Author A block level copy should be identical to an image. SystemRescueCD should be able to make either. Sorry about the delay in responding! I had to purchase another 3TB drive as the one I got from Best Buy to write my backup image to was DOA. I currently have the disk UnRAID disk being backed up through SystemRescueCD and should be done in a few hours. In the meantime, I was going to run the "reiserfsck --rebuild-tree --scan-whole-partition /dev/md4" command as suggested, but when I try to do that I just get an error that says "Failed to open the device '/dev/md4': No such file or directory". Is this because I do not have the array started? Assuming yes, but again, I'd like to tip toe on ice before assuming anything at all. I'm guessing I should start the array in Maintenance mode so it does not mount any disks, and then try again. If this is the case, then maybe I'll have more faith in myself henceforward Looking forward to hearing back, and THANK YOU ALL again for the help in navigating me through this maze! EDIT: Also, I had a question for dgaschk (or anyone else who may know!) regarding running recovery on my backup image I just created. Looking through the SystemRestoreCD documentation, I see there is a tool called "TestDisk". Is this the tool to run when you suggested running recovery on the image? I figure it must be, but again I just want to make sure before doing anything. Thank you again!!
April 15, 201412 yr The array must be started in order for /dev/md4 to exist. testdisk or part image can be used to image the disk. testdisk can also fix the file system and recover files on the image.
April 15, 201412 yr Author The array must be started in order for /dev/md4 to exist. testdisk or part image can be used to image the disk. testdisk can also fix the file system and recover files on the image. Great, thank you! I'm running recovery on the image currently, and my reiserfsck just finished this morning on /dev/md4. I'm not sure if there is a way to select any and all text that has scrolled outside of my telnet session window in PuTTY, but I copied all I could so I could post it here in a .txt file. I also pasted some of the report into a code block seen below. Most of what is in the report is what's seen on the first line below. vpf-10260: The file we are inserting the new item (695 691 0x780f001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) into has no StatData, insertion was skipped vpf-10260: The file we are inserting the new item (12 13 0x94b40001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) into has no StatData, insertion was skipped 100% left 0, 0 /sec Flushing..finished Leaves inserted item by item 222 Pass 3 (semantic): ####### Pass 3 ######### Flushing..finished Files found: 0 Directories found: 2 Pass 3a (looking for lost dir/files): ####### Pass 3a (lost+found pass) ######### Looking for lost directories: Flushing..finishede 1, 1 /sec Pass 4 - finished done 0, 0 /sec Deleted unreachable items 2122 Flushing..finished Syncing..finished ########### reiserfsck finished at Tue Apr 15 08:41:03 2014 ########### root@Tower:~# I am not sure if I can access the report the reiserfsck that is created as I think it usually writes it to the disk itself, but if anyone knows a possibly way to grab that I'd happily post it here. Hopefully what I've posted is enough information to move forward to the next step after Testdisk finished running recovery on my image. Also, it was suggested that i run reiserfsck on sdd1 as well after finishing it on md4. I just want to find out what label my "device" will be as I know it won't likely be sdd1. It no longer reports a label in UnRAID GUI or UnMENU, and when I run fdisk -l, it doesn't list the disk, which is what I expected. Thank you again so much and I am keeping my fingers crossed that this is a solvable problem! syslog-2014-04-15.txt 2014-04-15_-_reiserfsck_report.txt
April 16, 201412 yr I think I am in the same situation here. I removed an empty drive from the array (not physically) and started a parity check overnight. Checked on it this morning and saw errors. Parity never finished and has an Orange ball on the Parity drive. First drive has a red ball. I have not written to any data drive since I started the Parity check and I know drive 1 is OK. I just need to tell the system that the drive is fine and that parity needs to be rebuilt, right? I shut down, checked all controller cards seating, checked all cables, checked all power cables. Verified nothing was overheating, etc.. Rebooted and put the array in maint mode. Ran SMART test on Disk 1 and no problems found or noticed in the numbers. Ran file system check on Disk 1 and no problems found or noticed in the numbers. What should I do next? Re-init the config and let parity rebuild again? Drive_sdb_FSCHECK.txt Drive_sdb_SMART.txt
April 16, 201412 yr I think I am in the same situation here. I removed an empty drive from the array (not physically) and started a parity check overnight. Checked on it this morning and saw errors. Parity never finished and has an Orange ball on the Parity drive. First drive has a red ball. I have not written to any data drive since I started the Parity check and I know drive 1 is OK. I just need to tell the system that the drive is fine and that parity needs to be rebuilt, right? I shut down, checked all controller cards seating, checked all cables, checked all power cables. Verified nothing was overheating, etc.. Rebooted and put the array in maint mode. Ran SMART test on Disk 1 and no problems found or noticed in the numbers. Ran file system check on Disk 1 and no problems found or noticed in the numbers. What should I do next? Re-init the config and let parity rebuild again? The vast majority of times, these conditions where multiple apparently healthy drives are kicked from the array are caused by cabling issues to the drives. You should check all of the SATA and power connections to ensure that they are tight and secure. If you find everything is tight, you might want to replace the SATA cables. I prefer the locking cables which hold securely even with the vibrations and affects of occasionally moving or bumping into the server which happen over time. Your approach to recovery will work once this is corrected.
April 16, 201412 yr Yes, I too like the locking cables. I have 100% locking cables in this system. I even pulled all cables off 1 at a time and re-seated them just to make sure. I think I will try another parity check and if the same drive "fails" again, I will replace the cable. The cable is a SAS to 4x SATA and I do have a spare.
April 16, 201412 yr I think I am in the same situation here. I removed an empty drive from the array (not physically) and started a parity check overnight. Checked on it this morning and saw errors. Parity never finished and has an Orange ball on the Parity drive. First drive has a red ball. I have not written to any data drive since I started the Parity check and I know drive 1 is OK. I just need to tell the system that the drive is fine and that parity needs to be rebuilt, right? I shut down, checked all controller cards seating, checked all cables, checked all power cables. Verified nothing was overheating, etc.. Rebooted and put the array in maint mode. Ran SMART test on Disk 1 and no problems found or noticed in the numbers. Ran file system check on Disk 1 and no problems found or noticed in the numbers. What should I do next? Re-init the config and let parity rebuild again? Please do not hijack Nightstalk3rs thread. Start a new thread. Include a link to this thread if you believe they are related.
April 16, 201412 yr The array must be started in order for /dev/md4 to exist. testdisk or part image can be used to image the disk. testdisk can also fix the file system and recover files on the image. Great, thank you! I'm running recovery on the image currently, and my reiserfsck just finished this morning on /dev/md4. I'm not sure if there is a way to select any and all text that has scrolled outside of my telnet session window in PuTTY, but I copied all I could so I could post it here in a .txt file. I also pasted some of the report into a code block seen below. Most of what is in the report is what's seen on the first line below. vpf-10260: The file we are inserting the new item (695 691 0x780f001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) into has no StatData, insertion was skipped vpf-10260: The file we are inserting the new item (12 13 0x94b40001 IND (1), len 4048, location 48 entry count 0, fsck need 0, format new) into has no StatData, insertion was skipped 100% left 0, 0 /sec Flushing..finished Leaves inserted item by item 222 Pass 3 (semantic): ####### Pass 3 ######### Flushing..finished Files found: 0 Directories found: 2 Pass 3a (looking for lost dir/files): ####### Pass 3a (lost+found pass) ######### Looking for lost directories: Flushing..finishede 1, 1 /sec Pass 4 - finished done 0, 0 /sec Deleted unreachable items 2122 Flushing..finished Syncing..finished ########### reiserfsck finished at Tue Apr 15 08:41:03 2014 ########### root@Tower:~# I am not sure if I can access the report the reiserfsck that is created as I think it usually writes it to the disk itself, but if anyone knows a possibly way to grab that I'd happily post it here. Hopefully what I've posted is enough information to move forward to the next step after Testdisk finished running recovery on my image. Also, it was suggested that i run reiserfsck on sdd1 as well after finishing it on md4. I just want to find out what label my "device" will be as I know it won't likely be sdd1. It no longer reports a label in UnRAID GUI or UnMENU, and when I run fdisk -l, it doesn't list the disk, which is what I expected. Thank you again so much and I am keeping my fingers crossed that this is a solvable problem! Let us know how it goes.
April 16, 201412 yr Author Let us know how it goes. Thanks! I've been having some trouble figuring out how to run recovery on my backup image, so I'm not sure if I'm doing something wrong. Here is my process laid out from cradle to grave: [*]Purchased new 3TB Western Digital Green HDD to backup image to [*]Plugged both the new drive and the problem UnRAID drive into my HTPC [*]Created bootable USB drive for SystemRecoveryCD and booted into "StartX" [*]Opened up GParted and formatted the new WD 3TB drive as NTFS file system [*]Created a folder within /mnt called "unraid" (/mnt/unraid/) and mounted it to the NTFS partition on the new 3TB drive via the line "ntfs-3g /dev/sdb2 /mnt/unraid" in Terminal [*]Opened up Partimage and selected the partition on the problem UnRAID drive (sda1) and for the "Image file to create/use" section, I entered "/mnt/unraid/unraid" specifically not putting a file extension as I wanted the image to be uncompressed as I had the free space for it. I then chose the "Save partition into a new image file" option and hit F5 to go to the next screen. I then chose "None" under Compression Level and entered in a large enough GiB number in the "Image Split Mode" section so that it would just be one large image. [*]The image was created and can be found in "/mnt/unraid/unraid.000". From what i read, the .000 extension is because I chose no compression. I also noticed an image.dd file, which I'm assuming is related in some way. [*]Launched Testdisk through Terminal against the image file by typing "testdisk /mnt/unraid/unraid.000". [*]This is where I believe I'm having issues. In Testdisk, the Disk is selected by default "/mnt/unraid/unraid.000 - 1986 GB / 1850 GiB". I hit "Proceed" and then on the screen where it asks what partition table type, I select "Intel" as that is the one I read is for NTSF formatted partitions. Please correct me if I'm wrong! I chose GPT and None as well and they both did not seem to work. It takes a long time to scan the partition so I just want to make sure I'm selecting the right partition type!!!. [*]I then select "Analyse". On the next screen it says "Partition sector doesn't have the endmark 0xAA55 (not sure what that means), but I then hit enter on "Quick Search" which results in no partitions being found. Then I select "Deeper Search" with the same results. Some of my questions are: [*]I read somewhere that said to backup the partition tables for the primary partitions via Partimage, which I did not do. Is that possibly why I'm having this issue? [*]Is it maybe because I formatted the drive in NTFS, not realizing it's still experimental in Linux? [*]Did I maybe just skip a step in TestDisk? [*]Maybe I'm supposed to run TestDisk against the "image.dd" file that was created along with my "unraid.000" image file? For now, I will wait to see if anyone can offer some advice for me. Everyone has been very helpful and I am not in doubt that someone will know! I am not very familiar with Linux per se, but I figured out a lot of this on my own as I did not want to waste anyone's time. Thank you so much in advance!
April 17, 201412 yr Author What are the sizes of the files? Try the image.dd file. I will double check once I get home from work, but I believe the UnRAID.000 image file is roughly 1.7 TB, and the image.dd file was like several hundred MB. I did try running testdisk on image.dd with no luck however, but I will try again tonight! In the meantime, is there anything else I should be doing within the UnRAID server? I ran the reiserfsck on md4 (virtual device) , but wasn't able to figure out how to get the device label to run it against the physical device. I don't believe it is sdd1 as another drive had that label when doing an fdisk -l check. Is it because the drive is not currently in my server maybe? Thanks again as always for the friendly advice and assistance! You and everyone else have been more than helpful! Sent from my SCH-I535 using Tapatalk
April 17, 201412 yr I used the GUI to make and work with images. It's been a while so I don't remember the steps exactly.
April 17, 201412 yr Author I used the GUI to make and work with images. It's been a while so I don't remember the steps exactly. I used the Partimage GUI to create the image, which seemed to work okay, but maybe I didn't do it properly. I really wanted to make an uncompressed image since I have the space and it's faster to do. I then tried using the Testdisk GUI as well to run recovery on the image, which is where I'm stuck at now. Maybe I need to format the new 3TB drive I used to store the backup image as ReiserFS instead of NTFS, and the recreate the backup image again. It's hard to find any documentation on whether or not it matters regarding the backup disk file system. I just have read that NTFS is not 100% supported, at least it wasn't as of a few years ago. At this point, I'm tempted to put the original drive back into my UnRAID server and try to re-enable it now that I ran the "reiserfsck --rebuild-tree --scan-whole-partition" command, but I'm not sure if reiserfsck fixed anything after it finished, and if there is corruption on any of the data. If I did lose some of or even all of the data on that disk, I could hopefully obtain some of it back through my friends. I'm just not sure what to do next, and I hate to keep bothering everyone on here with my specific problem
April 18, 201412 yr The target disk file system makes no difference. The image file will be the same size as the source disk. Do a block level copy.
April 18, 201412 yr Author The target disk file system makes no difference. The image file will be the same size as the source disk. Do a block level copy. Okay great, I'll reformat the drive to remove the backup I did make and then do a block level copy of the troubled drive to my backup drive. Is there any best way to do a block copy? I see a lot of different ways to do this, including: "DD" "cp -a /path/from /path/to" "Rsync (Remote Sync)" I know Midnight Commander allows you to copy data as well. Whatever method you suggest is probably the one I'll go with since I trust your opinion Thanks again! You've been more than helpful with this whole process of assisting me!
April 18, 201412 yr Use DD. All the other methods are file level copies. You need to copy every sector (or block) on the source disk to the destination image file. The resulting file will be 2T (or whatever to size of the source disk is) in size. The disk system has several abstraction layers. Files and directories are the highest layer and disk sectors/blocks are the lowest accessible layer. These abstractions layers are built on the actual individual bits recorded on the disk. ( And these bits are actually an abstraction based on the state of electrons in the magnetic media. These abstraction layers descend into the realm of quantum mechanics. You decide where reality begins.) A block level disk image will allow recovery attempts of files missing or rather lost or misplaced by the corrupt file system because is contains all of the bits of the entire surface of the disk.
April 18, 201412 yr Author Use DD. All the other methods are file level copies. You need to copy every sector (or block) on the source disk to the destination image file. The resulting file will be 2T (or whatever to size of the source disk is) in size. The disk system has several abstraction layers. Files and directories are the highest layer and disk sectors/blocks are the lowest accessible layer. These abstractions layers are built on the actual individual bits recorded on the disk. ( And these bits are actually an abstraction based on the state of electrons in the magnetic media. These abstraction layers descend into the realm of quantum mechanics. You decide where reality begins.) A block level disk image will allow recovery attempts of files missing or rather lost or misplaced by the corrupt file system because is contains all of the bits of the entire surface of the disk. Awesome, thanks for the detailed explaination! I'm going to delete the current image and then do a block level copy of the troubled drive over to the backup drive. Is there any need to "shred" the device to wipe it of the data, even though I deleted the image? Regardless, once it's done, I'll run recovery on the backup. Should the backup be tested with Testdisk as I tried to do with the image file? Since I expect this to take over 24 hours, is there anything else I should be doing? I did not have logging on in PuTTY when I ran "reiserfsck --rebuild-tree --scan-whole-partition", and the terminal screen only scrolls so many lines. I did not see anything in the syslog regarding it, so I'm not sure if there is any important information I missed. I pasted what I was able to copy from PuTTY into a previous post in case there is any valuable info in there. Also, lets say that ultimately once recovery is finished running and it finds nothing wrong, can I just add the drive back to my server and reassign the disk back to the disk4 spot? I think I read where you need to start the array with no drive assigned first, then stop it and reassign the disk, then start the array again. I figured I would ask to not only just keep positive, but also to to plan ahead. Thanks again for all of the help because I'd be pretty lost if it weren't for it!
April 19, 201412 yr Here some examples: http://www.cgsecurity.org/wiki/Damaged_Hard_Disk http://www.cgsecurity.org/wiki/ReiserFS_File_Undelete_HOWTO The destination disk will be written with 2T of data from the source partition. The file system is unimportant as are the contents. Any content on the target disk will be overwritten by the new data. Do not focus on the destination drive. Do you wipe the drive on your laptop before writing a new file? Think of the image file as a really big text file.
April 21, 201412 yr Author Here some examples: http://www.cgsecurity.org/wiki/Damaged_Hard_Disk http://www.cgsecurity.org/wiki/ReiserFS_File_Undelete_HOWTO The destination disk will be written with 2T of data from the source partition. The file system is unimportant as are the contents. Any content on the target disk will be overwritten by the new data. Do not focus on the destination drive. Do you wipe the drive on your laptop before writing a new file? Think of the image file as a really big text file. Thanks again for all the info! The backup is complete. By reading the one link you sent regarding ReiserFS Undelete, it looks like I'm supposed to run reiserfsck on the backup partition, similarly as I did in UnRAID except now it would be (reiserfsck --rebuild-tree --scan-whole-partition /dev/sdb1). Once that's complete, I need to mount the drive and check "/mnt/recovery/lost+found" for lost files. If that is not the correct next step, please let me know, but I see that as the appropriate next step. Thank you again!!!
April 23, 201412 yr Use /dev/md4 to keep parity in sync. You can run reiserfsck or an equivalent recovery utility on the image file. rieserfsck requires the image be mounted first.
April 23, 201412 yr Author Use /dev/md4 to keep parity in sync. You can run reiserfsck or an equivalent recovery utility on the image file. rieserfsck requires the image be mounted first. Thanks for the response!!! So you are suggesting using reiserfsck or something equivalent on /dev/md4 (virtual device in UnRAID) as opposed to /dev/sdb1 (The partition on the new 3TB backup drive where the block level copy of my troubled 2TB drive is)? The only reason I ask is I wasn't sure if the backup partition I made of the troubled disk on the new 3TB would be known as /dev/md4 as It only shows as /dev/sdb1 in Partimage. When using DD, I did not create an image but just copied the entire partition from the troubled drive to the new 3TB backup drive. That shouldn't matter, correct? I am assuming it's as easy as just mounting the backup partition. Thank you again for all of your help with this! I just want to make sure I don't do the wrong thing and mess up what I have done so far. Your help has been greatly appreciated!!!
Archived
This topic is now archived and is closed to further replies.