April 11, 200818 yr Ok, am I missing something here... I thought I used to have the option to erase/format disks (or even the whole array, can't remember). But I am digging around now and I can't find it anywhere. I'm at the point that I'd like to just format all the drives and start from scratch. Anyone have a suggestion how to do this easily... or if not easily, at least quickly? I started off trying to select everything on disk1 and simply delete it... but after quite a while of "preparing to delete" I gave up and decided to try to track down the erase/format. Thanks.
April 11, 200818 yr There is no "erase" disk command available from the web-management page. You can "erase" a disk from the command line prompt, but make sure you have the right disk, since erasing can bring tears to your eyes if you accidentally erase the wrong disk. Also, you need not erase the parity drive. It will get completely erased and re-built when parity is re-enabled. To disable parity, un-assign the parity drive on the devices page and then back on the main page use the "Restore" button. That button does not restore data, but instead Saves a new configuration based on the currently assigned drives. To re-enable parity, assign a parity drive on the Devices assignment page. To erase a drive you must know its "device" as assigned by Linux. The devices are shown on the Device assignment page as the labels within the () parens. The device names are hda, hdb, hdc, .... for IDE based disks. and sda, sdb, sdc, ... for SATA based disks. Note: the USB flash drive will also have a name in that same sequence... usually sda, but if your flash drive initializes slowly, or is scanned by your motherboard's hardware last, it could be anything higher in the letter sequence. Another way to see the affiliated "device" name is to use the following command: ls -l /dev/disk/by-id | cut -c39- | grep -v part1 On my array it produces this list. The disk model and serial numbers are listed with the "device" name at the end of each line.[pre] ls -l /dev/disk/by-id | cut -c39- | grep -v part1 ata-HDS725050KLAT80_KRVA03ZAG3V5LD -> ../../hdb ata-HDS725050KLAT80_KRVA03ZAG4V99D -> ../../hdd ata-MAXTOR_STM3500630A_5QG00FTK -> ../../hdi ata-Maxtor_6Y250P0_Y63KH45E -> ../../hdj ata-Maxtor_6Y250P0_Y63KH8FE -> ../../hdk ata-ST3400620A_5QH00PF4 -> ../../hde ata-ST3400620A_5QH00QPN -> ../../hdc ata-ST3400633A_3PM0BE0T -> ../../hdg ata-ST3400633A_3PM0LZ3D -> ../../hdf ata-ST3500641A_3PM147P2 -> ../../hdh ata-ST3750640A_5QD2AX3G -> ../../hda usb-SanDisk_Corporation_MobileMate_200445269218B56190D7-0:0 -> ../../sda usb-USB_2.0_Storage_Device_00042222200000118226-0:0 -> ../../sdb [/pre] It is critical you properly identify the drive you wish to clear. There is NO undo. Check and double-check. Assuming you have stopped the array and un-assigned the drive you wish to erase, use the following command to clear your drive's partition table and initial portion of the file-system (in this example, "sdz" should be the device name of your disk): dd if=/dev/zero of=/dev/sdz count=10 You have been warned, giving the wrong device name as the output file of the "dd" command will zero out the wrong disk, and there is NO UNDO command. This could be very bad if the disk you accidentally named held all your movies or music. Make sure you have the correct "device" name before you perform these steps. Once all your disks are "cleared" you can use the Devices assignment page to assign them to slots in the array and return to the main management web-page, then press the "Restore" button after checking the checkbox. This will save the new configuration. Joe L.
April 11, 200818 yr Author Thank you Joe L., I don't know why, but I thought I remembered ( back in the 3.0 or earlier days) being able to take a drive that was not part of the previous array and erase it. I guess I must be wrong. I'm not concerned about deleting the wrong drive... since now that I've upgraded to 4.3 and have user shares available, I want to erase them all and start with a new system. I have everything copied (twice) onto my local computer. I will give this a shot and see if it works better than selecting/deleting. I'm afraid I may have a bad drive, since the first disk I was selecting/deleting stuff from eventually popped up an error (cannot find a certain file) and now when I try to reconnect to it, I get a path invalid (check permission) error.
April 11, 200818 yr Thank you Joe L., I don't know why, but I thought I remembered ( back in the 3.0 or earlier days) being able to take a drive that was not part of the previous array and erase it. I guess I must be wrong. I'm not concerned about deleting the wrong drive... since now that I've upgraded to 4.3 and have user shares available, I want to erase them all and start with a new system. I have everything copied (twice) onto my local computer. I will give this a shot and see if it works better than selecting/deleting. I'm afraid I may have a bad drive, since the first disk I was selecting/deleting stuff from eventually popped up an error (cannot find a certain file) and now when I try to reconnect to it, I get a path invalid (check permission) error. If all you wish to do is delete all the files, that is much quicker. (actually, the dd command is very quick, but the subsequent "clear" that unRaid will perform when you go to set up the drives again in your array will take hours for large drives) To remove all the files from your disks... rm -rf /mnt/disk1/* rm -rf /mnt/disk2/* rm -rf /mnt/disk3/* etc... I also would run reiserfsck on all your drives, before you run "dd" to clear the partitioning. It might just find something it can fix. Instructions here in the wiki: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems When you respond to its prompt, type "Yes" with a capital "Y" and lower case "es" for it to continue. Any other response will have it abort and not run. Joe L.
April 11, 200818 yr Author well... can this be done from telnet? I thought perhaps I was missing a step or two (I had tried something similar you posted in another thread), but it appears that this doesn't do anything for me. After I bring everything back online, I still have all my info (never thought I'd be disappointed saying THAT!). Does this look correct?
April 11, 200818 yr Author Thank you Joe L., I don't know why, but I thought I remembered ( back in the 3.0 or earlier days) being able to take a drive that was not part of the previous array and erase it. I guess I must be wrong. I'm not concerned about deleting the wrong drive... since now that I've upgraded to 4.3 and have user shares available, I want to erase them all and start with a new system. I have everything copied (twice) onto my local computer. I will give this a shot and see if it works better than selecting/deleting. I'm afraid I may have a bad drive, since the first disk I was selecting/deleting stuff from eventually popped up an error (cannot find a certain file) and now when I try to reconnect to it, I get a path invalid (check permission) error. If all you wish to do is delete all the files, that is much quicker. (actually, the dd command is very quick, but the subsequent "clear" that unRaid will perform when you go to set up the drives again in your array will take hours for large drives) To remove all the files from your disks... rm -rf /mnt/disk1/* rm -rf /mnt/disk2/* rm -rf /mnt/disk3/* etc... I also would run reiserfsck on all your drives, before you run "dd" to clear the partitioning. It might just find something it can fix. Instructions here in the wiki: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems When you respond to its prompt, type "Yes" with a capital "Y" and lower case "es" for it to continue. Any other response will have it abort and not run. Joe L. OK, I'll try this... I was trying (from some googling) to use the rm command, but I was trying to do it to the folders, not the "drive1", "drive2" etc... so I never figured that out. Thanks.
April 11, 200818 yr (I only mention this because you are going through the trouble of erasing and starting over). if you are going to erase and do the dd why only do count=10. If you want to be sure the drives are good and re-allocate any bad sectors remove the count=10 and write zero's to all the sectors. That will insure everything is erased and you have written to every sector (and possibly remapped any bad ones if they exist). Note: this will take a LONG time but it will ensure you are starting from scratch. Thereafter I would suggest doing a smartctl test smartctl -d ata -tshort /dev/sdX where x is the drive. You can also do a LONG test for more thorough testing. smartctl -d ata -tlongt /dev/sdX where x is the drive. This will also take a long time. Then look at the smartctl logs with smartctl -d ata -a /dev/sdX | less At that point you will be sure you now what you are starting with.
April 11, 200818 yr I suggested the count=10 option only because I figured that unRaid would clear the drives and in doing so write zeros to all the blocks. Your advice is very good... it just takes a bit longer. Joe L.
April 11, 200818 yr Author Well, I don't know if I have a new bug... or if somehow I just have a damaged file... but each of my drives with data on it seem to have problems. When I go to erase (be it selecting files in explorer and deleting, or the dd command, or the rm command) it never works. I was hoping this was just with my first disk in the array... but now 3 of them do this. I was able to do the reiserfschk as suggested, and that fixed the problem. Which isn't that big of a deal. I just upgraded from 3.0 to 4.3 beta6. I don't know if 3.0 would have shown this issue, as I did not try to clear everything off until after the upgrade. Anyhow, if I try to delete (or dd, or rm) it slows the system down to a crawl... then after a while I can usually get back into the OTHER disks, but not the one I was trying to delete/clear. Also, it appears that most commands will not work after this point (shutdown, web browser is slow or stops responding, etc). I seem to get this a lot when that happens: root@Tower:~# rm -rf /mnt/disk2/* Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: ------------[ cut here ]------------ Segmentation fault root@Tower:~# Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: invalid opcode: 0000 [#1] Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: Process rm (pid: 1924, ti=dea46000 task=d97c54c0 task.ti=dea46000) Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: Stack: dd977770 005c8000 c017b2bd 00001000 00000000 c0133d42 00000 000 e081c000 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: dfa5fe00 dd9680e0 00008000 000000b9 c017b627 e0838000 00001 000 005cf3a8 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: dfa5fe00 dea47f08 00000001 00000001 005cf3a8 dda198e4 dea47 f08 c1712000 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: Call Trace: Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c017b2bd>] reiserfs_read_bitmap_block+0xbb/0xc5 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0133d42>] mempool_alloc+0x1c/0x93 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c017b627>] _reiserfs_free_block+0xa6/0x176 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0191e05>] prepare_for_delete_or_cut+0x4e3/0x605 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c019230e>] reiserfs_cut_from_item+0xed/0x7e3 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0113623>] dequeue_entity+0xb/0x2a Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c02eba92>] schedule+0x1f3/0x20d Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0123ded>] autoremove_wake_function+0x0/0x35 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c01816d4>] make_cpu_key+0x3f/0x46 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0192d8e>] reiserfs_do_truncate+0x38a/0x4ee Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0192f22>] reiserfs_delete_object+0x30/0x5f Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0184d89>] reiserfs_delete_inode+0x4c/0x8d Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0184d3d>] reiserfs_delete_inode+0x0/0x8d Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c015afc8>] generic_delete_inode+0x54/0xa8 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c015a800>] iput+0x60/0x62 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0153d49>] do_unlinkat+0xb5/0x125 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0155692>] vfs_readdir+0x64/0x74 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c01554b0>] filldir64+0x0/0xc5 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c015573d>] sys_getdents64+0x9b/0xa5 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: [<c0103b4e>] syscall_call+0x7/0xb Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: ======================= Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: Code: ff 89 d8 e8 ec e0 fd ff a1 c0 34 41 c0 5b 8b 40 08 83 c0 6c e9 d1 0c 17 00 56 89 d6 53 89 cb 8b 4e 14 8b 52 18 8b 02 a8 01 75 04 <0f> 0b eb fe 8d 0c 0a c7 03 00 00 00 00 eb 24 8b 01 85 c0 75 05 Message from syslogd@Tower at Fri Apr 11 10:56:01 2008 ... Tower kernel: EIP: [<c017b0d6>] reiserfs_cache_bitmap_metadata+0x12/0x50 SS:ESP 0068:dea47b3c It is a problem with a specific file (or files), because when I start selecting and deleting individual files/folders on the drive... it works great, until I hit a specific folder... then it slows down and craps out on me. So far on 2 disks, it has been a jpg file (I think 2 different ones... but not 100% sure on that, as I erased the first one).
April 11, 200818 yr well... can this be done from telnet? I thought perhaps I was missing a step or two (I had tried something similar you posted in another thread), but it appears that this doesn't do anything for me. After I bring everything back online, I still have all my info (never thought I'd be disappointed saying THAT!). Does this look correct? The reason the "rm" commands are failing is because you clobbered the partition information when you ran the "dd" commands. The "rm" commands would only be useful before you wrote zeros to the disks to wipe them. The reason some files still exist is that you only overwrote the first 10 blocks of data on the disks... enough to clobber the partition table, but not enough to wipe out all your data. Just re-boot the array, and you will probably get lots of errors. (the disks are probably still assigned on the Devices page.) If they are, you might need to un-assign them all, start the array to let it see you have unassigned them all, then re-assign them. If not, assign them to their slots. Then use the "Restore" key to reset everything back the initial steps of clearing and formatting.
April 11, 200818 yr There's too much being done to allow you to use all the different commands. Once you have done the dd, anything else you do is working with a corrupt filesystem rm resierfsck mount unmount etc, etc. Even the partition table has been erased. So it's no wonder that the kernel is going bonkers when accessing the filesystem. if you dd the whole drive, everything will be set to zeros and you can reboot and start from scratch. I would still suggest doing the smartctl manually to double check your drives.
April 11, 200818 yr There's too much being done to allow you to use all the different commands. Once you have done the dd, anything else you do is working with a corrupt filesystem <snip> Even the partition table has been erased. So it's no wonder that the kernel is going bonkers when accessing the filesystem. I'm surprised reiserfs is doing even that and letting you get to some files... I cannot imagine windows doing as well if I wiped out its partition tables. I would still suggest doing the smartctl manually to double check your drives. Very good advice.
April 11, 200818 yr I'm surprised reiserfs is doing even that and letting you get to some files... I cannot imagine windows doing as well if I wiped out its partition tables. I think the filesystems are still mounted, so there's a bunch of data still in memory.
April 11, 200818 yr Author There's too much being done to allow you to use all the different commands. Once you have done the dd, anything else you do is working with a corrupt filesystem rm resierfsck mount unmount etc, etc. Even the partition table has been erased. So it's no wonder that the kernel is going bonkers when accessing the filesystem. if you dd the whole drive, everything will be set to zeros and you can reboot and start from scratch. I would still suggest doing the smartctl manually to double check your drives. OK, thanks guys. the reiserfsck seems to fix the problem (stopping samba, umounting, and then resiesrfsck --fix-fixable)... but if there's a faster way (rebooting like Joe suggest, or dd the entire drive) I'm all for it! I'll give the reboot a shot first, then if that doesn't clear everything I'll retry the dd without the count 10 on it. For clarification, should I only do the smartctl after the dd, or if rebooting cleans them off, is it ok to do this without first running dd? and FYI, I had the jpg issue on drive 1 before I tried any of those commands (when I was manually deleting stuff thru windows), but I can see how I've probably screwed everything up nicely
April 11, 200818 yr For clarification, should I only do the smartctl after the dd, or if rebooting cleans them off, is it ok to do this without first running dd? Do it after the drive has been cleared either via dd to the whole drive or after you have installed it into unraid, cleared and formatted it. It's up to you. When you look at the log after the test, look for any errors reported and/or numbers in "current pending sectors".
April 11, 200818 yr Author I'm trying the dd without the count now... Amazingly enough, after ALL the crap I did... the array booted up properly and everything works (except deleting some specific files). I can browse and copy and add files... just not erase. Usually I would consider this a GOOD thing, but not at the moment.
April 11, 200818 yr Author OK, I started the dd and after several hours, I had thought that perhaps it was not responding... so I ctrl C stopped it... it had only finished 120GB. So I changed the dd to be 100000 instead of 10 and THAT made the drives show up as unformatted. (and took a LOT less time). Now the system is reformatted and we're good to go. Thank you guys for your help. I'm going to try the smartctl as suggested... but if it takes as long as the dd takes, I think I'm going to have to skip it. I've got 2.4 terabytes of stuff to sort thru and backup so that I can reinstall the workstation and get back to work.
April 11, 200818 yr You can run the smartctl after your data is restored. The short test is 2 minutes, the long test is more thorough, but will be hundreds of minutes. I would recommend you do it anyway just to be sure. At the very least do a smartctl -data -a /dev/sd? | less Then review the S.M.A.R.T. logs and see if there are any current_pending_sectors. If there are none you're probably ok
April 12, 200818 yr I am SURE there is a better way to reformat a disk than what I am about to suggest. *** WARNING: DO NOT DO THIS FOR ANY DRIVE THAT CONTAINS DATA THAT YOU WANT TO KEEP *** 1. Stop the array 2. Remove all disks from your array 3. Assign a single disk to the array (that you want to reformat) in the PARITY slot 4. Start the array --- unRAID will start clobbering the disk - in a few seconds it will be "unformatted" 5. Stop the parity build 6. Go to Step 1 until all disks are unformatted You can then add these disks to an array and unRAID. unRAID will see them as unformatted. This will avoid a lengthy recursive file delete that can take forever. This worked in 4.2.1, I have not tried it in the latest BETA. Tom may have put some defensive code around this condition, but there is nothing in the release notes about it. *** BE VERY CAREFUL NOT TO ALLOW A DISK WITH IMPORTANT CONTENTS TO BE ASSIGNED TO THE PARITY SLOT ***
April 12, 200818 yr Author You can run the smartctl after your data is restored. The short test is 2 minutes, the long test is more thorough, but will be hundreds of minutes. I would recommend you do it anyway just to be sure. At the very least do a smartctl -data -a /dev/sd? | less Then review the S.M.A.R.T. logs and see if there are any current_pending_sectors. If there are none you're probably ok Gonna try this next... I'm currently resynching parity, because I had so many sync errors (on formatted drives), but I can't figure out what that means. I was hoping that was a glitch, but the new sync is pulling up sync errors too. After that is done, I'm going to do the smartctl...
Archived
This topic is now archived and is closed to further replies.