May 6, 200818 yr I've had a lot of problems with my recent unRAID server build and needed to buy some new hardware. I'm wondering if there's a way to clear all the hard drives. If I remember correctly when a new un-formatted drive is added to the system it is first cleared and then formated. This clearing process puts all 0's into the drive. That's exactly what I'd like to do. I want to scrub the disks clean and then re-format them. Is there a way to do this via telnet? I know this is how to check the file system: http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems So I'd assume I'd stop samba, un-mount the disks, and then run some command to clear each disk one by one. I can then re-mount the drives and start samba. Once I log in via HTTP it should give me the option to format the drives and bring the array online.
May 6, 200818 yr Stop the array. umount the drives umount /mnt/disk? where ?=1-16 of drive you want to clear (keep in mind these are the md drives). dd if=/dev/zero of=/dev/sd? bs=1024 where ? = a-z of drive letter you want to clear. Here are some ways to correspond a /dev/md? (/mnt/disk?) is to look at the devices page. parity device: pci-0000:01:01.0-scsi-0:0:0:0 (sda) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0437718 disk1 device: pci-0000:01:01.0-scsi-1:0:0:0 (sdb) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0353226 disk2 device: pci-0000:01:01.0-scsi-2:0:0:0 (sdc) ata-MAXTOR_STM3500630AS_5QG00ENH Parity is always /dev/md0 and notice in my installation it is /dev/sda disk 1 is /dev/md1 and in my installation it is /dev/sdb disk 2 is /dev/md2 and in my installation it is /dev/sdc etc, etc, or you can check with root@unraid:~# cat /proc/mdcmd | strings | egrep 'diskName|rdevName' diskName.0= rdevName.0=sda diskName.1=md1 rdevName.1=sdb diskName.2=md2 rdevName.2=sdc
May 6, 200818 yr Author Awesome... that's just the info I needed. It's the first thing I'll do once I get my new server setup with new hardware Thank you for taking the time to help me (both here and a bunch of times in my other thread).
May 6, 200818 yr Here is a procedure that I used to burn in some new disks before putting them in my array. You may be able to do something similar. http://lime-technology.com/forum/index.php?topic=1887.msg13312#msg13312 The only reason that drives are filled with binary zeros by unRAID is so they can be added to the array without affecting parity. There is no other particular benefit. In order to do an effective test on a drive and let the SMART system do its thing, you would want to READ the entire drive, then WRITE the entire drive, then READ the entire drive. - The first READ will test readability of the entire drive. If there are marginal sectors detected, SMART will remap them. If there are hard read errors, SMART will queue them to be remapped on the next write. - The WRITE will test writeability of the entire drive. If there were sectors marked for remapping, they will be remapped. - The final READ will confirm that the disk is fullly functional. Together, these represents a pretty decent "stress test" on the drive. You might also find this wiki article informative about hard drive troubleshooting. http://lime-technology.com/wiki/index.php?title=Troubleshooting#Hard_drive_failures
May 6, 200818 yr Keep in mind that the clearing method mentioned above, while good for testing, and perhaps useful for knowing all previous info is cleared, will not save you any time when adding the drive to unRAID. And this is actually a good thing. The parity information has to be perfectly reliable, every bit in every sector, so it can't rely on assumptions of cleared sectors, or our word that they are zeroed. It needs to KNOW the state of every single bit of every single sector of every single drive. If a drive fails, a perfect rebuild depends on perfect parity, and a wrong idea about any sector on any other drive will cause corruption on the rebuilt drive. The only safe way therefore for unRAID to add a drive is to examine/clear every sector of it. In the future, Tom may add a Clearing option, which would be relatively safe, but it would have to involve putting the MBR and first tracks into a known state, plus adding a special signature somewhere. This would be necessary, to know with certainty that no other program has changed the drive in any way.
May 6, 200818 yr Sorry if my post implied that filling a drive with zeros would save time adding a drive to unRaid. RobJ is 100% correct - unRAID will re-fill the drive with zero unless you use a special tool which has not been released as of 4.3b6.
May 7, 200818 yr Also, check and double-check that the command line is referencing the correct drive! Once 'dd' starts writing zeros, you will not be able to recover any data**. **Exception that proves the rule: someone did exactly this, realized their mistake and immediately hit 'control-C' to kill the 'dd'. Of course the disk would not mount, but after running reiserfsck with appropriate options, we are actually able to recover a great deal of the file system
May 16, 200818 yr Author Stop the array. umount the drives umount /mnt/disk? where ?=1-16 of drive you want to clear (keep in mind these are the md drives). dd if=/dev/zero of=/dev/sd? bs=1024 where ? = a-z of drive letter you want to clear. Here are some ways to correspond a /dev/md? (/mnt/disk?) is to look at the devices page. parity device: pci-0000:01:01.0-scsi-0:0:0:0 (sda) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0437718 disk1 device: pci-0000:01:01.0-scsi-1:0:0:0 (sdb) ata-WDC_WD10EACS-00ZJB0_WD-WCASJ0353226 disk2 device: pci-0000:01:01.0-scsi-2:0:0:0 (sdc) ata-MAXTOR_STM3500630AS_5QG00ENH Parity is always /dev/md0 and notice in my installation it is /dev/sda disk 1 is /dev/md1 and in my installation it is /dev/sdb disk 2 is /dev/md2 and in my installation it is /dev/sdc etc, etc, or you can check with root@unraid:~# cat /proc/mdcmd | strings | egrep 'diskName|rdevName' diskName.0= rdevName.0=sda diskName.1=md1 rdevName.1=sdb diskName.2=md2 rdevName.2=sdc I'm currious... why did you mention /dev/md? Reading your post I stop samba and then unmount disk 1 by doing: umount /mnt/disk1 I then fill the disk with 00's by doing: dd if=/dev/zero of=/dev/sdb bs=1024 Out of those two commands, none involve using /dev/md?
May 16, 200818 yr I mentioned the md device in case you've already configured the drive as part of the array. Using the output of /proc/mdcmd status to determine the md device vs disk number device to physical drive
May 16, 200818 yr Author I mentioned the md device in case you've already configured the drive as part of the array. Using the output of /proc/mdcmd status to determine the md device vs disk number device to physical drive I'm still a bit confused. I can ls /dev/md* and see a list of my 4 drives... but I can also ls /mnt/disk* and see the four drives there as well. Which is the correct one to umount and run the command to place nothing but 00's on the drive? Or does it even matter?
May 16, 200818 yr umount can work with mount point and/or drive (/dev/md?) For safety sake use mount point /mnt/disk? then do a df or a mount to make sure it was unmounted. NOTICE WHERE /BOOT IS !! root@unraid:~# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sde1 505752 115024 390728 23% /boot /dev/md2 488371640 131714212 356657428 27% /mnt/disk2 /dev/md1 976732736 448430432 528302304 46% /mnt/disk1 shfs 1465104376 580144644 884959732 40% /mnt/user tmpfs 1024 0 1024 0% /unraid root@unraid:~# mount proc on /proc type proc (rw) sysfs on /sys type sysfs (rw) usbfs on /proc/bus/usb type usbfs (rw) /dev/sde1 on /boot type vfat (rw) fusectl on /sys/fs/fuse/connections type fusectl (rw) /dev/md2 on /mnt/disk2 type reiserfs (rw,noatime,nodiratime) /dev/md1 on /mnt/disk1 type reiserfs (rw,noatime,nodiratime) shfs on /mnt/user type fuse.shfs (rw,nosuid,nodev) tmpfs on /unraid type tmpfs (rw,size=1m,mode=0775) nfsd on /proc/fs/nfs type nfsd (rw) root@unraid:~# cat /proc/mdcmd | strings | egrep 'diskName|rdevName' diskName.0= rdevName.0=sda diskName.1=md1 rdevName.1=sdb diskName.2=md2 rdevName.2=sdc NOTICE WHERE /BOOT IS NOW!! root@Atlas:~# df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sdb1 505012 77884 427128 16% /boot tmpfs 1024 0 1024 0% /mnt /dev/sda2 3927748 295248 3632500 8% /mnt/ram tmpfs 1024 0 1024 0% /Atlas /dev/md1 976732736 63227800 913504936 7% /mnt/disk1 /dev/md2 488371640 122780032 365591608 26% /mnt/disk2 shfs 1465104376 186007832 1279096544 13% /mnt/user root@Atlas:~# cat /proc/mdcmd | strings | egrep 'diskName|rdevName' diskName.0= rdevName.0=sde diskName.1=md1 rdevName.1=sdd diskName.2=md2 rdevName.2=sdc be sure you do the following to double check the MD vs SD device nameing. cat /proc/mdcmd | strings | egrep 'diskName|rdevName' if you want to clear the drive while keeping parity intact then use the md device. If you want to clear the drive right from underneath unraid then use the sd device. using the sd device clears the partition table completely, this will cause unraid to put a new partition on the drive and a new filesystem, then run a full parity build. I mention md vs disk vs sd so you can be sure you do not write to the wrong drive. md1 = disk1 = an SD device which can range depending on other devices in your system/ I.E. it is not always sda or sdb. in my system it moves depending on where I have the USB key installed and if I have more then one. Look at the different positions of my /boot position vs what drives are defined in my system.
May 16, 200818 yr WeeboTech - you are an amazingly knowledgable guy. But I fear that this is a very dangerous thread. It explains how to do things that no typical user should ever do. If a person messes up one of these commands, their data will be going to be toast in a New York minute. I wish this thread focused more on the "why" and less on the "how". The "how" should be sent via PM only if a user has a really good reason. Just because a person asks how to play Russian Roulet, we don't have to give them a loaded gun! TO ANY SANE NEWBIE READING THIS THREAD - THERE IS NO GOOD REASON TO FILL YOUR DRIVE WITH BINARY ZEROS!
May 16, 200818 yr TO ANY SANE NEWBIE READING THIS THREAD - THERE IS NO GOOD REASON TO FILL YOUR DRIVE WITH BINARY ZEROS! I happen to agree (Again). I think some of this should just be put in the emhttp. There seems to still be allot of manual things required when situations go awry. basic things link grabbing the syslog, dumping the smart log, testing via smart, fixing a filesystem, clearning a drive. > I wish this thread focused more on the "why" To add, Why... 1. To erase the partition completely removing any trace. 2. To write binary 0's to every sector so that if a sector is bad S.M.A.R.T. might remap it elsewhere. 3. When the MBR, Parition Table and data is erase, the emhttp interface will now present options to clear/format the drive as a fresh new drive. FWIW, when I need to clear a drive. I pull out one of my cheesy spare motherboards, hook up the drive and boot dban. I never do it on a live system so I am sure I do not mess up.
May 16, 200818 yr /mnt/md* are empty directories. Empty directories are used as mount points in Unix/Linux. Devices, such as /dev/md1, /dev/md2, etc are the actual devices... they are mounted on mount points (the empty directories). When a device is mounted on a mount-point, the files in the file-system on the device appear as if they are in the mount-point directory. Typing "mount" will show you the currently mounted devices and the mount-points they are mounted on. The "umount" command is smart enough to use either the device or the mount-point name when un-mounting a device, so for it you can use either name. Before "wiping a disk" you will want to un-assign it, AND you will want to use the "Restore" button to eliminate it from the array. Otherwise, unRAID will probably treat it as a failed drive and want to restore a replacement when you restart the array. Writing zeros to the empty directory, once the device is unmounted, will fail, since you cannot write to a directory. Writing zeros to the mount-point device /dev/md1, /dev/md2, etc while it is mounted will work just as well as writing zeros to it when not mounted but will throughly confuse the SAMBA shares trying to use it. (SAMBA expects a file-system, zeroing the file-system out from under it will cause all kinds of errors as it attempts to read it) So.. log in and type "samba stop" then un-mount the file system you are zeroing first, to keep both samba and unRAID happy. If you zero out the /dev/md? devices parity will stay correct. If you instead zero out the /dev/sda, /dev/sdb, etc devices, then parity will be incorrect and you MUST use the check parity button, or the "Restore" button to force a parity re-calculation. As already said, unless you are doing something very unusual, there is no reason to write zeros to drive. About the only reason I can think of is to make unRAID think a drive is unformatted, and personally, I'd just use "fdisk" on the drive and delete the partitions to accomplish that task. It would be done in minutes and, it I accidentally did it to the wrong drive, there might be a chance of fixing the partition table and putting it back the way it was. so unmount the drive, then fdisk /dev/sda use fdisk commands to delete the partition, then to write the superblock back out to the disk. Then you are done and the drive is ready to be re-assigned. Just be aware, deleting the partition using "fdisk" does not delete the data on the disk... If you have sensitive data on the disk you should write zeros (or random data) to all of it. On larger disks this can take 4 or more hours to perform. Joe L. Edit: Since this is an old thread, and since there is now a preclear_disk.sh script that will write zeros to an entire drive while attempting to protect yourself from overwriting a drive assigned to your array, I thought I'd add a link to the thread that describes it. It is here: http://lime-technology.com/forum/index.php?topic=2817.0 The preclear_disk.sh script writes the special signature and MBR exactly as needed for unRAID to skip its own lengthy "clear" operation on a drive being added to the array. It also exercises the disk to attempt to find any sectors that might be bad and need re-allocation.
Archived
This topic is now archived and is closed to further replies.