October 3, 201015 yr So, In haste I installed a 1 TB WD green drive and moved my data to it and then installed another 1 TB as parity. These drives were previously used in a Windows environment. One thing though I didn't pre-clear or delete or whatever using the pre-clear script. Any harm done here ? Is it worth my while to remove the 1 TB being used as parity and at least do that one? Thoughts? Am I paranoid? Kryspy
October 3, 201015 yr You won't discover the sectors the disk cannot read until you attempt to read them. (not when you load them but most likely when you do your first full manual parity check) If you trust the drives no problem.... At least perform a SMART report on each of the two drives to get a baseline... you'll know what you are starting with. Then, press the "check" button on the management web-page. When it is done, perform a second parity check. Look for sectors re-allocated or pending re-allocation. Joe L.
October 3, 201015 yr Doesn't the format do the clearing? According to the wiki it does: http://www.lime-technology.com/wiki/index.php?title=Add_One_or_More_New_Data_Drives "The format operation consists of two phases. First, the the entire contents of the new disk(s) is cleared (written with zeros), and then it’s marked active in the array. Next, a file system is created. unRAID Server uses the ReiserFS journalled file system."
October 3, 201015 yr Doesn't the format do the clearing? According to the wiki it does: http://www.lime-technology.com/wiki/index.php?title=Add_One_or_More_New_Data_Drives "The format operation consists of two phases. First, the the entire contents of the new disk(s) is cleared (written with zeros), and then it’s marked active in the array. Next, a file system is created. unRAID Server uses the ReiserFS journalled file system." Writing the drive with zeros does not mark any sectors as un-readable since no "read" operation occurs. You will not learn of "read" errors until you've performed your first parity check. Furthermore, when initially setting up an array, no disk is cleared (written with zeros). It is only after you've initially calculated parity will subsequent data disks have zeros written to them if not already pre-cleared and the pre-clear signature is detected. The pre-clear process first reads every sector to allow the disk to identify any it cannot read, then writes zeros to clear the drive AND allow the un-readable sectors to be re-allocated, then re-reads the entire disk to ensure that all contents are zero (were written properly) and any re-allocated sectors can be properly read. It is most definitely not the equivalent.
October 3, 201015 yr So I haven't done a pre-clear at all then on my parity or 1st data drive. I guess then as what you were saying earlier, I can just use the 'check' button and this will accomplish the same thing?
October 3, 201015 yr So I haven't done a pre-clear at all then on my parity or 1st data drive. I guess then as what you were saying earlier, I can just use the 'check' button and this will accomplish the same thing? Sort of... If you press the "Check" button, all the sectors on all the disks will be read. Let's say a sector representing a part of one of your movies is un-readable, it should report an error to the unRAID driver. It will then re-construct the missing data and re-write it to the data disk. This should re-allocate the sector if it is unable to re-write it in its original location. (It first tries to re-use the original location in case it was just written poorly)
October 3, 201015 yr When I did the check for the first time and before adding any data, it should no errors. I can assume I am good? I just want to make sure I am understanding this correctly. The 3rd drive I am adding now I will do the pre-clear but I want to make sure my first 2 drives are good also.
October 4, 201015 yr i have the same thoughts. i am planning on building an array this tuesday when my parts arrive from newegg...i did a trial with my current gigabyte board with 1 parity and 4 data drives, it was still running parity check. i didn't put any data on it..when i setup up my new system with these drives in the new case, can i just run unraid the first time and let it setup the array? will that not do the preclear stuff? i looked at the script but i am confused as to how to designate each drive to preclear? do i do it form the root command line? how do i know that designation to use for the individual drives attached? also, is there anyway to run the script from the webgui?
October 4, 201015 yr can i just run unraid the first time and let it setup the array? You can (a) Setup new RAID and kick off parity generation. this will exercise all data disks through READ and parity disk through WRITE. (b) Once parity generation has finished and unRAID is online, do another parity check. this will once again exercise READ on all disks including parity disk.
October 4, 201015 yr i have the same thoughts. i am planning on building an array this tuesday when my parts arrive from newegg...i did a trial with my current gigabyte board with 1 parity and 4 data drives, it was still running parity check. i didn't put any data on it..when i setup up my new system with these drives in the new case, can i just run unraid the first time and let it setup the array? will that not do the preclear stuff? i looked at the script but i am confused as to how to designate each drive to preclear? do i do it form the root command line? how do i know that designation to use for the individual drives attached? also, is there anyway to run the script from the webgui? There is no way to run the pre-clear script from the web-gui. To determine the IDs of the disks, type ls -l /dev/disk/by-id or, stop the array and go to the "devices" page. The three letter device name is in the drop-down list of disk names for each slot in the array (in parentheses). Don't assign the disks if you intend to pre-clear them, as disks assigned to the array may not be pre-cleared. Just note the three character device name in the parentheses. If you use the ls -l /dev/disk/by-id command, the disk devices are the three letter names at the ends of the lines. Each line has the affiliated model/serial number. the full device names needed when you invoke the preclear script will be /dev/sda, /dev/sdb, /dev/sdc, etc... and, if you have any IDE drives, or a disk controller emulating IDE drives /dev/hda, /dev/hdb, /dev/hdc etc... Those disks with existing partitions will also have additional lines with 4 character names ending with a number (usually 1) Here is a partial listing from one of my servers... lrwxrwxrwx 1 root root 9 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1101YAH0EAEV -> ../../sdd lrwxrwxrwx 1 root root 10 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1101YAH0EAEV-part1 -> ../../sdd1 lrwxrwxrwx 1 root root 9 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1130YAG162AT -> ../../sdb lrwxrwxrwx 1 root root 10 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1130YAG162AT-part1 -> ../../sdb1 lrwxrwxrwx 1 root root 9 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1130YAG6XPMT -> ../../sdc lrwxrwxrwx 1 root root 10 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK1130YAG6XPMT-part1 -> ../../sdc1 lrwxrwxrwx 1 root root 9 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK11A1YAJBBMAV -> ../../sda lrwxrwxrwx 1 root root 10 Oct 3 22:25 ata-Hitachi_HDS722020ALA330_JK11A1YAJBBMAV-part1 -> ../../sda1
October 4, 201015 yr thanks so much for the reply, this is great.. So i should be able to place the preclear script onto my flash drive with my current old setup, access the root console, run the preclear script after id'ing the drives and then once the preclears are complete, i can just take those drives out of the old build, transfer them and the USB flash drive to the new build and reboot. the new build should then have the precleared drives assigned according...sound about right? can i run preclear on my 5-6 drives at one time? they are 2tb x2, 1tb x3, 640gb x1. will this take several days and if so, am i better off just building the server tomorrow(when the parts come in) and running the parity check (preclear equivalent if i undertand correctly) which will take that long anyway...i don't want to be waiting for drives to preclear for days when i want to build the new server tomorrow...i really don't need to absolutely transfer data right away so maybe i have the luxury of building the server and doing the checks then. i want to make sure that doing it all after HDD installation with parity check will give me the same benefit as preclear beforehand...thanks, sorry of the newbie questions but just trying to build right the first time.
October 4, 201015 yr am i better off just building the server tomorrow(when the parts come in) and running the parity check (preclear equivalent if i undertand correctly) which will take that long anyway...i don't want to be waiting for drives to preclear for days when i want to build the new server tomorrow... The process of initially calculating parity and then checking parity is NOT the equivalent of the process used by the preclear_disk.sh script. Yes, the initial parity calculation will linearly read all the sectors of the data disks, and it will write linearly all the sectors of the parity disk. The subsequent parity check will linearly read all the sectors of all the disks. Nothing in the above two steps check if all the sectors on the data disks are writable, or that the data disks can read what was written using the unRAID server. It only tests that it can read what currently exists on the disks. Nothing in the above two steps will show how the disks work when subject to access that is not linear. (random tracks from all over the disk.) Do not be fooled into thinking that an initial parity calc combined with a subsequent parity check is equivalent to the pre-clear script. It is not. The pre-clear "read" cycles do not just read the disk linearly, but also reads multiple random blocks and the first and last block on the disk constantly keeping the disk heads moving and seeking. It was designed to exercise the disk and burn it in to detect physical and electrical problems. It you do not need all your drives initially assigned to the array, run the pre-clear script on them. Yes, it will take time, a lot of time actually. But, you can potentially save a lot of time if you have the misfortune of a new disk that fails soon after you put your precious data on it. Joe L.
October 4, 201015 yr Author Okay so I removed my 1TB parity disk and am currently running the pre-clear script on it. I installed the smartctl package after it had started but I assume that will be okay. Now I read about other missing packages. Am I good or do i need to install more packages? I am running 4.5.6. Kryspy
October 4, 201015 yr good to know Joe L. i was under the mistaken impression that the two processes were equivalent...now my question: can i run preclear on multiple disks at one time? also can i install the drives into their new case/motherboard, and then boot into the root console and run preclear? the reason why i ask is that i'm leaving town on business and would be happy to run preclear on all the drives at one time while i'm away for a week...and then return to set up the array on precleared drives... if so, what would the command line be for multiple disks? thx
October 4, 201015 yr He details in the preclear thread how to run multiple instances. I just did 5 at one time.
October 4, 201015 yr Okay so I removed my 1TB parity disk and am currently running the pre-clear script on it. I installed the smartctl package after it had started but I assume that will be okay. Now I read about other missing packages. Am I good or do i need to install more packages? I am running 4.5.6. Kryspy Not sure why you installed the smartctl package... It has been a part of unRAID for a long time now. You did not hurt anything, but it was an un-necessary step. No package needed by the pre-clear script is missing in 4.5.6. (In a few of the early 4.5-beta series, there was a support library missing, but that too has been fixed for a long while) About the only package that will help if you wish to detach and leave pre-clear sessions running is "screen" It lets you detach and re-attach as you need. It allows multiple "virtual screens" to run multiple concurrent pre-clear processes. Joe L. Joe L.
October 4, 201015 yr Author Once this drive is done I will then move my data to a 250 and 500 GB drives and then pre-clear my other 1 TB I plan to use as disk1 and then add it to the array. Then I will add my 250 GB drive that I use as disk2 and mainly just for receiving torrent data so I don't much care about it. Kryspy
October 4, 201015 yr his forum is awesome!!! thanks for all the links...currently have my old rig preclearing 6 disks !! will be curious to see how long it will take...2tbx2,1tbx3, 640gb....when i transfer these precleared disks into my new build, do i need to put them back in the same order there were in for the preclear? sda, sdb...etc..or can i put them in any order into the new array..
October 5, 201015 yr This forum is awesome!!! thanks for all the links...currently have my old rig preclearing 6 disks !! will be curious to see how long it will take...2tbx2,1tbx3, 640gb....when i transfer these precleared disks into my new build, do i need to put them back in the same order there were in for the preclear? sda, sdb...etc..or can i put them in any order into the new array.. You can put them in any order you like. the device names are assigned to the disks in the order in which they are discovered when the hardware is scanned at boot time. At times specific hardware might be detected in a different order so a given disk might be /dev/sda one time, and a different device name the next time you reboot. unRAID uses the PCI hardware address internally to deal with that issue, since a disk will not typically move from one disk controller port to another between boots. (and it will notice if you do move one and as long as all disks are present it will start after prompting you that things had changes)
October 5, 201015 yr if the drive being precleared had partitions beforehand does that matter? won't it just erase all partitions? I ask this because one of my drives was used as a WHS system drive and when it worked throught the step 4 it states "clearing MBR for partitions 2,3,4"??
October 5, 201015 yr if the drive being precleared had partitions beforehand does that matter? won't it just erase all partitions? I ask this because one of my drives was used as a WHS system drive and when it worked throught the step 4 it states "clearing MBR for partitions 2,3,4"?? It will erase any prior partitions and create the one partition unRAID expects.
Archived
This topic is now archived and is closed to further replies.