need to pre clear a drive on already full array..

[loady]

I have just aquired 2 x 3tb seagate drives which i intend to swap out of my array for 2 x 2tb drives, my array already has all available slots filled and was just wondering what the best way was to go to start pre clearing one then letting the array rebuild onto it and then pre clear the other one, i have an external caddy i can mount them in and i also have SNAP installed, so if there is a quicker way to go about this i would be greatful for pointing in the right direction..not sure if it is possible to mount the drive(s) via snap and then use screen to preclear them ?, if i can..would i then just remove the drive i intend to replace then let the array rebuild onto it ?

[Joe L.]

I have just aquired 2 x 3tb seagate drives which i intend to swap out of my array for 2 x 2tb drives, my array already has all available slots filled and was just wondering what the best way was to go to start pre clearing one then letting the array rebuild onto it and then pre clear the other one, i have an external caddy i can mount them in and i also have SNAP installed, so if there is a quicker way to go about this i would be greatful for pointing in the right direction..not sure if it is possible to mount the drive(s) via snap and then use screen to preclear them ?, if i can..would i then just remove the drive i intend to replace then let the array rebuild onto it ?

You cannot preclear a drive if it is mounted (or in use in any way, or assigned to the array), therefore, SNAP is completely useless in your situation. 

 

If you have a spare SATA port, connect the drive and run the pre-clear script on it.  Then, stop the array, un-assign the disk being replaced, assign the new drive in its place, and start the array (after checking the I'm sure box)

 

There is no quicker way.  Rinse and repeat with the second drive.

 

[loady]

i have a proliant server, i used all available sata ports to be able to get two more drives into it, however, one of the ports is the cache drive, can i use that ? it does have some apps installed to it, namely plex, CP and SB...how would i go about using this post as an interim and then returning the drive back in safely ?

[dgaschk]

Is you external caddy eSATA? http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

[loady]

no, but i can get power to it from an external source and then plug drive into esata port..or what i plan to do is to remove the cache drive which is on the esata port and replace it with the drive that needs preclearing...the cache is not part of the array so this should be fine ?

 

Device Identification Temp. Size Used Free Reads Writes Errors View

  Parity

SAMSUNG_HD203WI_S1UYJ1CZ316726 (sda) 1953514552 * 2 TB - - 58.932 49.818 0

  Disk 1

SAMSUNG_HD204UI_S2H7J1BZ912023 (sdb) 1953514552 * 2 TB 1,75 TB 250 GB 78.592 7690 0

  Disk 2

SAMSUNG_HD204UI_S2H7J1BZ912031 (sdc) 1953514552 * 2 TB 1,76 TB 239 GB 31.443 65 0

  Disk 3

SAMSUNG_HD103SJ_S246JDWZ125760 (sdd) 976762552 * 1 TB 938 GB 62,6 GB 5140 11 0

  Disk 4

SAMSUNG_HD203WI_S1UYJ1CZ316736 (hdb) 1953514552 * 2 TB 1,59 TB 412 GB 89.595 38.909 0

  Cache

WDC_WD3200BPVT-00ZEST0_WD-WX91A71E1730 (hda) 312571192 29 °C 320 GB 221 GB 98,8 GB 324.118 2.207.627 0

Total Array of five protected disks 29 °C 7 TB 6,04 TB 963 GB 587.820 2.304.120 0

Flash

JD_FireFly (sde) 15663072 16,0 GB 283 MB 15,8 GB 2947 65 0

 

 

[hawihoney]

Is you external caddy eSATA? http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

 

Does not support 3TB drives as requested by OP. Thermaltake says "up to 1TB", Amazon says "up to 2TB".

 

EDIT: Newegg says "up to 3TB" and Thermaltakes second page (Specification) says "All capacities". Four different specifications found on four different pages. I wouldn't buy that beast.

 

[loady]

its not relevant, i have no intention of using a caddy. i want to temporarily use the cache drives esata port

[loady]

'i have connected up the drive, removing the cache drive and all seems well...im not sure though what switches i need to use on this 3tb drive..i dont think my other are 4k aligned, will it hurt to utilise the -A option ? and are there many benefits in doing so?..think i need to update screen as well 

[Joe L.]

'i have connected up the drive, removing the cache drive and all seems well...im not sure though what switches i need to use on this 3tb drive..i dont think my other are 4k aligned, will it hurt to utilise the -A option ? and are there many benefits in doing so?..think i need to update screen as well 

both -A and -a are completely ignored on ANY drive over 2.2TB.  You can use them, or not, (since they are ignored). They only have meaning for drives 2.2TB and smaller that use an MBR based partition.

 

Joe L.

[dgaschk]

Is you external caddy eSATA? http://www.newegg.com/Product/Product.aspx?Item=N82E16817153071

 

Does not support 3TB drives as requested by OP. Thermaltake says "up to 1TB", Amazon says "up to 2TB".

 

EDIT: Newegg says "up to 3TB" and Thermaltakes second page (Specification) says "All capacities". Four different specifications found on four different pages. I wouldn't buy that beast.

 

It works with all capacities. There is no reason in the specification for a limit. Vendors usually only indicate the capacity they have tested. I have used several with various 3T drives..

[loady]

Thanks fellas. Another querie just arisen, clearing is going fine on the port where cache drive was attached, however, just been reading about adding the drives when they are precleared, it states that my parity drive must be 'as' large or 'larger' than the largest drive in the array..so i need to make one of these 3tb drive the parity which i already have..do i just carry on as normal ?...unassign the parity drive, remove it, insert the 3tb, assign it, let it format and then it will rewrite parity onto it automatically ?..after which i can carry on with replacing one of the other drives in the array ?

[Joe L.]

Thanks fellas. Another querie just arisen, clearing is going fine on the port where cache drive was attached, however, just been reading about adding the drives when they are precleared, it states that my parity drive must be 'as' large or 'larger' than the largest drive in the array..so i need to make one of these 3tb drive the parity which i already have..do i just carry on as normal ?...unassign the parity drive, remove it, insert the 3tb, assign it, let it format and then it will rewrite parity onto it automatically ?..after which i can carry on with replacing one of the other drives in the array ?

Basically correct, except that the parity drive is never formatted since it does not contain a file-system.  Other than that, you've described the sequence of steps to be performed.

 

It is HIGHLY recommended you perform a parity check before moving or replacing ANY disks to identify issues.  Also, before doing ANYTHING, cleanly stop the array and make a copy of the "config" directory on the flash drive with the array in a stopped state and take a screen print of the drive assignments.  You can restore that set of drives and that config directory contents to get back to a safe point in time if you really get messed up.

I see too many posts with people who replace a drive only to learn a different drive has issues and there is no easy way to recover.

[loady]

ok, so still good practise to preclear the new parity ?, i have already taken a screen dump of drive assignments from :8080, however, i can never get the array to just stop without it completley powering down, if i opt to just 'stop' array it goes into a wobbly loop, when i restart the system i set UNraid  to start the array straight away so i am never in a state of 'stopped'  .

 

I think my preclear/screen has hung, i keep closing and opening a window and the bytes wrote value has not moved, in fact, nothing has changed since 16%...has this fubered up and i need to restart it ? not sure if it is supposed to look like this untill it reaches its next phase.

 

================================================================== 1.13

=                unRAID server Pre-Clear disk /dev/hda

=              cycle 1 of 1, partition start on sector 1

= Disk Pre-Clear-Read completed                                DONE

= Step 1 of 10 - Copying zeros to first 2048k bytes            DONE

= Step 2 of 10 - Copying zeros to remainder of disk to clear it

=  **** This will take a while... you can follow progress below:

=

=

=

=

=

=

=

=

Disk Temperature: 36C, Elapsed Time:  11:00:42

232338+9 records in

232338+9 records out

487258673152 bytes (487 GB) copied, 16340.3 s, 29.8 MB/s

Wrote  487,258,673,152  bytes out of  3,000,592,982,016  bytes (16% Done)

 

[garycase]

"...  i can never get the array to just stop without it completley powering down, if i opt to just 'stop' array it goes into a wobbly loop ..."  ==>  Is this what happens if you click on "Stop" in the web interface?

 

If so, clearly something isn't working correctly.    You should resolve that BEFORE you make ANY drive changes.

 

The first thing I'd suggest is upgrading to the latest Release Candidate (rc12a).  You're MANY versions behind with your Beta version.

 

Then stop the array and save the contents of the flash drive [i always just copy the entire contents, but the Config folder is all you really need]

 

Then run a Parity Check (I'd just do a correcting one;  but some like to do a non-correcting one first ... although you NEED to have it correct before you do any drive replacements, so if it finds errors you'll almost certainly want to just repeat it as a correcting check).

 

... and, finally, you can start the drive replacement process.    You're correct that the FIRST drive you need to replace is your parity drive, as it needs to be the largest drive in the system (others can, of course, be the same size).    THEN you can replace others as needed.

 

r.e. Pre-Clears ==>  In this case what you're really doing with Pre-Clear isn't saving time (since both the parity drive and the subsequent data drive have to be completely written anyway -- unlike simply adding a new drive to the system, where the Pre-Clear saves the clearing operation);  but the Pre-Clear is still a good idea as it does a very thorough "scrubbing" of the drive and confirms it's good before you install it in your array.

 

[garycase]

r.e. the "hung" pre-clear window.    The byte count should occasionally update (not every second, but certainly within a minute)

 

Are you monitoring this from a console attached to the UnRAID system (best);  or are you logged in via Telnet?    If Telnet, you have to keep the connection open for the duration of the pre-Clear.  I think if you use Screen you're supposed to be able to close it and come back later (I've never done this, so I'm not sure exactly how it works) ... but if you do, I'm not sure how to confirm you're logged into the correct session.    Personally, I always use a console for pre-Clears ... Telnet works well; but I don't want to use it for an extended operation like pre-Clear [Plus with the console it's easy to switch sessions and have several pre-Clears going at once].

 

[loady]

r.e. the "hung" pre-clear window.    The byte count should occasionally update (not every second, but certainly within a minute)

 

Are you monitoring this from a console attached to the UnRAID system (best);  or are you logged in via Telnet?    If Telnet, you have to keep the connection open for the duration of the pre-Clear.  I think if you use Screen you're supposed to be able to close it and come back later (I've never done this, so I'm not sure exactly how it works) ... but if you do, I'm not sure how to confirm you're logged into the correct session.    Personally, I always use a console for pre-Clears ... Telnet works well; but I don't want to use it for an extended operation like pre-Clear [Plus with the console it's easy to switch sessions and have several pre-Clears going at once].

 

 

Firstly, thanks for your input and sorry for the delay in replying. Drives have cleared now, i had quite a few screens going for some reason and now i have found out how to close them it seemed to jump into life, both drives have passed pre clear now, i was using screen via telent and it worked well for me.

 

 

"...  i can never get the array to just stop without it completley powering down, if i opt to just 'stop' array it goes into a wobbly loop ..."  ==>  Is this what happens if you click on "Stop" in the web interface?

 

If so, clearly something isn't working correctly.    You should resolve that BEFORE you make ANY drive changes.

 

The first thing I'd suggest is upgrading to the latest Release Candidate (rc12a).  You're MANY versions behind with your Beta version.

 

Then stop the array and save the contents of the flash drive [i always just copy the entire contents, but the Config folder is all you really need]

 

Then run a Parity Check (I'd just do a correcting one;  but some like to do a non-correcting one first ... although you NEED to have it correct before you do any drive replacements, so if it finds errors you'll almost certainly want to just repeat it as a correcting check).

 

... and, finally, you can start the drive replacement process.    You're correct that the FIRST drive you need to replace is your parity drive, as it needs to be the largest drive in the system (others can, of course, be the same size).    THEN you can replace others as needed.

 

r.e. Pre-Clears ==>  In this case what you're really doing with Pre-Clear isn't saving time (since both the parity drive and the subsequent data drive have to be completely written anyway -- unlike simply adding a new drive to the system, where the Pre-Clear saves the clearing operation);  but the Pre-Clear is still a good idea as it does a very thorough "scrubbing" of the drive and confirms it's good before you install it in your array.

 

Ha! for the first time, it has stopped the array !! normally it would just keep trying to unmount disks, not sure if this has something to do with the cache drive not being attached as i have my apps installed to it, ie, .apps/plex so they dont get moved off the cache to the array...could this be the issue ? guess ill find out when it back on.

 

I have just updated my sig, my actual revision is 5.0-rc8a so not to far behind but i think i will update anyway.

 

my last parity check was Last checked on Sat Apr 13 08:35:00 2013 BST (two days ago), finding 0 errors. This is always the case..not had any errors as yet in nearly two years i think. Though i will leave it doing a check overnight.

 

[loady]

i have replaced the parity drive with its new pre cleared 3tb one, assigned it and started parity sync, its saying.. Estimated finish: 19 days, 15 hours, 27 minutes  ..is that right ?

[trurl]

r.e. the "hung" pre-clear window.    The byte count should occasionally update (not every second, but certainly within a minute)

 

Are you monitoring this from a console attached to the UnRAID system (best);  or are you logged in via Telnet?    If Telnet, you have to keep the connection open for the duration of the pre-Clear.  I think if you use Screen you're supposed to be able to close it and come back later (I've never done this, so I'm not sure exactly how it works) ... but if you do, I'm not sure how to confirm you're logged into the correct session.    Personally, I always use a console for pre-Clears ... Telnet works well; but I don't want to use it for an extended operation like pre-Clear [Plus with the console it's easy to switch sessions and have several pre-Clears going at once].

You can switch sessions in screen too, as well as disconnect and reconnect. I only temporarily attach a monitor when I need to get to the BIOS.

Configuration_Tutorial - Preclearing_With_Screen

[dgaschk]

i have replaced the parity drive with its new pre cleared 3tb one, assigned it and started parity sync, its saying.. Estimated finish: 19 days, 15 hours, 27 minutes  ..is that right ?

 

What is the current time estimate?

[loady]

I dont remember parity taking this long to create on a 2TB drive..would that be because i have accumulated a lot more data over time ?

 

[itimpi]

That parity check speed looks abysmally low.  I would be looking at what could be causing that.  Are there any errors showing in the syslog?

 

On my system I never get below 50MB/s, and once I am beyond the slower small disks it jumps up to 100+MB/s

[loady]

its not a check, it is a new parity drive, a pre cleared 3tb jobby..so its writing the parity data..is this why its so slow ?..but even so.. 18 days !!..come on

 

[garycase]

trurl => Thanks for reminding me of the Screen tutorial.    Next time I need to do some pre-clears I'll give Screen a try.

 

Loady =>  Something's clearly wrong with your parity generation process.    I would cancel your parity generation;    Unassign the parity drive;    Reboot the server;    ... and then re-assign the parity drive and restart the generation.

 

It's difficult to assess just how long it might take to create the initial parity, but with your mix of drives, assuming they're all attached to motherboard SATA ports or a PCIe x4 card, I'd expect a parity check to take on the order of 8-9 hours;  and an initial parity generation to take no more than twice that.

 

[itimpi]

its not a check, it is a new parity drive, a pre cleared 3tb jobby..so its writing the parity data..is this why its so slow ?..but even so.. 18 days !!..come on

I know I said parity check - but I meant calculate new parity after a new parity drive has been installed or you have told unRAID to recalculate parity.  Parity checking is faster II am guessing because no writes are involved).

[loady]

just logged into server to facilitate reboot and reassignment of drives as you suggested and before i did so i noticed that the time has reduced to 14 hours...the server uptime is down to a few hours so i can only assume it has rebooted itself at some point as i never did...weird..but hey, its doing its stuff.

 

Thanks all.