Jump to content
Biggy2872

Re: preclear_disk.sh - a new utility to burn-in and pre-clear disks for quick add

1381 posts in this topic Last Reply

Recommended Posts

I am using putty to telnet and run the script...

 

Is it supposed to update the complete %, elapsed time, temp, bytes etc in real time while running?

 

Thanks

 

Well pretty sure the drive is not liking unraid anymore... completely locks up when any sort of constant read/write is done (perm script, preclear etc).

 

Still would like to know if the status updates in realtime... though I guess I will find out when I get my replacement.

Share this post


Link to post

I'm about to buy another 2 4TB drives and intend to preclear them both at the same time.

 

I've got a machine that can be dedicated just to preclearing and it will be running virtually stock unRAID 5.x apart from the addition of a few small-footprint apps like unMENU and screen.

 

It has 3GB of RAM. The last time I precleared a single 4TB drive, on a machine running a ton of add-ons, but with 8GB of RAM, I was getting lots of memory errors (but no crash of the preclear process). I imagine that's because I was running out of lowmem while using the other apps.

The command line I used was:

 

preclear_disk.sh -r 65536 -w 65536 -b 25600 /dev/sdb

 

Can someone please advise what would be the optimum or suggested command line parameters to preclear 2x 4TB drives with 3GB RAM?

I guess you'd want a combination of speed + memory safety.

 

Cheers!

Share this post


Link to post

I'm about to buy another 2 4TB drives and intend to preclear them both at the same time.

 

I've got a machine that can be dedicated just to preclearing and it will be running virtually stock unRAID 5.x apart from the addition of a few small-footprint apps like unMENU and screen.

 

It has 3GB of RAM. The last time I precleared a single 4TB drive, on a machine running a ton of add-ons, but with 8GB of RAM, I was getting lots of memory errors (but no crash of the preclear process). I imagine that's because I was running out of lowmem while using the other apps.

The command line I used was:

 

preclear_disk.sh -r 65536 -w 65536 -b 25600 /dev/sdb

 

Can someone please advise what would be the optimum or suggested command line parameters to preclear 2x 4TB drives with 3GB RAM?

I guess you'd want a combination of speed + memory safety.

 

Cheers!

I wrote it, and I would not even guess.  I would not use more than -b 2000 personally.  You are asking it to read in 1.6 GB chunks.

 

65536 * 25600

 

 

Share this post


Link to post

OK, thanks. Well, if I'm running 2 preclears at once, I think halving that number seems safer. I'll try it when the time comes.

 

Cheers.

Share this post


Link to post

I'm not sure what the defaults are (Joe will know) ... but I've NEVER used any parameters except the disk and just let preclear use its defaults, and have not had any issues with as many as 4 pre-clears at once while still using the system [although I don't use any CPU or memory intensive add-ons].

 

i.e. a simple preclear_disk.sh /dev/sdx

 

Share this post


Link to post

Changing the parameters significantly decreases the preclear time on large drives.

 

I have heard of OOM issues when running multiple preclears on 4TB drives. It may be that OOM is all about lowmem and difficult to work out.

Share this post


Link to post

Hi Joe, found a quirk.  Running unRaid 5.0-rc16c and preclear 1.13.

 

In the unRaid disk settings, set Enable Auto Start to No, then reboot.  Run preclear_disks.sh -l.

 

All disks, whether assigned to the array or not, are listed as available for clearing.

 

Start the array.  Only the correct disks are listed for clearing.

 

Stop the array.  The correct disks are still listed.

 

Share this post


Link to post

Hi Joe, found a quirk.  Running unRaid 5.0-rc16c and preclear 1.13.

 

In the unRaid disk settings, set Enable Auto Start to No, then reboot.  Run preclear_disks.sh -l.

 

All disks, whether assigned to the array or not, are listed as available for clearing.

 

Start the array.  Only the correct disks are listed for clearing.

 

Stop the array.  The correct disks are still listed.

I'll check it out.  I have a server running 5.0rc16c.  Thanks for the report.

Share this post


Link to post

Joe -- I mentioned this a couple weeks ago -- guess you missed it.

 

By the way, the behavior is the same even if the array is set to Autostart.    Until the Web GUI is actually accessed, preclear lists all of the disks.

 

 

Share this post


Link to post

Joe -- I mentioned this a couple weeks ago -- guess you missed it.

 

By the way, the behavior is the same even if the array is set to Autostart.    Until the Web GUI is actually accessed, preclear lists all of the disks.

 

 

Just encountered this today - and your results are consistent to mine.

 

Set to autostart. Don't touch the web UI at all. Preclear lists all disks, but doesn't list my disk that is available for preclearing

 

Load the web UI, do another preclear list, and expected behavior.

Share this post


Link to post

Hi Joe et al!

 

I had ordered 3 drives to be delivered today, from 2 different vendors. One of the vendors screwed up and their two drives won't be here until tomorrow. Therefore I am going to start preclearing the first 4TB drive today.

 

I have a machine currently dedicated to preclearing, so it has no drives in an array, but is running unRAID. The BIOS has been upgraded to support hotswapping drives.

 

So my question is, I know unRAID per se doesn't support hotswapping, but if I'm running a preclear on one drive, can I hotswap (insert/add) another drive and start a preclear on that? Has anyone tried it?

 

Cheers,

 

Neil.

Share this post


Link to post

So my question is, I know unRAID per se doesn't support hotswapping, but if I'm running a preclear on one drive, can I hotswap (insert/add) another drive and start a preclear on that? Has anyone tried it?

UnRAID does not support hot swapping of array drives while the array is started.  However as long as your hardware + BIOS supports it there is no problem with hot swapping array drives with array stopped, or drives that are not part of the array at any time.

Share this post


Link to post

Hi,

 

Recently started to get unraid setup with my N54L, having never used it before I was following the guide on the wiki and regarding the preclearing, I used the following command:

 

preclear_disk.sh -r 65536 -w 65536 -b 2000 -A /dev/sdX

 

Is this going to be a problem? As after I set the pre-clear's going, I read somewhere that I don't need the "-A" flag as I am running the latest beta?

 

Thanks

Share this post


Link to post

So my question is, I know unRAID per se doesn't support hotswapping, but if I'm running a preclear on one drive, can I hotswap (insert/add) another drive and start a preclear on that? Has anyone tried it?

UnRAID does not support hot swapping of array drives while the array is started.  However as long as your hardware + BIOS supports it there is no problem with hot swapping array drives with array stopped, or drives that are not part of the array at any time.

 

FYI, this seems to have worked. I inserted the two other drives and the first drive is continuing to preclear. Good times!

 

lg7renn.png

Share this post


Link to post

Hello there...

 

I'm running the preclear script (v1.13) on a 4TB HGST Deskstar HDS724040A through Screen in a telnet session. (5.0-rc13)

 

The issue is the telnet windows stopped receiving updates. After an hour or so the screen just stop reseting the temperature, the elapsed time clock, etc. I tried twice running it in this drive with the same result but at different times. Fisrt when writing zeroes and the second time while the initial reading. The first time I thought the script halted, I panicked, and I restarted the box. The second time I looked deeper - I still see the preclear_disk process taking CPU time at intervals and I also see the dd command doing the same. Is there any other process I should check before assuming the script failed? Have anyone else experienced this before? I tried searching and I found only one other thread with a similar situation but it didn't help much. Should I just let it run until the light in the box go off?

 

Your advice is greatly appreciated.

Share this post


Link to post

You do, I presume, know that it's fairly long between updates ... so be sure you wait at least several minutes before concluding they've stopped.

 

But the best way to resolve this is simple:  don't do a preclear in a Telnet session.    Either use Screen (install it through UnMenu);  or simply attach a keyboard and monitor to the UnRAID box and do it at the local console (by far the best way).

 

Share this post


Link to post

By the way, there's a very simple way to "attach a monitor" to your UnRAID box if it's located anywhere near a PC you normally use ==>  Just run a long (15-50') VGA cable from the UnRAID box to your desktop (a 50' cable is only $13.88 at Monoprice);  and use a little 2-way video switch to select between that PC and your normal one.    You don't need to keep a keyboard connected either -- you can hot-plug a USB keyboard and the system will recognize it fine.

 

I do exactly that -- when I need to do something at the console level I just plug in a USB keyboard;  then go switch the nearby PC's display to the UnRAID input; and I'm ready to go.

 

Share this post


Link to post

Actually this morning I resumed Screen and everything seems to be right. Updates are showing in intervals as usual but after a while it stopped receiving them again... very weird. I'm just letting it run until HD activity light goes off.

 

 

 

Share this post


Link to post

Discovered that I am unable to stop the array while the preclear is running with UNRAID version 5.0 (final). It hangs on the sync command.

 

Cancel the pre-clear, UNRAID sync command completes.

Share this post


Link to post

Discovered that I am unable to stop the array while the preclear is running with UNRAID version 5.0 (final). It hangs on the sync command.

 

Cancel the pre-clear, UNRAID sync command completes.

 

Interesting result.  Were you trying to Stop the array or Shut Down ??

 

I thought pre-clear was an entirely separate Linux task, so I'm surprised it hangs UnRAID.

 

Share this post


Link to post
Interesting result.  Were you trying to Stop the array or Shut Down ??

 

I thought pre-clear was an entirely separate Linux task, so I'm surprised it hangs UnRAID.

 

I was trying to stop the array, invoked by the GUI. It is repeatable for me - reseted the box twice before I realized what was going on.

Share this post


Link to post

I've never tried to stop the array during a pre-clear, so hadn't noticed that.    I wonder if it's a long-standing problem, or something new with v5.0

 

Yet another good reason to use a separate machine for pre-clears.

 

Share this post


Link to post

I have a known working drive. I know, I know...

 

Anyhow, is there a "quick" preclear so I can add it to the array quickly?

Share this post


Link to post

Run it with the -n parameter and it will only zero the drive and write the signature.  I do that all the time when I transfer drives from one unRAID server to the other.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.