New disk stress testing (burn in)


Recommended Posts

Does UnRaid provide a way to stress test a new drive? I'd like to test each disk for, say, 24 hrs hard labor before using it for data. - I found 'preclear', buts it seems to be un-supported.

Note, I've not yet installed UnRaid, i'm waiting for some disks to arrive in the post....

Link to comment
27 minutes ago, witalit said:

the dev is no longer working on it.

 

He is, sort of. Like everyone else apart from the small team at LT, he's a volunteer and real life sometimes gets in the way. The trouble with the Preclear plugin is that the last time he was called away it was left in a non-working state due to updates made to unRAID and the GUI - in fact, it interfered quite badly with the basic functioning of the system and many people lost faith in it and uninstalled it.

 

Preclearing disks has a long and controversial history amongst unRAID users. It started life as a bash script at a time when adding an un-prepared disk to an existing array took the array off-line for several hours while the disk was cleared. A cleared disk is one that is filled entirely with zeros and such a disk can be added to an existing array without invalidating parity - hence the idea of pre-clearing a disk in advance in order to allow it to be added without interrupting the array's availability. The script evolved, as useful things do, into a tool that not only cleared a disk but read back all those zeros to make sure they had been written correctly. In order better to detect bad sectors a optional full read pass was also added before the clearing pass, so read-write-read became the norm. Many people applied the procedure several times over and a routine for comparing SMART parameters before and after each pass was added. The script had evolved into a disk testing and stressing tool. More recently, in keeping with the modern GUI, a plugin was written that removed the need for typing in a command terminal and potentially making a mistake and zeroing the wrong disk. More recently one of the core functions of unRAID itself was overhauled and adding a new disk to the array became much easier. The disk still needs to be cleared before it can be added but the process takes place without affecting the array's availability. At that point the original purpose of Preclear, both the script and the plugin, became redundant, though their potential use as disk testing tools remained.

 

Over the years, as new Linux kernels and tool-sets have been used both the original script and the plugin have needed patches and maintenance in order to keep them working. If that doesn't happen problems arise and earlier this year the general advice was that preclear was dead - use another tool. The trouble is that other tools fall into two categories and neither is ideal. On the one hand there are the tools that hard disk manufacturers or third parties supply that run on other operating systems, such as Windows. On the other there are Linux tools that run from the command line. I use the badblocks command that is included in the unRAID distribution and find it both thorough and effective. The downside is that it involves typing and the consequent possibility of making a mistake and over-writing the wrong disk. I thought badblocks was an ideal candidate for wrapping into a simple plugin but I chose a bad time to make the suggestion. The broken state of the Preclear plugin was causing a lot of support issues for another plugin and emotions were running high on that day and I received a mauling for my "stupid" suggestion. The Preclear plugin itself was marked as incompatible. It has since been updated but I don't know its current state.

 

I test all disks before I use them and I use badblocks from the command line in destructive read/write mode:

badblocks -wv /dev/sdX

This writes and then reads different bit patterns in four passes and leaves the disk full of zeros. This is almost the same state as a precleared disk, except that it lacks the "signature" necessary to allow it to be introduced into the array without being cleared again. If I had the skill needed to write a plugin I'd write one based on badblocks with options for destructive read/write mode that automatically adds the precleared signature, for non-destructive read/write mode and for read-only mode. That would make it easy to use and safer. But I wouldn't dare suggest that someone else might like to rise to the challenge again.

Link to comment
1 hour ago, superloopy1 said:

No problems at all.

 

Assuming you're referring to the plugin, I no longer have it installed but looking at the syslogs of people who clearly do shows a lot of spamming by a component called rc.diskinfo that makes the logs difficult to read. Have you noticed that yourself?

Link to comment
 
Assuming you're referring to the plugin, I no longer have it installed but looking at the syslogs of people who clearly do shows a lot of spamming by a component called rc.diskinfo that makes the logs difficult to read. Have you noticed that yourself?
Cant say i have but thereagain i dont take much notice of the syslog entries either. Unless of course it fills due to the aforementioned 'spamming' as you call it which i've not experienced since reinstalling the plugin. It serves my purpose, is working aok so it stays for the present.

Sent from my LG-D855 using Tapatalk

Link to comment
1 hour ago, Zonediver said:

I use unraid since 8 years now and never did a preclear on any disk...

But what i "always" do is to zero the first 100GB of each disk i connect to unraid - thats all.

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear.

Link to comment
2 hours ago, itimpi said:

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear. 

 

That is not completely correct.

Unraid already recognizes the disk as empty and inserts it into the array without deleting it. The disc is just formatted and available immediately.

 

 

 

 

Edited by Zonediver
Link to comment
5 hours ago, Zonediver said:

I use unraid since 8 years now and never did a preclear on any disk...

But what i "always" do is to zero the first 100GB of each disk i connect to unraid - thats all.

Zeroing the first 100GB doesn't guarantee a clear disk.

 

1 hour ago, Zonediver said:

 

That is not completely correct.

Unraid already recognizes the disk as empty and inserts it into the array without deleting it. The disc is just formatted and available immediately.

Unraid only skips the clearing process when adding a drive to an existing parity protected array if it detects a valid preclear signature.

 

"Empty" is not a well defined term. To some people, it means a drive with no partitions defined, to others, it means a valid partition table, with formatted partitions containing no file references. In no case is "empty" a valid term to describe  a precleared drive.

 

4 hours ago, itimpi said:

That is a perfectly acceptable approach, but in such a case you need to wait until unRAID itself clears the disk before it is ready to be used.   This is less of an issue than it used to be as you can now continue to use the array while the clear is happening whereas before unRAID v6 the array was offline during the clear.

 

This is completely accurate. Unraid will never skip the clearing process without a valid preclear signature on the drive. "Empty" drives do not have the preclear signature, and are not guaranteed to be full of zeroes.

 

 

 

Link to comment

Lots here that has gone way over my head . I'm sticking with the preclear plugin until such time as it fails me. So far, every disk that hits my array has had a 3 cycle 'burn in' and, touch wood, no failures to date in over 4 years That'll do for me ... each to their own.

 

Sent from my LG-D855 using Tapatalk

 

 

 

 

Link to comment
On 5/15/2018 at 11:55 PM, John_M said:

 

He is, sort of. Like everyone else apart from the small team at LT, he's a volunteer and real life sometimes gets in the way. The trouble with the Preclear plugin is that the last time he was called away it was left in a non-working state due to updates made to unRAID and the GUI - in fact, it interfered quite badly with the basic functioning of the system and many people lost faith in it and uninstalled it.

 

Preclearing disks has a long and controversial history amongst unRAID users. It started life as a bash script at a time when adding an un-prepared disk to an existing array took the array off-line for several hours while the disk was cleared. A cleared disk is one that is filled entirely with zeros and such a disk can be added to an existing array without invalidating parity - hence the idea of pre-clearing a disk in advance in order to allow it to be added without interrupting the array's availability. The script evolved, as useful things do, into a tool that not only cleared a disk but read back all those zeros to make sure they had been written correctly. In order better to detect bad sectors a optional full read pass was also added before the clearing pass, so read-write-read became the norm. Many people applied the procedure several times over and a routine for comparing SMART parameters before and after each pass was added. The script had evolved into a disk testing and stressing tool. More recently, in keeping with the modern GUI, a plugin was written that removed the need for typing in a command terminal and potentially making a mistake and zeroing the wrong disk. More recently one of the core functions of unRAID itself was overhauled and adding a new disk to the array became much easier. The disk still needs to be cleared before it can be added but the process takes place without affecting the array's availability. At that point the original purpose of Preclear, both the script and the plugin, became redundant, though their potential use as disk testing tools remained.

 

Over the years, as new Linux kernels and tool-sets have been used both the original script and the plugin have needed patches and maintenance in order to keep them working. If that doesn't happen problems arise and earlier this year the general advice was that preclear was dead - use another tool. The trouble is that other tools fall into two categories and neither is ideal. On the one hand there are the tools that hard disk manufacturers or third parties supply that run on other operating systems, such as Windows. On the other there are Linux tools that run from the command line. I use the badblocks command that is included in the unRAID distribution and find it both thorough and effective. The downside is that it involves typing and the consequent possibility of making a mistake and over-writing the wrong disk. I thought badblocks was an ideal candidate for wrapping into a simple plugin but I chose a bad time to make the suggestion. The broken state of the Preclear plugin was causing a lot of support issues for another plugin and emotions were running high on that day and I received a mauling for my "stupid" suggestion. The Preclear plugin itself was marked as incompatible. It has since been updated but I don't know its current state.

 

I test all disks before I use them and I use badblocks from the command line in destructive read/write mode:


badblocks -wv /dev/sdX

This writes and then reads different bit patterns in four passes and leaves the disk full of zeros. This is almost the same state as a precleared disk, except that it lacks the "signature" necessary to allow it to be introduced into the array without being cleared again. If I had the skill needed to write a plugin I'd write one based on badblocks with options for destructive read/write mode that automatically adds the precleared signature, for non-destructive read/write mode and for read-only mode. That would make it easy to use and safer. But I wouldn't dare suggest that someone else might like to rise to the challenge again.

 

 

Interesting read thanks for that. Yes of course I don't blame the dev, I thank him for the great script he created.. but its just a shame no one else can pick it up :-).

Link to comment
3 hours ago, witalit said:

but its just a shame no one else can pick it up

 

You'll see from the tone of the discussion in the support thread (it starts around page 74) that many seasoned users and potential co-developers consider it toxic, which is a shame. I don't comment in that thread any more.

Link to comment
1 hour ago, John_M said:

 

You'll see from the tone of the discussion in the support thread (it starts around page 74) that many seasoned users and potential co-developers consider it toxic, which is a shame. I don't comment in that thread any more.

 

Ouch - I had to break reading after a while. Felt like a bit of a witch hunt there.

 

When debating what is best processor, best way to test a disk, best ... there are quite often multiple views. And quite often the different views have merit.

 

It isn't as simple as claiming that there is no use or need testing a disk  before starting to use it. The clear functionality in unRAID just makes the drive content compatible with use in the unRAID array - it represents a quite limited test of the drive. And a significant percentage of drives dies during the first days or weeks.

 

Not sure if unRAID is a hard environment to write plugins for. The basic tests done by the plugin are quite simple to implement. Most work would be code to find which drives aren't already claimed by the array, cache or UD and in a safe way make sure the user only starts data-destructive tests on the intended drives, and fixing an interface that can display progress and final result.

Link to comment
  • 1 month later...
On 5/18/2018 at 7:22 PM, pwm said:

It isn't as simple as claiming that there is no use or need testing a disk  before starting to use it. The clear functionality in unRAID just makes the drive content compatible with use in the unRAID array - it represents a quite limited test of the drive. And a significant percentage of drives dies during the first days or weeks. 

Would a preclear help us to find out possible issues in brand new disks and somehow replace in time the disks with Early "Infant Mortality" Failure?

I will receive soon 3x WD Red 4TB disks and I am wondering if does make any sense to test the disks thoroughly with preclear before first use or it will not help at all.

 

image.jpeg.e928f3f23792e4f4f12b9635232fc0f1.jpeg

Source: https://www.extremetech.com/computing/170748-how-long-do-hard-drives-actually-live-for

 

 

Edited by FlorinB
Link to comment
2 minutes ago, FlorinB said:

Would a preclear help us to find out possible issues in brand new disks and somehow replace in time the disks with Early "Infant Mortality" Failure?

I will receive soon 3x WD Red 4TB disks and I am wondering if does make any sense at all to test the disks thoroughly with preclear before first use or it will not help at all.

It is a good idea to test the disks before use, but preclear is just one way of doing that. You can use wddiag on a windows box, or any of the disk exercise and testing programs available.

 

With all the angst surrounding the preclear plugin, I think the most prudent thing is to use the patched command line version if you are comfortable with the unraid console, or use windows tools if not.

 

Whatever test method you use, be sure to keep the drives physically protected from impact and cool for the duration, if that means temporarily mounting them in another pc, so be it.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.