Frank1940

I doubt if this is really a preclear problem in-and-by itself. If you are running the latest Stable (or Release Candidate) version, you might want to post up this problem in that announcement thread and see if anyone has any thoughts there. If those other cores aren't being used, why not...

Am I correct in assuming that you have looked at CPU utilization to make sure that you don't have a bottleneck there? Does the Docker container eventually stop? Is it all Dockers? Oh, how many drives are you preclearing at one time? (I understand that (some?) preclear script is a bit of a memory hog.)

Why don't one of you start a thread in Feature Request section of the forum and post a link to that new thread here. Then each of you should post up your reason for wanting this feature. I seem to recall (very vaguely) that here are a number of video resolutions/modes baked into the standard Linux package. However, the details of how to access them completely escapes me but I remember that it was not really an easy thing to do. To open these modes up to the more casual user will probably require some sort of interface... EDIT: google set video resolution in linux command line to see what is involved.

I recall reading that @Joe L. was in contact with Tom at the time when he was writing the preclear script and Tom gave @Joe L. the 'magic cookie' on a confidential basis. (Tom has been known in the past to provide people with proprietary information and trust they will not reveal it to the general public.) I think that Tom (and LimeTech) feels that the majority of unRAID users are trustworthy and will bend over backwards to provide them with any information and service necessary to further the usability and functionality of unRAID. He does not seem to have a 'Not invented here' type of mentality toward innovation.

While I do not speak for LimeTech, I think one may safely infer that they are neutral on the subject. They do not feel that testing of new drives is necessary. Otherwise, they would be obligated to include a testing module as a part of the basic unRAID system. And 99%+ of the time, that is correct. Pretesting of hard drives are not required if they don't fail in the near term. (All Hard drives will eventually fail and it is only a matter of when...) However, what is your comfort level if you happen to get one of those outliers that fail in less then a week after you put the drive into your system? If you find this unacceptable, then you should be preclearing. If you are comfortable dealing with the problem, you scan skip oit.

A disk that was working fine in an other computer may have surface defects (in an unused area of that disk) that would render it unusable for unRAID! For some reason, the SMART attributes were not reported for this disk. It appears that their reporting was turned off for some reason. Perhaps, someone else might have some inslight into what your next course of action might be.

Here is a link to the update required for @Joe L.'s script to work with unRAID versions after 6.2.0. I have tested and verified that this modified script works several times but not with the most recent releases. The last time that I tested the script was with V6.5.0. https://lime-technology.com/forums/topic/12391-re-preclear_disksh-a-new-utility-to-burn-in-and-pre-clear-disks-for-quick-add/?page=53#comment-460592 Hope this helps...

+1. If you have a disk with data and it and you want to dispose of it by giving it away, selling it or tossing it in the trash, you probably want to remove any data that might be recoverable from the disk before doing so. The "erase' and the 'erase and clear' are the two ways provided to do exactly that. ('Erase and clear' is like 'belt and suspenders'...)

You can us the method described here: https://lime-technology.com/prerelease-support/how-to-install-prereleases-r8/ This does require that you install one of the stable releases first.

Disk shares are normally turned off!!! This is because there is a very easy way to lose data by copying between disk shares and User shares. See here: https://lime-technology.com/forums/topic/32836-user-share-copy-bug/#comment-316512 User Shares are the recommended method for all file operations If you are using disk shares, you had best be a experienced Linux user and understand exactly what you are doing and how to do it. ANY top level folder automatically becomes a User Share. It does not matter how it was created. If you are creating folder under Linux, you have to be careful about capitalization of Folder names. Linux considers movies and Movies to be two different folders/directories. Window only recognizes (and shows) only one of them! Plus, it is a turkey shoot as to which one. Any files in the other folder will be inaccessible!

Double check the smart report on this drive and see what it reports. You said the drive was used. What are the power-on hours? Did the seller give any reason for selling? By the way, if you are using USB for preclearing, you want to use a UBS3 port on the server and a UBS3 housing for the drive. UBS2 will be very slow...

And don't forget the size of the disks. If all of the servers in a section were equipped with 2TB disks when 10TB driveshave gone main stream and lack of storage space is becoming an issue, replacement of all the servers in a section of data center would make sense. It would also seem that when doing an upgrading on a massive scale, that it makes sense that the replacements be reconfigured and tested before placing them in service. Think about another factor, the cost of those two or three SATA cards are small change when you consider that some servers now have SIXTY hard drives in them! Here is a link to the Backblaze for their most recent server build: https://www.backblaze.com/blog/open-source-data-storage-server/ Note that they have even provide their cost analysis! You can buy one of these for a modest $12,849.40 (less hard drives)!

Plus, many of these are cards that were removed when big server complexes (or farms) with capacity issues were replaced with bigger, faster equipment using larger hard drives and faster CPU's and more memory. Much of the old gear was sold and some of it was savaged for parts. Many of these servers were built by companies like IBM, Hp and Dell. These are the used parts that you are looking at. The important point is that the gear being sold is not on the market because it worn out (or was giving operational problems) but because it was obsolete to do the current job required of it.

I can only tell you that I have seen more posted problems with the Marvell-based SATA cards then I have seen from using a used LSI card...

I would not be looking at the cheap 4-port cards. Most of them are based on the Marvell chip set and some users have had problems with them. The basic recommendation today is to go with the LSI cards. Many are getting them new and used from E-Bay. You can get an eight port card beginning at $50US that may require crossflashing or you could buy a ready-to-use card beginning at $70US. (As with all E-Bay purchases carefully vet your choice of vendor. I, personally, would avoid the China based vendors and get one from a vendor in my country.) You may only need four ports now but you will be getting the extra ports at a very small price per port. IF you do go with the cheap four port card, go with one that has a large heatsink on it. Most folks tend to use these cards to give a few more SATA ports in their PC's and only have one drive active at a time. unRAID, on the other hand, has long periods when every drive on the system will be running. All SATA chip sets run HOT and cooling can be a real problem!

Folks really need to understand that what a CRC error is that the data going in one end of a wire is not coming out of the other end of the wire with the identical data. It is not a software failure! It is a hardware issue! And it is relativity uncommon, but it is not totally unexpected. (Otherwise, there would be no need for checking using CRC!) The error itself is correctable, and that is always done. Better than 90% of the time, multiple error situations are cable related--Loose cables or crosstalk between cables being common causes. There could be problems in the decoding and checking hardware but again that is rare and can be a bear to figure out exactly where it is. (I have had a flakey SATA expansion card in the last six months that threw up a couple of hundred CRC errors in a few hours as well as an End-to-End error. So that possibility must always be kept in mind.) For these reasons, each one of you who has a problem should post up in a new thread about the issues in your system. Keeping track of several different investigations with only a slight chance of any two have the same cause and solution is very confusing.

Virtually all of the longtime users who are active on this forum encourage everyone to turn on the Notification System. The reason being is that it can alert the user that there are problems with the server before the system deteriorates to the point that data will be lost unless it can be restored from another source. HOWEVER, Notification requires that the user setup it up with things like E-mail addresses, mail server and other setup information for various other notification services that the user might be setup for. So, it is necessary that the Notification Service be turned off until these setups are completed.

I would suggest that you post up your Diagnostics file. If the GUI and SSH are unavailable then connect up a monitor and keyboard, login , and type diagnostics on the command line. That will write the diagnostics file in the logs folder/directory on your flash drive.

You can install the Tips and Tweaks plugin and on the 'Tweaks' tab, set the "Turn on PHP warnings?" to 'No' and that will suppress the PHP Warning: entries.

If you really want to make sure that everything is truly OK, you should run a second parity check (either correcting or non-correcting). It should finish with zero errors.

As I understand it, the CRC error count is a count of errors that occur between when the SATA data is encoded on the hard drive and when it is later decoded (a few microseconds later) on the SATA port on either the MB of a SATA expansion card. So it a basically an error that occurred in data transmission. It can be a simple random event. It can also be caused by tying all of the SATA cables together to make the inside of the case look 'neat'! (Tying cables together can cross-talk between the cables if the cables are not shielded and 99%+ of all cables today are not!) It can be caused by a bad cable or loose connection. (See here for an possible loose connection cause: https://support.wdc.com/knowledgebase/answer.aspx?ID=10477 Please note that If WD made a design to their SATA connector that must have been some reason behind it and other manufacturers may have done the same thing and not posted about it.) It could be a bad SATA controller on the MB but that is a most unusual occurrence. EDIT: A CRC error is not a fatal error in any sense of the word. The data will simply be resent. The only way it becomes a real problem is if a high percentage of the packets have errors. That could impact performance. It can also be a indicator of a potential fatal error in the future.

Results of that preclear test with your new version. NOTE: the End-to-End and the CRC Errors have exist for quite some time now and were the product of a bad SATA expansion card. This disk is now one that I currently use just for test purposes and has probably been through ten preclear cycles in the past couple of years. I might use it in an array in a real bind but with crossed fingers! New Text Document.pdf

That's OK. So glad you are back. I have already started with ver6.5.1-rc2. I am now 5+ hours into a 3TB drive. Will report back when done.

It would appear that he is back to working on it. This is the first few lines from the link above: <?xml version='1.0' standalone='yes'?> <!DOCTYPE PLUGIN [ <!ENTITY name "preclear.disk"> <!ENTITY repo "unRAID-plugins"> <!ENTITY author "gfjardim"> <!ENTITY version "2018.03.29"> <!ENTITY launch "Tools/Preclear"> <!ENTITY gitURL "https://raw.githubusercontent.com/&author;/&repo;/master"> <!ENTITY pluginURL "&gitURL;/plugins/&name;.plg"> ]> <PLUGIN name="&name;" author="&author;" launch="&launch;" version="&version;" pluginURL="&pluginURL;"> <CHANGES> ##&name; ###2018.03.29 - Fix: early unRAID 6.5 compatibility Thanks, @gfjardim, for your time and effort in writing and maintaining this plugin. It is so good to see that you haven't abandoned this plugin. I, for one, feel that preclearing a disk (While it does not often find a bad one) is a step that can prevent headaches a few days down the road when either (1) a disk that should have never been put into service in the first pl;ace starts throwing errors or (2) a disk fails from the infant mortality phenomenon.

Perhaps, "Force" is a bit strong. Setting up notification should really be a part of the standard setup procedure for unRAID. The problem with that is that there is basically no setup procedure to start unRAID. It will basically run from the minute it boots up. (I know that you don't get off that easy the first time you start up a Windows computer! And I would bet you don't with any other OS either!) Back when unRAID was a simple NAS device, this simplicity of setup may have been a nice feature but with the increased complexity of its capabilities, this overly simple approach needs to be rethought. Take a look at what the manual (From the link in the lower right corner of the GUI has to say about notifications and there is about as much information on most of the rest of the settings). http://lime-technology.com/wiki/UnRAID_6/Getting_Started#Notifications_Settings One thought would be, in a future stable release, there be a script that runs the first time it boots where the upgrading user is shown each current setup parameter with a bit of Help Text explaining what the parameter does. If an problematic setup parameter is detected, the script would point this out and encourage the user to fix it. In the cases where is a actually a clean install, the default setting could shown. EDIT: This is a rather simple proposal and, obviously finds a lot more input and refinement before it could (or should) be actually implemented.