RobJ

It means the drive tested each of the 8 sectors, when written zeroes to, and decided they were fine so put them back online. It doesn't mean those 8 are perfect, although they could be, but for now the drive believes each of them can be trusted. The drive appears to be fine now, but not perfect. The best advice we give users when this happens is to Preclear it one or two more times, and see if anything changes. If no further changes, then the drive should be good. All other numbers look OK.

Nothing to worry about, at all. I've made some suggestions to Joe L about removing these near_thresh's from the reporting, but I suspect he's been too busy lately.

It takes time to understand SMART reports, and the inconsistencies within them. For the error rate attributes, the important number is the VALUE, which for both is perfect at 100. Their RAW numbers are internal use only, ignore them. For other attributes, the RAW numbers are sometimes useful, but you have to understand how each attribute is used. They are all different.

What is the significance of installing in /boot/plugins? Does it affect the way that the plugin operates? As far as I can tell, it is only a simple way to control timing. Some things have to be installed before others. So far, it only appears to have been used to load base plugins, the platforms upon which other plugins will install. The distinction 'system' has so far just meant that it should load before other plugins. Tom, dlandon, and others may know more though.

High-fly-writes don't have a threshold or valid scaling routine. The RAW is an actual count of how many high-fly-writes have occurred, but VALUE is just 100 minus the count in RAW until it hits one where it will stay. It never reaches zero, but the RAW count will continue to increase. The THRESH number is zero, which basically means it is not used, since it cannot be reached, which also means this attribute can never FAIL a SMART test. Joe's point is good though. 77 high-fly-writes already (only 50 operational hours) is rather unusual, so probably something to monitor. However, I do not know of a single case where a drive failed with a very high or growing high-fly-writes count, so it does not seem to be a significant item. It was added several drive generations ago to the SMART attribute tables of many drives, but always with the same dummy scaling routine, and always with a THRESH of zero. I prefer to call it an experimental attribute, perhaps one they wanted to monitor to see if there was any association with drives failing. So far, I haven't seen one, and the fact the handling of it hasn't changed seems to indicate it's not an important factor to them either. I would use the drive without any concerns. There are no other indications of issues at all.

I was going to suggest a wheel chock, but it's too low. Another possibility, more practical, use it as a decoy for thieves. Set it on an appropriate shelf, visible, with a label indicating "All my financial info", and underneath that the word "(hidden)" to totally interest any hacker. Load it with gigabytes of trash videos (any kind) and other garbage, and that should give you a big laugh if it is ever taken. It is really really time-consuming and frustrating trying to recover data from a bad drive, especially if it is getting worse while you work on it. Or for the less technical thief, label it as "Very private videos, hidden on here". Should waste a lot of their time.

Drive is fine. Final temp is 33, nothing wrong with that.

Yeah, lots of bad sectors. I would start a Preclear but skip the Pre-read (use preclear_disk.sh -W), since you already know there are issues. The write-zeroes phase should force the drive to deal with every sector, remapping them if necessary. It will probably be slow. Then the Post-read will test the drive, provide a final SMART report, with hopefully all Pending sectors fixed. From the last report and the Preclear report, you should be able to decide what to do with the drive. I do apologize for my tone above, somewhat myopic, only looked at the one post.

Edit: I apologize, I've just noticed elsewhere that you have included a syslog. [respectful chide mode on] It was hard to know how to respond. You have a drive that had been red-balled, and was now running extremely slowly, and yet it does not appear that you thought to check or include a SMART report and the syslog, which would normally be stuffed with reasons for the problem. I do understand this may be your first drive issue, but it still was very disappointing. I'm not blaming you, those of us who are veteran UnRAID users, including LimeTech staff, have to take some responsibility for not doing a very good educational job. We need to do a much better job making sure ALL users, both new and experienced, know what a syslog is, why it's so useful, and how to capture it, and the same with SMART reports. They are admittedly hard to interpret, but there are many here ready and willing to help with that. If we had done a good job, and a problem like this arose, every users first thought would be "something is wrong, let's see what the syslog says". Perhaps we need more stickied troubleshooting help posts. Perhaps the initial ReadMe needs an 'UnRAID Troubleshooting 101' section... There have been a number of cases like this lately, where the syslog was needed yet the user never thought of it. This too was disappointing. It's like someone has a shed on fire, and someone else says "I wonder if that strange foreign family down the street did this", leading to rumors and distrust. Please don't cast aspersions on something without a single shred of evidence. Beta 9 is working fine for most, and I don't know of any reports of slow drives with it, and slow drives are a relatively common issue. I'd venture to say that at this very moment, around the entire world, there are hundreds of thousands of very slow drives occurring, using many different operating system versions. Many of them are problems with the drives, and the others are problems with the interface to those drives. A syslog or comparable diagnostic will clarify the issue. [chide mode off]

Looks good.

It does look good, Precleared correctly too, ready to go on line.

Using /mnt/ cannot work, because includes must be top level folders on any of the data drives or the cache drive. The test is something like checking for the existence of /mnt/disk1//mnt/, which could not exist (note the double slash).

You are running Dynamix, check its settings, perhaps it is set to run CacheDirs also?

It looks like something is configured to start CacheDirs (using the at command) at 8:40pm??? The spin downs don't seem related.

SMART report looks fine. It's hard to say what the times should be for a 6tb drive, until other reports come in. Doesn't look wildly wrong though. You used Preclear 1.9, which is very old, probably too old for very large drives. I'm not sure you can trust the result on drives over 2TB using any Preclear less than 1.13, and perhaps Preclear 1.15 (the latest). You should definitely upgrade Preclear, and you may want to try it again. At least, try a Preclear -t test on it, with the latest Preclear.

The first Preclear forced the drive to deal with all of the marginal sectors, and when it tested them the drive found that ALL of the sectors were physically good, so put them back online. The drive is fine.

All 3 drives look great.

That's probably the best explanation of the various SMART attributes. Just remember that it is general info concerning many drives from many vendors, both older and newer. Each drive has its own differing subset of the attributes, and they may not be used consistently. THERE IS NO STANDARD USAGE! I started such a wiki tool, found at Understanding SMART Reports. Unfortunately, like almost all of my projects (I'm severe ADHD and either Aspergers or brain damaged), it was not finished, is incomplete especially in the attribute section, but it will give you (I think!) a good background on SMART and SMART reports. At the bottom of the wiki page are more reference links about SMART. At the moment, the wiki is locked, read-only, but once editable again, I'll try to complete the page. I would like to ask Joe L to consider adding the changes I suggested, that included among other things, removing the spurious 'near_thresh' warnings from the Preclear reports. We've understood the problem since almost when they first began appearing, and I *think* we all agree they are a source of confusion.

Since it's happening on the last data drive to be scanned, I suspect it is trying to start a new scan before the current one is finished, that is, timing out on the scan polling interval. As an experiment, try increasing the minimum scan time (eg. /boot/cache_dirs -w -m 7). If that doesn't help then increase it to 8 seconds, then 9, then 10 seconds. If it does appear to be working correctly, then decrease it a second at a time until it doesn't, then change it back to the lowest minimum scan time that does work.

Correct. Just a question ... Those of us with SMART and Preclear experience could easily see that the Preclear was successful and the drive is excellent, but many users like yourself don't have that same experience and often post here with their results requesting confirmation of success. That would seem to indicate we could do a better job of communicating. In your opinion, are the Preclear reports too confusing, or just too much information? What if they started with something like - *** In summary *** Preclear was successful, drive is excellent. - or - Preclear was successful, but see Details for very minor issues - or - Preclear was not successful, see Details for issues *** Details *** [and full Preclear and SMART reports continue here]

RobJ thank you for your help! I attach the entire preclear report. As you've probably already determined for yourself, the drive is currently showing 22,702 hours of operation. So while there was a problem at 3535 hours, a long time ago, it apparently wasn't serious and was completely resolved by the drive, without having to remap a single sector. Enjoy your drive, assign it anyway you like, as its SMART report looks fine, and the Preclear report was clean.

We need to see the entire SMART report. This excerpt just shows the ATA error section, displaying the last 5 ATA errors, no matter how long ago they occurred. In this case, the drive reported at least 5 UNC errors, bad sectors that probably affected the reallocated sector counts (Reallocated_Sector_Ct) and/or the current pending sector (Current_Pending_Sector) counts. However these errors occurred at the 3536 and 3535 operational hour points, and you need to check the current Hours (Power_On_Hours) number, in order to know if that is recent or a very long time ago. If a long time ago, ignore it. If recent, then they should have been reported by Preclear and/or the SMART report as changes in Reallocated_Sector_Ct and/or Current_Pending_Sector. Note: this may have been a spurious error, as all 5 displayed are at the same 'apparent' sector number, with mask of 0x0fffffff, "UNC at LBA = 0x0fffffff = 268435455".

Might be time for a simple explanation of CacheDirs, for newer users. In the beginning was UnRAID, and it was good. And there was a vast expansion of space. And drives and User Shares were brought forth, and made shareable by all creatures. And a way to spin down drives was made, for days of rest. But our media players and our needs insisted on constant access, and the drives had no rest. So Joe created CacheDirs to keep directories loaded, and let the drives rest. with apologies to any biblical scholars! CacheDirs is a script of simple shell commands to periodically access selected directories, thereby keeping them loaded in kernel buffers. In DOS(1) and Windows, you could do the same in a batch file of 'dir' commands(2), looping back again and again to repeat the same dir commands, but without displaying anything on the screen. Why would you do that? The kernel has limited buffer space, used for all disk I/O (both the folder info and other metadata, and the actual file data buffers). When more buffer space is needed, the kernel drops anything that has not been accessed recently. By CacheDirs constantly accessing the drive directories, we indirectly force the kernel to keep them loaded in RAM, not drop them. Apart from a PID file in a single system folder (to keep CacheDirs from installing itself more than once), CacheDirs does not write anything anywhere. It also does not directly manage memory, but depending on how it is configured, may load a few small directories or load many large directories, thereby causing the use of a little memory or a lot of memory. UnRAID version 6 is 64bit, with almost no limit to memory for CacheDirs. UnRAID version 5 and lower use 32bit kernels, that divide memory into LOWMEM and HIGHMEM. LOWMEM is typically a little less than 900MB, no matter how much RAM is installed. The directory buffers are limited to a small portion of LOWMEM, and that explains why it is possible for CacheDirs to apparently cause "out of memory" issues in UnRAID v5 and lower, if too many files and folders are cached (use Excludes and the depth parameter to limit them). (1) A simple DOS version of CacheDirs, CacheDirs.bat: @echo off :top dir folder_1 >null dir folder_2 >null dir folder_3 >null ... dir folder_n >null sleep 3 seconds goto top (2) Instead of the 'dir' command, CacheDirs uses the Linux 'find' command, similarly to the way 'dir' works. Written fairly quickly, with little checking; please inform me of any inaccuracies, and I'll correct them.

In addition the cat /proc/meminfo command may be useful, because it shows low and high memory totals. It also shows Slab, SReclaimable, and SUnreclaim, which may be a way to see what cache memory can and cannot be reclaimed, on request. I'd be interested in what testers find with these numbers, especially when comparing 32bit and 64 bit versions, under differing loads. I've added more info to the Console commands wiki page, hopefully to get more users started.

Linux memory management is very different than Windows memory management, which is what most of us are used to. We need to read more closely what WeeboTech is saying: (and I hope we hear more from him) We're used to thinking of memory in a capitalistic sense - you need memory, so you get it and now you own it and no one else can use it. Linux seems more like a hippie sharing economy - you need memory, so you use what you need until someone else needs some of it. That makes the memory reporting tools somewhat useless. The top command seems particularly useless because it doesn't even break down what is being used in lowmem as opposed to highmem. In my experience, the disk caches appeared to be stored in lowmem, which has always appeared to be limited to less than the first gigabyte only, typically 893MB or less. So comparing top results between 32bit and 64bit releases seems useless. And throwing more memory at a 32bit system only increased highmem, made no difference at all at alleviating any of the lowmem limitations. This statement is correct from a Windows standpoint, but not in the Linux world, since we have no idea how much is actually available if we make a request. What would be useful is knowing how much can actually be given back by all current memory consumers and pools, but that does not appear to be available by any tools I could find. Any memory pool must have some minimum requirement, beyond which it cannot give back. Some additional commands that may be useful are 'slabtop', 'slabtop -s c', 'vmstat', and 'vmstat -m'. They do break down the cache usage into its parts. Perhaps further research and testing here could use these to find particular numbers that are more indicative of OOM danger?