Joe L.

Correct except that the number of disks you can concurrently pre-clear is more limited by your memory than anything else. It has nothing to do with the three disk limit of the free version of unRAID since you are never assigning the disks to the array. The three disk limit is how many disks you can assign to an unRAID array on the disk assignment screen, not how many you can attach to disk controllers in a given server. Joe L.

the reports all look fine. Yes, I would pre-clear the cache drive to ensure it does not have any issues. (not because it will save time when adding it to an array)

I am tempted to say no... I have spent some time browsing the forums on SMART results, what I have come to understand is that what your drive is doing could possibly be ok, but parts of the drive ARE failing so it comes down to wanting to take a risk... Looking at the prices of disks I would advise taking out this disk, using it for some other kind of storage (its not broken yet) and replace your unraid drive with a fresh one that comes out of multuple preclear cycles without any issues.. I would try another pre-clear cycle. Each cycle so far has uncovered additional un-readable sectors. If that continues, the drive is not one I'd want in my array. On the other hand, I've got one drive with 100 re-allocated sectors that has never changed from the first pre-clear I ran it through. Since it is stable, I trust it. Joe L.

all three drives look fine.

zip it, (they zip really well) or, use ext host as you described. For pre-clear results I really do not need to see the entire syslog. You can attach only the pre-clear reports as found in /boot/preclear_reports Joe L. Ok, got the report on both drives. thx in advance both look fine.

zip it, (they zip really well) or, use ext host as you described. For pre-clear results I really do not need to see the entire syslog. You can attach only the pre-clear reports as found in /boot/preclear_reports Joe L.

The third disk shows 38 sectors pending re-allocation. There would normally be none as the writing of zeros should have re-allocated all the sectors. There were no sectors re-allocated, so I'd suspect the 38 un-readable sectors were discovered in the post-read phase. (that is not good) I'd run another pre-clear on that disk. If it continues to show sectors pending re-allocation, I'd not trust it.

Those are the categories those attributes belong to. Not failure unless they also say FAILING_NOW on the same line. As an example, run-time-hours would be an old_age indicator of a disk. Un-correctable-disk-read-errors will be in a category of pre-failure. High run-time hours does not indicate the drive will fail, just that it is getting older. Un-correctable errors can occur at any age. A large number, or increasing numbers of them might indicate a pending failure (once the disk runs out of spare sectors to re-allocate in place of the un-readable ones) You just need to compare the normalized value with the failure threshold for any given attribute. That will tell you of the drive's health.

Not knowing what you have on your server, but theorizing. If each directory entry occupied 1000 bytes of data (I doubt this is the case, but again, just guessing it has to have the file name, the modification/access/creation dates, and permissions. ) If you had 100,000 files, the directory entries would occupy 100Meg of space in RAM. That is nowhere near 4Gig. Something else is spinning up the drives. In addition, I suspect it is less than 100 bytes per directory structure, so 100Meg should handle about a million directory entries with relative ease. Joe L.

I guess that isolates the issue to the only common hardware involved. (the disk itself) Not DOA, but Zombie. (keeps coming back from the dead) I would not trust my data on it. Not unless you want to keep power cycling the server to get to a file.

Lots of errors in communicating with the disk in the syslog. Many are CRC errors (bad checksums in communications with the disk) Mar 25 22:49:45 Tower1 emhttp: get_config_idx: fopen /boot/config/shares/Pix2012.cfg: No such file or directory - assigning defaults Mar 25 22:49:45 Tower1 emhttp: Restart SMB... Mar 25 22:49:45 Tower1 emhttp: shcmd (46): killall -HUP smbd Mar 25 22:49:45 Tower1 emhttp: shcmd (47): ps axc | grep -q rpc.mountd Mar 25 22:49:45 Tower1 emhttp: _shcmd: shcmd (47): exit status: 1 Mar 25 22:49:45 Tower1 emhttp: Start NFS... Mar 25 22:49:45 Tower1 emhttp: shcmd (48): /etc/rc.d/rc.nfsd start |& logger Mar 25 22:49:45 Tower1 logger: Starting NFS server daemons: Mar 25 22:49:45 Tower1 logger: /usr/sbin/exportfs -r Mar 25 22:49:45 Tower1 logger: /usr/sbin/rpc.nfsd 8 Mar 25 22:49:45 Tower1 logger: /usr/sbin/rpc.mountd Mar 25 22:49:45 Tower1 mountd[2091]: Kernel does not have pseudo root support. Mar 25 22:49:45 Tower1 mountd[2091]: NFS v4 mounts will be disabled unless fsid=0 Mar 25 22:49:45 Tower1 mountd[2091]: is specfied in /etc/exports file. Mar 25 22:49:45 Tower1 emhttp: shcmd (49): /usr/local/sbin/emhttp_event svcs_restarted Mar 25 22:49:45 Tower1 emhttp_event: svcs_restarted Mar 25 22:49:47 Tower1 kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x380100 action 0x6 Mar 25 22:49:47 Tower1 kernel: ata7.00: irq_stat 0x08000000 Mar 25 22:49:47 Tower1 kernel: ata7: SError: { UnrecovData 10B8B Dispar BadCRC } Mar 25 22:49:47 Tower1 kernel: ata7.00: failed command: READ DMA Mar 25 22:49:47 Tower1 kernel: ata7.00: cmd c8/00:08:47:ae:00/00:00:00:00:00/e0 tag 0 dma 4096 in Mar 25 22:49:47 Tower1 kernel: res 50/00:00:46:ae:00/00:00:00:00:00/e0 Emask 0x10 (ATA bus error) Mar 25 22:49:47 Tower1 kernel: ata7.00: status: { DRDY } Mar 25 22:49:47 Tower1 kernel: ata7: hard resetting link Mar 25 22:49:48 Tower1 kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Mar 25 22:49:53 Tower1 kernel: ata7.00: qc timeout (cmd 0xec) Mar 25 22:49:53 Tower1 kernel: ata7.00: failed to IDENTIFY (I/O error, err_mask=0x4) Mar 25 22:49:53 Tower1 kernel: ata7.00: revalidation failed (errno=-5) Mar 25 22:49:53 Tower1 kernel: ata7: hard resetting link Mar 25 22:49:53 Tower1 kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Mar 25 22:49:53 Tower1 kernel: ata7.00: failed to IDENTIFY (I/O error, err_mask=0x100) Mar 25 22:49:53 Tower1 kernel: ata7.00: revalidation failed (errno=-5) Mar 25 22:49:58 Tower1 kernel: ata7: hard resetting link Mar 25 22:49:59 Tower1 kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Mar 25 22:49:59 Tower1 kernel: ata7.00: configured for UDMA/33 Mar 25 22:49:59 Tower1 kernel: ata7: EH complete Mar 25 22:49:59 Tower1 kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x380100 action 0x6 Mar 25 22:49:59 Tower1 kernel: ata7.00: irq_stat 0x08000000 Mar 25 22:49:59 Tower1 kernel: ata7: SError: { UnrecovData 10B8B Dispar BadCRC } Mar 25 22:49:59 Tower1 kernel: ata7.00: failed command: READ DMA Mar 25 22:49:59 Tower1 kernel: ata7.00: cmd c8/00:08:47:ae:00/00:00:00:00:00/e0 tag 0 dma 4096 in Mar 25 22:49:59 Tower1 kernel: res 50/00:42:00:00:00/00:00:00:00:00/a0 Emask 0x10 (ATA bus error) Mar 25 22:49:59 Tower1 kernel: ata7.00: status: { DRDY } Mar 25 22:49:59 Tower1 kernel: ata7: hard resetting link Mar 25 22:49:59 Tower1 kernel: ata7: SATA link up 1.5 Gbps (SStatus 113 SControl 310) Mar 25 22:49:59 Tower1 kernel: ata7.00: configured for UDMA/33 Mar 25 22:49:59 Tower1 kernel: ata7: EH complete Mar 25 22:50:40 Tower1 kernel: ata7.00: exception Emask 0x10 SAct 0x0 SErr 0x380100 action 0x6 Mar 25 22:50:40 Tower1 kernel: ata7.00: irq_stat 0x08000000 Mar 25 22:50:40 Tower1 kernel: ata7: SError: { UnrecovData 10B8B Dispar BadCRC } Mar 25 22:50:40 Tower1 kernel: ata7.00: failed command: IDENTIFY DEVICE Mar 25 22:50:40 Tower1 kernel: ata7.00: cmd ec/00:01:00:00:00/00:00:00:00:00/00 tag 0 pio 512 in Mar 25 22:50:40 Tower1 kernel: res 50/00:ff:00:00:00/00:00:00:00:00/40 Emask 0x10 (ATA bus error) Basically, the disk is ceasing to communicate with the disk controller somewhere in the process of reading and writing to the disk. A power cycle seems to get it to re-initialize and communicate once more. The 10 steps in the preclear are being performed, but when the verification is performed, the expected values are not there. (because the disk stopped responding to commands somewhere in the middle) It could be a disk controller port issue, or a cable issue (noise on the cable causes CRC errors) or just a bad disk drive. It could even be a marginal power supply for that drive. The process of elimination is tedious, but I'd try a different PC first. If it fails there, it was the disk. Joe L.

exectly. The cache_dirs is just keeping the directory entries in cache by accessing them frequently. Your browser or whatever you are using to access the directory is opening up the media files. Those are not in the cache. (and unless you have a *very small* media collection, all your media files are very unlikely to fit in your RAM at the same time) Perhaps you can set a listing mode that shows text names only, and does not display thumbnail images of the media in the directories. The issue is in your file-explorer, not in how the directories are cached. The name of the script unfortunately is not "cache_all_my_hard_disk_content" as that is not possible in most cases. (I only have 512 Meg of ram on my original server, and 6TB of disks. No way will it all fit, even if I use small bits .)

The directory entries are ALREADY being cached. If your visiting the individual directories is accessing the files in the directory themselves, then that is why the disks themselves need to be accessed. Sorry, but there is nothing that will help you unless you know more of exactly what is being accessed. You can install the inotify tools package and see exactly what is being accessed. Then you can really see why your disks are being spun up. You can run the cache_dirs program with the -F option on the command line to see what it is doing. It will print to the screen the directories as they are being scanned. Joe L.

in a folder on the flash drive /boot/preclear_reports (I think)

I don't love the seek-error-rate. The normalized value of 38 is uncomfortably getting close to the SMART failure threshold of 30. (typically parameters start at 100, or 200 when factory new. Your other parameters are mostly 100, so I'm guessing that is the initial value.) It is an indication of drive wear, so if it gets worse over the next few months consider a replacement. That seek-error-rate might also explain the clicking...or vice versa... Of course, it could be an effect of the old firmware, but keep an eye on it.

It looks for the lock file once each time through it "find" loop. Have it loop faster ( by having it search fewer directories, or less deep in your hierarchy ) and it will terminate faster. Or, don't use it... Or... re-write it in C. It is only a shell script.

Interesting, but the preclear script does nothing that the unRAID array will do itself when calculating parity or re-constructing a failed drive. It reads and write to the failed disk. (exactly like a parity check/calc or disk re-construction, it reads/writes the entire disk) The preclear script does invoke the smartctl program on the drive hundreds/thousands of times on the drive. unRAID uses native system calls to do the same. That is the only significant difference. The hardware will likely cause you the same errors once you put the disk into service. I wish you luck but I have nothing to suspect anything other than a bad drive. I've seen drives stop responding before... It seems like this is a case of this. We'll soon see. As you said, unRAID is clearing your drive. I would, if I were you, once assigned, see if you can get through a manual parity check. Joe L.

If you've already added it to the array, no. It is no longer pre-cleared as unRAID has written to it. If you had not yet assigned it to the array, then it is simple to change the partitioning using the -C option in preclear_disk.sh However, it is perfectly fine as you have it. The ONLY drives with the performance issue are Western Digital EARS drives. They have firmware that works less efficiently if not aligned on a 4K boundry. No other drives have this issue. Please post the preclear report for that drive. you'll find it in /boot/preclear_reports I'll be curious to what it says.

Does that mean the drive corrected itself? Or is this a leading indicator of future failure? Should I RMA the disk? Also, for the first disk I posted above about, 2 sectors had been re-allocated before the preclear and then there were still 2 re-allocated. The number did not increase, so the disk is fine? Or should I RMA that one? Thanks for your help!!! I would use both of them in the array. All disks will experience un-readable blocks over their lifetime. That the number of un-readable blocks did not increase is good. (if it increased each time you ran a preclear cycle it would be an indication of a disk starting to fail) Yes, that first drive corrected itself. Perhaps when the sectors that were marked as un-readable were written the drive experienced some vibration, or a power fluctuation, or something else to make the write perform poorly. The "write" in the preclear did work. There was no need to re-allocate the sector.

Apparently, on this drive, the sectors were able to be re-written to their original locations and then read successfully. There was no need to re-allocate them. Joe L.

yes

User-shares data structures that are performing the union of all the disks, and presenting you with a consolidated view, are already in memory. No need to cache them... should never have to spin up a disk to read them. But the option is there to cache_dirs if you want to see if it makes a difference. I use user-shares. I never use the -u option. Joe L.

You cannot change the internal SSD structure. Use it as it is. NO DRIVE presents anything other than 512 byte sectors to the OS. (not yet, anyways) If they did, there would not have been a need for the GPT partitioning for disks over 2.2TB. Some disks, such as yours, organize the data internally in 4096 byte sectors. Some just say 512 externally and use 1024, 2048, 4096, or something else internally. Forget what it is doing internally... You've started the partition on a 4096 byte boundary. That is also a multiple of 512 bytes, so it works for just about anything. You did nothing to set sector size. (and you cannot) You only indicated to unRAID where you would like the start of the data partition. The reiserfs, by default, uses 4096 byte blocks in its structures. Therefore, it will read, and write in multiples of 4096bytes, each which will involve eight 512 byte sectors. The unRAID "stripe" of data, is much larger, composed of many blocks being read and written at a time. Those are involved when calculating ans checking parity.

I do not think either drive is completely safe to use, but, on the other hand, there has only been one re-allocated sector and they did pre-clear. SMART values do not usually recover from FAILING_NOW. I have no idea what an "end-to-end" error is. Both drives have been in an environment of very high heat. (55C = 131 F). I would not trust the pre-clear under such high heat. (things expand under high heat, and track alignment is just too critical) Get the temperature under control, then perhaps preclear again. Joe L. Joe L.

It means the blocks needed are not in memory so it has to go to the disks. You can always install "inotify tools" and then: To track activity under /mnt/user, type:inotifywait -mr /mnt/user To track activity on a specific disk (/mnt/disk1), type:inotifywait -mr /mnt/disk1 they will show you exactly what is being accessed.