Joe L.

Completely up to you. You could stop now if you like. You can run it again, either with or without the "-c" option. (as many times as you like) Eventually you'll want it in your array, so I'd only do one more pass, unless something shows up. You are welcome.

change the line in the config/go file from /usr/local/sbin/emhttp & to /usr/local/sbin/emhttp -p XX & where XX = the port you wish it to use, then reboot the server.

Did invoking that command at least say something like There is no screen to be resumed. (that is what it does on my server if there is no screen session running.)

screen is not connected to the telnet session. (or rather, does not die when telnet disconnected) Did you try running screen -r to see the existing "screen" sessions so you can re-connect to it? About the only other thing you can do is type top and see if the screen and preclear process is still running. (you would see "dd" constantly being invoked) or you can look in the /tmp directory. The preclear process writes its status files there.

Other than you used a version of the preclear script that is 4 versions old, all looks fine. Current version is 1.13, you used 1.9. Your older version does not work properly on disks > 2TB.

unless you invoked the preclear process under "screen", it stopped. There is no way to see its progress if you log in again, since it stopped. All you can do is start it again from the beginning. If you did use "screen" you can log back in and type screen -r to resume your screen session.

Because the section posted does not look at any of the other SMART parameters. It is NOT the report output from the pre-clear, but only part of it. It does not say that the disk was pre-cleared successfully, but only that there were zero re-allocated sectors and zero sectors pending re-allocation. (granted, that is good, but not unless you look at and compare the pre-and-post SMART reports.) So "no", it is not all that you need to look at. It is only part of it.

no

looks fine.

You can boot from ANY device. It is not necessary for it to be the one labeled UNRAID. There is at least one post on the forum where a user described how they use two flash drives, and one is just for the GUID check. Found it: See this thread for how one user boots from one flash drive, but uses the second "official UNRAID" one for the license check. http://lime-technology.com/forum/index.php?topic=3846.msg34018#msg34018 Basically, you boot from one flash drive labeled FLASH Then, unRAID mounts the flash labeled as UNRAID at /boot, but you then use the mount --move option to move it out of the way, and move the other into place at /boot. Joe L.

No worry... Not unless their normalized values go below their affiliated failure thresholds.

The inability to clear the drive had nothing to do with thr existing partitions, or the starting. sector. Existing partitions would have been cleared, that had nothing to do with the error message you saw. Do not fall under a false impression all is well. You will probably experience random parity errors. They are very hard to isolate, so keep this disk in mind if it happens to you. And do not forget to test your memory. It is critical as anything.

Yes, and it told you why, you just missed it. The post-read expected the disk to be all zeros. It was not. The actual cause could be almost anything except the "mouse" It could be a bad disk, or bad memory, or a flaky power supply, or motherboard, or disk controller. I'd start with a memory test, preferably overnight. Then, tests using badblocks on the disk itself. Then go from there. Joe L.

Do not use the drive if it has pending sectors that cannot be re-allocated. RMA it (although you could try another preclear cycle or two and see if they go to zero sectors pending re-allocation on the first, and stay at zero on the second, and no additional re-allocated sectors appear.). Joe L.

Both look great. Enjoy.

Would a swap file help with this? I have 4GB memory and don't usually run with a swap file because creating it seems to be one of the longest parts of boot time. But it might be worth it for those rare occasions when I need to do a preclear. no, it would not help at all. (if you needed one, and did not have one, processes would have been killed in an attempt to free memory and your server would have effectively "crashed" Therefore, if you did not crash, you did not need one. ) Most people will never need a swap file on an unRAID server unless they have very limited RAM and are running memory intensive programs. (where the program itself needs the memory, not the disk-buffer-cache) My older server only has 512 Meg of RAM, and unless I'm compiling a ffmpeg (a huge program) I do not need a swap file. Joe L.

Linux ... feh!! ... FYI, I first used, and did kernel development on, Unix before it even had dd (v4, 1973). Were you one of those who used "adb -w" on the kernel while loaded in memory? I fully respect anyone with that skill. The description of the count= option has not changed in its entire 38+ year lifetime (except in a negligibly semantic sense): May 1974: copy only n input records August 2012: copy only N input blocks The fact that it is a copy precludes any concern about buffering. It isn't really keeping it (in an active sense); it has just not yet overwritten it with anything else. True. I understand the concepts involved. My early involvement with computers was roughly about the same time as you, but I was fixing them, at the hardware level, and running hand-coded machine code routines to test them. There was no "motherboard" back then on the TSPS system I was working... It was all DTL logic. (ICs were not yet in common use) My involvement with UNIX did not begin until 1979/1980. It was a version of PWB-Unix... and prior to the Borne shell. (Its "Mashey" shell actually had labels and "goto") Years later (late 80s) I also was involved in writing custom kernel level "device driver" code in SVR3 UNIX (on a 3B2) for a very special interface to hardware manufactured by a supplier of "smart-phones" back when I was working on project for AT&T. Their customer at that time had specified the hardware, and I needed to communicate with it, and it was not possible through any existing interface. I wrote the device driver. It was years later that I first ran Linux, and re-wrote the scsi/sound-card driver on it to work on my hardware. (So I could play "doom" ) I'll agree with you there. (on both.. the Politicians , and the kernel developers) I have a feeling, like you, that one of the cache systems is not configured to be able to handle a large traversal of files using "find" at the same time performing a "dd" of zeros to an entire disk. Since the conditions needed to experience the issue are rare, nobody in the kernel-dev team have fixed it (if they even know of it) The issue seems to be related to "low memory" exhaustion. (a concept we never had to worry about on true UNIX with a single linear address space, and swap space available if memory was insufficient.) It seems to not occur if smaller block-sizes are used with the "dd" command. I never encounter "low memory" issues, since all I have on both of my servers is "low" memory. 512 Meg on one, 4gig on the other) As you said, the "dd" operation does not involve anything but the disk-buffer cache, but running it concurrently with a "find" of a large hierarchy certainly does involve the dentry and inode cache systems. I'm sure the user-share file-system is complicating the issue, it being entirely in memory. In an ideal world, we would not need to deal with any of this. Linux has its quirks when allocating memory for cache and processes, even more in SMP environments. I suspect it only shows on some hardware/drivers. As Linux/unRAID changes, we'll just have to adapt to the environment. 5 years ago I never had to think about concurrently writing using large buffer sizes to multiple 3TB disks. I guess I will eventually just change the preclear script to use a fixed buffer size, and hope it will work in all situations. Joe L. PS. Great to see another old-time-Unix-geek on here.

Granted. I was younger and far more innocent when I wrote the preclear utility. FYI, I first used "dd" many many years before Linux was created. It has not changed much over the years... ( It was on version 1.0 of CB-Unix. (barely out of Bell-Labs)) I knew it issued reads to the OS sized at the block size. Never gave much thought to the "count" and its buffering prior to output. I'll have to look at linux source to see what it does these days. Regardless of what "dd" is doing, the disk buffer cache will be keeping much of what it had recently accessed simple because it was most recently accessed. In the same way, cache_dirs is just using the "find" command, and it will force a lot of the disk buffer cache to be involved if you have a deep/extensive directory hierarchy. Between them, you can run out of "low" memory. Joe L.

Cylinder? As a design criterion (such as you allude), the cylinder has been obsolete for ~20 years, even moreso in the last 5-8 years. Just change your default to something that "feels" right. And, totally forget about "disk geometry"--all that does for you is create a chunk size that is NOT a multiple of 4K. --UhClem "The times they are a'changing." I fully understand that "cylinders" have not been used for 20 years (probably much more) The issue I was faced when originally writing the preclear script was in selecting an appropriate "block size" when reading and writing to the disks. I used the output of the "fdisk" command as my guide. I figured the disk geometry would probably report a size it could handle. "fdisk" presented a line like this (from a sample IDE disk on one of my servers): Units = cylinders of 16065 * 512 = 8225280 bytes The preclear script then read, by default, 200 "units" of data at a time with a "dd" command looking something like this (for that disk) dd if=/dev/sdX bs=8225280 count=200 ......... The amount of memory used for a single read request was then 8225280 * 200 = 1,645,056,000 bytes. Now, with larger 3TB disks, and a much larger different "Unit" you can easily run out of memory. The use of Units worked for many years, with disk sizes from 6 Gig upwards. It is only now with 3TB drives are the sizes out-growing the available RAM. I agree, there needs to be a limit, but a multiple of 4k makes no practical difference at all when you are asking for 8225280 bytes or more at a time. In the interim, use the "-r" and "-w" options as I previously indicated, and you'll probably not run out of memory. Joe L.

basically, you ran out of free memory. Apparently, on the 3TB drives you will need to use the -r, -w, and -b options to limit the memory used by the preclear process. This is because those parameters were originally sized when 1 TB drives were common. At that time the preclear script was designed to read and write a cylinder at a time. With larger disks, their geometry has gotten large enough that it may not leave enough memory for other processes. Try something like this: preclear_disk.sh -w 65536 -r 65536 -b 200 /dev/sdX

Thanks Joe! Also, I suppose there's no difference if I do one cycle at a time and repeat three times, rather than 3 cycles in one go? three at once is quicker, since the post-read phase of the first acts as the pre-read phase of the second, and the post-read phase of the second acts as the pre-read phase of the third. You save two "read" phases, so probably 6 to 8 hours or more with large disks is saved.

No, for those parameters the manufacturer simply has set the failure threshold high. (only a few failures to spin up in a timely manner would cause the drive to fail SMART tests) The "normalized" values are still at their starting values from the factory and the raw value has no errors. Some disks are faster, others slower... (same for disk controller ports)

ALL drives have raw read errors. Every one... some report them, some just retry. You can ignore the "raw" read-errors parameter UNLESS the normalized value reaches the affiliated failure threshold.

I have no idea why you are including the "diff" for our analysis. There is a report that summarizes the effect of the preclear. It shows how many sectors were pending re-allocation and already re-allocated both before and after the preclear. It also shows those attributes that are failing, or near their failing threshold. Basically, if it does not show re-allocated sectors, and no sectors are pending re-allocation,and there are no attributes failing_now, and the post-read was successful, the disk should be fine. The reports from the preclear are in /boot/preclear_reports. You can look there to see the results. (not in the syslog) Joe L.

Basically, you are looking for no sectors pending re-allocation at the end of the process, and few, if any sectors that have been re-allocated. Also, you are looking for no other parameters where the normalized value has reached its affiliated failure threshold. (Those would be showing FAILING_NOW on their report line) no