Joe L.

That's really good information! Writing a historical pre-clear byte there is problematic, though. Let's say you had to quickly install a replacement drive (new, never written to) and didn't have time to pre-clear it. So it's installed and being restored by the system via the parity drive. Now, since the pre-clear byte from the original (now failed) disk is part of the restore, you now have an indicator that says this new drive was pre-cleared, when in fact it wasn't. You'd need to manually clear that byte if it was to be accurate. Correct... The byte you are trying to use to indicate the drive status is part of parity protected data and would be restored. For that reason, it is not a place to put a drive specific indicator. (unless you added intelligence to it. If it was the drive "serial number" and it did not match the physical drive serial, you can be pretty certain it was not the actual pre-cleared drive.) Correct. Only partition 1 is part of the "md" device. The initial unused blocks are not currently part of it, although personal e-mail from Tom has indicated that will probably have to change in the future when he changes unRAID to be able to handle other file-system types. No, sector 0 is the area usually used for the master boot record and the partition table. unRAID does not protect it with parity, but completely re-creates it if rebuilding a disk, as its contents are known and completely based on the drive geometry. Sectors 1 through 62 are unused... and not part of the parity protected data. I'm going to guess unRAID would not touch any prior contents in those sectors, even when re-constructing onto a new disk. Those sectors are probably cleared when unRAID does its own clearing of drives, and I know my preclear_disk script does clear them. I think you are right... Sectors 1 through 62 are potential places for an indicator of some kind that will not be restored from a parity reconstruction. Exactly correct. It is why a drive can't just be "added" to the array without either clearing it entirely, or completely rebuilding parity. All bits are seen by parity... correct. It is why unRAID does not need to pre-clear newly added disks if you do not have a parity disk assigned. When a parity disk is eventually assigned, it calculates parity on the bits it finds, regardless if they are parts of current files, or un-referenced parts of old deleted files. You could, as long as the parity drive was added after the data drive. If you have an established array, and add a properly formatted reiserfs disk with data to it, I'm going to guess the array will either come on-line with no parity initially, and a parity calc under-way, or.. stay off-line until a full parity calc is performed, or, force you to use the "restore" button to force a full parity calc as it stores a new configuration. All bits are used, not just the allocated ones in the file-system. Oh yes... I go back a bit more than that... Do you remember 8" floppy disks? (or loading programs via punched paper tape on a teletype machine?) Talk about slow... I go back far enough to where the first computer I worked on was all discrete transistors and diode logic. There were no IC chips... The "motherboard" equivalent was racks and racks of equipment that filled a room about 50 by 100 feet in size. The I/O device was a mechanical teletype machine. We serviced at the individual "bit" level. The dual-core-CPU itself was 7 feet high, and 12 feet wide. (It was early computing, but real-tiime-redundant processing and way ahead of its time. It used parity and hamming error detection on its memory. it could detect and correct any single bit error, and detect any double bit errors...but be unable to correct them... All this back in the early 70s. The two CPUs ran in parallel, on the same program, each from their own set of "RAM" and compared results with each other many times during each instruction cycle. Any difference and diagnostics would kick in...) Either CPU could use either set of RAM, or any part of it. If a bank of ram were to fail, both CPUs would share the mirrored bank, and we'd start working to fix the faulty one. a single 16k bank of memory was 4 feet wide and 7 feet tall. It was 47 bits wide... two 20 bit half words, and 7 bits of hamming and parity. We had 32 banks of memory ... each generated about 2000 watts of heat. If the room AC was turned off, the room temperature rose about a degree a minute. Lots of things stopped working well when the room temperature got to 115 degrees... (equipment AND people) We know because we did a heat-stress test before the system went into production mode. It was lots of fun. My first home "terminal" was home-made... with the "firmware source code" on it supplied by the vendor on an 8" floppy disk... in CP/M format... How many people today have the source code of the firmware of their terminal... It emulated an adm-3 or a vt100 terminal... Joe L.

Actually, no. But cumulatively on one boot, yes. I pre-cleared two 1.5T, then a 500G. For some reason it didn't show problems until the end of the 500G drive. I have 4G RAM installed (2.9G available to Linux), but I failed to notice how much was freed each time. Gotcha. Ok, thanks. I'll try that. --Bill Actually, yes... Your 4Gig of RAM (2.9Gig available for programs/buffers) is most certainly less than even the smallest of your disk drives (500gig) Your buffer cache used for disk I/O will be used at a rate of about 75-80 MB/second (however fast you are reading and/or writing to the disk). It is only freed when another program needs memory.

Until the array is started, yes. There's no indication of what is new and cleared. But as soon as you start the array, the drives that were pre-cleared and newly installed show up as needing formatting. At least that how it was on the last two I just did with the 9.6 version of preclear_disk.sh. Why would it matter if there was a parity drive or not? It would not matter if there was a parity drive or not. Don't think I understand this. If the drive had something extra written to say, sector 2, and that was consistent throughout the life of the drive in that array, the parity check would always be consistent, wouldn't it? Why would you not be able to restore? Basically, it would... but not for the reason you think. Sectors 0 through 62 are unused, and not part of any partition, and NOT part of the protected data. The protected data in unRAID does not start until the first partition. (You can write anything you want to sector 2, but it will be gone when you restore.) The good news however, is the first 64k of a reiserfs partition is unused by reiserfs, and specifically for boot records, etc. So... you can write to /dev/md1 to its first sector (which is actually the first sector on the first partition) and you will then get it recorded on the disk, and protected by parity. Never write directly to /dev/sdX1 as it will cause a parity error. Oh yes. I can't tell you the number of times I've had to enter the old drive parms into a BIOS manually, in the (not so) 'good old days'. But we were in heaven then with the new 386-25 or 33Mhz motherboard and our 50-100MB full size drives... Screamers, you know. --Bill Oh yes... I go back a bit more than that... Do you remember 8" floppy disks? (or loading programs via punched paper tape on a teletype machine?) Talk about slow...

It is using the cache, and linux will let it use as much as it can. It is not specifically "dd", but the fact that you are reading/writing every block of the disk being cleared, and each block is being held in cache in case you will be referencing it again shortly... It has no way to know the usage patterns of an unRAID server It just frees the least recently used buffer memory when it is needed and not currently being referenced. (And odds are pretty high that your disk size is greater than the amount of memory you have in the server, no matter how much RAM you have. ) You might set the cache pressure to allow it to re-use the buffer cache more quickly rather than to keep entries in the cache. sysctl vm.vfs_cache_pressure=100

Hmmm. It seems obvious that I must have done something wrong in the sequence, but I'm quite sure I didn't format any drives. In fact I recall being surprised that the supposedly pre-cleared ones didn't show up needing to be formatted, since with all zeros just written there can't be a filesystem structure there. Do the disks show as available in the management interface? (with a size, and free space?) Not that I'm aware of, but then most people have a parity drive assigned... I don't know what happens if a pre-cleared set of disks is present with no parity drive assigned. I would have expected the "Format" button, but who knows. the "Format" button would format all disks that were assigned and unable to be mounted.... It is not necessary to do it for each disk in turn. No, all of the bytes in the MBR (the first 512) are significant, and are tested by unRAID. Interestingly, the remainder of the first cylinder (up to sector 62) are currently not used at all... for historical reasons that allow the disk to be recognized as partitioned in older BIOS and in Windows) So, you could write something there... as a "note." I'd just write something down on paper... If you accidentally made a drive look like it was pre-cleared, and it had anything other than zeros on the remainder of the disk, you would completely break all the parity calculations and be unable to restore from a disk failure unless you did a full parity check and fixed all the parity ... prior to the failure ... It has been a long time since the old C/H/S geometry worked as originally defined.... In fact, it could not handle disks > 8 Gig. (At that time, 8 Gig was a dream... disk sizes were measures in Megabytes, not Terabytes. and a (tiny by today's standards) 20 Megabyte drive was nearly $1000. See here: http://www.mattscomputertrends.com/harddiskdata.html) Joe L.

Nice looking rig. Be aware, even though the hardware is hot-swap, unRAID is not hot-swap. At least one user caused a series of failures when hot-plugging an additional drive into an array caused it to re-assign drive IDs. He attempted to recover on his own, and ended up making an error that caused a loss of data on an entire drive. Always power down when plugging and un-plugging drives.

It certainly sounds as if you've already created file-systems on the disks. If you have them assigned to your array, can you store files on them? (They would have shown as "Un-formatted" if unable to be mounted, and if they are mounted, they have a file-system.) I somehow think you pressed the "Format" button... at least once. I have never heard of unRAID doing it on its own, but then again, most of the time it is not presented with a pre-cleared disk before parity is established. Edit: I've since figured out that the act of assigning the drives to the array will write the MBR's partition table in preparation for the creation of a file-system on it. As far as the "warning" from fdisk about the partition not ending on a cylinder boundary. Yes, very normal. The Cylinder/Heads/Sectors notation is not used by modern disks. It is "faked" and reported so very-old BIOS can still interact with the disk. see here: http://www.pcguide.com/ref/hdd/geom/geomPhysical-c.html and here: http://www.pcguide.com/ref/hdd/geom/geomLogical-c.html Modern disks use a linear addressing scheme... Their bit density on the platters guarantees more than 63 sectors per cylinder, but that is as many as the field in the MBR would allow... The number of heads can be as high as 255, but again, no modern disk has that many.... typically they have 2 per platter, one on top and one on the bottom. Many disks will therefore report 1 head, 63 sectors per cylinder, and a much higher number of "cylinders" than actually present. The number of "sectors" per cylinder is not constant on modern disks... There are more on the outer tracks than on the inner ones to keep the bit density equal. (otherwise, the bits on the inner tracks would be much closer together, and harder to read accurately) Your disk "reported" 46,512,336 cylinders. If true, the arm holding the disk head would have to be able to be positioned to 46 million different positions... accurately... It reported 1 head... difficult to read the top AND bottom of the platter with only 1 head. In fact, most 1.5TB drives have either 3 or 4 platters... Impossible to read 6 (or 8 ) surfaces with 1 disk head. Don't worry about the "partition not ending on cylinder boundary" warning... it has no meaning on current drives. Joe L.

Thanks Joe, here it is: Disk /dev/sde: 1500.3 GB, 1500301910016 bytes 1 heads, 63 sectors/track, 46512336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sde1 2 46512336 1465138552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 7af1 aea8 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 Disk /dev/sdg: 1500.3 GB, 1500301910016 bytes 1 heads, 63 sectors/track, 46512336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdg1 2 46512336 1465138552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 7af1 aea8 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 Disk /dev/sdb: 750.1 GB, 750156374016 bytes 1 heads, 63 sectors/track, 23256336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 2 23256336 732574552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 66b1 5754 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 --Bill I'm not sure how you managed it, but all three have the file-system type set to "83" (linux) at offset 450. This is not the normal value for a pre-cleared disk, but is what you would expect to find after a linux file system was created on the first partition. With that value unRAID would be very likely to assume a reiserfs exists and not do anything to the drive as the remaining bytes all seem to match what would exist AFTER a formatting step had occurred and a reiserfs file-system created on the drives. What did you do to the disks after you pre-cleared them? Did you format them? Did you attempt to mount them? Did you use "fdisk" on them? Something put the "83" file system type byte in there for the first partition... and it was not the pre-clear script. Joe L.

To analyze what the script did, or did not do, please supply the output of the following two commands for each of the drives it says are not pre-cleared. fdisk -l /dev/??? and dd if=/dev/??? count=1 | od -x -A d (substitute ? ? ? for your correct drive designation) Joe L.

New version of preclear_disk.sh does log the difference to the syslog. There was no good reason not to, just did not think of it until you mentioned it. Thanks for the suggestion. New 0.9.6 version has a few other improvements in reporting via e-mail (if you have a mail program configured on your server) and also reports drive temperature as the process is under way (in case the drive is overheating from lack of cooling air movement) New version is attached to first post on original thread here Joe L.

Not normally, at least not before /etc/profile is run, and usually never for the "root" user to prevent abuse (although Tom has edited /etc/profile to add it, even for root). Since no login has occurred when the "go" script is being invoked, it is not using /etc/profile at all.

When the "go" script is being run, the current working directory is probably "/" and the file you are trying to invoke is not in the search path. Try using either a full path to the script nohup /boot/whatever/s3.sh & or change directory to it first, and then use "nohup ./s3.sh" cd /boot/whatever nohup ./s3.sh &

It sounds as if your motherboard BIOS, (or disk controller) does not think there is any ability to use the hardware write-cache, but you know better. I'd just add the line to the end of the "go" script, exactly as you said. Glad you figured out how to enable the write-cache on your hardware. Also glad you got rid of the squeaky disk... It can only indicate friction as parts rub on each other... and that is never good for a hard-disk. Joe L.

Look here for how to interpret your results: http://en.wikipedia.org/wiki/Self-Monitoring,_Analysis,_and_Reporting_Technology#Known_ATA_S.M.A.R.T._attributes As already described, your drive looks good. it does not show uncorrectable errors. But now you can verify for yourself.

I still don't like what the disk is doing. Far too many uncorrectable errors, and they seem to be continuing. From what it appears, the pending-reallocation sectors were successfully written in their original locations and not re-allocated... interesting. Joe L.

I see an RMA in your future. Not sure why the 331 sectors were not re-allocated already, unless the failures were in the post-read, and no subsequent "write" has happened since then to those sectors. Joe L.

After 14+ hours your disk temperature went from 25C to 28C. I'd say that in itself is not too serious but it does say you have some serious fans. The S.M.A.R.T. wiki here indicates that attribute 200 is You started with the default initialized value of 253, and after a full 14+ hour pre-clear cycle, it has a normalized value of 100. The failure threshold is 0. You are nowhere close to the failure threshold value, so unless it changes over time, you are fine there too. Joe L.

As far as the hot-plug causing harm... Look through this thread After a hot plug, and a reboot when it did not work as expected, the user accidentally started a "parity check" with a drive that was not mounted. It ran for a minute or two before he stopped it It read "zeros" from the un-mounted drive and changed parity accordingly... Later, when a replacement drive was installed, those zeros were written to it instead of the normal file-system structures. Basically, he had wiped his data, from both parity and the drive. That hot-plug initiated actions that resulted in one of the few cases I know of where unRAID lost data. All that said, stop your array, reboot, and you'll probably be fine. Oh yeah... don't hot-plug... always stop the array and power down. Joe L.

It sounds as if the out-of-memory kernel process is killing processes on your server. deleting the syslog does not free the space it uses if there is still a process that has an open file-descriptor writing to it. The blocks are freed only after there are no more references to it, and a open file-descriptor is a reference. Some programs actually take advantage of this behavior and create a temp file, open it for reading and writing, then delete it. Until the file-descriptors are closed, the temp file is still readable and writable by that program... The memory (and space) is automatically freed when the program exists. To stop the old syslog process, and restart it, type the following: /etc/rc.d/rc.syslog restart It should free up the memory and you should then see the new syslog file you created start to be used.

That message indicates that the cleared "signature" expected in the first 512 bytes of the disk was not found when the disk was read after being written to. Your error messages seem to indicate that reading the drive is failing. I'd stop the array, power down, and check the cabling. Odds are either it is not seated properly, or one of the cables to the drive is defective. You can use the preclear_disk.sh -t /dev/sdk command to test if the pre-clear was successful. (With all the errors, it might not have been) It will run in a few seconds and let you know if the disk is cleared. It also sounds as if you are hot-plugging the external disks... DO NOT... the SATA drives may be, but unRAID is NOT. You could cause yourself all kinds of grief. (I apologize if you had both plugged in at the same time, but it sounds as if you had one disk connected, and then the other.) The syslog filling with disk errors is not a good sign... I'd run smartctl -d ata -a /dev/sdk on the drive, to see what the full SMART report says. Joe L.

Well... sorry about causing you extra "trouble" but then I figure you might want to avoid extra issues that can be uncovered before you move your files... The cost of a few new drives is small compared to the amount of time and effort needed otherwise. I hope your data transfer goes smoothly once you have a set of disks to move it to. From what you've said, your RAID-5 array would have had to deal with the defects on those two old disks at some point... and it might not have been as easy to swap in a new larger drive. Joe L. I appreciate the help and the abilities of your tools - I didn't complain, just reported back my experience. Please don't misunderstand me - I am happy to discover the problems in advance instead of having the trouble later and yes, you're completely right - the price of a new disk is nothing compared to trouble of a machine and the data on it - that's why I replaced the failing drives quickly with new ones... I did not misunderstand... Just wanted to save you, and any others reading this thread from problems you might otherwise avoid. I've spent many hours loading my array, I'm sure every unRAID owner's experience is much the same. I appreciate feedback... good and bad... Most important, I learn from everyone's experience... there is just no way for me to duplicate everyone's hardware and errors experienced. if the script needs improvement, I'm the first to admit it. I saw the "smileys" in your previous post and understood their meaning. I know your plans for a quick migration of data were put aside when the old disks you intended to use did not test well... but with new replacements in place it should be much better. I am hoping by now you are starting your data migration. Joe L.

From what I've read, the "VALUE" is the current value. The "WORST" is frequently initialized at the factory with a starting value of 253, and the "THRESH" is a value where (when reached) that parameter will be considered as failed. So, for your drive, the "VALUE" would need to go down to 51 for the RAW_READ_ERROR_RATE parameter to be an issue. Since you have just put the drive into service, you should see the RAW_READ_ERROR_RATE "VALUE" will stay pretty stable over time. If, at some point in the future you find it getting closer to the THRESH value, it would indicate some kind of problem getting worse. At that point in time, you can replace the drive proactively. All that has happened in the pre-clear process is that the "VALUE" and "WORST" are now changed from the initialized factory values to those that are reflecting how your drive is actually performing. As far as the "raw" value, it may never change from 0 for that drive. The internal method used to calculate most parameters is only known by the drive manufacturer. I see no reason why you should not add the drive to your array. Looks pretty good to me. Joe L.

Well... sorry about causing you extra "trouble" but then I figure you might want to avoid extra issues that can be uncovered before you move your files... The cost of a few new drives is small compared to the amount of time and effort needed otherwise. I hope your data transfer goes smoothly once you have a set of disks to move it to. From what you've said, your RAID-5 array would have had to deal with the defects on those two old disks at some point... and it might not have been as easy to swap in a new larger drive. Joe L.

Don't you mean... just installing and being surprised, if when something fails :( On MS-Windows, these disk issues are not visible. We happily go along until the OS will no longer boot, or we cannot open the critical document we need... Now, I'm sure many of those issues are bugs in programs... but some might not be... some are disk sectors that become unreadable. Remember, there are only two types of disk drives. No, not IDE and SATA. The two disk drive types are: 1. Those disks that have already failed. 2. Those disks that have not YET failed, but will... it's just a matter of time. Joe L.

The three columns of values are all internal to the drive. Each manufacturer has their own "normal" values and threshold for failures. The "raw" value is also only known to the manufacturer. All we can go by is "trends" ... and, of course, if a value meets its threshold, the drive will then fail the SMART test. (but may still be perfectly normal... just reached an end-of-life wear limit thought to be appropriate by the manufacturer.) Interpreting is difficult for everybody... For the most part, re-allocated sectors is the one thing we know we can focus on... but even there, according to seagate, modern drives have thousands of spare sectors. If 6 are re-allocated when you first exercise the drive, in my mind it is fine, unless the numbers of re-allocated sectors increase more and more each time you use the disk. Then, it is replacement time. Look here for a good summary of what and how to interpret what you are seeing: http://en.wikipedia.org/wiki/S.M.A.R.T. Joe L.