purko Posted August 29, 2009 Share Posted August 29, 2009 Your 1TBand 2TB WD drives are set as UDMA/133 and 3G SATA. I don't quite understand: If it's 3G SATA then why does it say "max UDMA/133"? Quote Link to comment
BRiT Posted August 29, 2009 Share Posted August 29, 2009 That's just the way Linux's libata works. The 3G / 1.5G Sata only refers to the maximum burst rate allowed. No disk's can sustain that speed, not even SSDs. Quote Link to comment
Joe L. Posted August 31, 2009 Share Posted August 31, 2009 As promised, there is now a updated version of the preclear_disk.sh script attached to the first post in this thread. It now is version 0.9.6: 5th edit: August 31, 2009. New version 0.9.6 ( 0.9.4 and 0.9.5 were internal versions as jbuszkie and I tested) # Version .9.4 - Enable SMART monitoring, just in case it is disabled on a drive. # Version .9.5 - Added disk temperature display and disk read speed display. # Version .9.6 - Enhanced the mail reporting to include some statistics (time, MB/s temp .. ect) # - Fixed a bug with using zero.txt and concurrent tests. Each test will use it's own file. # - Changed read block size to be bigger than 1,000,000 if smaller, to improve read speed There is nothing wrong with the previous version, but the newer one provides more statistics as it runs, especially if have configured mail on your server and you use the -m mailaddress@mailhost option to mail progress reports to yourself. This time, a lot of the improvements are credited to jbuszkie. He took my original script and made it even more informative. I just prettied things up once he made his improvements and sent them onward to me for review. Oh yes, the "diff" of the smart reports is now logged to the syslog... Makes it easier to see the changes to the results all in one spot. As always, if you spot anything we missed, or have a suggestion for an improvement, let me know. Joe L. Quote Link to comment
WeeboTech Posted August 31, 2009 Share Posted August 31, 2009 Any chance of setting this up on google code? Quote Link to comment
purko Posted September 1, 2009 Share Posted September 1, 2009 This time I am attaching my syslog, in hope that it could help resolve this problem. One of your disks is running in ATA/33 mode Aug 28 12:58:35 Tower kernel: Probing IDE interface ide0... Aug 28 12:58:35 Tower kernel: hda: WDC WD3200BEVE-00A0HT0, ATA DISK drive Aug 28 12:58:35 Tower kernel: hda: host max PIO4 wanted PIO255(auto-tune) selected PIO4 Aug 28 12:58:35 Tower kernel: hda: host side 80-wire cable detection failed, limiting max speed to UDMA33 Aug 28 12:58:35 Tower kernel: hda: UDMA/33 mode selected Aug 28 12:58:35 Tower kernel: ide0 at 0x170-0x177,0x376 on irq 15 Aug 28 12:58:35 Tower kernel: hda: max request size: 512KiB Aug 28 12:58:35 Tower kernel: hda: 625142448 sectors (320072 MB) w/8192KiB Cache, CHS=38913/255/63 Aug 28 12:58:35 Tower kernel: hda: cache flushes supported Now this is one thing that drives me crazy: The system is trying to detect a 80-wire cable, and I can find no way to override that! See, the disk in question is a brand new 2.5" mini-IDE disk from Western Digital. These disks don't have 40-pin or 80-pin cables. Their cables are 44-pin! I expected that Linux should be smart enough by now to know how to work with 2.5" PATA disks. As documented in: http://www.kernel.org/doc/Documentation/kernel-parameters.txt I tried adding libata.force=80c in syslinux.cfg, but the system replies with "unknown command: libata.force=80c, ignoring." Any ideas how I can make the system ignore the cable test, and just ask the disk what itis capable of? Yours, Purko Quote Link to comment
purko Posted September 25, 2009 Share Posted September 25, 2009 Hi Joe! A quick question: On the second read, doesn't it make sense to verify that you are actually reading zeros from the disk instead of dumping it straight to /dev/null ? If it makes sense, then maybe it can be a configurable command-line option, like -v for 'verify' or something? Yours, Purko Quote Link to comment
Joe L. Posted September 25, 2009 Share Posted September 25, 2009 Hi Joe! A quick question: On the second read, doesn't it make sense to verify that you are actually reading zeros from the disk instead of dumping it straight to /dev/null ? If it makes sense, then maybe it can be a configurable command-line option, like -v for 'verify' or something? Yours, Purko Funny you should mention that, as this past week I made a improvement that does exactly what you suggested. I did not post it, as I had just finished posting a new version a day or so before and wanted to test it a bit myself... The new version does the verification of zeros as you described as its default behavior in the post-read phase. It will take about 10-15% more time, as it has to evaluate what it is reading. It only does it in the post-read phase, as it is there where the zeros are expected. You can elect to NOT do the post-read verification of all zeros with a new -N option... (if you are in a hurry) I'll attach the new version to the first post in this thread. Edit: Version .9.7 is now attached to the first post in this thread... Enjoy. Joe L. Quote Link to comment
purko Posted September 26, 2009 Share Posted September 26, 2009 You're the best! Quote Link to comment
stoner Posted September 28, 2009 Share Posted September 28, 2009 Hi Joe, Just a quick query. If I use the pre-clear script to exercise a few old HDDs, can I still use the disks in a Windows-based system after that? As in, I will be able to just partition as per normal using Windows Disk Management or some other third party tool right? Quote Link to comment
Joe L. Posted September 28, 2009 Share Posted September 28, 2009 Hi Joe, Just a quick query. If I use the pre-clear script to exercise a few old HDDs, can I still use the disks in a Windows-based system after that? As in, I will be able to just partition as per normal using Windows Disk Management or some other third party tool right? Sure, Windows will probably recognize the partition it creates. I see no reason why it would not... In fact, it is specifically partitioned to be compatible with the BIOS on most MB. The partition the pre-clear script creates starts on the second cylinder (usually at sector 63, depending on the "reported" disk geometry) and goes to the end of the disk. The practice of leaving the first full cylinder un-used except for the MBr in the first 512 bytes is a holdover from DOS days. You need to create a file-system in the partition preclear_disk.sh creates, or delete the one the pre_clear script creates and re-create the partition if you want to make it smaller, or to create multiple partitions and use it for windows. Joe L. Quote Link to comment
stoner Posted September 28, 2009 Share Posted September 28, 2009 Sounds easy enough. Thanks! Quote Link to comment
purko Posted September 30, 2009 Share Posted September 30, 2009 Hi Joe! How about a new command line option to specify the block sizes dd will use? The reason I'am asking this is that the preclear script is still capable of totally rendering a weak machine unusable: processes get stuck at 'disk wait' state for looooong time, samba transfers time-out and abort with errors, etc., etc.. I think that experimenting with smaller dd block sizes at a time will solve all these problems, albeit at the expense of taking much longer time to complete the preclear script. I contemplated making some experiments myself, but I was stopped short when I saw how many actual numbers (rather than variables) are scattered all over the script. It makes any experimentation by anyone other than yourself quite risky. So it occured to me that an option to specify the bs on the command line would be really nice. (Or at least a nicely defined variable with the default bs value in the first few lines of the script) Seriously, I would rather have the preclear script run three times longer than now, rather than having it put so much strain on a weak box. Yours, Purko Quote Link to comment
Joe L. Posted September 30, 2009 Share Posted September 30, 2009 Hi Joe! How about a new command line option to specify the block sizes dd will use? <snip> Seriously, I would rather have the preclear script run three times longer than now, rather than having it put so much strain on a weak box. Yours, Purko Good idea. I'll put it in the next version. Quote Link to comment
purko Posted September 30, 2009 Share Posted September 30, 2009 How about a new command line option to specify the block sizes dd will use? Better yet, just a new command line option like -s (for "slow mode") When started with this option, the script will use a bs number much smaller than the default. Also, the number of blocks it does at a time must be greatly reduced, not just the block size. Quote Link to comment
Joe L. Posted October 7, 2009 Share Posted October 7, 2009 Hi Joe! How about a new command line option to specify the block sizes dd will use? The reason I'am asking this is that the preclear script is still capable of totally rendering a weak machine unusable: processes get stuck at 'disk wait' state for looooong time, samba transfers time-out and abort with errors, etc., etc.. I think that experimenting with smaller dd block sizes at a time will solve all these problems, albeit at the expense of taking much longer time to complete the preclear script. I contemplated making some experiments myself, but I was stopped short when I saw how many actual numbers (rather than variables) are scattered all over the script. It makes any experimentation by anyone other than yourself quite risky. So it occured to me that an option to specify the bs on the command line would be really nice. (Or at least a nicely defined variable with the default bs value in the first few lines of the script) Seriously, I would rather have the preclear script run three times longer than now, rather than having it put so much strain on a weak box. Yours, Purko Version .9.8 of preclear_disk.sh is now attached to the first post in this thread. It now has three new options that can be specified on the command line to accomplish exactly what you requested. You can now specify the block size for reading, for writing, and the number of blocks to read at a time. The sizes are all in bytes. So if you want 1 thousand byte blocks, and to read 100 blocks at a time, specify preclear_disk.sh -r 1000 -w 1000 -b 100 /dev/sdX if you want 32k byte blocks, use: preclear_disk.sh -r 32k -w 32k -b 100 /dev/sdX or preclear_disk.sh -r 512 -w 32768 -b 1k /dev/sdX For the most part, using smaller block sizes will lengthen the time it takes to clear a drive. It just will use the sizes you supply and continue until done. When specifying the sizes, you can use 1k for 1000, or 1M for 1000000. The "K" or "M"suffix will be expanded by multiplying the supplied value by 1000 or 1000000. I did not put an upper limit on size, so if you specify a size bigger than your available ram, you could run out and crash your server. You've been warned... Don't use -r 1000000000 -b 10000000000. You will run out of ram... :'( If you do not use the new -b, -w, or -r options the preclear_disk will use the default values used in all the prior versions of the script. -w size = write block size in bytes. If not specified, default is 2048k. -r size = read block size in bytes. If not specified, default is one cylinder at a time ( heads * sectors * 512)) -b count = number of blocks to read at a time. If not specified, default is 200 blocks at a time. As requested, these options are for running the script on a server with a limited amount of free ram, and limited processor power, where taking longer is acceptable. Joe L. Quote Link to comment
purko Posted October 7, 2009 Share Posted October 7, 2009 If you were a girl, I'd totally ask you out! Quote Link to comment
Joe L. Posted October 8, 2009 Share Posted October 8, 2009 If you were a girl, I'd totally ask you out! I'm sure that is a compliment... You'll see it really wasn't much work to add the parameters. There's more code validating the new options than actually using them. Quote Link to comment
jd7001 Posted October 10, 2009 Share Posted October 10, 2009 I am currently running Preclear...but an error message keeps on flashing on the screen below the elapsed time. "smartctl: error while loading shared libraries: libstc++.so.6: cannot open shared object file: no such file or directory" Can you please tell me what I have done wrong. Quote Link to comment
Joe L. Posted October 10, 2009 Share Posted October 10, 2009 I am currently running Preclear...but an error message keeps on flashing on the screen below the elapsed time. "smartctl: error while loading shared libraries: libstc++.so.6: cannot open shared object file: no such file or directory" Can you please tell me what I have done wrong. Yes, re-read the first post in this thread. It describes how the support library needed for SMART reports is missing on some unRAID releases and points to instructions on where to download it and how to install it. That can all be done even as the pre-clear continues to run. Or look here: http://lime-technology.com/forum/index.php?topic=4407.msg39409#msg39409 Joe L. Quote Link to comment
jd7001 Posted October 10, 2009 Share Posted October 10, 2009 Yes, re-read the first post in this thread. It describes how the support library needed for SMART reports is missing on some unRAID releases and points to instructions on where to download it and how to install it. That can all be done even as the pre-clear continues to run. Or look here: http://lime-technology.com/forum/index.php?topic=4407.msg39409#msg39409 Joe L. Thank you. Quote Link to comment
spinbot Posted October 14, 2009 Share Posted October 14, 2009 When I run the command: ls -l /dev/disk/by-id to locate the drive names ( so I can pre-clear the right one ), I notice some follow one naming pattern and the others a different one (ie. hdx and sdx) I thought I distinguished SATA from IDE as I read this elsewhere: The actual drive would depend on your hardware, since "hdX" drives are IDE, and "sdX" drives are SATA, I'm pretty sure it will not be /dev/hdk. Grin for an SATA drive, it will be /dev/sdX (where X will depend on your hardware) Here is my output. Although it has 21 entries, I actually have 6 physical drives + Flash Drive ( 1.5TB X 2 Seagates, 1.5TB WD , 1TB Seagate, 750GB Seagate and 20GB Seagate ) lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-ST31000528AS_9VP0H3TD -> ../../sdd lrwxrwxrwx 1 root root 10 Oct 14 18:12 ata-ST31000528AS_9VP0H3TD-part1 -> ../../sdd1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-ST31500341AS_9VS2L1L4 -> ../../hdc lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-ST31500341AS_9VS2LXAF -> ../../sda lrwxrwxrwx 1 root root 10 Oct 14 18:12 ata-ST31500341AS_9VS2LXAF-part1 -> ../../sda1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-ST320011A_3HT0GTF1 -> ../../hda lrwxrwxrwx 1 root root 10 Oct 14 18:12 ata-ST320011A_3HT0GTF1-part1 -> ../../hda1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-ST3750330AS_5QK00GT5 -> ../../sdc lrwxrwxrwx 1 root root 10 Oct 14 18:12 ata-ST3750330AS_5QK00GT5-part1 -> ../../sdc1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 ata-WDC_WD15EADS-00P8B0_WD-WMAVU0072618 -> ../../sdb lrwxrwxrwx 1 root root 10 Oct 14 18:12 ata-WDC_WD15EADS-00P8B0_WD-WMAVU0072618-part1 -> ../../sdb1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 scsi-SATA_ST31000528AS_9VP0H3TD -> ../../sdd lrwxrwxrwx 1 root root 10 Oct 14 18:12 scsi-SATA_ST31000528AS_9VP0H3TD-part1 -> ../../sdd1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 scsi-SATA_ST31500341AS_9VS2LXAF -> ../../sda lrwxrwxrwx 1 root root 10 Oct 14 18:12 scsi-SATA_ST31500341AS_9VS2LXAF-part1 -> ../../sda1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 scsi-SATA_ST3750330AS_5QK00GT5 -> ../../sdc lrwxrwxrwx 1 root root 10 Oct 14 18:12 scsi-SATA_ST3750330AS_5QK00GT5-part1 -> ../../sdc1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 scsi-SATA_WDC_WD15EADS-00_WD-WMAVU0072618 -> ../../sdb lrwxrwxrwx 1 root root 10 Oct 14 18:12 scsi-SATA_WDC_WD15EADS-00_WD-WMAVU0072618-part1 -> ../../sdb1 lrwxrwxrwx 1 root root 9 Oct 14 18:12 usb-Lexar_JD_FireFly_AA04011000035322-0:0 -> ../../sde lrwxrwxrwx 1 root root 10 Oct 14 18:12 usb-Lexar_JD_FireFly_AA04011000035322-0:0-part1 -> ../../sde1 I've just installed the one recognized as "hdc". I still have to pre-clear it and it will become my Parity drive. The Question: Why are all my other SATA drives labeled "sda, sdb, sdc, sdd" and this new one not called "sdf" ? You can see that "sda" is the exact same make of drive as "hdc" (I've noticed the Flash drive is sde/sde1) I can say for sure, it is SATA as I had to search for an hour to find a darn SATA cable for it Quote Link to comment
Joe L. Posted October 15, 2009 Share Posted October 15, 2009 When I run the command: ls -l /dev/disk/by-id to locate the drive names ( so I can pre-clear the right one ), I notice some follow one naming pattern and the others a different one (ie. hdx and sdx) I thought I distinguished SATA from IDE as I read this elsewhere: The actual drive would depend on your hardware, since "hdX" drives are IDE, and "sdX" drives are SATA, I'm pretty sure it will not be /dev/hdk. Grin for an SATA drive, it will be /dev/sdX (where X will depend on your hardware) I thought the naming would be consistent too. Here is my output. Although it has 21 entries, I actually have 6 physical drives + Flash Drive ( 1.5TB X 2 Seagates, 1.5TB WD , 1TB Seagate, 750GB Seagate and 20GB Seagate ) <snip> I've just installed the one recognized as "hdc". I still have to pre-clear it and it will become my Parity drive. The Question: Why are all my other SATA drives labeled "sda, sdb, sdc, sdd" and this new one not called "sdf" ? You can see that "sda" is the exact same make of drive as "hdc" (I've noticed the Flash drive is sde/sde1) I can say for sure, it is SATA as I had to search for an hour to find a darn SATA cable for it The make/model of drive is not what determines hdX vs sdX, but how the disk controller chipset on the MB or controller card identifies itself to Linux. There might be a BIOS option to get a disk to identify itself as one or the other, and it really does not matter unless the disk is accessed in PIO mode (really slow) instead of a DMA mode of some kind. Do not be surprised if the actual assigned devices change as you add or remove drives, or update to different versions of Linux, as the devices might be scanned in a slightly different order with a new Linux version. Post a copy of your syslog so anything odd might be identified in your configuration. Joe L. Quote Link to comment
spinbot Posted October 15, 2009 Share Posted October 15, 2009 The new drive is added on my motherboard directly. It in the 5th of 6 ports. Syslog is attached. I'll hold off on starting the preclear until I hear back Thanks Quote Link to comment
Joe L. Posted October 15, 2009 Share Posted October 15, 2009 The new drive is added on my motherboard directly. It in the 5th of 6 ports. Syslog is attached. I'll hold off on starting the preclear until I hear back Thanks The port is being found on the MB as follows: Oct 14 18:12:22 Tower kernel: hdc: ST31500341AS, ATA DISK drive Oct 14 18:12:22 Tower kernel: hdc: host max PIO4 wanted PIO255(auto-tune) selected PIO4 Oct 14 18:12:22 Tower kernel: hdc: UDMA/100 mode selected Can you do a hdparm -i /dev/hdc so we can confirm the mode being used to access it. It looks like it will be using UDMA/100. Go ahead and start the pre-clear. It too will give you an idea of the speed the disk can be accessed. Joe L. Quote Link to comment
spinbot Posted October 15, 2009 Share Posted October 15, 2009 root@Tower:~# hdparm -i /dev/hdc /dev/hdc: Model=ST31500341AS, FwRev=CC1H, SerialNo=9VS2L1L4 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% } RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4 BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=16 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=18446744072344861488 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120} PIO modes: pio0 pio1 pio2 pio3 pio4 DMA modes: mdma0 mdma1 mdma2 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 udma6 AdvancedPM=no WriteCache=enabled Drive conforms to: unknown: ATA/ATAPI-4,5,6,7 * signifies the current active mode I started preclear to get an idea of speeds and everything looks normal ( i think ) unRAID server Pre-Clear disk /dev/hdc = cycle 1 of 1 = Disk Pre-Read in progress: 0% complete = ( 1,645,056,000 bytes of 1,500,301,910,016 read ) 116 MB/s = = = = = = = = = = Disk Temperature: 30C, Elapsed Time: 0:00:15 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.