July 5, 201115 yr My setup is here (think ended up with a variant of this board because of a NIC compat issue): http://lime-technology.com/forum/index.php?topic=9644.0 I have had some read/write issues: http://lime-technology.com/forum/index.php?topic=12278.0 I brought it into work and hooked it up to an enterprise gig switch and was able to read 60-70mb and write at 10-15mbs. Got a different 1gig switch and took that home but haven't tried read/write tests yet. Instead, for continuity's sake I decided to preclear (...one...more...time). Mind you, these are the exact same drives and are both connected to the built in SATA on the board (haven't gotten the add-in cards yet... this is a slow process ): Drive 1: == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 8:37:43 (64 MB/s) == Last Cycle's Zeroing time : 13:39:11 (40 MB/s) == Last Cycle's Post Read Time : 15:33:42 (35 MB/s) == Last Cycle's Total Time : 37:51:44 == == Total Elapsed Time 37:51:44 == == Disk Start Temperature: 26C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdb /tmp/smart_finish_sdb ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Seek_Error_Rate = 100 200 0 ok 0 Temperature_Celsius = 118 124 0 Drive 2: == Using :Read block size = 8225280 Bytes == Last Cycle's Pre Read Time : 15:55:11 (34 MB/s) == Last Cycle's Zeroing time : 28:11:07 (19 MB/s) == Last Cycle's Post Read Time : 21:09:27 (26 MB/s) == Last Cycle's Total Time : 65:16:53 == == Total Elapsed Time 65:16:53 == == Disk Start Temperature: 26C == == Current Disk Temperature: 32C, == ============================================================================ ** Changed attributes in files: /tmp/smart_start_sdc /tmp/smart_finish_sdc ATTRIBUTE NEW_VAL OLD_VAL FAILURE_THRESHOLD STATUS RAW_VALUE Raw_Read_Error_Rate = 197 195 51 ok 935 Temperature_Celsius = 118 124 0 ok 32 No SMART attributes are FAILING_NOW Attached is a screen shot from the SMTP shout-outs sent from the server, you can see the span of time and where they start at 'together' but then SDC starts to lag. I also attached the PASS emails for each drive as well. The other emails seemed insignificant other than then the one drive taking much, much, much longer. Anyone have any ideas why it might do this? If it was a resources issue I would think that SDC would kick it up a notch when SDB finished over a day earlier... if it was a cable or actual drive failure you would think that would show up in SMART in the preclear reports or just a general failure. unRaid and unMenu don't show any read/write errors, etc in any of the display screens that I could see. I do notice that the line Raw_Read_Error_Rate is much different on C than B, but both say 'ok' so I am not sure what that means... sdb_disk1.txt sdc_disk2.txt
July 6, 201115 yr BTW, check the EARS drive sticky here: http://lime-technology.com/forum/index.php?topic=11842.0
July 6, 201115 yr Author Thanks cyrnel. I checked last night and the jumpers are off both drives. They were new when I installed them and the first thing I did back then was preclear them. One (presumably the same one) took longer than the other but I didn't pay too much attention. Is there something particular about the RAW_VALUE #?
July 6, 201115 yr Is there something particular about the RAW_VALUE #? Only meaningful to the manufacturer. We have no way to know what it represents. It is not the first time we've seen drives with drastically different performance. Are they on the same disk controller? Are they the same firmware version? (perhaps they made a fix in the newer firmware, or introduced a feature )
July 6, 201115 yr I had one WD 2TB green drive that took almost twice as long to complete a preclear cycle. I took it back and my dealer replaced it. There were some mentions here: http://lime-technology.com/forum/index.php?topic=12616.msg119971#msg119971 Stephen
July 6, 201115 yr From what little I know all drives these days use mad amounts of ECC so the only part of raw read errors that should be important (and are even intelligible) are the value and worst vs. threshold. Even with the other reports the difference in performance between the two drives is strange. It might be one of those proprietary things we'll never know, but if time allows you could exchange the drives (both position and cables) and test again. You could save time and probably get comparable numbers by simply running raw dd commands yourself, outside of preclear.sh.
July 21, 201114 yr Author cyrnel, What do you mean run the DD commands myself? Sorry, new to this. I am just running the preclear commands from one of the online guides (bookmarked on other machine) - I do like how you can just start it and walk away and it will inform you what's going on via email. I ordered 2 more WD Green drives on that deal yesterday. I am going to RMA the 'bad' one with WD and when I get that back that will be 4 total which is more than enough for now. Found an 80gig Raptor (10k) laying around - contemplating using that as a cache drive, just because I can - but I think I would need to use that script to move the data quite frequently, especially at first as I fill the drives up. I reseated all the cables, checked all the power and then organized all the SATA and power cables with zip ties - feel pretty confident that everything is solid. I think tonight I will slide drive0 into the spot where the bad one is and preclear again the way I did before. If that starts and maintains a decent speed (35 hours total or so) then I will just assume the drive was bad and that the cables, the carriage, etc are all fine.
July 21, 201114 yr Are the HDs set to AHCI mode (in the motherboard BIOS) as in mine own experience 38 to 65 hours for preclear is awful (should be around 26-28 hours on a decent hardware like yours). Perhaps you should use new data cables with locking clips and not not use zip ties at all for your test to see if the speed will increase (you may have damaged the cables in your quest for nice cabling job)
July 21, 201114 yr Nothing magic. Your plan sounds solid. I was just suggesting an alternative in case time was short. Of course if it means more of your time to get comfortable with the commands and not wipe out everything else on the system, it may not be worth the effort. Joe's preclear script is a terrific packaging of system tools for the specific purpose of preparing and gaining confidence in drives. If I'm not mistaken the performance numbers it displays originate with those underlying tools. For example, a really simple read test: This command: dd if=/dev/sdb of=/dev/null bs=1K count=1MB Produces this output: 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 11.0558 s, 92.6 MB/s You have to determine the correct input device, /dev/sdb in this example. You have to take into account effects of sleeping drives or other activity. (run the command multiple times) It doesn't test writes. It can, but using your drive as the output destroys whatever was there. Again, your plan sounds good. Also, bcbgboy13's comment about the zip-ties is quite valid. Keep them loose-ish. Zip-ties are taboo for production wiring because they're too easy to over-cinch which permanently damages the underlying conductors. Every kink in the cable creates signal reflections that tax the tranceiver and protocols which can hurt throughput, if it doesn't mess up transfers completely.
July 22, 201114 yr Author Thanks for the replies guys. They are AHCI, yes. Right now I am just on two of the onboards right now. The two SATA cards don't have any drives on them, so I didn't go into those BIOSs (BIOS's?). I am using the SATA cables that came with the carriages. The quality seems good, no better or worse than the Dell ones I having laying around - attach securely. I just zipped them together today, nothing tight. Just to keep things up and organized. The power and SATA cable groups bend easy with minimal criss-crossing/touching. More of a house cleaning. If I have issues I can always try some other SATA cables as a test. I am not worried about existing data, there is none. So I have my old parity ('good' disk) in the 2nd bay now, former home of the 'bad' disk. I turned the system up and it looks like this currently: parity Missing WDC_WD20EARS-00M_WD-WCAZA1273904 - - GOOD DRIVE 1,953,514,552 - - - - disk1 Missing WDC_WD20EARS-00M_WD-WCAZA0540067 - - BAD DRIVE 1,953,514,552 - I having nothing assigned at the moment. If I assign GOOD disk on bay 2 (going to put BAD disk in bay 1 for one more preclear to see if it has the same speed as GOOD drive did in that bay - might allude to cable issues) I get this: parity Missing WDC_WD20EARS-00M_WD-WCAZA1273904 - - GOOD DRIVE 1,953,514,552 - - - - disk1 WDC_WD20EARS-00M_WD-WCAZA1273904 GOOD DRIVE WDC_WD20EARS-00M_WD-WCAZA0540067 missing BAD DRIVE I don't see any place to remove that drive, that one will eventually go away completely (WD RMA'd it and sent a new one out today). Is there a way to remove that reference? Seems to me that as you replaced drives you would eventually replace a drive in a bay with a different one (one goes down, bigger drives, etc) and that reference in the config to the old one would need to go away... Thanks EDIT: looks like INITCONFIG or //your_server_name/blank.htm?cmdInit=apply is what I need...? Again, I just want it wiped out - not looking to save data. Seems to be working (cleaned it up, rebooting now)
July 22, 201114 yr What motherboard and ports? Some motherboards will have one controller running 4 sata ports and another controller running another 2. Josh
July 22, 201114 yr Author What motherboard and ports? Some motherboards will have one controller running 4 sata ports and another controller running another 2. Josh It is an Asus M4A78LT-M ( the 'LE' is the bad one w/unraid unfriendly NIC). I believe it does operate as a 4+2 config, settings in the BIOS led me to that conclusion with the wording they used when setting up RAID (which, of course, I am not - the 6 ports are on with AHCI and that's about it). Here is what is confusing: Bad disk in slot 1 *seems* to be going along on pre-clear at a decent pace. Now, I have already RMA'd the thing so I have to send it back at this point (WD has already shipped out the replacement). At 16 hours in it is on post read, it zero'd fine. Previously this is where it faltered though (well, it slowed down - when I was preclearing two disks at once this 'bad' disk would fall way behind and finish at 30-36 hours when the other finished earlier). I guess it could have been the SATA cabling - I cleaned it up and reseated them. Maybe that helped. I am not a computer newbie, new to unraid and it has been a bit since i built any machine but I deal with servers, routers, switches - run cat5 and terminate by the 1000's each year. I really, really doubt that the cabling has much to do with it - but that's what confuses me. SATA is so simple. There are issues with crosstalk/noise/interference -in every situation you are going to have some, but it generally cable runs fault/fail or they work. At the end of the day the fault failure threshold and the raw data values on the bad drive are there - where there are not values on the good drive. If I get those at the end of the preclear again then I will just assume the drive is bad and hope the new one works better.
July 31, 201114 yr Author Getting back to this thing... At this point I am about to toss those whole thing into the river - that's how frustrating this f*ing thing is becoming. Update: Got 4 WD 2tb EARS drives. One was pre-clearing very slow and registering some errors. I RMA'd that one and have the replacement in hand. Up until today I have only been running preclear on the built in SATA ports on the ASUS board. This is the M4A78LT-M and apparently is working in other setups for people. A couple days ago I tried preclearing two drives at once on the AOC-MV8-SASLP add-in card. They both started and I got the two confirmation emails that the next notice is at 25%, etc, etc. Hours and hours later when I didn't have any new emails I figured something was up. Connecting back up the the SCREEN sessions via Telnet didn't show anything - it's like they were stuck or something. The web interface was up but the syslog was last written to when I started preclear orginally... so anyway, I turned it off and came back to it today. Basically the same thing, same two drives on the same two drive bays - start preclear and then check back after a few hours. It's strange, the telnet/screen session barely updates (previously, it seemed like it was more 'responsive' with the %/time updating frequently. Now it seems to 'burst' update, a few minutes at a time...). ================================================================== 1.9 = unRAID server Pre-Clear disk /dev/sdb = cycle 1 of 1, partition start on sector 64 = Disk Pre-Read in progress: 1% complete = ( 31,256,064,000 bytes of 2,000,398,934,016 read ) 31.5 MB/s = = = = = = = = = = Disk Temperature: 30C, Elapsed Time: 0:15:54 I attached the syslog at the point from boot --> start of preclear on one (1) disk on the SuperMicro card (first SATA port). A few things: Why is this such a PITA? I know other folks have better luck with this than me, but frankly, I am not even sure if I would trust this if I got it going. This is all new gear, vetted from the list of stuff that is supposed to work together compiled for studying other examples on this site. So damn frustrating. Even though it is SUPPOSED to work does the Super Micro AOC-MV8-SASLP card really not work that well in 4.7??: http://lime-technology.com/forum/index.php?topic=10618.0 The above post ends with someone more or less saying, 'doesn't work for me either'. Should I RAM yet another thing and send back the SuperMicro card??? I went into the BIOS for the card itself, not much you can change. Mode is JBOD, didn't see where I could change anything of substance (had some spin up delays, etc). Any ideas before i just part this thing out or go a different direction??? syslog.pdf
August 1, 201114 yr Author Not sure if this helps but in looking around (while waiting for the next preclear stage at 25% - it's at 10% of pre-read now after several hours) I noticed the following, in read under /MyMain on the web interface - again, these lines are in red. Jul 31 17:19:14 Tower kernel: [<c112b0f8>] ? get_disk+0x4a/0x61 (Errors) Jul 31 17:19:14 Tower kernel: [<c101b028>] ? kmap_atomic+0x14/0x16 (Errors) Jul 31 17:19:14 Tower kernel: [<c11334a5>] ? radix_tree_lookup_slot+0xd/0xf (Errors) Jul 31 17:19:14 Tower kernel: [<c104a179>] ? filemap_fault+0xb8/0x305 (Errors) Jul 31 17:19:14 Tower kernel: [<c1048c43>] ? unlock_page+0x18/0x1b (Errors) Jul 31 17:19:14 Tower kernel: [<c1057c63>] ? __do_fault+0x3a7/0x3da (Errors) Jul 31 17:19:14 Tower kernel: [<c105985f>] ? handle_mm_fault+0x42d/0x8f1 (Errors) Jul 31 17:19:14 Tower kernel: [<c108b6c6>] block_ioctl+0x2a/0x32 (Errors) Jul 31 17:19:14 Tower kernel: [<c108b69c>] ? block_ioctl+0x0/0x32 (Errors) Jul 31 17:19:14 Tower kernel: [<c10769d5>] vfs_ioctl+0x22/0x67 (Errors) Jul 31 17:19:14 Tower kernel: [<c1076f33>] do_vfs_ioctl+0x478/0x4ac (Errors) Jul 31 17:19:14 Tower kernel: [<c105dcdd>] ? do_mmap_pgoff+0x232/0x294 (Errors) Jul 31 17:19:14 Tower kernel: [<c1076f93>] sys_ioctl+0x2c/0x45 (Errors) Jul 31 17:19:14 Tower kernel: [<c1002935>] syscall_call+0x7/0xb (Errors) Jul 31 17:19:14 Tower kernel: ---[ end trace aa65922ad6df94b5 ]---
August 1, 201114 yr Those lines are normal with 2 AOC-MV8-SASLP's. It could be that you have had bad luck with HDD's. I once had the problems you are describing.... It took 3 HDD replacements and multiple SATA cable replacements for me to work out the kinks in my setup. All the hard drives were 2TB EARS WD GREEN's and all the "bad" cables were brand new from newegg.com.
August 3, 201114 yr Author Update: Cleared 4 new WD 2TB EARS drives with the onboard SATA. Two individually, two at the same time. Average speed was 100Mb/s even with the two going at once. Final run time was in the 28 hour range for the two at a time and 26 hours for the two going by themselves. No errors on either drives and I would say they work fine. BIOS on the ASUS board is 802 (or 8.02, however they designate it) which is the latest according to ASUS - late 2010, when I bought the board, actually. SuperMicro card is .21, which is the newest BIOS for that. I reseated it and reattached the cables. I have a ton of 80gig 7200 rpm drives and 5 or 6 340gig drives as well. I think I will get this thing setup on the built in SATA ports, make my shares, etc and start using it. I can experiment with the SuperMicro card and the smaller drives at that point and see if I can replicate the issues. Unfortunately I think the two sets of 1 to 4 SATA cables were $20 each (anyone know some 'good' ones, high quality?). I am weary of ordering more just to test but I may have to. I have a 2-port card for the final SATA drive (#15) and that one clears fine. So even if the AOC, for some reason, can't preclear but reads/writes fine I may just use #15 to clear drives as I get them (a ways off from that, I can run 4 2TB and two 340gb drives on the on-boards for awhile). AHCI was setup correctly for the on-board. Does anyone familiar with the SuperMicro card know if you need to adjust that BIOS in any way???
August 9, 201114 yr You may not need to do this....... -------------------------------------------------------------------------------- Title: Re: Issue with booting from usb with the SUPERMICRO AOC-SASLP-MV8. Post by: Rajahal on October 06, 2010, 09:51:34 AM -------------------------------------------------------------------------------- I just learned this trick recently: go into the SuperMicro card BIOS (press ctrl-m at the right time) and disable INT 13h for all your SuperMicro cards (if you have more than one). This setting makes it so that none of your hard drives on the SuperMicro card can ever steal boot priority! HDDs on the motherboard still can, though. -------------------------------------------------------------------------------- This is only needed if you have a hard drive robbing your boot priority. I did it before I let unRAID try and boot. As for cables there is a thread in good deals. I think it's dead now but, worth a look. I picked up four more..... http://lime-technology.com/forum/index.php?topic=14043.45
Archived
This topic is now archived and is closed to further replies.