September 13, 200916 yr Just to confuse myself, I moved the wiring around. It turns out, Disk 1 gives good parity no matter where it is plugged (PCI or mobo). Therefore, it might actually be the filesystem on the Disk 2 drive, or perhaps (unlikely) the disk itself. I say unlikely because it is not reporting any errors. Suggestions? I might try changing the disk on Drive 2 to something else and see if it works on PCI. It's all very confusing to narrow this one down, as it could be so many different things. Thanks!!
September 13, 200916 yr YAY! I've finally tracked it down--and it doesn't mean replacing my motherboard, nor my PCI cards! Sometimes... yes, sometimes, diagnosis is a big pain in the bum, because when I tell you what I found, we'll all think, frig... why didn't we check that first? LOL So... as I mentioned, as part of the diagnosing procedure, I decided I'd move wires around some more... So, this meant putting Disk 1 on a PCI card, but moving Disk 2 onto the motherboard. I was surprised to find then, that writing to Disk 1 still gave no md5 errors. I'm sure you can guess where I'm going with this. So, I wrote to Disk 2, and sure enough, md5 error. So, is it the motherboard bus, as we originally thought might be the case? I moved Disk 1 onto the same header as Disk 2, removing Disk 2 completely. No md5 error on Disk 1. So, I move Disk 2 onto another header, and still, an md5 error on the copy to Disk 2. (PS - I'm continually calling it 'disk 2' just to keep things simple). Rebooting into Ubuntu still had the same errors on ... you guessed it... "disk 2" (sdf1) So, the ultimate test: rule out everything about the unRAID box. I pulled "Disk 2" completely, removed it from its rack tray, and plugged it into my BlacX. Mounting the ReiserFS drive to my Ubuntu desktop, I ran a local version of the test (copied the test file from my desktop to the "Disk 2" which is sitting on the dock station connected directly by USB 2.0, and ... MD5 error! WOOT! I've never been so happy to see an error in my life! So, I popped a 320 GB drive into my unRAID box, and am currently clearing it. My guess is, it's going to work just fine. "Disk 2" is not reporting any smart errors or anything like that, and it's reading/writing fine (other than md5 errors), so my guess is a bad format. I'll try pre-clearing it or otherwise trying to re-format it on my unRAID box, and will see if I can get it to report good MD5 hashes. But I'm going to keep it out of production until I feel it is safe. All in all, I "lose" possibly about 250 GB of files here. Thankfully, most of them are not irreplaceable. Some are, but I'll get over it; it's my stuff, not family stuff... and of course, some of it will be okay; but larger files will fail CRC check and may have 'glitches'. Fortunately for me, I do not keep anything in compressed format... so a jpg is a jpg... not a zip file with 1000 jpgs. So I'll be pretty fortunate in getting the data back off this drive, for the most part. So, moral of the story is, if you're getting parity errors, the first thing you should try is test that data drive on another computer and check the MD5 sums. If the md5's are good, continue to check other stuff... but in my case, it would have saved a lot of time to do that first (and an md5 test only takes a few moments as opposed to other tests which take hours). And for the record, this issue could be a hardware problem on the drive itself. It could be a bad format (which can happen), but it should be said that unRAID is spectacular; this was not an unRAID issue -- technically -- but I want to thank the unRAID community for helping me through the trial-and-error nature of this problem. Unless this 320 GB drive reports bad sums, I believe my problem to be resolved. Thank you everyone!! Robbie
September 13, 200916 yr Well, that proved it. The problem is not unRAID's fault -- it is hardware. I booted Ubuntu 8.04 live (100% bypassing unRAID, but using the same hardware), mounted the drive connected to my PCI card (Sil3112), cp'd the file, and the MD5 failed. At this point, I'm assuming it's the motherboard's PCI bus, as unRAID's compatibility chart shows Silicon Image chips as being perfectly compatible, plus I'm getting the same problem with two different cards (the other is an Sil3512, which Joe reported worked fine on his system) Any other thoughts? After I made my final post last night I thought about trying a "live" cd, just as you tried... I was going to suggest it today, but you beat me to it... Sorry about the hardware issue... perhaps the person who "traded up" did not tell you of all the problems they were having with the motherboard. (their true underlying reason for their desire to upgrade) You will certainly have a story to tell on your next video program... (basically... how to make tests first, to ensure old-cast-off-hardware works before trusting your data to it... and why you should... ) You have learned a HUGE advantage to unRAID, in that in almost any scenario you can boot up a live CD and get to your data... even with a brain-damaged MB or disk controller. I strongly suggest you use that MB DISK with MS-Windows... At least then you will be certain the blue-screens-of-death are not all Bill Gate's fault. Don't put Linux on it... as you will then think that Linux is unstable too... OOps... you posted while I was writing this... I edited my response... I hope it was a simple as the "disk" itself. I need to go back and look at the "SMART" report on that disk and see if it had any clues in retrospect of its behavior. Joe L.
September 13, 200916 yr Joe; sometimes I think these things happen specifically to me, just so I can help other people I was sticking my feet into the water of unRAID, and got sucked out by the tide. Not unRAID's fault... it's just the position of the moon. Man, that's deep. LOL. Thanks!! Robbie
September 13, 200916 yr Joe; sometimes I think these things happen specifically to me, just so I can help other people I know how you feel... having helped a few people myself, with knowledge based on my own past experiences... Did you read above, I found the trouble? Yes, I edited my response... If the new disk behaves itself, I'd take the one that can't be read reliably and see if it improves when given a wheel-chock test. The wheel-chock test involves placing the drive behind the wheel of your car and repeatedly driving over it, testing its ability to stop your car. Although the test is intended to uncover a mechanical fault, it "could" fix it so you never read a bad MD5 from that disk ever again. Thanks!! Robbie You are welcome. Joe L.
September 13, 200916 yr For the record: Internal, Desktop source file: 9c3b3eb65e00c18cffb058a91c0903e7 /home/robbie/Desktop/testfile15.zip unRAID Server, replaced Drive 2, connected to PCI Sil3112: 9c3b3eb65e00c18cffb058a91c0903e7 testfile15.zip A match! I am so very thankful to have found the problem, and even more so to see it was not an unRAID issue at all. My confidence and love for the unRAID system is through the roof. And now that I've got it all good, I'm gonna be very happy Thanks to all for your help!! Robbie
September 13, 200916 yr Glad you found your problem & extra glad I was wrong about the MB.. Yeah; I'm glad you were wrong too! haha! Although it was going to be the next step A hard drive is a LOT easier to swap than a motherboard. And it would have been really frustrating to go to all that trouble and expense only to find that it was still doing it I'm copying all the data from the old Disk2 to the new Disk2 now. The checksums will be bad, but it's reading fine. Then, I'll try re-formatting the drive, and if it does not give good parity at that point, I'll get the warranty replacement. Robbie
September 14, 200916 yr Then, I'll try re-formatting the drive, and if it does not give good parity at that point, I'll get the warranty replacement. I'm sure you know this, but the type of failure you are seeing is not usual at all for a disk drive. I would have thought the internal SMART firmware on the drive would have detected the errors and showed them somehow. I doubt if formatting will help... all it does is initialize the bitmaps in the rieserfs and puts into place the "superblock" structure. I doubt if it writes more than a few sectors. A full preclear_disk.sh "might" but only because it reads and writes to all the sectors and possibly might let SMART firmware detect a problem. To show this, I pre-cleared a drive, then checked the raw disk I/O statistics, then formatted it with a reiserfs, and checked the statistics again. My test showed the mkreiserfs command performed writes to 66,192 sectors... out of the total 1,651,104 ... representing about 4% of that disk 8 Gig disk. I'd also be interested in you performing a "long" SMART test on the bad drive. It would be interesting if it detects the errors in itself. smartctl -t long /dev/sdX wait a few hours for it to run, then get a new SMART report That drive presents a great opportunity (in a twisted way) to learn. Maybe you should hold off on the wheel-chock test for a day or two... Joe L.
September 14, 200916 yr smartctl -t long /dev/sdX wait a few hours for it to run, then get a new SMART report This is an an important test to do with any drive you plan on using for critical data. It does seem odd that a drive is writing the data in a corrupt manner. A google search did reveal some hints that a Silicon image driver in a older kernel had write corruption. One of the tests used was to write all 0's via DD, then read it back to make sure nothing changed. In their case, there were changes. In any case the smartctl long test should reveal surface defects. I would do a long test then use the unRAID pre-clear script to write 0s. Check the smart logs for re-allocated sectors. Then read back the drive to insure it was still 0's. Then do a long test again. Which drive is this? (Go by model and/or serial).
Archived
This topic is now archived and is closed to further replies.