October 6, 200817 yr I was unable to access my unraid server recently (running great for many months), and when I rebooted the server and logged into the web config, I saw that disk 1 and the parity disk had an orange status indicator(see picture attached). I thought perhaps restarting the array would begin the rebuild process, it would hang on the "starting..." for at least an hour. Then I read in the forums sometimes you need to remove the disk and start the array, then stop the array, and add the disk back. That also did not work, because physically removing the disk and rebooting the webGUI shows the missing disk, but still will not start the array, hangs on the "starting..." screen. I have attached a picture of my webgui screen, and the syslog. I have a brand new spare 750GB drive waiting, because I suspect that it is disk1 that is giving me the problems. But I fear that with two orange lights, it means that parity is gone, and disk1 is gone, so I lose disk1. Please tell me I am wrong.
October 7, 200817 yr Did you check your cables? There seems to be a pattern here with the layout of the controllers and the drives. Oct 6 19:42:39 tower emhttp: pci-0000:00:1f.2-scsi-0:0:0:0 (sde) ata-ST3750640AS_3QD02DB5 ** Oct 6 19:42:39 tower emhttp: pci-0000:00:1f.2-scsi-1:0:0:0 (sdf) ata-WDC_WD7500AAKS-00RBA0_WD-WCAPT0373203 Oct 6 19:42:39 tower emhttp: pci-0000:01:01.0-scsi-1:0:0:0 (sda) ata-ST3750640AS_3QD02EY8 ** Oct 6 19:42:39 tower emhttp: pci-0000:01:01.0-scsi-3:0:0:0 (sdb) ata-ST3750640AS_5QD4HH0C Oct 6 19:42:39 tower emhttp: pci-0000:02:00.0-scsi-0:0:0:0 (sdc) ata-Maxtor_6V300F0_V60C2AGG Oct 6 19:42:39 tower emhttp: pci-0000:02:00.0-scsi-2:0:0:0 (sdd) ata-Maxtor_6V300F0_V6038XWG Before attempting the array startup, Perhaps you could do a smartctl -d ata -a /dev/sdx where x=drive letter of the offline drives. >> But I fear that with two orange lights, it means that parity is gone, and disk1 is gone, so I lose disk1. Please tell me I am wrong. if the two drives really died, they you are not wrong... However it may be temporary. see what smartctl says. Also you may want to run a short test if it seems to be responding. smartctl -t short /dev/sdx where x=drive letter.
October 7, 200817 yr I was unable to access my unraid server recently (running great for many months), and when I rebooted the server and logged into the web config, I saw that disk 1 and the parity disk had an orange status indicator(see picture attached). I thought perhaps restarting the array would begin the rebuild process, it would hang on the "starting..." for at least an hour. Then I read in the forums sometimes you need to remove the disk and start the array, then stop the array, and add the disk back. That also did not work, because physically removing the disk and rebooting the webGUI shows the missing disk, but still will not start the array, hangs on the "starting..." screen. I have attached a picture of my webgui screen, and the syslog. I have a brand new spare 750GB drive waiting, because I suspect that it is disk1 that is giving me the problems. But I fear that with two orange lights, it means that parity is gone, and disk1 is gone, so I lose disk1. Please tell me I am wrong. I don't think your disks are gone... but I would not be surprised if one was interfering with the other in some way. Are they on the same cable? I would power down, re-seat the power and data cables, and if connected to a disk-controller card, re-seat it too. If it still does not come up, disconnect physically one drive, try to come up online, if no luck, put it back and disconnect the other and try to come online. Whatever you do, DO NOT PRESS a "Restore" button. It would eliminate any chance of recovering via parity if you find you do need to replace a drive. When you finally get the array back on-line, the "Start" button is the only button you should think of pressing, even if you replace a defective drive with a new one. Joe L.
October 7, 200817 yr Author > Did you check your cables? If the drives are detected in BIOS properly, and are seen by unraid in the devices page, could this still be an issue? I reseated the cables and controllers anyway, just to be sure, but I did not replace any cables. > There seems to be a pattern here with the layout of the controllers and the drives. What is the pattern? The **'s on the drives that I believe are effected? Parity and Disk1? > Before attempting the array startup, Perhaps you could do a > smartctl -d ata -a /dev/sdx where x=drive letter of the offline drives. I was able to run smartctl -d on SDA (DISK1) and SDE(parity drive), results are attached, however, I could not run a short test for any drive (sda, sdb, sdc, sdd, sde,sdf) gave me a "Short offline self test failed [unsupported field in scsi command]" In the smartctl -d test, I did see a "failing_now" in the WHEN_FAILED section just for SDE(parity drive) > Whatever you do, DO NOT PRESS a "Restore" button. Thank you for this, I remember reading a post while back when installing un-raid that it was either never press restore or never press start, I couldn't remember, this makes me feel better that I didn't lost the parity (given that the drive is still alive) > If it still does not come up, disconnect physically one drive, try to come up online, if no luck, put it back and disconnect the other and try to come online. INTERESTING, after reseating the cables: 1. Leaving all drives in, when I use the webGUI and press start, the web pages refreshes and nothing happens not even "starting..." 2. Removing the SDE (parity drive) and restarting the server, I can't start the array because of too many missing disks (red dot on the parity drive, orange dot on disk1) 3. Removing the SDA (disk 1) and restarting the server, when I use the webGUI and press start, the web pages refreshes and nothing happens not even "starting..."
October 7, 200817 yr > If the drives are detected in BIOS properly, and are seen by unraid in the devices page, could this still be an issue? I reseated the cables and controllers anyway, just to be sure, but I did not replace any cables. Yes the cables can creep out. I had a drive seemingly work. I replaced everything in the chain piece by piece. Mount, Controller.. Then it turned out it was the cable all along. Drive /dev/sde is failing. It has not failed completely, but it will soon. These drives are not that old eh? See section saying. === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: FAILED! Drive failure expected in less than 24 hours. SAVE ALL DATA. See vendor-specific Attribute list for failed Attributes. ------ The other drive says === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED The returned values look OK. Chances are the data is intact. I don't know why you cannot do a self test. try adding -d ata as in smartctl --d ata -t short /dev/sda and do it on /dev/sde Then wait 2 minutes or so and grab the smartllogs again. If you want you can run a long test and come back in a few hours (takes up to 2 hours to run). smartctl --d ata -t long /dev/sda >> What is the pattern? The **'s on the drives that I believe are effected? Parity and Disk1? They are first on each controller.... Which is why I suspected cable as in position of socket. I could be totally wrong, it's just trying to look at everything. I don't see why /dev/sde does not show up. It may be mountable without the parity drive. Let's see what the long test shows, if successful, you can try and mount it directly in readonly however that would invalidate parity. drive serial 3QD02DB5 is having issues and may also be hanging the bus. When you remove the disk, how are you removing it? Are you removing the cable from the motherboard?
October 7, 200817 yr I hope I'm not going to confuse you, when you have had such veteran troubleshooters helping, but I have a little different view. 1. Leaving all drives in, when I use the webGUI and press start, the web pages refreshes and nothing happens not even "starting..." 2. Removing the SDE (parity drive) and restarting the server, I can't start the array because of too many missing disks (red dot on the parity drive, orange dot on disk1) 3. Removing the SDA (disk 1) and restarting the server, when I use the webGUI and press start, the web pages refreshes and nothing happens not even "starting... Your syslog looks fine, except that the array can't start (obviously a major problem). Disk 1, sda, tested fine, not perfect, but pretty good. The Parity drive, sde, on the other hand, is worthless, with a Reallocated_Sector_Ct of over 6700 failed and remapped sectors. I have to wonder if a head or platter is failing. And the fact that when ever it is installed (as in #1 and #3 above), the system is essentially unresponsive, tells me it is hanging the system trying to read from it. The reason you have a red dot in #2 is because you did not un-assign the parity drive. What removes a drive from unRAID is not physically removing a drive (although you can, and should in this case), but un_assigning it from the array. I believe you will see a huge difference in response once you un-assign sde. The reason that it is usually advised to avoid using the Restore button, is because it invalidates parity, which you don't normally want to do. In your case, the parity drive is already worthless, unusable, so once un-assigned, you should be able to click the Restore button without worry. Before you do though, there are a couple of other steps I would recommend. We don't yet know the state of Disk 1, but I have not seen any reason to think that there is anything seriously wrong with it, so far. I think there may be something minor wrong with it, that originally caused the Orange state, but that is probably fixable, perhaps with reiserfsck. And it is possible that it was simply a casualty of the unRAID crash you reported when you first started this thread, a side effect of the serious failure of the parity drive. So here are the steps I would advise: Un-assign the Parity drive (sde) At the console, run reiserfsck on Disk 1: reiserfsck -y /dev/sda1 Verify a successful result from reiserfsck. If it instructs you to re-run with a special parameter, do so, follow its instructions exactly. Again, verify success Reboot Assign the other Seagate 750GB (sdb) to the Parity drive Click the Restore button, after clicking the 'I'm sure I want to do this' box (I think that's what the box says) You should have 5 drives, all blue balls (I think!) Click the Start button, and let it build a new parity drive Verify the files on Disk 1 are intact
October 7, 200817 yr I would still suggest doing the smartctl test before anything is written to the drive. This may require booting up and not starting the array at all, doing the hardware tests with smartctl. Either that or boot up with the seatools diskette. As RobJ believes, I also believe, the data disk in question should be OK and recoverable. The parity disk is probably shot and is timing out. The steps presented are sound.
October 8, 200817 yr Author Thanks for the suggestions. I will try the smartctl test first just to be sure. Actually, un-assigning the parity drives makes sense. I will post when I run the test.
October 8, 200817 yr Author Seriously, I owe you guys a beer or something. Thanks ROBJ and WeeboTech, I learned a lot through this process. After following ROBJ steps I am indeed rebuilding the parity drive, and the data on disk 1 is fine. I have attached the reiserfsck -y /dev/sda1, and a shot of the victory screen! I did attach the syslog after running the smartctl short test on both drives. I was wondering should I replace drive1 as well? You said it tested good but not great. Of course I will wait a while after the parity is rebuilt.
October 8, 200817 yr I was wondering should I replace drive1 as well? You said it tested good but not great. Of course I will wait a while after the parity is rebuilt. Your Disk 1 is fine. Almost all drives have small defects, that you would not even know about without a SMART report. What is good is that you now have a base line to compare with in the future. It is a good idea to keep SMART reports for ALL of your drives, so that when you check them several times a year, you can compare the newest SMART report with previous ones, and be able to tell if a hard disk may be starting to fail. What you don't want to see are increasing numbers of defects. By the way, it's a good quality root beer for me! And I'd do almost anything for someone who brought me a root beer float! Edit: Forgot to mention that Joe's Unmenu has a great and easy way to obtain SMART reports, and WeeboTech has a great script somewhere that saves SMART reports for all of your drives. I should find it, and make sure it is a priority in the Wiki, for hard drive maintenance.
October 8, 200817 yr Edit: Forgot to mention that Joe's Unmenu has a great and easy way to obtain SMART reports, and WeeboTech has a great script somewhere that saves SMART reports for all of your drives. I should find it, and make sure it is a priority in the Wiki, for hard drive maintenance. Well I was wrong, credit goes to JoeL for the script, which can be found here.
October 8, 200817 yr I was wondering should I replace drive1 as well? You said it tested good but not great. Of course I will wait a while after the parity is rebuilt. Drive: 3QD02DB5 needs to go through the RMA process which may or may not require you to run diagnostics from the manufacture. The other drive is fine so far. I'm sure the short and long test will show it's OK. See section saying. === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: FAILED! Drive failure expected in less than 24 hours. SAVE ALL DATA. See vendor-specific Attribute list for failed Attributes.
Archived
This topic is now archived and is closed to further replies.