February 15, 201412 yr Hi everyone whos taking the time to help me out, thanks in advance! So, back in July/August my unraid 4.7 machine was being a pain in the butt. Something was wrong with my USB stick and it wouldn't boot. Unfortunately I can't remember the error message I was getting. I tried a bunch of troubleshooting and couldn't get it to work so I just said screw it. Fast forward to today, I decided I'm finally going to do something about it. I re-formatted my flash drive, copied the config directory back in, and put unraid back on it. I decided maybe 5.x would help some things, so i put 5.0.5 on there. My machine booted right up, no issues there. Network config is still there, things seem to be good. One issue though, on two of my 6 drives are saying "wrong". Disk 2 and 3 in particular, the parity drive is okay. At the bottom the array status says: Stopped. Two or more disks are wrong. The two disks are correct with their serial numbers, so I'm not sure why it says its wrong. I've attached a screenshot of it as well as my syslog.txt. Any help you all can give me would be super appreciated! Thanks again! syslog.txt
February 16, 201412 yr The drives have HPA. See here: http://lime-technology.com/forum/index.php?topic=10866.0
February 16, 201412 yr Author The drives have HPA. See here: http://lime-technology.com/forum/index.php?topic=10866.0 Hi dgaschk, thanks for the quick response. I did the hdparm commands and it did show an HPA. I disabled it and rebooted my server. Now, when doing hdparm -N /dev/sdc and sdd I see this: root@Tower:~# hdparm -N /dev/sdc /dev/sdc: max sectors = 3907029168/3907029168, HPA is disabled root@Tower:~# hdparm -N /dev/sdd /dev/sdd: max sectors = 1953525168/1953525168, HPA is disabled root@Tower:~# I checked my new syslog and I don't see anything regarding HPA...but my drives still say "wrong". Any idea? syslog2.txt
February 16, 201412 yr Author Whoops, just realized theres more to that post that I didnt see. Let me try the rest of this and I'll get back to you.
February 16, 201412 yr Author Before I go any further, I've been following this thread: http://lime-technology.com/forum/index.php?topic=5072.msg46903#msg46903 I wasn't sure if I needed to do the reiserfsck stuff, and the commands on the next page look significantly more severe than what I'm having. Can you just give me a quick pointer on the stuff I do and don't need to run? I think rebuilding an entire partition table seems a bit much, and I don't want to make it worse than it already is
February 16, 201412 yr Author Hi again everyone, Just for reference, here was the other thread I was looking at. Perhaps this would help my issue as it thinks the partitions are different sizes? Not sure, could use some direction. Thanks! http://lime-technology.com/forum/index.php?topic=5072.msg47122#msg47122
February 17, 201412 yr The disks need to be fixed one at a time using the last version of unRAID that they worked in. Revert the system to the older version. Undo the hdparm commands by running hdparm, setting the disks to the original size. All of the array disks must be green and the system must pass a parity check before proceeding. Once the array is green, run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. Repeat with the other disk. Once the array is all green, and after the parity check, upgrade the system.
February 18, 201412 yr Author The disks need to be fixed one at a time using the last version of unRAID that they worked in. Revert the system to the older version. Undo the hdparm commands by running hdparm, setting the disks to the original size. All of the array disks must be green and the system must pass a parity check before proceeding. Once the array is green, run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. Repeat with the other disk. Once the array is all green, and after the parity check, upgrade the system. Okay sounds good. I just put 4.7 back on there, and it booted properly. One of the drives gave "input/output error" when i went to remove the HPA (still in 5.0). Now one of the drives shows a blue circle next to it, and it is now doing a data rebuild. Things are looking up hopefully, we'll see how it looks in the morning. Thanks again for all of your help.
February 18, 201412 yr Author Okay, morning update: The array finished and now all shows green, parity says valid. I can't access my main media share, but after a bit of googling looks like it might be because of user shares? Not sure. I tried doing an ls on the /mnt/use/media directory and that just kind of hung on me (maybe it was taking a while, not sure). The other issue is, I just did an hdparm on my two drives with hpa, and this is what I get: root@Tower:~# hdparm -N /dev/sdc /dev/sdc: max sectors = 1953525168/7368112(1953525168?), HPA setting seems invalid (bug gy kernel device driver?) root@Tower:~# hdparm -N /dev/sdd /dev/sdd: max sectors = 3907029168/14715056(18446744073321613488?), HPA setting seems i nvalid (buggy kernel device driver?) root@Tower:~# Interestingly, it shows the same thing if i tried sda or sdb as well, which don't show HPA issues in the log. Speaking of logs, I've attached the new one (this one running 4.7 now). syslog47.txt
February 18, 201412 yr Once the array is green, run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. Repeat with the other disk. Be careful not to work ahead... "Once the array is green: run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. and then Repeat with the other disk."
February 18, 201412 yr Author Once the array is green, run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. Repeat with the other disk. Be careful not to work ahead... "Once the array is green: run the hdparm command on 1 disk. Rebuild the disk and the run a parity check. and then Repeat with the other disk." Yep, I think I inadvertently ran it on one disk, as one of the two disks didn't go back to the original value (gave the input/output error). I rebuilt it last night, but now I can't seem to set the HPA on the other one (or, maybe I could force it, but its giving weird results when i check the status, see my last post). I was thinking, would it be possible now to upgrade it to 5.0, then (since I should only have one disk with HPA) set the remaining HPA disk and rebuild from parity? Running from 4.7 obviously would be better, but I'm not sure how to interpret the hdparm results from my last post.
February 18, 201412 yr Yes. You can leave HPA on 1 disk. It does no real harm; just a few bytes smaller. Just don't try to use that disk for parity or try to replace a slightly larger disk with the HPA disk.
February 21, 201412 yr Author Okay, I moved to 5.0, and it had the one disk left to rebuild, which I did. My array is all showing green. Heres my current problem: I'm having trouble accessing my shares. If I go to browse them it seems like bad perms. On telnet, I can get to /mnt/user, but when i go to /media and try an ls, it just hangs forever. I tried the new perms script in utilities, it gets past the first disk, then hangs on the 2nd disk. Like this: /usr/local/sbin/newperms processing /mnt/disk1 ... chmod -R u-x,go-rwx,go+u,ugo+X /mnt/disk1 ... chown -R nobody:users /mnt/disk1 ... sync processing /mnt/disk2 ... chmod -R u-x,go-rwx,go+u,ugo+X /mnt/disk2 Its been like that for about 20 hours now. Any ideas? EDIT: I forgot to mention, top still shows that the script is running at 100%, but at the same time, all of my drives are spun down, so they aren't actually doing anything, correct? I was thinking maybe something was specifically wrong with disk2? Hopefully theres a more elegant solution out there, but I thought about the possibliity of pulling drive2 and having the parity take over, then seeing if i can set perms?
February 22, 201412 yr Author Attach a syslog. Sorry, was in a rush this morning, I knew i forgot something! syslogperms.txt
February 22, 201412 yr The syslog doesn't indicate a problem. Reboot and try again. Post a new syslog and note the time of the problem.
February 23, 201412 yr Author Okay, heres after a fresh reboot. I also just started up two instances of putty. one is running top, and the other just did an ls command on /mnt/user/media (which is my main share, and hangs forever). My other 3 shares, none of which have any files in them (which is correct) are responding fine and i can create files in them no problem. After I did the ls command, my top looked like this: top - 00:32:42 up 3 min, 2 users, load average: 0.62, 0.25, 0.10 Tasks: 98 total, 1 running, 97 sleeping, 0 stopped, 0 zombie Cpu(s): 8.8%us, 41.3%sy, 0.0%ni, 49.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 4148844k total, 262332k used, 3886512k free, 1420k buffers Swap: 0k total, 0k used, 0k free, 198580k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1281 root 20 0 8096 844 552 S 100 0.0 0:55.00 shfs 1325 root 20 0 2440 1048 824 R 0 0.0 0:00.13 top 1 root 20 0 828 280 240 S 0 0.0 0:05.77 init 2 root 20 0 0 0 0 S 0 0.0 0:00.00 kthreadd 3 root 20 0 0 0 0 S 0 0.0 0:00.01 ksoftirqd/0 4 root 20 0 0 0 0 S 0 0.0 0:00.00 kworker/0:0 Before I rebooted, top looked like that as well, it had been at 100% cpu for basically 36 hours. the cpu numbers in the top right were showing 10.XX on all 3. Seems like something is wrong with the "media" share, which is the same share where all my data is. syslogfeb22.txt
February 23, 201412 yr Author You know, I just did a random google search on ls hanging on a directory...they mentioned NFS mount points. I did have NFS working from my esxi cluster to my unraid box way back when, but it wasn't to the media share, it was to the vmware share. Still, could that have something to do with it? I just looked on the settings page of unraid 5, and nfs is set to no.
February 24, 201412 yr Author See Check Disk Filesystems in my sig. hey dhaschk, I ran the reiserfsck command on my md2 drive (which is the one that the perms kept getting hung on) and it looks pretty bad. heres the end of it: bad_stat_data: The objectid (21626) is marked free, but used by an object [21623 21626 0x0 SD (0)] bad_stat_data: The objectid (21627) is marked free, but used by an object [21623 21627 0x0 SD (0)] bad_stat_data: The objectid (21628) is marked free, but used by an object [21623 21628 0x0 SD (0)] bad_stat_data: The objectid (21629) is marked free, but used by an object [21623 21629 0x0 SD (0)] finished Comparing bitmaps..vpf-10640: The on-disk and the correct bitmaps differs. Bad nodes were found, Semantic pass skipped 3 found corruptions can be fixed only when running with --rebuild-tree ########### reiserfsck finished at Sun Feb 23 20:02:28 2014 ########### root@Tower:~# The wiki says to be very cautious running the rebuild-tree command, so I figured I would check with you experts before starting that. What do you think? EDIT: I was able to run it again and put it in a log file so I could capture everything. I've attached the full log from md2. md2.txt
February 24, 201412 yr The warning is to make sure that people are using the correct device identifier. As long as you're using /dev/md2 then run the command with --rebuild-tree.
February 24, 201412 yr Author The warning is to make sure that people are using the correct device identifier. As long as you're using /dev/md2 then run the command with --rebuild-tree. Okay, thats what I was hoping. Sounds good, I'll give it a shot!
February 24, 201412 yr Author Okay, the repair of md2 worked, and the rest of the drives looked okay. I'm now able to access the share properly! Heading into work, but I'm starting the new perms script again, but I think once that is done I should be okay. We'll find out!
February 24, 201412 yr Author Well, the perms script finished, and I just tried accessing my data: Everything is working great! Thanks everyone for the support, and especially dgaschk for his (im assuming his, not her) help through all of this mess. Wouldn't have been able to figure it out without you! Thanks again!
Archived
This topic is now archived and is closed to further replies.