September 3, 200817 yr I pulled a drive (disk3) to replace it with a large one but am having problems. After I booted with the new drive, I selected it for disk3. I then proceeded to allow the unRAID to restore that new drive (disk3). The progress for this new 750GB drive (with 1TB parity) was about 400+ minutes. In the morning, the restoration was complete but disk3 was listed as unformatted. Per the thread http://lime-technology.com/forum/index.php?topic=2092.0 and the wiki http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems, I ran reiserfsck --fix-fixable and --rebuild-tree. In the morning it still lists disk3 as unformatted. Any suggestions?
September 3, 200817 yr I pulled a drive (disk3) to replace it with a large one but am having problems. After I booted with the new drive, I selected it for disk3. I then proceeded to allow the unRAID to restore that new drive (disk3). The progress for this new 750GB drive (with 1TB parity) was about 400+ minutes. In the morning, the restoration was complete but disk3 was listed as unformatted. Per the thread http://lime-technology.com/forum/index.php?topic=2092.0 and the wiki http://lime-technology.com/wiki/index.php?title=Check_Disk_Filesystems, I ran reiserfsck --fix-fixable and --rebuild-tree. In the morning it still lists disk3 as unformatted. Any suggestions? Did you press the "Start" button to reconstruct the contents of the old data drive onto the new, or did you press the "Restore" button? It makes a HUGE difference. (Hint: you should have pressed the "Start" button.) Also, what version of unRAID are you running? Please attach a syslog. It will help in diagnosing what is happening.
September 3, 200817 yr Author Hmmm, start or restore? I honestly cannot recall which I clicked. I would guess, if I were given the choice, I would have pressed restore. Was that wrong? I'm away from the unRAID until tonight, so no syslog until then. [bTW, at least I still have the original, smaller drive, in case I screwed this up]
September 3, 200817 yr Hmmm, start or restore? I honestly cannot recall which I clicked. I would guess, if I were given the choice, I would have pressed restore. Was that wrong? I'm away from the unRAID until tonight, so no syslog until then. [bTW, at least I still have the original, smaller drive, in case I screwed this up] Well, if you pressed "Restore" you asked the unRAID server to replace your existing super.dat file with a new one.. And then to rebuild parity with the existing working drives as defined in your devices page. In other words, you asked it to restore itself to the same state as when you purchased it. Not to reconstruct the old data on the new drive. To get back to where you were, you now need to go through the following steps: Shut down the array and power down. Put the original smaller drive back in the array, and remove the newer larger drive. Power up the array. It will probably not start on its own, but present you with the "Start" and "Restore" choices. Since you do want to throw away the configuration with the newer drive, you do want to press "Restore" after checking its affiliated checkbox. After pressing restore, I think you still have to start the array. Press Start. Let it finish calculating parity. You want a good valid parity check with your old drive in place before doing anything else. Once the parity calculation is complete, you are back to where you were before replacing the smaller drive with the new, Now, lets try this one more time... Stop the array, power down, replace the old drive with the new larger one, power back up. Check the checkbox next to the "start" button and press it. DO NOT PRESS THE RESTORE BUTTON. It does not restore data. It resets an initial configuration. It is very poorly named... See the wiki for further details (under Evils of the Restore Button): http://lime-technology.com/wiki/index.php?title=Best_of_the_Forums#Newbie.27s_Corner Once the Start button is pressed, you will see the new drive being rebuilt with the contents of the old. When it is done, you will be back up an running with full parity protection on all your drives. Fortunately for you, you have your old smaller data drive... If instead it had failed, and you were attempting to rebuild it on a replacement drive, your use of the poorly named "Restore" button would have caused you to erase all knowledge of the data on the failed drive. Not a good thing. Joe L.
September 4, 200817 yr He hit the start button not the restore. I only know because I was on the phone with him when he did it.
September 4, 200817 yr Let me add a bit more as to what happened. The proper steps were taken to add a newer larger drive to the system. The box was shut down. The drive was replaced. The box was turned back on. The check box was checked to say yes I want to do this etc. Then the start button was hit. The drive ran for hours just as it should and once done came up as unformatted. Then the reiserfsck was run and found errors. The "fix" commands were run and the result was a lost + found folder with some small files and a large single file (about 130gb). Although he is just using YAReG to keep a backup on a windows box and then recopied back the array, we would like to figure out the root cause. Once the drive was rebuild was there anything else that could have been done besides the fsck?
September 4, 200817 yr From what you say, it sounds like it was done correctly, and therefore you should have had a new Disk3 that was identical to the old, except a different model and serial number in the device list, and more free space. I would not attempt trying to repair the drive. Once it indicated any problem at all, then I would assume the rebuild failed, and I see no point in trying to fix a failed rebuild (unless it was all you had). I would start over, revert back to what you had, just as Joe said. Hopefully, this second attempt will prove successful. But if not, capture a syslog and let us know. Also, could you list the drives involved, and the version of unRAID? And attach a syslog.
September 4, 200817 yr Author I figured before going any further, a parity check was in order. It finished last night with over 34,000 errors. Why, I don't know, but I figure this could have a lot to do w/ why my disk upgrade failed, correct? And those errors are automatically corrected, right?
September 4, 200817 yr Correct - but with so many I would recommend running it again to get a clean report before proceeding to rebuild the drive. The nature of unRAID is that there is zero reliance on accurate parity unless there is a problem (or you are trying to upgrade a disk). This makes it VERY easy for a problem with parity to go completely undetected for long periods of time. There have been some bugs that can cause parity to get out-of-sync. Although these have all been fixed, the fixes don't result in correcting the parity errors caused by those bugs in the past. The only way to find and fix these is running a parity check. If parity is broken and you go to rebuild a drive, you will get crap. That's apparently what happened. It is HIGHLY recommended that ALL USERS perform a periodic parity check (e.g., MONTHLY) to make sure that parity is accurate and can be relied upon in case of a failure. (Running parity check also has the beneficial affect of giving each drive's SMART system a chance to remap marginal sectors to prevent future problems). It is also HIGHLY recommend that you run a parity check immediately BEFORE upgrading a disk. The chances of having a problem are dramatically reduced if you do. Good luck!
September 4, 200817 yr Author Excellent suggestions. I'm now running another parity check. In the minute after starting, it already said 103 errors. I've copied a few files into my unRAID since the last check, but that shouldn't have caused a problem. You say there were some bug fixes? I'm using firmware 4.3. I was planning to upgrade my firmware once I solved this current issues. Maybe this is too much to ask from Lime, but it might be nice to be able to schedule automatic parity checks.
September 4, 200817 yr Excellent suggestions. I'm now running another parity check. In the minute after starting, it already said 103 errors. I've copied a few files into my unRAID since the last check, but that shouldn't have caused a problem. You say there were some bug fixes? I'm using firmware 4.3. I was planning to upgrade my firmware once I solved this current issues. Capture a syslog after this parity check and post it. There was one user that had a problem that they could never get a good parity check. Each check resulted in new errors at random locations. I'm not sure there was ever a resolution. I always throught it was an issue with the cabling / backplanes. But most users get to a clean parity check after one or two runs. Truth is, though, that one parity check should be enough to get everything clean. Your syslog might help Tom figure out why a second check is sometimes needed. Maybe this is too much to ask from Lime, but it might be nice to be able to schedule automatic parity checks. This has been suggested, and there is a script to set up a cron job to do exactly that. I will try to find and post a link.
September 4, 200817 yr Maybe this is too much to ask from Lime, but it might be nice to be able to schedule automatic parity checks. This has been suggested, and there is a script to set up a cron job to do exactly that. I will try to find and post a link. The script to set up a monthly scheduled parity check is here: http://lime-technology.com/forum/index.php?topic=2250.msg17378#msg17378 It should never take more than one parity check to fix all the parity errors unless you are writing directly to the raw drives with the server stopped. If you get continued random parity errors on subsequent checks, it is usually cabling, power supply, memory, or motherboard related. Joe L.
September 5, 200817 yr Author Okay, ran a 2nd parity check. This one 24k errors. Changed the 1 IDE ribbon in my server (rest are SATA cables). Upgraded firmware from 4.3 to 4.3.3 Ran 3rd parity, now 17003 error. Now starting parity check #4. Can someone help direct me to how get a syslog so I can post it.
September 5, 200817 yr Author Apparently can't create a syslog during a parity check. Says "no space left on device". I'll try again once parity check's done. On another note, in regards to the smaller disk3 I removed from my unRAID server and started this whole mess, can I simply put it back as a new drive but not lose the original data, or do I have to use reiserfstool to drag-and-drop all the files (which would take forever)?
September 5, 200817 yr Apparently can't create a syslog during a parity check. Says "no space left on device". I'll try again once parity check's done. saving a syslog and a parity check are different operations and one should not affect the other. Where you are saving the syslog may be the issue. i.e. is /boot full do df -v to check. is the /var/log/syslog file ginourmous? do ls -l /var/log/syslog to check
September 5, 200817 yr Author df - v produces drive sizes for md1-4 and shfs. Is shfs the boot? It says 872572408 available, "Use%" "63%" syslog is 1409045, which is clearly smaller.
September 5, 200817 yr No, boot would be mounted separately. As in bash-3.1# df -v Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 505012 132692 372320 27% /boot tmpfs 1024 0 1024 0% /mnt /dev/md2 976732736 928553232 48179504 96% /mnt/disk2 /dev/md4 976732736 688729416 288003320 71% /mnt/disk4 /dev/md3 976732736 964854900 11877836 99% /mnt/disk3 /dev/md1 976732736 540351132 436381604 56% /mnt/disk1 shfs 3906930944 3122488680 784442264 80% /mnt/user /dev/sdb2 3927748 295248 3632500 8% /mnt/ram tmpfs 1024 0 1024 0% /Atlas bash-3.1# Ignore my tmpfs mounts. do a ls -l /dev/disk/by-label/ and a cat /proc/mounts
September 5, 200817 yr Author Is this what you need? Sorry for my ignorance on telnet. Filesystem 1K-blocks Used Available Use% Mounted on /dev/md3 732552188 34920 732517268 1% /mnt/disk3 /dev/md4 488371640 352351036 136020604 73% /mnt/disk4 /dev/md2 390699424 390532772 166652 100% /mnt/disk2 /dev/md1 732552188 728684304 3867884 100% /mnt/disk1 shfs 2344175440 1471603032 872572408 63% /mnt/user
September 5, 200817 yr it looks like your boot flash is not mounted or it went off line do a ls -l /dev/disk/by-label/ and a cat /proc/mounts
September 5, 200817 yr Author Here's what it spit out: rootfs / rootfs rw 0 0 proc /proc proc rw 0 0 sysfs /sys sysfs rw 0 0 tmpfs /dev tmpfs rw 0 0 devpts /dev/pts devpts rw 0 0 usbfs /proc/bus/usb usbfs rw 0 0 /dev/disk/by-label/UNRAID /boot vfat rw,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1 0 0 fusectl /sys/fs/fuse/connections fusectl rw 0 0 /dev/md3 /mnt/disk3 reiserfs rw,noatime,nodiratime 0 0 /dev/md4 /mnt/disk4 reiserfs rw,noatime,nodiratime 0 0 /dev/md2 /mnt/disk2 reiserfs rw,noatime,nodiratime 0 0 /dev/md1 /mnt/disk1 reiserfs rw,noatime,nodiratime 0 0 shfs /mnt/user fuse.shfs rw,nosuid,nodev,user_id=0,group_id=0 0 0
September 5, 200817 yr OK it says it's mounted via this line /dev/disk/by-label/UNRAID /boot vfat rw,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1 0 0 but the df disagrees with it. Can you do an ls -l /boot and report what you see.
September 5, 200817 yr Author Parity is done. 9830 errors. /dev/disk/by-label/UNRAID /boot vfat rw,fmask=0022,dmask=0022,codepage=cp437,iocharset=iso8859-1 0 0 but the df disagrees with it. Can you do an ls -l /boot and report what you see. first line reports "permission denied" second line reports "total 0"
September 6, 200817 yr Two more variations of the same commands: mount and df -aT. Try attaching a normal flash drive to your unRAID machine, in order to copy the syslog to it, and transport it to a Windows machine. See this and this for help in mounting it. However, you will need to know what file system the flash drive is formatted with, and use ntfs or ntfs-3g if it is formatted as an NTFS drive, and vfat if it is formatted with FAT or FAT32. I'm not positive but I believe ntfs-3g is now included with unRAID, so you should not need to install it. ntfs-3g is preferred over ntfs. We do need to see a syslog. Yours will be too large to attach, but zip it, and it should be around 7% of original size, and the zip file can be attached. You may (or may not!) wish to read another thread, where 2 users were repeatedly getting very large numbers of parity errors, in large bands. The thread is somewhat disheartening, and I don't believe there was a final happy resolution in either case. I've kept a Firefox tab open to it, because I had hoped to find a large block of free time to analyze the parity error patterns in their syslogs further, because the patterns do seem regular enough to be instructive, but I haven't had the time. This large parity error count problem is NOT related to your flash drive setup problem.
Archived
This topic is now archived and is closed to further replies.