January 23, 201016 yr I have an unraid box that recently started randomly crashing on me. Here is what I have done so far Ran Memtest over night, no errors Removed hard drives and used Spinrite to look for bad sectors none were found. Plugged hards drives in windows box to scan with Hdtune, no errors found, 1 drive has 6 relocated sectors appears to be the only thing that sticks out in smart Reporting.. I am running 4.5, but when problem originally started I had been running the previous 4.4 final build. Often times after a crash Disk 4 shows as unformated while the system does something with that drive until it finally catches up. Note its the parity drive that shows the 6 relocated sectors. So now I know you guys want to know what error is being displayed.... Well I made the mistake of assuming a logfile would be created. So I can't provide that right now until the next time it crashes... Whats the best way to retrieve log files from Unraid? Unmenu Syslog appears to only display the most current logs from the last bootup. So if unraid hides logs anywhere please let me know.
January 23, 201016 yr The syslog is created on an in-memory file-system. It is lost if the server hard-crashes. The only way to capture it is to follow it on the system console using tail -f /var/log/syslog Other than hardware issues causing crashes, the most common "software" caused crash is a corrupt file-system. You might want to run a file system check on each of your data disks. The procedure is described here in the wiki. Second most common is running out of memory as the syslog uses it all, filling with errors. You'll see those with the "tail -f" as described above. After a hard-crash and a reboot the reiser file-system takes some time to replay the transactions in its journal. during this time it is not yet mounted. Until it is mounted it will (inaccurately) show as un-formatted. Once mounted the correct status will be shown. Just be patient when restarting after a hard crash... and don't press the "format" button. Joe L.
January 23, 201016 yr Author Thanks for the input, I did that tail thing before for syslog but I had done it in a telnet session, i will try again at the console.. I think I'm going to start with the file system checks as I have had so many hard crashes lately I would be very surprised if their were not errors.. Right now I'm at a point where its almost always crashing when doing a Parity check or copying data to the array.
January 23, 201016 yr Author So even with that Syslog thing running on the console nothing is written to log files still correct? Had another hard crash during the parity rebuild. The screen on the console looked the same as always did with nothing extra from having sys log written.. I need to get a camera to take a full picture of all the error being displayed but here are some bits from the error that may be helpful Md_Recovery Call Trace get_active_stripe unraid_sync md_do_sync autoremove_wakefunction md_do_recover md_thread auto_removewakefunction md_thread kthread Kthread kernel_thread_helper Code EIP end trace B117207c86b61930 All the above lines were all surrounded by random numbers and cryptic coded information I didn't understand. Is there still no log file to look up? Is a photo going to be my only option to share the entire error message? I did the file system check, 1 drive had 2 errors recovered, the other was fine.
January 23, 201016 yr Author No my 3 drives include WDC_WD10EADS-00L5B1_WD-WCAU45468634 WDC_WD10EAVS-00D7B1_WD-WCAU46543731 WDC_WD10EADS-00L5B1_WD-WCAU46900940 This is my motherboard http://www.e4allupgraders.info/dir1/motherboards/socket754/msi7145.shtm And I had been using the Onboard SATA ports but in troubleshooting I switched to a Promise TX4 adapter to see if that helped anything out.. Only thing the adapter has done so far is make the parity checks move a lot slower before still hard crashing.
January 23, 201016 yr Author I have attached my syslog from the last bootup just in case some problem sticks out here.. syslog-2010-01-232.txt
January 23, 201016 yr You would not be the first who posted a "picture" of a crash. Sometimes it is the only way. (Actually, one user did hand-transcribe the entire screen. A lot more work than a picture)
January 23, 201016 yr Author Yeah its kind of lame that no logging takes place. You would think something could be setup to log to the USB flash drive or something...
January 24, 201016 yr Yeah its kind of lame that no logging takes place. You would think something could be setup to log to the USB flash drive or something... That's easy. Add the following lines to your 'go' script. mv -f /boot/flashlog.current /boot/flashlog.last 2>/dev/null cat /var/log/syslog > /boot/flashlog.current echo "*.debug /boot/flashlog.current" >> /etc/syslog.conf /etc/rc.d/rc.syslog restart Reboot your server. Now, even if your server crashes completely, you'll still have the logs on your flash key. A "picture" of the crash! Purko
January 24, 201016 yr Author Good to know...... Despite my memtest running for many hours successfully I have decided to remove a single stick of memory and see if things work any better for me. So far I am upto 60% on my Parity check which is farther than I have reached in a week or two now so I am crossing my fingers, but not very hopeful that this was actually my fix.
January 24, 201016 yr So far I am upto 60% on my Parity check which is farther than I have reached in a week or two now so I am crossing my fingers, but not very hopeful that this was actually my fix. In that case you should really make the above modification to yout 'go' script. Otherwise, when it crashes we won't know what happened. Then after a couple of months of crashless run you can remove those lines.
January 24, 201016 yr Please try a diferrent network card and report back. I had this issue. Plagued me for a year. I even swapped mother boards and that did not work. But, a new network card and all has been good.
January 29, 201016 yr Author So far I haven't had any crashes since removing a single stick of RAM... Again Memtest came back clean but one of the sticks I had was a 'performance grade' and I have had issue with those before in that certain motherboards just don't like em. So I'm crossing my fingers and hopping my problem is fixed.. I am doing File system checks and another Parity check today in prep for a new 2tb parity drive... -Josh
January 29, 201016 yr Joe L. made an interesting point about mixing ram sticks in this post: http://lime-technology.com/forum/index.php?topic=5122.msg47872#msg47872 Since one of your sticks is performance grade, the other stick of RAM might not like the more aggressive timings of the performance grade stick.
Archived
This topic is now archived and is closed to further replies.