Greetings,

 

My server suffered a blown power supply and went down. I replaced it and starting the system initiated a parity check as expected. All seemed well and it ran all last evening without showing any drama.

 

Disk 15 was showing some sign of errors under SMART (prior to this), and I'm was going to replace it after this parity check.

 

When I got up this morning, the Tower webgui was unreponsive. I used unMenu and it loaded. I checked the syslog (included here) and it showed lots of Disk 15 errors, but the parity reading at the top of the screen still shows none (see below)

 

Total Size 1,953,514,552 KB

Current 820,323,848 (42.0%)

Speed 3,516 KB/sec

Finish 5350 minutes

Sync Errors 0 (corrected)

 

I do noticed the speed is way slower than it should be (It's usually 14 kb/sec).

 

Now unMenu will not load and I cannot check what is going on.

 

Is there some way through the cmd at the terminal to check it?

Will the parity check be invalid with all these errors?

Should I stop the check? (it was due to complete in about six hours from now)

What should my next steps be.

 

Thx.

 

syslog_17-12-2013_-_Disk_15_Read_Error.txt

Reboot your desktop/laptop -- NOT your UnRAID server -- and see if you can then access the WebGUI and/or UnMenu.

 

It's possible to encounter read errors without having any sync issues.    When a read error occurs on a disk, UnRAID re-writes the failed sector, which it restores by using the data from all the other disks plus the parity drive.  As long as that write is successful, the disk remains in service.  This does not result in a sync error on a parity check ... although clearly that specific sector's "slice" in the parity computation is assumed to be correct, since it's used for the reconstruction.

 

I suspect you're having a lot of reconstructed sectors on disk 15 ... which is why the check has slowed down so much.    Note that 5350 minutes is almost 90 hours -- not "six hours from now".    So if you want to wait for it to complete, I'd wait at least 4 days ... 5 to be a bit safer.    As long as there is disk activity ongoing, I'd let it finish if possible.

 

Do you have a monitor attached to the UnRAID system?  Does it still show "Tower Login" ??  ... or have there been error messages from Linux?

 

Hey Gary,

 

First thing I did was reboot my desktop, and both LT webgui and unmenu are still not connecting to the server.

 

I do have a monitor attached to the server and TowerLogin is available. I can ping the server from my desktop and it replies.

 

I've never used the time shown in the GUI to figure the finish time. It has never taken longer than 22 hours to complete anything. There could be a first time though

 

I was planning to wait until this evening to see what happens. Obviously, you can't find out what is going on at the terminal??

 

Can I trust the parity check to rebuild my replacement disk?

 

Thankfully, there isn't much on that disk and I thought I might just copy it off before rebuilding the replacement - just as a reassurance measure.

 

Thx for the input.

I would definitely copy off the data from the failing disk before doing the rebuild if you don't have backups of your data (I keep a complete set of backups, but not everyone does).

 

Whether or not you wait for it to finish is up to you.    You could simply press the power button on the server (just a momentary push) and it should do a correct shutdown.  That will, of course, stop the parity check.  I'm not sure if it will save the log off or not when it's aborted.  You can check the log files on the flash drive when you reboot to see if there's any data available.

 

I'm not a "Linux person" ... so I don't know if there's a way to log in to the server from the login prompt and check the status or not => hopefully somebody will chime in and let you know if there's a way to do that.  I'd at least wait a few hours to see if you get some guidance on this from one of the Linux experts.

 

IF parity is good, you can trust the system to rebuild the failed disk;  but given the power surge you likely had from the PSU failure, there may be some doubt about the validity of your parity drive.    Personally, I'd be inclined to (a) attach drive #15 to your PC and see if you can read it okay (use the free LinuxReader utility so Windows can "see" the data);  (b)  if you can read #15 okay, then put a new drive in its place in the array; do a New Config and simply let it build a good parity for the new configuration; and then copy the data from the old #15 to the array from your PC.

 

 

I figured copying the data off first would be the right thing to do. I do have all the disks for backups, but copying the info off disk15 is a better option I think.

 

I think I'll wait until 18:00 tonight. It should've finished by then (in theory). I can still access the data through network places, so it hasn't hung.

 

Yeah, I wish I knew if you could check things at the terminal. Would be nice, because the webgui seems to always go down when there is an issue

 

Hopefully, if it completes it's parity check, then the webgui will come back - but I'm not holding my breath!

 

Can you save the current log (doesn't seem it's available on the USB stick until after you reboot), so I can see what it's doing. Surely this can be done from the login prompt?? Perhaps even pull the current up on the monitor. That would at least show me what's happening.

 

Much appreciated.

Follow this to get a copy of the current syslog: http://lime-technology.com/wiki/index.php/Troubleshooting#Capturing_your_syslog

 

I'm also no Linux expert, but I see no reason why you shouldn't be able to do this through the console.

Follow this to get a copy of the current syslog: http://lime-technology.com/wiki/index.php/Troubleshooting#Capturing_your_syslog

 

I'm also no Linux expert, but I see no reason why you shouldn't be able to do this through the console.

 

Thx - that helped.

 

Shows the disk errors went on for a couple of minutes, stopped then started again 3 hours later, then stopped. Now it shows a couple of the smaller drives have finished (because they have spundown). I'll leave it running. The webgui is still unavailable.

 

 

 

 

Isn't it funny how we have these tech issues at the busiest times...

 

We suffered an extended power cut yesterday that was beyond the capacity of my UPS, so the server shutdown gracefully before it had completed it's check. At that point, there were no errors.

 

Later, I started copying the video data off Disk15 to a backup disk and I got more Disk15 Read Errors. They showed in the unRAID gui under the errors column. Like last time, the read errors only lasted for a couple of minutes over two parts of the disk.

 

I began a verify parity check (without correction) and so far no parity errors.

 

I think I will stop that and just rebuild the drive.

 

What does data corruption look like? Virtually all the files on my server are DVD ISO files (+ info files). Will DVDShrink open a corrupted ISO file??

 

Is there a BEST way to determine if the data is okay at the completion of the rebuild?

 

Thx.

Is there a BEST way to determine if the data is okay at the completion of the rebuild?

Binary compare with your backups.

I do have all the disks for backups,

I know imgburn can do compares between an iso and a disk.

Solved:

 

The new HDD has rebuilt and a parity check is now in session. Seems everything is as it should be. Much appreciated for the assistance. Thx