unRAID Version: 5.0b11

Hardware: One of these filled with these (bought when they were a decent price ).

 

Hi unRAID Community o/

 

I have been messing with unRAID a little for some time and had an odd error that occurs intermittently and I have had some difficulty nailing down what it is exactly. What I observe is a period of normal activity (reading, writing and parity checking) between a couple of days and a couple of weeks before I will encounter 'permissions errors' in a couple of apps which write to the server.

 

I have learnt that permissions errors usually involve some problem with the filesystem so after I check the syslog (the attached syslog is typical for what happens) I run the reiserfsck tool on the affected drive and it will always tell me that --rebuild-tree is necessary. Running that will eventually (up to 48 hours later) end up with some kind of loop error that I do not recognise but it spectacularly fails and I am forced to halt the reiserfsck tool, reboot and completely pre-clear the drive and start again.

 

This has happened on a number of different unRAID beta releases (including b8, b10, b11, b12a & b13) and each time I simply assumed it was an error with the beta I was using and have up/downgraded to test. b11 seems to be the most stable for me; I get ~1-2 weeks of uninterrupted usage.

 

I am often lost when reading the syslog but to me it looks like some kind of RAID card error. Has anyone seen/experienced this error before? Just to be clear, my server contains two AOC-SASLP-MV8 cards.

 

Many thanks for any assistance/insight offered and *ahem* ignore the dubious files my mover script is shifting. They are all legitimate Linux ISOs ;-)

syslog-2012-03-28.zip

Post SMART reports.

S.M.A.R.T. reports are attached for all drives in use in the machine.

 

/dev/sda is my cache drive.

/dev/sdb is my parity drive.

/dev/sdd - sdi are data drives.

smart_reports.zip

As an added note, none of my drives are redballed and I was able to complete a parity check last night. I see 160 errors in the errors column for my parity drive (sdb) & 64 errors on one of my data disks (sde) which have me worried.

 

I have attached a screen grab for clarity.

Post a new syslog.

Attached is a new syslog, it continues from my previous one at line 2342.

 

Is this the parity sync completing?

 

Mar 29 05:12:57 passionfruit kernel: md: sync done. time=38473sec
Mar 29 05:12:57 passionfruit kernel: md: recovery thread sync completion status: 0

 

Note, I have increased my DHCP lease time to make the syslog a little more readable

syslog-2012-03-29.zip

Attached is a new syslog, it continues from my previous one at line 2342.

 

Is this the parity sync completing?

 

Mar 29 05:12:57 passionfruit kernel: md: sync done. time=38473sec
Mar 29 05:12:57 passionfruit kernel: md: recovery thread sync completion status: 0

 

Note, I have increased my DHCP lease time to make the syslog a little more readable

Have you performed an overnight memory test recently?

No. You mean a test like memtest or something, right?

No. You mean a test like memtest or something, right?

yes.

 

Continued random parity errors are usually memory, or a power supply issue, or a defective disk.  Memory is the easiest to test.

Alright, I have powered down the server and will run memtest when I boot it again. How long should I run the tests for usually?

Alright, I have powered down the server and will run memtest when I boot it again. How long should I run the tests for usually?

We usually suggest overnight, but at least several cycles.

Not a worry, I'll leave it for 12 hours and check on it tomorrow. Do I need to do anything to save a report it spits out or will it make it pretty obvious if I have memory issues?

Not a worry, I'll leave it for 12 hours and check on it tomorrow. Do I need to do anything to save a report it spits out or will it make it pretty obvious if I have memory issues?

It runs only on the screen.  All you could do is take a picture, but if it passes, there is no need to.  If it fails, there is no need either.  (If you have not verified the memory voltage/timing, and clock speed is correct for your specific memory make/model then now is the time to fix that if the memory test fails)

if you have multiple memory strips you can try moving them around, some-times that makes a difference, or try with only one strip at a time to isolate a defective strip.

It hasn't been 12 hours but I haven't seen an error yet. If the RAM checks out, what should I test next?

Alright. I left the memory check running for a good while (40+ cycles) and all completed with no errors.

 

Is anyone able to shed light on the errors in my syslogs and/or potential problems in my S.M.A.R.T. reports?

The first few drives failed the smart tests with read errors. Each one seemed to fail that, the last few drives have never been tested. I would think this is either a seriously defective port(or ports) or a power/ram issue, just as Joe L suspects. Since the RAM checked out good, I'd move on to the next likely, being the power supply.

 

It could be the times of "normal read/write/parity" the system isn't stressed but when running parity checks or just the intermittent errors are when the system is stressed/requiring more power.

 

If you have another power supply I'd test the system with it.

 

I'm kind of leaning against a faulty disk as multiple disks are giving errors. If bought at or around the same time it is possible, but I'd like to think unlikely.

I bought the drives at different times and from a couple of dealers (whomever had the best deal at the time) so I hope it isn't the disks. I have another power supply that I'll be testing as soon as I can this weekend.

 

Is there anything I can do to deliberately stress the power supply and potentially force an error like I have been seeing? I am not wanting to break stuff but I am also unsure about how to test a power supply.

Do a parity check.

Just in case an explanation is needed, a parity check spins all the drives up at once and runs them as it checks every sector, so they stay spinning for a pretty good length of time.

 

I'd make sure to do a NON-correcting parity check.

Alrighty, I have run a couple of non-correcting parity checks with a new power supply. As far as I can tell this power supply is solid (has been running for ~12 months with no issues) and everything seems to be OK.

 

I'll run it for a while longer to see if I run into any issues.

Regarding power supplies, it is important to remember that the hotter they run, the less efficient they become.  They also become less efficient over time.  I just try to keep in mind if I'm running into really strange computer issues on older hardware (2+ years), memory tests all check out, and a CPU stress test doesn't indicate any problems, the PSU is likely at fault. 

 

Buying a cheap PSU for a computer is a really bad idea (no matter the computer in question).  You're relying on it to power every part of the PC, if it goes bad, it may impact/damage every component in your build.