So a scheduled parity check went off tonight and right away finds 128 errors on seven disks, hmmm seems a bit odd. I had this happen last month as well but just thought it was a one off weird thing, guess not. I stopped the parity check then started it manually again and let it run for a little over an hour, it didn't register any errors. I know those disks are fine, they've passed every parity check I've run before so I'm just wondering what could be triggering them. Diags attached.

tower-diagnostics-20170430-0116.zip

Edited April 30, 20179 yr by ashman70

Were both of them the correcting parity check kind? If you didn't run in correction mode then there's really no way the errors would be corrected unless you rewrote those exact troubled sectors on each drive

Apr 30 00:03:47 Tower kernel: md: disk12 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk15 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk17 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk18 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk20 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk21 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: disk22 read error, sector=0
Apr 30 00:03:47 Tower kernel: md: recovery thread: multiple disk errors, sector=0
Apr 30 00:03:47 Tower kernel: md: disk12 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk15 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk17 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk18 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk20 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk21 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: disk22 read error, sector=8
Apr 30 00:03:47 Tower kernel: md: recovery thread: multiple disk errors, sector=8

Do these disks have share anything in common? backplane/cables/etc?

The box to correct errors was ticked. All of those drives share the same backplane, cable and HBA. I have only one HBA in my system, all of those drives are on the front backplane of my server.

Then possibly a problem with some of the backplane slots.

The only reason I doubt that is that it the errors only show up when a scheduled parity check goes off, I believe is the same slots and it finds the same amount of errors, doesn't that strike you as weird? Otherwise, I get no errors or any indication that there is any kind of hardware issue. Could it be some kind of bug or is it something about my system?

Those are not parity sync errors, they are read errors, i.e., those sectors are not being read.

Right but I don't believe they are real, I believe they are false positives because other than this instance, I have no other indications that there is anything wrong with those drives. The question is why would the errors register at all?

The read errors are definitely real and a real problem, probably hardware related, sync errors they cause are likely false positives and a consequence of the read errors..

See I don't agree. You are saying I have seven disks with read problems that only show up when a scheduled parity sync occurs? If I do a short smart test on the drives and they come back ok will you be convinced there is nothing wrong with them? Or is this a HBA problem? But if so, why only with these seven drives and why only when a scheduled parity sync occurs?

I never said it's a disk problem, in fact I'd say it's practically impossible to have so many disks fail at the same time in the same sectors, but there's definitely a hardware problem somewhere.

Ok lets assume for a minute that you are correct, there is a hardware problem somewhere. If its not the drives, by process of elimination it can only be:

 

1. The HBA

2. The backplane

3. The cable (which was replaced with a brand new one a few months back when I was having other problems)

 

Now lets bear in mind that outside of when these errors show up when a scheduled parity check goes off, the system functions normally without any other errors or problems. Data is written to the array on a daily basis and read on an almost daily basis (plex).

 

How would one go about trying to narrow down where this problem is?

I would start by swapping slots from one of the disks with errors to one without errors to see if the errors stay with the backplane slot.

So to do that I would have to move the disks and do a new config, correct?

You don't need to do a new config at all or even mess with the config. Just adjust how the 2 data drives are physically connected.

So just swap the drives into different slots? That is easy.

5 minutes ago, ashman70 said:

So just swap the drives into different slots? That is easy.

 

Yes