Hi,

 

Before I mess anything up, just looking for advice how to proceed.  I have 24 drives and 1 parity (I know I need to add another one, but my case is full.)

 

System has been stable for 12+ years.  Last parity check a couple of months ago with no issues (0 errors).  I was going to upgrade a drive or 2 to get rid of old drives and increase the size of the 2tb to 8tb.

 

I started a parity check to make sure it was good to go before the upgrade, and about 41% in, I got 1 drive disabled and 4 others with errors. (all read errors)

 

My gut tells me its a loose connection issue.  My webGUI has has paused the operation.

 

Should I powerdown and re-seat all connections and do another parity check? What about the current disk that is disabled?

 

Thanks,

 

-R

 

syslog.1.txt syslog.2.txt syslog.txt

attach diagnostics to your NEXT post in this thread.

 

2 hours ago, visionmaster said:

Should I powerdown and re-seat all connections

Probably. Since multiple disks are involved might also reseat controller. Could also be a power problem, you have a lot of disks.

 

2 hours ago, visionmaster said:

do another parity check?

Since you have single parity and a disabled disk, it won't do a parity check. It would only do a read check. Not much point.

 

2 hours ago, visionmaster said:

What about the current disk that is disabled?

It will have to be rebuilt, but need to see if the emulated filesystem is mountable first.

 

Many other things to check before giving further advice. If it isn't already, start the array in normal (not Maintenance mode), then

1 hour ago, trurl said:

attach diagnostics to your NEXT post in this thread.

 

Thanks for the replies so far.

 

Is there a way to reenable disk 16 which got disabled (if it were related to a loose connection) and trust the parity.  And recheck everything from there?

 

Thanks,

-R

tower-diagnostics-20230214-0728.zip

33 minutes ago, visionmaster said:

Is there a way to reenable disk 16 which got disabled (if it were related to a loose connection) and trust the parity.  And recheck everything from there?

 

Unraid disables a disk when a write to it fails. After a disk is disabled, the physical disk isn't used again until rebuilt. The emulated disk can be read from the parity calculation by reading all other disks. The emulated disk can be written, including that initial failed write, by updating parity. So, physical disk16 is out-of-sync with parity.

 

You could New Config and rebuild parity instead of rebuilding the data disk. You could New Config and trust parity, but you would need to run a correcting parity check to get the array in sync.

 

Best idea is to rebuild the data disk and recover the emulated writes. If you don't those will be lost, and possibly some of the emulated writes were filesystem metadata necessary to keep the filesystem from corrupting.

 

Will look at diagnostics and comment on next post.

Emulated disk16 is mounted and has plenty of data. Should be OK to rebuild after you have taken care of whatever hardware problems caused multiple disk errors.

03:01.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller [11ab:6081] (rev 09)
03:02.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller [11ab:6081] (rev 09)
04:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)
05:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)

Marvell controllers are NOT recommended these days

 

Feb 14 07:29:03 Tower kernel: sas: ata31: end_device-24:0: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata32: end_device-24:1: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata33: end_device-24:2: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata34: end_device-24:3: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata35: end_device-24:4: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata36: end_device-24:5: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata37: end_device-24:6: dev error handler
Feb 14 07:29:03 Tower kernel: sas: ata38: end_device-24:7: dev error handler

These are all on

05:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)

 

Disk16 is on this other one

04:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)
	Subsystem: Super Micro Computer Inc MV64460/64461/64462 System Controller, Revision B [15d9:0500]
	Kernel driver in use: mvsas
	Kernel modules: mvsas

 

No SMART report for disk16, so after you fix your hardware problems, post new diagnostics, or at least a SMART report for that disk.

 

Do any of your other disks have SMART warnings ( 👎 ) on the Dashboard page?

No SMART reports for disks 17, 13, 18, 14, 15, 12 either.

 

No point in answering this question

11 minutes ago, trurl said:

Do any of your other disks have SMART warnings ( 👎 ) on the Dashboard page?

Until those are all fixed.

 

If it has been working OK up to now, unlikely all these controllers would go bad at the same time.

 

Reseat all controllers. Check all connections, both ends, including power and splitters.

 

All disks are going to be running with parity checks and rebuilds, and you have a lot of disks. Maybe you have a power supply problem?

OK thanks for the guidance.  So far I've changed the power supply from the 750w to a 1000w that I had.  I checked all connections and reseated everything.  I changed out disk 16 to a spare and started rebuild.  It mounted fine and the data was emulated.  About a minute in, disk 15 started racking up read errors and then no data was visible on that disk or 16. 

 

I stopped and shut down and moved those 2 drives onto a different controller and booted up again.  Everything appears good with no errors.  Rebuild restarted and the data is emulated again and visible on both drives.  I guess I'll let it finish, and hopefully with no errors. 

 

I'm going to research a whole new build to get this 13 year old hardware updated.  I'd like a big case like a rack mount 36 bay Supermicro hot swap. 

 

Thanks again for the assistance.

-R 

Going forward, you need more capacity, and you need to get the remaining reiserfs disks converted by 2025.

 

I recommend upsizing your smaller xfs disks, that way you will have more space on disks that are already the correct filesystem, and you can use that space to copy data from your reiserfs disks so you can reformat them to xfs.

 

At some point maybe you will have enough data space that you could remove a small reiserfs disk after copying its data. Then you could add parity2.