Hi everyone,

I’ve been struggling with recurring issues on my ZFS setup. My main storage pool keeps throwing errors every now and then.

To rule out hardware problems, I’ve already:

• replaced all the RAM,

• installed a new HBA card,

• and swapped out all the cables.

Despite that, the issue keeps coming back (see attached photo). After a reboot, the affected drive comes back online and everything seems to work fine again.

Unfortunately, the diagnostics I captured are from after the reboot, so the issue itself isn’t visible anymore. Still, maybe someone can spot something that might explain why this keeps happening?

Thanks in advance for taking a look and sharing your thoughts!

beanserver-diagnostics-20251203-1129.zip

15 minutes ago, beantje said:

the diagnostics I captured are from after the reboot

Since it's saving the previous syslog to the flash drive, it still shows the issue:

Dec  2 19:19:58 BeanServer kernel: sd 10:0:1:0: Power-on or device reset occurred
### [PREVIOUS LINE REPEATED 1 TIMES] ###
Dec  2 21:01:59 BeanServer kernel: sd 10:0:1:0: device_block, handle(0x0009)
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: device_unblock and setting to running, handle(0x0009)
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: [sdc] Synchronizing SCSI cache
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: [sdc] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=DRIVER_OK
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: mpt3sas_transport_port_remove: removed: sas_addr(0x4433221100000000)
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: removing handle(0x0009), sas_addr(0x4433221100000000)
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: enclosure logical id(0x56c92bf0000390f0), slot(3)
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=16185069023232 size=8192 flags=3145856
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=270336 size=8192 flags=1245377
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=26000623935488 size=8192 flags=1245377
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=26000624197632 size=8192 flags=1245377

The disk dropped offline and then reconnected again with a different letter. This is typically a power/connection issue. Any power splitters in use? Could also be a bad/weak PSU.

28 minutes ago, JorgeB said:

Since it's saving the previous syslog to the flash drive, it still shows the issue:

Dec  2 19:19:58 BeanServer kernel: sd 10:0:1:0: Power-on or device reset occurred
### [PREVIOUS LINE REPEATED 1 TIMES] ###
Dec  2 21:01:59 BeanServer kernel: sd 10:0:1:0: device_block, handle(0x0009)
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: device_unblock and setting to running, handle(0x0009)
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: [sdc] Synchronizing SCSI cache
Dec  2 21:02:00 BeanServer kernel: sd 10:0:1:0: [sdc] Synchronize Cache(10) failed: Result: hostbyte=0x01 driverbyte=DRIVER_OK
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: mpt3sas_transport_port_remove: removed: sas_addr(0x4433221100000000)
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: removing handle(0x0009), sas_addr(0x4433221100000000)
Dec  2 21:02:00 BeanServer kernel: mpt2sas_cm0: enclosure logical id(0x56c92bf0000390f0), slot(3)
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=16185069023232 size=8192 flags=3145856
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=270336 size=8192 flags=1245377
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=26000623935488 size=8192 flags=1245377
Dec  2 21:02:08 BeanServer kernel: zio pool=mainstorage vdev=/dev/mapper/sdc1 error=5 type=1 offset=26000624197632 size=8192 flags=1245377

The disk dropped offline and then reconnected again with a different letter. This is typically a power/connection issue. Any power splitters in use? Could also be a bad/weak PSU.

I did swap the PSU before all this shit happend,

Going to replace that one, thanks