Hi all,
I've got a pushover notification a few minutes ago telling me there was 1 read error, but it appeared on all my drives at once.
I stupidly decided to start a parity sync and noticed non of my drives were reachable anymore :
https://app.screencast.com/3rYhKTK4EHjzQ
I also tried to access /mnt/user ->
root@Tower:~# cd /mnt/user/
-bash: cd: /mnt/user/: Input/output error
here is an extract of what syslog tells me:
```
Mar 17 10:54:51 Tower root: ddns-updater: Could not download icon https://github.com/DiamondPrecisionComputing/unraid-templates/blob/main/templates/img/ddnsgopher.png?raw=true
Mar 17 11:16:54 Tower kernel: md: disk1 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk2 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk3 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk4 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk5 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk6 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk7 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk8 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk9 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk10 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk11 read error, sector=0
Mar 17 11:16:54 Tower kernel: md: disk0 read error, sector=0
Mar 17 11:16:54 Tower kernel: XFS (md1): metadata I/O error in "xfs_alloc_read_agfl+0x84/0xc3 [xfs]" at daddr 0x3 len 1 error 5
Mar 17 11:16:54 Tower kernel: md: disk1 read error, sector=12886351208
Mar 17 11:16:54 Tower kernel: md: disk0 read error, sector=12886351208
Mar 17 11:16:54 Tower kernel: md: disk1 read error, sector=12886351216
Mar 17 11:16:54 Tower kernel: md: disk0 read error, sector=12886351216
Mar 17 11:16:54 Tower kernel: XFS (md1): log I/O error -5
Mar 17 11:16:54 Tower kernel: XFS (md1): Corruption of in-memory data (0x8) detected at xfs_defer_finish_noroll+0x479/0x503 [xfs] (fs/xfs/libxfs/xfs_defer.c:573). Shutting down filesystem.
Mar 17 11:16:54 Tower kernel: XFS (md1): Please unmount the filesystem and rectify the problem(s)
Mar 17 11:17:33 Tower kernel: mdcmd (36): check correct
Mar 17 11:17:33 Tower kernel: md: recovery thread: check P ...
Mar 17 11:17:33 Tower kernel: md: recovery thread: multiple disk errors, sector=0
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=8
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=16
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=24
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=32
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=40
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=48
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=56
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=64
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=72
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=80
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=88
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=96
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=104
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=112
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=120
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=128
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=136
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=144
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=152
Mar 17 11:17:33 Tower kernel: md: disk1 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk2 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk3 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk4 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk5 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk6 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk7 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk8 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk9 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk10 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk11 read error, sector=160
Mar 17 11:17:33 Tower kernel: md: disk0 read error, sector=160
```
Can someone help me out figuring out what's happening and what broke ? It's pretty urgent, all my dockers are running from there
I've attached full diagnostics zip.
thanks !
tower-diagnostics-20230317-1141.zip