I don't even know where to start with this one, but it's a real pain.
The server will simply stop responding to any traffic and, because it's headless (no gpu, no display ports on motherboard), I have no choice but to cold restart.
You may note that drive 5 is missing. This drive had quite a lot of smart errors, so I took it out to see if that was somehow causing issues. Ideally I would think that drives misbehaving may result in it getting disabled (and certainly shouldn't take out the server), but it doesn't seem to have helped.
lime-diagnostics-20231013-0844.zip