Hi all.
I have been trying to diagnose a problem for over a week now. It has sorted of creeped up so I am not sure what exactly caused it to start and everytime I think I have solved it, it comes back.
Its always marked by a high load average, 20+ or in one case 6000+ when I woke up this morning. The load average is not the problem itself, there is little to no CPU usage and/or memory usage. I firmly believe its an IO problem with the array. I just don't know where the IO problems exists, everytime I think its a disk it seems not to be.
What I have tried so far:
*A reboot
*Memtest overnight, no errors
*Nearly done, have 4 disks left of converting from XFS to BTRFS, I was thinking maybe it was filesystem corruption.
The symptons of the problem is basically the unRaid box becomes useless unless I hard reboot it. The web interface remains available as does SSH but anything with the filesystem becomes useless.
This makes the dockers etc. not work as they can no longer read/write from the array.
The system is currently sitting in the state described. Load average is over 20 and the array is un-usable, no directory listings etc. vis ssh.
Diagnostics attached.
Will update thread as I go.
damonster-diagnostics-20160322-2203.zip