Athelstan

Well, moving everything off the cache in safe mode by mounting it in read only mode to a disk in the array worked.  Ran a full memtest (passed) and restarted the system with auto start array turned off and removed the cache disks, erased the cache, then built a new one and it worked fine.  Copied everything back and found everything was owned by root:root (docker permissions was totally borked because of this).  At least that is fixable and a much better place to be than where I was.
 
I've seen some other people (including the one I posted above and possibly DingusKahn) with the same issue.  I'm questioning if going to ZFS to get compression on my cache was such a good idea (don't get me wrong, performance and compression are great) but only time will tell.  I guess if it happens again I'll roll it back to XFS.  This was no fun, haha.

Well, I followed up this post with a post on the r/Unraid on Reddit about the issue.  I had someone reply that said doing a full server reboot fixed their issues (they had it happen more than once, apparently).  I had already rebooted the server twice, but I thought why not.  Third server reboot fixed my issue.  No idea what the issue was or why it even existed unless it was an app update that went south somewhere recently.