mckagse

My apologies. I'll share what I can in hopes it will help others. I think I grabbed the Diagnostics too late as the event occurred around 17:00 and the logs start at 19:00. I'll recover, there was nothing of irreplaceable value, just lots of effort lost. I've learned a lot due to this. When I brought the array back online in maintenance mode I checked each drive individually, including the cache drive, looking for files I knew I had just created. I can't give you a quantitative value of how much remained and how much was missing, but it was mixed. When the event occurred, following a manual invoking of the mover, which likely cause a PSU failure as the disks where spinning up, I observed warnings of multiple disks failing, however the mover continued to run and no option was available to cancel. I just watched as the write errors started to add up on the remaining parity disk and array. 1 of 2 Parity disks lost power, 1 of 3 array disks lost power, unassigned devices lost power, the cache disk remained powered. I believe the mover was attempting to move data to the disk in the array that lost power leading to the error count. The reason I posted here is that it seemed to be writing the data to a location that didn't exist, then erases from the cache disk without verification the data was transferred. I attempted to intervene by requesting a shutdown, as no option to stop the mover exists. I lost connection with the server, however, when I physical accessed the server it was still running and I forced a shutdown. I almost think a complete loss of system power would have been easier to recover from. I probably won't completely disable the cache, but I'll up the frequency to hourly to lower the chance of me filling up the cache drive and running into this issue again. tower-diagnostics-20220530-2103.zip

This is a legitimate concern that just caused me to loose a lot of data. I was dumping lots of documents to my array today and I got a notice that my cache disk was low on space. I invoked the mover and half my drives disappeared. A single rail on my PSU failed during the moving process. I replaced the PSU and started back up and checked my data to learn much of the data I had dumped and already erased from the original media earlier that day was gone. Once I rebuild the now disabled devices, perform a manual parity check, and invoke the mover to completion, I will make sure to disable the cache disk until I can determine a work around for this issue. This issue should be readdressed.