Last few days were rough for my Unraid server.

 

Day 1: it started with a container (Manyfold) that filled the docker.img (by unzipping multigigabyte files in a temp folder inside the image), locking down the entire system. I solved by backing up the .img, then changing the docker setting to folder. Rebuilt my containers via the "recent apps" function.

 

Day 2: I woke up to all some of my containers not starting up after the backup, error 403. Restarted Unraid, seems solved.

 

Day 3: after a failed Appdata Backup (error 403), I woke up to the SSD pool not mounting. Formatted the pool, and started again. Rebuilt my containers via the "recent apps" function, recreated my VMs, restored everything via backups. Changed the problematic container temp folder to be a share on the array.

 

Day 4 (today): woke up to an unreachable web UI for Unraid, but my Home Assistant VM was running perfectly (and was reachable). Restarted Unraid, the array and cache pool seems fine, but all my containers are gone. Appdata is still there. I'm not sure what happened.

 

At this point, I'm not sure what to look for. I will rebuilt the containers again via "recent apps", but it's not something I can do everyday. The docker directory in /mnt/share/system seems fine, the "containers" folder is full of folders, the volumes seem to be all there.

 

I'll attach the diagnostics I run a few minutes ago.

nickfury-diagnostics-20241117-1002.zip

Edited November 17, 20241 yr by Dreadino

Data corruption is being detected in both pool members, run a correcting scrub and post the results.

Will this remove the data on the SSDs?

 

EDIT: performed the scrub, this is the status
 

UUID:             0c6ff596-13dc-449a-bc07-06cbc7e94c13
Scrub started:    Sun Nov 17 13:56:09 2024
Status:           finished
Duration:         0:00:32
Total to scrub:   143.97GiB
Rate:             4.50GiB/s
Error summary:    no errors found

 

Edited November 17, 20241 yr by Dreadino

Reset the stats on the pool and keep monitoring it:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=700582

 

If it happens again, save the diagnostics before rebooting, or enable the syslog server.