Today started off great. I was removing duplicate photos from Immich when suddenly, I got an error within the application. I then noticed that some exposed services (Overseerr, File Browser, etc.) were down. The first thing I did was restart Traefik. While restarting, I received a popup error: 403. I tried restarting other containers and received the same error. Rebooting unRaid helped for a minute, but the errors returned. It seemed that the recommended next step was to recreate the docker.img. Deleting it via the GUI and command line did nothing... the image file was read-only. After rebooting again, I was able to delete the docker.img file and recreate it.

 

After reinstalling almost all my containers using "Previous Apps," I received a popup that I believe read, "Server error." Checking the unRaid log, I saw a lot of red. The Fix Common Problems plugin also reported that the cache drive was unwritable, although that error is no longer showing after a reboot.

 

MemTest seemed to have passed, although I stopped it as soon as I saw the large, green "PASSED" message and did not leave it running for very long.

I'm wondering if there's anything I'm missing before I start checking physical connections. I haven't moved or physically touched the server in months.

 

Any help would be greatly appreciated! Attached are the diagnostics.

 

heavyd-diagnostics-20250319-1522.zip

See if you can scrub cache.

https://docs.unraid.net/unraid-os/manual/storage-management/#btrfs

if that won't work you can try to back it up and reformat

Either way, disable Docker and VM Manager in Settings until you get cache going again.

5 hours ago, trurl said:

See if you can scrub cache.

https://docs.unraid.net/unraid-os/manual/storage-management/#btrfs

Thanks @trurl .. I've just finished running the scrub and this is the result on the cache drive:

 

UUID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

Scrub started: Wed Mar 19 21:57:38 2025

Status: finished

Duration: 0:05:45

Total to scrub: 152.66GiB

Rate: 453.10MiB/s

Error summary: csum=5

Corrected: 0

Uncorrectable: 0

Unverified: 0

 

I'm guessing the checksum mismatch is an issue... but would this be the root cause of all the errors and permission issues?

The data corruption is not the only issue, but that happening suggests there may be a RAM problem, and that may also have caused the other filesystem corruptions, recommend running memtest first, then backup and reformat the pool

OK so after scrubbing, SMART diagnosis, and memtest, everything passed. I also checked all cables. Next, I moved all the files in my cache pool to another SSD drive via rsync and changed all settings under each docker container and share to point to the new cache pool and rebuilt docker.img. It's been almost 24 hours now and everything is running flawlessly. I'm filing this under a hardware failure.

For the record, the old SATA SSD cache drive was a Crucial MX500 1tb purchased in 2023 (2 years ago). New cache drive is a Samsung 870 EVO 2tb SATA SSD.