granjerox

Hello guys, I'm back to document the (up to now) solution to my problem. Diagnosis and solution supported by Claude (claude.ai)" UPDATE - Root Cause Found & Resolved After further investigation, the Docker daemon crashes (goroutine dumps, CopyToPipe FIFO blocks, libnetwork DNS resolver hangs) were not primarily a Docker bug — they were a symptom of a failing USB flash drive. What we found: Running dmesg revealed the real culprit: sd 0:0:0:0: [sda] Sense Key : 0x3 ← MEDIUM ERROR sd 0:0:0:0: [sda] ASC=0x11 ASCQ=0x0 ← Unrecovered Read Error critical medium error, dev sda, sector 13500440 SQUASHFS error: Failed to read block 0x243b6a8: -5 SQUASHFS error: xz decompression failed, data probably corrupt sda was the Unraid USB boot drive (SanDisk Cruzer, the low-end model). The failing sectors were part of the OS squashfs — specifically the region containing the dockerd binary. When Docker tried to restart, it couldn't read its own executable from disk, resulting in a Bus error. The goroutine dump was a snapshot of the daemon's state at the moment of the crash — the FIFO and DNS goroutines were leaked/blocked, but they were a consequence of the abrupt daemon death, not the cause. Timeline reconstruction: Docker daemon degraded for 1+ hour with leaked goroutines (FIFO blocked 13 min, DNS resolver blocked 82 min) USB read errors caused dockerd to crash uncleanly docker.img remained mounted as an orphan (confirmed by is in-use, cannot mount in syslog) Subsequent restart attempts failed with Bus error due to continued USB read failures Resolution: Backed up /boot/config (including Plus.key) while the system was still running Prepared a new USB drive (SanDisk Extreme 64GB) on Linux with a proper 2GB FAT32 partition Took the opportunity to upgrade directly from Unraid 6.12.14 → 7.2.4 (Docker 24.0.9 → 27.5.1) Transferred the license to the new USB GUID via Unraid support After 15+ hours uptime: zero goroutine dumps, zero disk read errors, all containers stable Lessons learned: If Docker daemon crashes with goroutine dumps, always check dmesg for hardware errors first before assuming it's a Docker bug SanDisk Cruzer/basic USB drives are not suitable for continuous Unraid use — prefer SanDisk Extreme, Samsung FIT Plus or similar drives with proper wear leveling The Bus error on dockerd restart is a strong indicator of underlying storage/hardware failure Keep /boot/config backed up regularly — in this case it saved the migration Current status: Running stable on Unraid 7.2.4, Docker 27.5.1, new USB flash drive. Parity check completed with only 1 correction (expected after unclean shutdown).

Update on my case (last crash, March 16): Had another crash with the same symptoms. This time I was able to recover Docker without a full reboot by manually remounting the btrfs loop device and restarting the daemon: umount -l /var/lib/docker losetup -d /dev/loop2 killall -9 dockerd containerd containerd-shim-runc-v2 docker-proxy sleep 3 mount /dev/loop2 /var/lib/docker /etc/rc.d/rc.docker start The docker.log shows the daemon terminated with a full goroutine dump. Two goroutines stand out: 1. A CopyToPipe goroutine blocked indefinitely on a container stdio fifo: github.com/containerd/fifo.(*fifo).Read github.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1()2. A libnetwork.(*Resolver) goroutine blocked on a UDP read: github.com/miekg/dns.(*Server).serveUDP github.com/docker/docker/libnetwork.(*Resolver).Start.func2() Attaching full diagnostics this time. nighthawks-diagnostics-20260317-1733.zip

It keeps crashing here every 24–48 hours. I have no clue what could be causing it, and I'm not aware of having changed anything recently. I also wonder why there aren’t more people affected by this problem. I'm running: Unraid 6.12.14 Docker version 24.0.9, build 2936816 And this is my container list, in case you can find something in common: REPOSITORY_______________________________TAG_________IMAGE_ID_______CREATED_________SIZE binhex/arch-minidlna_____________________latest______ba4560c4ba57___5_months_ago____1_27GB cyfershepard/jellystat___________________latest______4f1399257ec0___5_weeks_ago_____809MB germannewsmaker/myspeed__________________latest______6c1fe29ccfd7___22_months_ago___188MB ghcr_io/goauthentik/server_______________2025_8_1____650da939c425___6_months_ago____1_11GB ghcr_io/mainfrezzer/adguardhome__________latest______3d02f027cec0___5_days_ago______123MB jc21/nginx-proxy-manager_________________latest______e3f9fd85ccab___3_weeks_ago_____1_14GB jlesage/handbrake________________________latest______6bec325104a9___36_hours_ago____631MB jlesage/jdownloader-2____________________latest______26f1b6cc8f0f___38_hours_ago____426MB lscr_io/linuxserver/deluge_______________latest______664c1c1f0f60___6_days_ago______147MB lscr_io/linuxserver/fail2ban_____________latest______d53ff225da95___3_days_ago______77_4MB lscr_io/linuxserver/jellyfin_____________latest______0803882db4eb___13_days_ago_____840MB lscr_io/linuxserver/mediaelch____________latest______c51019dc52a9___45_hours_ago____2_57GB lscr_io/linuxserver/plex_________________latest______33a8f98f8f05___13_hours_ago____371MB lscr_io/linuxserver/sonarr_______________latest______ddf896fd2d36___9_days_ago______206MB lscr_io/linuxserver/tautulli_____________latest______82026c0d05e5___3_days_ago______146MB migalmoreno/tubo-backend_________________latest______55e7bff49d8b___6_weeks_ago_____661MB migalmoreno/tubo-bg-helper_______________latest______ebeb46067fcf___6_weeks_ago_____1_67GB migalmoreno/tubo-frontend________________latest______b7188d5cea63___6_weeks_ago_____67_5MB ngosang/amule____________________________latest______016ac5d1385e___4_months_ago____38MB postgres_________________________________16-alpine___108b27c919e6___2_weeks_ago_____276MB postgres_________________________________17__________f31394cb50b3___2_weeks_ago_____453MB qmcgaw/ddns-updater______________________latest______f85c951aea51___2_months_ago____13_2MB quay_io/oauth2-proxy/oauth2-proxy________latest______42f956651e34___2_weeks_ago_____40_6MB redis____________________________________latest______c234a24851ba___2_weeks_ago_____140MB xavierh/goaccess-for-nginxproxymanager___latest______22a4360f1376___3_months_ago____140MB

Hi, I'm facing similar symptoms lately. I'm not sure if I have the same root cause, but I end up with no containers running and having to reboot the whole server because of a “docker image in use” error when trying to restart the Docker service from the WebUI. In my case, the docker.log crash seems to be related to portmapper. I'm not using the alternate Docker proxy. I don't have the full diagnostics files right now because I didn't collect them before rebooting. I'll post them after the next crash. docker.log

In my case i just have to restart ssh server by disabling and enabling it in unraid web interface. settings -> use ssh -> no (apply), yes (apply)