Docker Corrupt?

[MitchTalmadge]

Good morning, I have been having some issues with Docker on my new Unraid (v6.11.5) build. I have resolved some of it, but I wanted to share my experience and get advice going forward.

Build Details

• No parity -- see next bullet.
• One 14TB HDD that I am transferring data onto from another computer. Once this is done I will move two more 14TB HDDs from that computer into this Unraid build, and use one of them as a parity disk.
  • I am constantly transferring ~100MB/s of data into the Unraid build during the events outlined in this post.
• One 512GB SATA SSD to be used as cache.
• Ryzen 5 3600 CPU
• 32GB RAM (not ECC at the moment, but ECC is coming).
• Unraid is installed on a brand new Samsung BAR Plus 64GB USB 3.1 Flash Drive (MUF-64BE4/AM)

Problem

Sometimes when I try to install a docker container, the web UI hangs during download of the images, then starts throwing up the "500 Internal Server Error" page. After a while (~10 minutes) it will resolve itself and the UI loads again. During this time I can still SSH into the server, I can transfer files in and out, etc. without issues. This happened when installing plex, but not when installing SWAG.

 

For my latest attempt, I tried to install pihole and the Unraid UI once again froze up and I could not open it up in another tab; I would get 500 Internal Server Error. It eventually came back up, but the Docker container was all sorts of messed up. I was getting errors such as "Error response from daemon: Failed to create btrfs snapshot: read-only file system" whenever trying to do anything with the container. I couldn't remove or re-create the container because of the "read-only file system."

Fixing "read-only file system"

I read somewhere here on the forums that the read-only problem can be a result of corruption when using btrfs (as I am) and the cache drive becomes full (as it did -- I accidentally filled it up with the mover). So I brought the array down, set minimum file sizes on the cache pool and the shares (30GB each), then brought the array back up. 

 

After this, the pihole docker image was working and the read-only messages were gone, but it had no way to edit the image. The UI told me that it believed the image was added manually (not via apps). I stopped and deleted the container + image, went back to the apps area, and re-downloaded/configured pihole. This time it worked with no issues and I have a working pihole instance.

 

I believe this problem is fixed but I would like to hear feedback about whether this fix was "proper" or not.

Fixing "500 Internal Server Error"

I don't really know what causes the 500 internal server errors, or how to fix it. I don't believe they are related to the cache drive; they happened even when the cache drive was mostly empty. I am worried that it could be a bad USB drive, from what I have read on the forums, but I don't know how to verify that.

 

Let me know what you think, or if more context would help. I greatly appreciate your time!

alpine-diagnostics-20230612-0927.zip

Edited June 12, 2023 by MitchTalmadge

[JorgeB]

Docker image is corrupt, delete and recreate.

[MitchTalmadge]

3 hours ago, JorgeB said:

Docker image is corrupt, delete and recreate.

 

Thank you @JorgeB, I'll try this out.

 

Also, I believe my USB drive is a lemon. I ran Check Flash on it and it had a read error:

 

I'm going to run a more thorough test to be sure. I probably need to find a new drive though.

 

----

 

Update: I ran more tests and didn't run into the error again. I am leaning towards that being a fluke. I'm going to try running Unraid again and see if the problem persists after a reboot. I did find a somewhat loose M.2 SATA adapter in my case which could have been part of the problem.

Edited June 12, 2023 by MitchTalmadge

[MitchTalmadge]

I reseated all my various peripherals (SATA ports, internal USB header, M.2 slots, etc.) and recreated my Docker image. I've had no problems since then! I guess it was a loose connection. (I am still using the same USB drive as before, I guess there wasn't really any problems with it.) Thanks for the help.

Edited June 27, 2023 by MitchTalmadge