[SOLVED] read-only file system error when creating container.

[YourNightmar3]

I recently changed from a single-SSD XFS cache to a dual SSD BTRFS cache. I stopped VM's and Docker, moved everything off the cache drive, reformatted it to BTRFS and added a second one, then moved everything back. I deleted my docker.img file and redownloaded all docker images from the "previous apps" tab as has been mentioned many times on this forum.

 

I must have done something wrong because when i try to install a new docker container i now get the following error:

 

Error: open /var/lib/docker/tmp/GetImageBlob484538067: read-only file system.

 

I do not know where to start solving this. My diagnostics file is attached to this post. Many thanks in advance for any help.

tower-diagnostics-20201020-1526.zip

Edited October 20, 2020 by YourNightmar3

[JorgeB]

Pool fs is fully allocated, you need to run a balance, see here for more info, king of strange this is happening in a new filesystem, did you fill it up completely before?

[YourNightmar3]

8 minutes ago, JorgeB said:

Pool fs is fully allocated, you need to run a balance, see here for more info, king of strange this is happening in a new filesystem, did you fill it up completely before?

Thank you for the quick reply.

Now that i think about it, i did accidentally set a share that wasn't supposed to be on cache to cache and it filled up the cache completely. I didn't think much of it as i put the share cache usage back to normal and ran mover again.

 

I ran

btrfs balance start -dusage=75 /mnt/cache

with the following result:

Done, had to relocate 203 out of 934 chunks

But unfortunately when creating a docker container i am still getting "Error: open /var/lib/docker/tmp/GetImageBlob091507751: read-only file system"

Edited October 20, 2020 by YourNightmar3

[JorgeB]

2 minutes ago, YourNightmar3 said:

But unfortunately when creating a docker container i am still getting "Error: open /var/lib/docker/tmp/GetImageBlob091507751: read-only file system"

You'll need to re-start the docker service, that should do it, if it doesn't recreate the docker image.

[YourNightmar3]

8 minutes ago, JorgeB said:

You'll need to re-start the docker service, that should do it, if it doesn't recreate the docker image.

Thanks so much! It worked!

 

I did find that thread you linked when googling for the problem but i also found some people saying i had to reformat my entire SSD cache so im glad it was solved with this solution instead.

 

Why is this command nessecary when the cache fills up? Will i have to do this again in the future?

[JorgeB]

This should no longer happen with recent kernels, but in some circumstances it does, you're not the first.

[ajugland]

Thanks alot. My NVME cache filled up and I got same error after this.

[Cakroksh]

On 10/20/2020 at 10:00 AM, YourNightmar3 said:

Thank you for the quick reply.

Now that i think about it, i did accidentally set a share that wasn't supposed to be on cache to cache and it filled up the cache completely. I didn't think much of it as i put the share cache usage back to normal and ran mover again.

 

I ran

btrfs balance start -dusage=75 /mnt/cache

with the following result:

Done, had to relocate 203 out of 934 chunks

But unfortunately when creating a docker container i am still getting "Error: open /var/lib/docker/tmp/GetImageBlob091507751: read-only file system"

i think i am chasing a similar issue but not certain but when i tried to run the balance cmd i got the error 

ERROR: error during balancing '/mnt/cache': Read-only file system

 

pepper-diagnostics-20210413-0751.zip

[JorgeB]

36 minutes ago, Cakroksh said:

ERROR: error during balancing '/mnt/cache': Read-only file system

Cache filesystem is corrupt, you should backup and re-format.