Cache Marked as Read Only or Full (Solved)

[kpc180]

So let me preface this by saying I’m a simpleton.  Now let’s begin -- So I woke up this morning and my server was being odd.  I could not access any of my Dockers and I could see my shares and access things but I could not move anything, save anything new, or even create a new folder on any share, it would tell me there was no room.  I have TBs of space left on the array and the cache was showing over 100gbs of free space.  
I tried to restart but then it just got stuck doing that for forever.  I had to do a button press hard power down (This will result in a parity check I'm aware).  Then it had problems booting back up.  It would start and mount the disks but then it would get stuck on "starting services" for forever.  I could go to some of the web interface tabs but not all (could not open Dashboard or Docker for sure) .  
I also noticed the parity check did not start yet because services were still starting.  I did some searching and came across a thread that seemed to be have an answer for when the system gets stuck on starting services - that talked about changing the config file (but now reading it closer it seems that had nothing to do with me and was just about  VMs which I’m not even running – I changed the boot/config/domain.cfg file from disable to enable).  Anyways I made the changes and rebooted – and it once again hung while rebooting for a crazy long time.  So I did a hard button press reboot.  
This time it mounted the disks and started the services correctly however it is still acting like the cache is read only or totally full.   
Any help in simpleton language would be greatly appreciated!

unkevin-diagnostics-20180208-1306.zip

[JorgeB]

Array was not started yet when you grabbed the diagnostics, we need after start diagnostics, if you can't get them from the GUI type diagnostics on the console/SSH.

[kpc180]

Wowzers, sorry about that.

unkevin-diagnostics-20180208-1837.zip

[JorgeB]

Your cache filesystem is fully allocated resulting in ENOSPC errors, see here on how to resolve:

 

https://lime-technology.com/forums/topic/62230-out-of-space-errors-on-cache-drive/?do=findComment&comment=610551

 

After it's done you'll likely need to delete and recreate the docker image since it will most likely be corrupt.

 

 

[kpc180]

Thanks so much.  It seems to be running the rebalancing now. 

Just one more question when you say to "delete and recreate the docker image" -are you talking about a specific docker or all of them or do you mean something else alltogether?  

[JorgeB]

See here:

 

 

[Doommius]

On 2/9/2018 at 12:45 AM, JorgeB said:

Your cache filesystem is fully allocated resulting in ENOSPC errors, see here on how to resolve:

 

https://lime-technology.com/forums/topic/62230-out-of-space-errors-on-cache-drive/?do=findComment&comment=610551

 

After it's done you'll likely need to delete and recreate the docker image since it will most likely be corrupt.

 

 

This link is dead, are you able to repost the solution here ? i did this with my write cache on mistake.

[JorgeB]

13 minutes ago, Doommius said:

are you able to repost the solution here ?

Please post the diagnostics.

[Doommius]

1 hour ago, JorgeB said:

Please post the diagnostics.

This link here is dead. I found a thread on reddit. leading to here which helped solve the issue

 

Edited February 16, 2023 by Doommius