Docker Crashing After A Few Hours After 6.12.3

[jbrown705]

Hi all, hopefully this is in the right place because it's related to my docker crashing and causing issues with being able to shutdown the docker, restart it, remove it etc.  After upgrading to 6.12.3 from the last version of 6.11, my docker crashes after a few hours (most I have gotten was about 12hrs after updating).  I have tried 4x and the same thing keeps happening.  As soon as I downgrade, it works and remains stable.  I have checked and removed anything the compatibility check flags and all works perfectly fine for hours then just one after the other the dockers crash and become unresponsive.  I have attached the diagnostic file it prompts you to download when downgrading and am hoping can someone can take a look and let me know what the problem might be or how to fix it?

deathstar-diagnostics-20230729-0330.zip deathstar-diagnostics-20230728-1318.zip

[JorgeB]

Try switching to ipvlan (Settings -> Docker Settings -> Docker custom network type -> ipvlan (advanced view must be enabled, top right)).

 

There are also issues with you cache pool.

 

 

[jbrown705]

@JorgeB

I had to format and reinstall my main cache drive (I have 3 cache drives, nvme for all appdata, and other one for downloads only and a 3rd for just vm and the other stuff). The main nvme is the one I had to reformat. Seems ok now but if there is anything else you can share about it, I’ll try to fix. I reverted back and everything has been fine for about 2 weeks now. 
 

as for the ipvlan, I did try switching that but it was after I did the upgrade and i don’t know if that was what was causing the crashes. Should I switch that give it a day then try the upgrade or just switch the immediately so the upgrade?

 

Was there something in the diag files indicating the issue with with that?

[JorgeB]

55 minutes ago, jbrown705 said:

Was there something in the diag files indicating the issue with with that?

Yes, call traces:

Jul 28 22:01:18 DeathStar kernel: macvlan_broadcast+0x10a/0x150 [macvlan]
Jul 28 22:01:18 DeathStar kernel: ? _raw_spin_unlock+0x14/0x29
Jul 28 22:01:18 DeathStar kernel: macvlan_process_broadcast+0xbc/0x12f [macvlan]

 

[jbrown705]

@JorgeB, So I should switch to ipvlan then do the upgrade immediately or so i need to check anything before the upgrade?

 

Edited August 12, 2023 by jbrown705

[JorgeB]

Just the switch before or after updating should do it.

[WoS]

Hello @jbrown705,

i have problems with my docker Containers too. With 6.11.5 it works great for months but after upgrading to 6.12.2 and then 6.12.3, all Containers crashes after some hours.

 

I deleted the Containers and added them from my Templates. It worked for many hours and then the same... All Containers broken.

 

Downgrade to 6.11.5 and all Containers with new Images. Worked for hours. The Problem is still the same now with broken Containers from 10.08.23 in the evening.

 

I changed to IPVLAN before but nothing changed 🥺

 

Now i can't add the Containers from my Templates. I have to delete the docker folder again and then IT will work but i think that will BE only for hours again.

 

@JorgeB Can you see Something in the diagnostic File that can Help me?

 

Thank you and good luck @jbrown705.

 

Greetings 

WoSunraid-diagnostics-20230812-1630.zip

[JorgeB]

18 hours ago, WoS said:

Downgrade to 6.11.5 and all Containers with new Images. Worked for hours. The Problem is still the same now with broken Containers from 10.08.23 in the evening.

Do you mean you still have the same issue with v6.11.5?

[WoS]

 

34 minutes ago, JorgeB said:

Do you mean you still have the same issue with v6.11.5?

 

Yes. The same issue with v6.11.5 after downgrade.

 

But before i did the upgrade from 6.11.5 to 6.12.2 everything works great.

 

Now i have deleted the docker folder again and startet only 2 docker containers with new images from my templates.

[WoS]

Only two Containers running in docker since last docker-crash to test.

Today i woke up the server to check the funktionalität of the containers and they crashed again.

 

I pushed the update to each container but this ends in an error "No such file or directory" 🥺 (Screenshot)

 

I added the diagnostic file again. Hope someone can support me?

 

@jbrown705:

Are your docker containers running?

 

Thank you and greetings

WoS

unraid-diagnostics-20230817-2028.zip

[jbrown705]

5 hours ago, WoS said:

@jbrown705:

Are your docker containers running?

Yes, I first updated from macvlan to ipvlan and left run for a few days. All was fine. Then I updated to latest version and left run for a few days. All seems to be ok but I had one error that cause me all my dockers to say unavailable but they still worked. I did a force update and that fixed them one by one. I am having issues getting the new appdata backup to complete tho, but that’s my only issue so far.