Docker issues. How severe was the docker vulnerability that was just patched in the recent unraid versions? Thinking about downgrading

[strike]

TLDR; Having a lot of docker issues after updating beyond 6.12.6. Is it safe to downgrade to unraid 6.12.6 with the recent docker vulnerability that was patched after 6.12.6? I have multiple containers exposed to the internet through a reverse proxy.

 

Thinking about downgrading from 6.12.10 to 6.12.6 as docker and the unraid webui has become almost unusable after updating beyond 6.12.6. It's like docker itself has become very slow as updating or restarting a container now takes about 15-30 minutes. And the containers seems to run more slowly as well. Now I'm having problems with emby loading slow an movies stops to buffer as well and I have never had playback issues before. Emby is on the same version as it was on 6.12.6.

 

My Audiobookshelf container was litteraly unusable on their latest version on 6.12.10 and I had to downgrade that to an older version which works. Everything was super slow, and wasn't able to load most times. But the newst version of audiobookshelf shoud run just fine though slower than version 2.3.3 which I had to downgrade to and even that is slower then it was before.

 

And updating or restarting a container takes up to 15-30 min as I said and while doing that the unraid webui is unreachable, it's just loading. More often than not when restarting a container it comes back with an exceution error "server error", but it does restart anyway when it finanlly decides to.

 

Uploading my diags here from a couple hours ago when my emby and audiobookshelf container was unracahble again, it was just loading. Tried to restart emby but i gave up after about 30 min and decided to stop the array and start it again which also took about 15-20 more min. I guess restarting the docker service would have done the same thing getting my containers up again. 

 

And when starting the arrray wasn't all containers started simultaneously before? Now they start one by one from the top and takes ablout 30 min that too before all are started. Before everything was up in about 5 min after starting the array. Can this has something to do with the appdata backup plugin? I have that set to stop, backup and start for each container.

 

I can also add that I was using docker directory, but tried to switch to image again just to see if that would solve the issues, but it didn't. I've also changed all the hardware. But that didn't help. (i have two servers, but one isn't in use right now so I just swapped the all the drives to the other server.)

 

My next step before downgrading is to reboot in safemode to rule out any plugins. Docker runs in safe mode right?

 

Hoping to get some help diagnosing this and to tell me it safe to downgrade to 6.12.6 again if I can't figure this out.

 

 

tower-diagnostics-20240502-1116.zip

Edited May 2 by strike

[strike]

Well it's not caused by any plugins. The issue still remains when running in safe mode. After the array starts it takes almost exactly an hour from when the first container is starting until the last container is started. AN HOUR to start 22 containers. This used to take about 5 min.. Everything docker related is running ridiculously slow. I stopped the appdata backup yesterday as it was not finished after 5 HOURS..  Is it safe to downgrade to 6.12.6? If I were to start from scratch and redo the flash drive, which files do I need to copy over to keep all my settings? 

Edited Tuesday at 06:34 PM by strike

[JorgeB]

If you create a new docker image and add a couple of new containers, differentfrom the ones you are using, and without restoring the old ones initially, do you see the same?

[strike]

I will test this and report back

[strike]

On 5/15/2024 at 9:23 AM, JorgeB said:

If you create a new docker image and add a couple of new containers, differentfrom the ones you are using, and without restoring the old ones initially, do you see the same?

So I tested this. Nuked my docker image and installed a few containers I have never installed before. Worked like normal. Restart/stop of a container took about 30 seconds and no execution error either. Nuked my docker image again and started installing all my previous containers from CA. The first 10-12 installed like normal, then it started acting up again. Not finishing pulling all the layers and complaining about not finding the image locally so it started pulling all the layers again, See attached images.

 

 

 

Took me about 2 hours to install all the containers. Normally it would take less then half of that time IIRC. It downloads fast but the rest takes more time then it used to especially when it can't seem to finish pulling all the layers the first time. 

Then when everything was up an running again I restared one container just to check and yup, same behavior as I have struggled with ever since updating from 6.12.6. Took about 5 min to restart and I was greeted with an nice execution error in the end. I just love my life... 

 

Waiting for some new hardware which will arrive next week. Almost up to date CPU and MB. Hope that solves it. If not I'm downgrading or staring over with a new flash drive.

Edited Friday at 12:04 AM by strike

[JorgeB]

Is the docker image on an SSD or HDD?

[strike]

It's on an ssd pool. And this is just getting worse. Now mover is running very slowly as well. Tried to move some stuff last night, gave up after 2 hours as it just had moved about 100MB. Tried to stop the array, it wouidn't stop so I had to restart in stead. Booted up again and started all containers, started the mover again. Now after 8 hours is has moved another 100MB and it's still running apparantly. Starting to think it must be something wrong with my HBA or something. Or it could be that windows rebooted and aborted my manually started mover script. I have a script that pauses all my downloads, starts the mover and starts all my downloads again when it's finished. Looks like everythings is working as normal when I move stuff manually. Nothing wrong with the speed there. No error is the syslog either. Will try to move some stuff off my cache pool manually as well just to check that too. The test earlier was from array disk to array disk. If my test goes well, I have no clue what the issue with the mover is. Maybe start it manually without my script and see how it behaves.

[strike]

I think I figured it out. There must be something wrong with my cache pool. I deleted my docker image again and changed it's location to another ssd pool. 23 containers installed in under 20 min compared to 2 hours on my cache pool. I'll have to do some speedtest on my cache pool as all my appdata and downloads go there. Either it's the drives themself or my HBA. One of the pools is on my MB SATA connections I just don't remember which one. So it could be my HBA since the mover acted up as well. I ran the mover again manually earlier today and then it was fine. This was before I moved the dockerimage.