Docker Service failed to start after drive swap

[DesertCookie]

After replacing two 4TB drives with a single 12TB drive I just had started rebuilding parity when I noticed Docker would not start:

Warning: stream_socket_client(): unable to connect to unix:///var/run/docker.sock (No such file or directory) in /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php on line 682
Couldn't create socket: [2] No such file or directory
Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/dynamix.docker.manager/include/DockerClient.php on line 866

 

I had restarted the server after stopping the array took longer than 20 minutes; an occurrence I have noticed to happen about 50% of the time in my case.

 

What to do in this situation?

stower20-diagnostics-20220516-2122.zip

Edited May 16, 2022 by DesertCookie
added more detail

[JorgeB]

Docker image is corrupt, delete and re-create.

[unRAIDuser7]

Well that's a coincidence!

 

I just did a parity swap to get a larger drive and I'm now also getting that exact same message about Docker! Except that I never had any issues restarting to shutting down my server during the process. Just pressed the button to "Parity-Sync/Data-Rebuild", waited for the array to start, went to the docker tab and BAM error message.

[trurl]

20 minutes ago, unRAIDuser7 said:

BAM error message

If you want help attach diagnostics to your NEXT post in this thread 

[unRAIDuser7]

Sure. Much appreciated!

 

helios-diagnostics-20220601-1640.zip

[JorgeB]

Strange, not seeing an attempt to start docker after the rebuild started, once the rebuild finishes reboot and post new diags if it still doesn't start.

[unRAIDuser7]

Rebooted after the rebuild and Docker started fine…who knows. 

[macmus]

I just got same error  

 

Could someone have a look ?

 

tower-diagnostics-20230108-0650.zip

[trurl]

Your system share is all over the array. Not sure how you could even get it on so many disks, since it normally only has 2 files in it. What do you have in system share besides docker.img and libvirt.img?

 

 

 

[macmus]

I'm not sure what you mean ?

 

Is there anything wrong ?

[trurl]

4 minutes ago, macmus said:

Is there anything wrong ?

You said there is. I'm just trying to get an idea of how you have things setup so we can talk about how to fix.

 

28 minutes ago, trurl said:

What do you have in system share besides docker.img and libvirt.img?

 

[macmus]

I mean docker/libvm does not start after shaping parity and going to copy and rebuild process.

 

Index of /mnt/user/system

TYPE

NAME

SIZE

LAST MODIFIED

LOCATION

custom_ovmf<FOLDER>2021-04-04 13:54disk6

docker<FOLDER>2019-05-02 22:08disk1, disk4

libvirt<FOLDER>2019-05-02 22:08disk1, disk2

 

Index of /mnt/user/system/custom_ovmf

TYPE

NAME

SIZE

LAST MODIFIED

LOCATION

Parent Directory

.DS_Store6.15 KB2021-04-04 13:54disk6

Macinabox_CODE-pure-efi.fd3.65 MB2021-04-04 13:54disk6

Macinabox_VARS-pure-efi.fd131 KB2021-04-04 13:54disk6

readme.txt786 B2021-04-04 13:54disk6

 

ndex of /mnt/user/system/docker

TYPE

NAME

SIZE

LAST MODIFIED

LOCATION

Parent Directory

docker.img26.8 GB2023-01-07 14:09disk4

 

 

ndex of /mnt/user/system/libvirt

TYPE

NAME

SIZE

LAST MODIFIED

LOCATION

Parent Directory

libvirt.img1.07 GB2023-01-07 14:09disk2

 

 

I was swiping sdk with parity disk2.

 

 

 

Edited January 8, 2023 by macmus

[trurl]

5 hours ago, macmus said:

I was swiping sdk with parity disk2.

Looks like you were doing parity swap with parity2 and disk3. Post new diagnostics when that has completed.

[macmus]

2 hours ago, trurl said:

Looks like you were doing parity swap with parity2 and disk3. Post new diagnostics when that has completed.

I did that .. 

 

Stop failed .. so I hard to do hard reboot.. After it came back I was able to start array and it's now up and running with libvirt also up..

 

Strange and scary!

 

I'm planning to do more of that .. but the whole process took I think 40h!! + scary experience..

[trurl]

9 hours ago, trurl said:

Your system share is all over the array

Since I already explained these shares several times recently, I will just post a link.

 

 

[macmus]

15 hours ago, trurl said:

Since I already explained these shares several times recently, I will just post a link.

 

Thank you for the link... but if I hold it on the cache disk something goes wrong with this cache I will lost both files and won't be able to recover ?

[trurl]

Your new text was inside my quote and I almost didn't see it since I had to expand. You might be more careful in future.

 

2 hours ago, macmus said:

Thank you for the link... but if I hold it on the cache disk something goes wrong with this cache I will lost both files and won't be able to recover ?

You should have backups for appdata and libvirt whether it is on cache or not. The Apps page has plugins for that.

 

And, in general, parity is not a substitute for backup anyway.

[macmus]

3 hours ago, trurl said:

Your new text was inside my quote and I almost didn't see it since I had to expand. You might be more careful in future.

 

You should have backups for appdata and libvirt whether it is on cache or not. The Apps page has plugins for that.

 

And, in general, parity is not a substitute for backup anyway.

how much of performance ill get by moving system to cache ?

does it impact regular running of vms ?

 

i have 1 passthrough vm and few with vdisks on array 

Edited January 10, 2023 by macmus

[trurl]

Probably won't affect VMs as much as dockers since container executables are there. But array disks can't spin down with system on the array since those files are always open.

[macmus]

11 hours ago, trurl said:

Probably won't affect VMs as much as dockers since container executables are there. But array disks can't spin down with system on the array since those files are always open.

How important is spinning down the array ?

[trurl]

22 minutes ago, macmus said:

How important is spinning down the array ?

How important is it to you? If your server is inactive for hours at a time it can save power if they are spun down, and might prolong disk life. If your server isn't inactive for long periods, then spinning them up frequently might have the opposite effects.