6.9.2 Docker shim not getting created after hard reboot


Recommended Posts

After a graceful reboot, no issue, but after a hard reboot from watchdog or powerloss, the system will not start up with the docker shim in place resulting in no network connection between the unraid server and custom docker IP's. As soon as I shut down and re-start the docker service from the settings menu, the shim is generated and everything works as expected.

 

Confirmed that access to custom networks is in place. I've seen a few other mentions on the forums but nobody ever seemed to have a resolution (nor did they mention in the post they realized the shim wasn't generated).

 

Additionally, this is not new to 6.9.2. Pretty certain it's been going on since pre 6.9 but can't be certain.

 

fc-ps-urd1-diagnostics-20210512-2016.zip

Edited by ionred
Link to comment
4 minutes ago, jonp said:

Totally being honest here, I have no idea what a Docker shim even is ;-).  Any additional insights you can provide would help me better triage this issue.

Its a shim network created when you enable Docker > Host access to custom networks

This allows containers to communicate with the host, which is normally blocked as a security measure for docker

Link to comment

Here's a quick paste from my console. the first half is after a hard reboot, then I shut down docker and restarted it from the settings menu in unraid. Afterwards (I made a comment in the middle), there are new routes (the docker generated shims 10.0.0.0/25 and 10.0.0.128/25 to link the host and docker networks) and the network works properly again.

 

Relevant new ip routes added:

10.0.0.0/25 dev shim-br0 scope link

10.0.0.128/25 dev shim-br0 scope link

 

root@FC-PS-URD1:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
5: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
6: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
7: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
10: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
11: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff permaddr d0:50:99:d5:cf:27
12: bond0: <BROADCAST,MULTICAST,PROMISC,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
13: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
14: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
15: br-c31dbeb7d5c2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
    link/ether 02:42:43:73:59:a7 brd ff:ff:ff:ff:ff:ff
16: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 02:42:f6:f6:69:75 brd ff:ff:ff:ff:ff:ff
18: veth58ec1ca@if17: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether a6:76:5d:44:40:7c brd ff:ff:ff:ff:ff:ff link-netnsid 0
34: veth7c64111@if33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether e6:22:25:f9:dd:04 brd ff:ff:ff:ff:ff:ff link-netnsid 14
root@FC-PS-URD1:~# ip route
default via 10.0.0.1 dev br0
10.0.0.0/24 dev br0 proto kernel scope link src 10.0.0.246
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev br-c31dbeb7d5c2 proto kernel scope link src 172.18.0.1 linkdown
root@FC-PS-URD1:~# ping 10.0.0.187
PING 10.0.0.187 (10.0.0.187) 56(84) bytes of data.
From 10.0.0.246 icmp_seq=1 Destination Host Unreachable
From 10.0.0.246 icmp_seq=2 Destination Host Unreachable
From 10.0.0.246 icmp_seq=3 Destination Host Unreachable
^C
--- 10.0.0.187 ping statistics ---
4 packets transmitted, 0 received, +3 errors, 100% packet loss, time 3066ms
pipe 4
root@FC-PS-URD1:~# ## SHUTDOWN DOCKER, RESTARTED DOCKER
root@FC-PS-URD1:~# ## NO OTHER CHANGES
root@FC-PS-URD1:~# ping 10.0.0.187
PING 10.0.0.187 (10.0.0.187) 56(84) bytes of data.
64 bytes from 10.0.0.187: icmp_seq=1 ttl=64 time=0.024 ms
64 bytes from 10.0.0.187: icmp_seq=2 ttl=64 time=0.020 ms
^C
--- 10.0.0.187 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1014ms
rtt min/avg/max/mdev = 0.020/0.022/0.024/0.002 ms
root@FC-PS-URD1:~# ip route
default via 10.0.0.1 dev br0
10.0.0.0/25 dev shim-br0 scope link
10.0.0.0/24 dev br0 proto kernel scope link src 10.0.0.246
10.0.0.128/25 dev shim-br0 scope link
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
172.18.0.0/16 dev br-c31dbeb7d5c2 proto kernel scope link src 172.18.0.1 linkdown
root@FC-PS-URD1:~#
root@FC-PS-URD1:~#
root@FC-PS-URD1:~# ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: tunl0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
3: gre0@NONE: <NOARP> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/gre 0.0.0.0 brd 0.0.0.0
4: gretap0@NONE: <BROADCAST,MULTICAST> mtu 1476 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
5: erspan0@NONE: <BROADCAST,MULTICAST> mtu 1464 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
6: ip_vti0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ipip 0.0.0.0 brd 0.0.0.0
7: sit0@NONE: <NOARP> mtu 1480 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/sit 0.0.0.0 brd 0.0.0.0
10: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
11: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc mq master bond0 state DOWN mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff permaddr d0:50:99:d5:cf:27
12: bond0: <BROADCAST,MULTICAST,PROMISC,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
13: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether d0:50:99:d5:cf:26 brd ff:ff:ff:ff:ff:ff
14: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/none
15: br-c31dbeb7d5c2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default
    link/ether 02:42:43:73:59:a7 brd ff:ff:ff:ff:ff:ff
59: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
    link/ether 02:42:b9:5f:64:4d brd ff:ff:ff:ff:ff:ff
60: shim-br0@br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 62:a9:9c:7c:8c:ec brd ff:ff:ff:ff:ff:ff
62: vethfbb91df@if61: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether 82:aa:df:a5:64:6f brd ff:ff:ff:ff:ff:ff link-netnsid 0
64: veth7d3b0b2@if63: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master docker0 state UP mode DEFAULT group default
    link/ether de:80:0a:a4:85:de brd ff:ff:ff:ff:ff:ff link-netnsid 1

 

Edited by ionred
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.