Rebuild DNDC constantly rebuilding docker containers


Recommended Posts

Hi everyone! So I've been Googling my brains out on this and even read the code line by line on GitHub for the Rebuild-DNDC.sh file.

The issue is Rebuild-DNDC constantly restarts my vpn docker (bubuntux/nordvpn) with the report of:

# MASTER CONTAINER INFO
- CONTAINER-NAME: nordvpn
- ENDPOINT-ID: 12345---edited for safety?
- NETMODE-ID: 12345---edited for safety?
- CONNECTIVITY: BROKEN
---- restarting nordvpn container
---- nordvpn restarted

Based on the code, it matches up some long endpoint-id's that I've visually matched from this terminal output docker inspect nordvpn --format="{{ .NetworkSettings.EndpointID }}" to the mastercontepid.tmp file that it writes out but for some reason it NEVER considers them a match. I followed this tutorial and updated where needed accounting for docker updates but my setup is nearly similar (Rebuild-DNDC config is exactly the same as the screenshot).

 

In summary, this tool never finds a match for nordvpn docker and restarts every 5 min or whatever is set for the cron variable and I never see "SKIPPING: MASTER CONTAINER ENDPOINTID IS CURRENT" in the logs. Would anyone happen to know what I'm doing wrong?

Link to comment
  • 1 month later...

Once you deploy the vpn container, you need to make sure it's working/connected, only then start the rebuild-dndc container.

 

"- CONNECTIVITY: BROKEN" message suggests that there might be an issue with your vpn container connection i.e. rebuild-dndc is unable to ping (via your vpn container) any of the set dns servers (CF or Google) to check for an active connection, hence it's restarting your vpn docker hoping that would fix the connectivity issue.

 

 

 

 

Edited by eafx
Link to comment
  • 1 month later...

Hi @eafx! Thanks for the reply.

 

I ended up moving away from the NordVPN container and landed with binhex-delugeVPN. I got RebuildDNDC to sort of work now. As long as it creates the endpoints at first startup everything is good. BUT, if the EndpointID's don't match (ex: caused by a reboot) RebuildDNDC will start to repair the containers then just stop itself and never finish the rebuild. Last thing I see in the logs is this:

D. PROCESSING: IN-SCOPE CONTAINERS

----------------------------
Stopping: Rebuild-DNDC
----------------------------

At part B, I don't think it's able to rebuild the VPN container either as the VPN's logs do not show a restart in the container.

 

Not too sure what's going on at this point. Maybe a privilege issue that RebuildDNDC cannot kill and restart containers? Any thoughts would be greatly appreciated! Thanks so much for reading and your time!

Link to comment

Think I know what your issue is, the "network type" on rebuild-dndc container should be bridge mode/it shouldn't be using the VPN container network. The reason it's stopping itself is because it's using the VPN container network (or at least at some point it was using the VPN network) and has marked itself as a container in-scope for rebuild.

 

1. Make sure rebuild-dnc isn't using the VPN container network.

2. Stop the rebuild container and delete all the *.tmp files in config\rebuild-dndc folder, better to clear any any previous misconfiguration and then restart it.

3. Some other useful tips

  • Make sure the VPN container is the first container on your docker startup list
  • Move the Rebuild-DNDC container to the end of your container startup list.

 

Edit: Also, FYI the issue with bubuntux/nordvpn might've been resolved now. Problem was that container didn't have iputils-ping package installed and the rebuild container uses the ping command to check for connectivity on the VPN container.
https://github.com/elmerfdz/rebuild-dndc/issues/52 and seems like that has been implemented now: https://github.com/bubuntux/nordvpn/issues/163

Edited by eafx
Link to comment
Posted (edited)

Ok so I switched it out of using the VPN as its network mode and back to bridge. Now I get this error when the cron job runs. Maybe I didn't quite understand the readme and how to set this up.

 

/usr/bin/docker: Error response from daemon: Container cannot be connected to network endpoints: container:binhex-delugevpn, none.

 

I have each container setup with prerequisite #2. Can the network type not be set to "None"?

"--net=container:master_container_name"

 

Now I'm getting lots of orphaned containers upon Rebuild-DNDC running.

 

Edit: Turns out it does not like prerequisite #2. Adding a custom network with the container name gives me no issues. I followed SpaceInvader's tutorial on using --net=container: and setting network type to none, maybe I made a mistake in here?

Edited by JimmyGerms
Link to comment
  • 2 weeks later...
Posted (edited)

Hey @eafx! Thanks for all the help. I'll grab some screenshots this week and update my post.

 

Where I'm at now, everything is working and rebuilding correctly when the master container has an issue but I have noticed that the CPU pinning in the xml templates isn't getting applied when the in scope containers are rebuilt. I looked at ParseDockerTemplate.sh on GitHub and it's not grabbing the <CPUset>.

 

Also, let me know if I should be making bugs on GitHub for this. I'm a bit new to the workflow.

 

EDIT: Solved my issue by updating ParseDockerTemplate.sh with this code:

add_cpuset(){
	status=0
	xmllint --noout --xpath "//CPUset/text()" $xmlFile > /dev/null 2>&1
	status=$?
	if [[ $status == 0 ]]; then
		cpus=$(xmllint --xpath "//CPUset/text()" $xmlFile)
		docker_string+=" --cpuset-cpus=\"$cpus\""
		[ "$verbose" = "1" ] && echo "Found CPU Pinning:  $cpus"
	fi
}

 

Then altered the main call to add the function like so:

#Main - Call each function
#Each function adds to the 'docker run' argument
#list, based on what is in the .xml file.
docker_string="$docker"

add_name
add_net
#Adding CPU Pinning Detection
add_cpuset
add_ip	
add_privileged
add_envars
add_timezone
add_ports
add_volumes
add_extraparams
add_repository

 

Edited by JimmyGerms
Link to comment
  • 2 months later...
On 4/26/2021 at 3:06 PM, JimmyGerms said:

Where I'm at now, everything is working and rebuilding correctly when the master container has an issue...

 

@JimmyGerms – how'd you end up fixing this? I have the same problem as you (error below), using prerequisite #2 set up having previously followed the SpaceInvader tutorial as well.

 

/usr/bin/docker: Error response from daemon: Container cannot be connected to network endpoints
  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.