ZappyZap Posted October 4, 2022 Share Posted October 4, 2022 Overview: Support for Docker image Pihole Sync Application Name: Pihole Sync Application: https://github.com/ShiromMakkad/docker-pihole-sync Documentation: https://github.com/ShiromMakkad/docker-pihole-sync/blob/master/README.md Make sure to look at the complete documentation Please post any questions/issues relating to this docker you have in this thread. Quote Link to comment
rutherford Posted November 21, 2022 Share Posted November 21, 2022 I was looking into getting https://github.com/vmstan/gravity-sync going for my two piHole instances. I would like to use the hardware rasperrypi as the primary, and the docker instance as the secondary - I'm thinking that isn't what this docker does. I guess I could swap 'em around? thanks! Quote Link to comment
ZappyZap Posted November 21, 2022 Author Share Posted November 21, 2022 Ho this is the sender so for the primary i can creater a receiver template if you need .... Quote Link to comment
rutherford Posted November 21, 2022 Share Posted November 21, 2022 no no, that's OK! As I started to look into a little more: read/write to an SD card (which my raspberri pi has) it seems like too big a thing. I rarely make changes to the pihole anyhow, and I can just manually do it every once and a while. AND: thanks for your work and containers! Quote Link to comment
ZappyZap Posted November 21, 2022 Author Share Posted November 21, 2022 Ok No problem I will still make a receiver i think just in case Quote Link to comment
Nodiaque Posted December 14, 2022 Share Posted December 14, 2022 Hello, just to be sure, this is only the "sync" part of pihole and not pihole directly right? Meaning I still have to install a pihole server? Thank you Quote Link to comment
ZappyZap Posted December 14, 2022 Author Share Posted December 14, 2022 You are correct this is used when you have 2 pihole server and you want to keep them in sync Quote Link to comment
Nodiaque Posted December 14, 2022 Share Posted December 14, 2022 ok great. I just set it up yesterday. I see everything is getting sync, but is it normal I don't have the same amount of blocked domains? I though this would sync the gravity list when it's updated on the master, but it seems it's waiting on the gravity trigger that I triggered manually after 12 hours or so. There's no error in log from either receiver and sender, and when I look at the sender during a gravity update, it does says it's sending a lots of stuff. Quote Link to comment
Nodiaque Posted December 14, 2022 Share Posted December 14, 2022 Oh I forgot to add, there's an error in the receiver template, the default mode is sender and since it's a hidden value, you don't see it when configuring the template per default. Quote Link to comment
ZappyZap Posted December 14, 2022 Author Share Posted December 14, 2022 (edited) 2 hours ago, Nodiaque said: Oh I forgot to add, there's an error in the receiver template, the default mode is sender and since it's a hidden value, you don't see it when configuring the template per default. Thanks for the report , i will take a look and fix Edit : This has been fixed Edited December 14, 2022 by ZappyZap Quote Link to comment
geekazoid Posted December 31, 2022 Share Posted December 31, 2022 Hi, I'm just wondering how this is supposed to be operable. When I deploy the images they just drop into a reboot loop forever and I can't get a console long enough to set up ssh. Quote Link to comment
ZappyZap Posted December 31, 2022 Author Share Posted December 31, 2022 6 minutes ago, geekazoid said: Hi, I'm just wondering how this is supposed to be operable. When I deploy the images they just drop into a reboot loop forever and I can't get a console long enough to set up ssh. Can you please provide some log ? would be a good start Quote Link to comment
geekazoid Posted December 31, 2022 Share Posted December 31, 2022 (edited) I've deleted the verbose detail of my troubleshooting because its not relevant to the community. Instead I'm going to turn my findings into a Quick Start Guide for pihole-sync: Quick Start Guide Environment Primary pihole on the unraid docker host A (unraidA) has it's files typically in /mnt/user/appdata/pihole. There are two important directories there that this app is going to sync: /mnt/user/appdata/pihole/pihole /mnt/user/appdata/pihole/dnsmasq.d The pihole-sync sender is going to reside on the same docker host as the master pihole server as it has direct access to the same filesystem. A sensible path for this would be /mnt/user/appdata/pihole-sync-sender (on unraidA) Secondary pihole on unraid docker host B (unraidB) should probably have a similar filesystem setup for consistency. Of course the path for the receiver should be similarly /mnt/user/appdata/pihole-sync-receiver (on unraidB) NETWORKING: If you use Bridge mode for your containers, the sync will be employing TCP ports 22222. Defaults will work for you. If you use br0 mode for a dedicated IP per container, you can just use the standard ssh port 22 (change 22222 to 22). Remember that your sync hosts are going to be on different IPs than your piholes if you use br0 method, so don't confuse that in your setup. Operations: you should be able to ssh from unraidA to unraidB. Otherwise you can use the clipboard and web terminal. Install the Receiver On unraidB, use Community Apps to install Pihole-Sync-Receiver. In the Add Container dialog you are going to have the following fields to fill: Name: I recommend renaming the docker image to pihole-sync-receiver (lowercase) for consistency (this will make it less tedious in the CLI later) Network: as discussed above in Networking Remote Host IP: as discussed above in Networking Root directory: /mnt/user/appdata/pihole-sync-receiver/root Etc-ssh: /mnt/user/appdata/pihole-sync-receiver/ssh Pi-Hole Path: /mnt/user/appdata/pihole-sync-receiver/pihole (we're going to change this later to a symlink) Pi-Hole DNSmasq path: /mnt/user/appdata/pihole-sync-receiver/dnsmasq.d (we're going to change this later to a symlink) <click Show more settings> Note Type: receiver Remote SSH port: as discussed above in Networking <Apply> Stop the Receiver container for now. Open a Terminal on unraidB cd /mnt/user/appdata/pihole-sync-receiver mkdir root/.ssh later you will be symlinking the pihole and dnsmasq.d directories here to your secondary pihole server's paths one level up. Let's get this working first. Note: We installed the Receiver first because the sender is going to thrash against the receiver trying to connect until we complete the ssh key installation. Install the Sender and copy the ssh key to the Receiver On unraidA, use Community Apps to install Pihole-Sync-Sender. In the Add Container dialog you are going to have the following fields to fill: Name: I recommend renaming the docker image to pihole-sync-sender for consistency (this will make it less tedious in the CLI later) Network: as discussed above in Networking Remote Host IP: as discussed above in Networking Root directory: /mnt/user/appdata/pihole-sync-sender/root Etc-ssh: /mnt/user/appdata/pihole-sync-sender/ssh Pi-Hole Path: /mnt/user/appdata/pihole-sync-sender/pihole (we're going to change this later to a symlink) Pi-Hole DNSmasq path: /mnt/user/appdata/pihole-sync-sender/dnsmasq.d (we're going to change this later to a symlink) <click Show more settings> Note Type: sender Remote SSH port: as discussed above in Networking <Apply> On first startup, the container is going to generate a ssh host key. If you used the paths defined above, it will be located in /mnt/user/appdata/pihole-sync-sender/root/.ssh/ Open Terminal on unraidA docker logs pihole-sync-sender there is a message early on in the logs that mentions the steps to copy the ssh key to the receiver. I'll re-iterate these instructions below. cd /mnt/user/appdata/pihole-sync-sender/root/.ssh/ scp id_ed25519.pub root@unraidB:/mnt/user/appdata/pihole-sync-receiver/root/.ssh/authorized_keys enter the root password for unraidB when prompted to authenticate this secure copy Now start the Receiver on unraidB Open Terminal on unraidB docker exec -it pihole-sync-receiver /bin/bash cd / chown root root chgrp root root exit On unraidA cd /mnt/user/appdata/pihole-sync-sender touch pihole/psynctestfile touch dnsmasq.d/dsynctestfile Now start the Sender on unraidA docker logs pihole-sync-sender On unraidB docker logs pihole-sync-receiver cd /mnt/user/appdata/pihole-sync-receiver ls -a pihole (do you see your psynctestfile ?) ls -a dynsmasq.d (do you see your dsynctestfile ?) Once you know its syncing properly, the setup is ready to be connected to your primary and secondary piholes. stop both the sender and receiver containers delete the sync test files at both ends Use symlinks: on unraidA (sender) it would be something like this: ln -s /mnt/user/appdata/pihole/pihole /mnt/user/appdata/pihole-sync-sender/pihole ln -s /mnt/user/appdata/pihole/dnsmasq.d /mnt/user/appdata/pihole-sync-sender/dnsmasq.d on unraidB (receiver) it would be something like: ln -s /mnt/user/appdata/pihole/pihole /mnt/user/appdata/pihole-sync-receiver/pihole ln -s /mnt/user/appdata/pihole/dnsmasq.d /mnt/user/appdata/pihole-sync-receiver/dnsmasq.d Start the Receiver, then the Sender... check the logs! This is where the more specific pihole configuration begins (beyond scope of this quick start guide) Edited January 1, 2023 by geekazoid Made my post useful for others Quote Link to comment
ZappyZap Posted January 1, 2023 Author Share Posted January 1, 2023 I will have look and get back to you 1 Quote Link to comment
geekazoid Posted January 1, 2023 Share Posted January 1, 2023 (edited) I've fixed the ssh pubkeygen auth issue. Authentication refused: bad ownership or modes for directory /root cd / chown root root chgrp root root I made an issue on github for this, and I submitted an edit to the README. Above I've written a Quick Start Guide which I'll probably build upon later when I get the whole stack dialed. When done I'll bring the final product to github, but I figure being on first page it will be helpful for the next guy here. Have a Happy New Year! Edited January 1, 2023 by geekazoid updated to reflect final solution Quote Link to comment
Nodiaque Posted January 1, 2023 Share Posted January 1, 2023 Quick question, does it sync the gravity database? I notice that they don't have the same number of domain in the blocklist unless I sync both gravity at the sametime. I do see a lot of log saying it's sending files when I start a sync on the sender, but the receiver doesn't seems to "read" the new values (Although I did see all other setting getting updated). 1 Quote Link to comment
ZappyZap Posted January 1, 2023 Author Share Posted January 1, 2023 5 hours ago, geekazoid said: I've fixed the ssh pubkeygen auth issue. Authentication refused: bad ownership or modes for directory /root cd / chown root root chgrp root root I made an issue on github for this, and I submitted an edit to the README. Above I've written a Quick Start Guide which I'll probably build upon later when I get the whole stack dialed. When done I'll bring the final product to github, but I figure being on first page it will be helpful for the next guy here. Have a Happy New Year! Thanks @geekazoid Awesome Job , did not get this issue on mine but probably because my receiver is a docker <> Unraid.... 5 hours ago, Nodiaque said: Quick question, does it sync the gravity database? I notice that they don't have the same number of domain in the blocklist unless I sync both gravity at the sametime. I do see a lot of log saying it's sending files when I start a sync on the sender, but the receiver doesn't seems to "read" the new values (Although I did see all other setting getting updated). I notice that as well, but i am pretty sure the rsync send the db over.... i will investigate Happy New Year Quote Link to comment
geekazoid Posted January 1, 2023 Share Posted January 1, 2023 I built this again from scratch and it went off without a hitch using the procedure I laid out in my comment above. What a delight! I also figured out what causes the root permissions bug. If you create the path manually before installation, it doesn't happen. So I think I will update my guide to move those steps to the beginning and include the symlink steps as well. I also found a little nugget people might appreciate: If you want to set the hostname of your pihole, select Advanced View in the Edit page and append " --hostname=your_hostname" to Extra Parameters. Quote Link to comment
ZappyZap Posted January 1, 2023 Author Share Posted January 1, 2023 (edited) 31 minutes ago, geekazoid said: Question: what is the purpose of the ssh path since .ssh is in root? This is the directory in which the SSH server key files and the SSH daemon config will be stored, so it needs to be persistent. only relevant to receiver..... i might go head and remove from Sender .... Edited January 1, 2023 by ZappyZap Quote Link to comment
ZappyZap Posted January 1, 2023 Author Share Posted January 1, 2023 24 minutes ago, geekazoid said: I built this again from scratch and it went off without a hitch using the procedure I laid out in my comment above. What a delight! I also figured out what causes the root permissions bug. If you create the path manually before installation, it doesn't happen. So I think I will update my guide to move those steps to the beginning and include the symlink steps as well. I also found a little nugget people might appreciate: If you want to set the hostname of your pihole, select Advanced View in the Edit page and append " --hostname=your_hostname" to Extra Parameters. If you want make a PR in to https://github.com/devzwf/unraid-docker-templates/tree/main/guides and i will add the guide in the readme first section of the template perhaps ? let me know 1 Quote Link to comment
geekazoid Posted January 2, 2023 Share Posted January 2, 2023 Another configuration nugget for those using pihole's dhcp service. In dnsmasq.d create a file called 99-extra-dns.conf, contents: dhcp-option=6,pihole1_ip,pihole2_ip So your dhcp clients are aware of the primary and secondary dns servers. Quote Link to comment
ZappyZap Posted January 2, 2023 Author Share Posted January 2, 2023 33 minutes ago, geekazoid said: Another configuration nugget for those using pihole's dhcp service. In dnsmasq.d create a file called 99-extra-dns.conf, contents: dhcp-option=6,pihole1_ip,pihole2_ip So your dhcp clients are aware of the primary and secondary dns servers. this should go in pihole thread .... Quote Link to comment
axeman Posted February 2, 2023 Share Posted February 2, 2023 Thanks for your work! Would this be able to sync to piholes where one isn't in a docker? My primary pihole runs as a VM on an ESXi machine. I'd like to sync that to a potential docker container on Unraid (potential because I need to understand how to get them to sync before deploying). Also - does the "disable pihole for x minutes" sync or is it just the lists that are getting synced? Quote Link to comment
ZappyZap Posted February 2, 2023 Author Share Posted February 2, 2023 11 minutes ago, axeman said: Thanks for your work! Would this be able to sync to piholes where one isn't in a docker? My primary pihole runs as a VM on an ESXi machine. I'd like to sync that to a potential docker container on Unraid (potential because I need to understand how to get them to sync before deploying). Also - does the "disable pihole for x minutes" sync or is it just the lists that are getting synced? should doable but with some adjustment. I dont think the "Disable phole for x minutes" write anything to file , and all is in mem so i dont think this getting synced ... need to confirm Quote Link to comment
geekazoid Posted February 2, 2023 Share Posted February 2, 2023 1 hour ago, axeman said: does the "disable pihole for x minutes" sync or is it just the lists that are getting synced? PiHole Browser Extension supports multiple PiHoles. For the other stuff: You can run dockers on ESXi. Then you can deploy pihole in docker and use the shared storage to run pihole-sync. There may be licensing requirements from vmware. My last certification was vSphere 5; I don't know. Otherwise you'd need to set up some kind of shared storage (nfs) that your linux pihole VMs can mount. Note: this would not be mounted to the hypervisor but to the VMs directly. Basically a bunch of work and adding points of failure. Then you could fork the project and adapt the pihole-sync scripts to whatever linux you want to run etc. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.