Slappy Posted August 18, 2021 Share Posted August 18, 2021 Hello, For some reason NTP is always the bane of my existence, and unraid seems to be no exception. After being running for 90 days without a reboot I noticed the clock on my automation system (unraid docker) was a couple minutes off, which upon further debugging led me to discover the clock on unraid (verified with 'date' command) was off. I verified NTP was running with 'ntpq -p' and i get the following using the default configuration, and the default pool.ntp.org severs (pool, 0.pool, etc): :~# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== LOCAL(0) .LOCL. 10 l 6 64 1 0.000 +0.000 0.000 pscolka.of.pl .INIT. 16 u - 64 0 0.000 +0.000 0.000 laika.paina.net .INIT. 16 u - 64 0 0.000 +0.000 0.000 138.68.201.49 ( .INIT. 16 u - 64 0 0.000 +0.000 0.000 198.255.68.106 .INIT. 16 u - 64 0 0.000 +0.000 0.000 If I restart NTP I get the following in syslog: Aug 18 12:39:35 icecube root: Stopping NTP daemon... Aug 18 12:39:36 icecube ntpd[32271]: ntpd [email protected] Tue Oct 20 18:42:21 UTC 2020 (1): Starting Aug 18 12:39:36 icecube ntpd[32271]: Command line: /usr/sbin/ntpd -g -u ntp:ntp Aug 18 12:39:36 icecube ntpd[32271]: ---------------------------------------------------- Aug 18 12:39:36 icecube ntpd[32271]: ntp-4 is maintained by Network Time Foundation, Aug 18 12:39:36 icecube ntpd[32271]: Inc. (NTF), a non-profit 501(c)(3) public-benefit Aug 18 12:39:36 icecube ntpd[32271]: corporation. Support and training for ntp-4 are Aug 18 12:39:36 icecube ntpd[32271]: available at https://www.nwtime.org/support Aug 18 12:39:36 icecube ntpd[32271]: ---------------------------------------------------- Aug 18 12:39:36 icecube ntpd[32273]: proto: precision = 0.064 usec (-24) Aug 18 12:39:36 icecube ntpd[32273]: basedate set to 2020-10-08 Aug 18 12:39:36 icecube ntpd[32273]: gps base set to 2020-10-11 (week 2127) Aug 18 12:39:36 icecube ntpd[32273]: Listen normally on 0 lo 127.0.0.1:123 Aug 18 12:39:36 icecube ntpd[32273]: Listen normally on 1 br0 192.168.XXX.XXX:123 Aug 18 12:39:36 icecube ntpd[32273]: Listen normally on 2 lo [::1]:123 Aug 18 12:39:36 icecube ntpd[32273]: Listening on routing socket on fd #19 for interface updates Aug 18 12:39:36 icecube ntpd[32273]: kernel reports TIME_ERROR: 0x2041: Clock Unsynchronized Aug 18 12:39:36 icecube ntpd[32273]: kernel reports TIME_ERROR: 0x2041: Clock Unsynchronized Aug 18 12:39:36 icecube root: Starting NTP daemon: /usr/sbin/ntpd -g -u ntp:ntp ntpq -p stays stuck in .INIT. for days/weeks so it's not a matter of "just wait a bit longer." I've tried the following to debug: Restart NTPD on the command line; reboot server. Same symptoms Try alternate NTP servers, including entering an IP address from the ntp.org pool servers. Same .INIT. Opened port 123 for UDP on my router and mapped to my unraid server at 192.168.XXX.XXX which is the br0 interface it should be listening on. Verified UDP connectivity from the outside world using an online port checker (UDP Port Checker Online - Open Port) which reports that port 123 is open when hit from my external IP address Checked that I could hit port 123 from another server on the lan using nmap. Reported success Drank 2 Manhattans and cursed the NTP gods. That felt good but didn't really help much Anyone have any ideas? Quote Link to comment
Slappy Posted August 19, 2021 Author Share Posted August 19, 2021 From the "curiouser and curiouser" files, I have another server running as a VM on my unraid box that was migrated from physical hardware long ago. It's running the default ubuntu NTP setup, and NTP is connecting to the ubuntu pool servers just fine, so that seems to eliminate a network configuration problem or port blocking somewhere, and isolate this to a unraid problem. The only thing I can think of is that I have bridged interfaces... would that create a problem? ntpq -p from ubuntu VM on the same LAN: dude@server:~$ ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== 0.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000 1.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000 2.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000 3.ubuntu.pool.n .POOL. 16 p - 64 0 0.000 0.000 0.000 ntp.ubuntu.com .POOL. 16 p - 64 0 0.000 0.000 0.000 -muug.ca 216.218.254.202 2 u 449 1024 377 46.959 1.997 0.662 +ntp-ca.padevsof 97.183.206.88 2 u 528 1024 377 40.260 1.904 0.428 *voipmonitor.wci 66.220.9.122 2 u 768 1024 377 73.923 0.002 0.861 -time.nullrouten 132.163.97.1 2 u 631 1024 377 67.773 2.113 0.214 +time.cloudflare 10.106.8.9 3 u 655 1024 377 27.078 -0.763 0.118 Quote Link to comment
Slappy Posted August 19, 2021 Author Share Posted August 19, 2021 One more data point... I made my local ubuntu VM an ntp server for my unraid box, and it was able to connect and sync just fine. So it sppears that I can't connect to servers outside the LAN and get ntp to move beyond .INIT., so that rules out some sort of generalized network misconfiguration preventing NTP from working on the unraid side. This serves as a temporary fix but I'd sure like to get NTP to sync with external servers. Quote Link to comment
pants Posted February 11, 2022 Share Posted February 11, 2022 I had an issue with symptoms very similar to those described in the first post of this thread. In my case, I noticed the system time on my server was out of sync by a few minutes and discovered that all my ntp remotes were also stuck in the ".INIT." state. I was able to root cause my issue using tcpdump (installable via the nerdpack plugin) and the ntpdate command. In one UnRaid terminal I ran tcpdump to listen for ntp traffic while in another terminal I shutdown ntp, ran ntpdate, and looked for packets from both the ntp client and ntp server as described in this [external] post. I discovered that no packets from the ntp server were returning on the eth0 or br0 network interfaces. I have my server configured to send all traffic over a Wireguard VPN tunnel which is accomplished by introducing a new secure network interface wg0. I found that when I instructed ntp and tcpdump to listen for responses on wg0 they was able to see the packets returned from the ntp server. I suspect the rc.ntpd script written by Limetech is not properly accounting for ntp traffic transmitted via wg0. I filed a bug report here and was able to fix my issue by altering the rc.ntpd script (as described in the report). Quote Link to comment
eeemt Posted June 20, 2023 Share Posted June 20, 2023 (edited) In my case, I also experience the `ntpd` state stuck in `INIT`. And I fix it by hack into built-in script. I hava TWO physical internet interfaces on my motherboard, and bridge is trun on by default, named `br0`. When I SSH into unraid, execute `ntpq -p`, the result is just as @Slappy mentioned. And then, kill ntpd, execute `ntpd -gq`, It result in something like this: ``` root@WhiteTower:~# ntpd -gq 20 Jun 22:11:24 ntpd[11706]: ntpd [email protected] Fri Jun 3 04:17:10 UTC 2022 (1): Starting 20 Jun 22:11:24 ntpd[11706]: Command line: ntpd -gq 20 Jun 22:11:24 ntpd[11706]: ---------------------------------------------------- 20 Jun 22:11:24 ntpd[11706]: ntp-4 is maintained by Network Time Foundation, 20 Jun 22:11:24 ntpd[11706]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 20 Jun 22:11:24 ntpd[11706]: corporation. Support and training for ntp-4 are 20 Jun 22:11:24 ntpd[11706]: available at https://www.nwtime.org/support 20 Jun 22:11:24 ntpd[11706]: ---------------------------------------------------- 20 Jun 22:11:24 ntpd[11706]: proto: precision = 0.067 usec (-24) 20 Jun 22:11:24 ntpd[11706]: line 84 column 10 syntax error, unexpected T_String, expecting T_Drop or T_Ignore or T_Listen 20 Jun 22:11:24 ntpd[11706]: syntax error in /etc/ntp.conf line 84, column 10 20 Jun 22:11:24 ntpd[11706]: basedate set to 2022-05-22 20 Jun 22:11:24 ntpd[11706]: gps base set to 2022-05-22 (week 2211) 20 Jun 22:11:24 ntpd[11706]: Listen and drop on 0 v4wildcard 0.0.0.0:123 20 Jun 22:11:24 ntpd[11706]: Listen normally on 1 lo 127.0.0.1:123 20 Jun 22:11:24 ntpd[11706]: Listening on routing socket on fd #18 for interface updates ``` It stucked and even not send a packet (I also run a tcpdump on another SSH session). Here comes the important part. I modified `/etc/rc.d/rc.ntpd` from ``` ...... echo "# Generated entries follow:" >>$CONF echo "interface ignore wildcard" >>$CONF ...... ``` into ``` ...... echo "# Generated entries follow:" >>$CONF echo "interface listen br0" >>$CONF # explicitly specify listen on br0 echo "interface ignore wildcard" >>$CONF ...... ``` After that, by execute `/etc/rc.d/rc.ntpd reload` to re-generate the /etc/ntp.conf file. The conf file will look like this: ``` ...... # Generated entries follow: interface listen br0 interface ignore wildcard interface ignore ipv6 server some.ntp.server.com iburst ``` Finally, rerun `ntpd -gq` to test if it works, it could be something like this: ``` 21 Jun 13:03:47 ntpd[2288]: ntpd [email protected] Fri Jun 3 04:17:10 UTC 2022 (1): Starting 21 Jun 13:03:47 ntpd[2288]: Command line: ntpd -gq 21 Jun 13:03:47 ntpd[2288]: ---------------------------------------------------- 21 Jun 13:03:47 ntpd[2288]: ntp-4 is maintained by Network Time Foundation, 21 Jun 13:03:47 ntpd[2288]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 21 Jun 13:03:47 ntpd[2288]: corporation. Support and training for ntp-4 are 21 Jun 13:03:47 ntpd[2288]: available at https://www.nwtime.org/support 21 Jun 13:03:47 ntpd[2288]: ---------------------------------------------------- 21 Jun 13:03:47 ntpd[2288]: proto: precision = 0.067 usec (-24) 21 Jun 13:03:47 ntpd[2288]: basedate set to 2022-05-22 21 Jun 13:03:47 ntpd[2288]: gps base set to 2022-05-22 (week 2211) 21 Jun 13:03:47 ntpd[2288]: Listen normally on 0 lo 127.0.0.1:123 21 Jun 13:03:47 ntpd[2288]: Listen normally on 1 br0 192.168.50.160:123 21 Jun 13:03:47 ntpd[2288]: Listening on routing socket on fd #18 for interface updates 21 Jun 13:03:54 ntpd[2288]: ntpd: time slew +0.000075 s ntpd: time slew +0.000075s ``` Then restart the ntpd by `/etc/rc.d/rc.ntpd restart` or use the web-gui (recommended). At the end, this solution isn't graceful, but it's the only way I found. If any one got a better solution, remind me please. Edited June 21, 2023 by eeemt Correct it Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.