IanB Posted January 21, 2018 Share Posted January 21, 2018 Hi, I'm currently running a fairly new / clean install of Unraid 6.4 and am finding that the time (accessible via Settings > Date and Time) runs "slow". I will reset the time by either setting it manually, or by changing the UTP servers. After about ~20 mins though, it's already noticeably behind. If I leave it a few hours, it's a few minutes behind and the difference keeps getting larger and larger the longer I run it. I've attached some diagnostics in case that's useful. But any help would be greatly appreciated. As the time goes more and more out of sync, various services start failing (e.g. syncing to Amazon S3 can fail if the local and remote clock are too far out of sync). Thanks in advance Ian tower-diagnostics-20180121-1625.zip Quote Link to comment
pwm Posted January 21, 2018 Share Posted January 21, 2018 Does the erroneous clock speed survive a reboot? ntp that is responsible for keeping the time also keeps track of how much drift a specific hardware has. Once in a while, it can totally fail with this and compute a horrendous drift value making the clock run very much too fast or slow. ntpd keeps the drift value in /var/lib/ntp, but I have never checked if unraid saves the content of this directory on reboot. Quote Link to comment
bonienl Posted January 21, 2018 Share Posted January 21, 2018 5 minutes ago, pwm said: ntpd keeps the drift value in /var/lib/ntp, but I have never checked if unraid saves the content of this directory on reboot. yes, drift is stored on flash (see /config folder) and used on reboot. Quote Link to comment
pwm Posted January 21, 2018 Share Posted January 21, 2018 Just now, bonienl said: yes, drift is stored on flash (see /config folder) and used on reboot. I must be bad at searching - I do find the configuration settings in the /boot/config folder but no file that seems to store any drift value. Or maybe it gets deleted on every boot after the value has been copied to RAM? But I have two times had to clear the drift value for other machines where ntpd has goofed and mistaken some manual time update as drift that should be corrected. Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 (edited) Hey, Thanks for the responses! >Does the erroneous clock speed survive a reboot? Yeah, i've rebooted the machine multiple times and have seen this issue again and again. >drift is stored on flash I've checked the flash drive, and /var/lib/ntp and both locations have a "drift" file (which contains "0.000"). As mentioned above, this happens with NPT enabled in my settings, but also with it off and just a date / time set manually. I assume the contents of my drift file(s) means that it thinks the machine clock is completely in sync with the ntp server(s)? As you can see, it's gone out of sync by ~4 mins since I last synced (which was when I first created this forum post) Ian Edited January 21, 2018 by IanB Added screenshot Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 (edited) Browsing around, I thought this info would be useful: root@Tower:/mnt/user# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== LOCAL(0) .LOCL. 10 l 3 64 1 0.000 0.000 0.000 time3.google.co .GOOG. 1 u 1 64 1 138.324 300111. 85.763 li1535-196.memb .INIT. 16 u - 64 0 0.000 0.000 0.000 I noticed that when I modify the set of NTP servers, the drift file disappears (from /var/lib/ntp/) so I guess it gets re-created later? This was run right after I added another NTP server, which means my clock is now apparently in sync. I can run it again in ~20 mins to see if it's drifted out. Edited January 21, 2018 by IanB Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 (edited) Actually, only leaving it a few minutes shows that it's drifting: root@Tower:/mnt/user# ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== LOCAL(0) .LOCL. 10 l 29 64 77 0.000 0.000 0.000 time3.google.co .GOOG. 1 u 48 64 17 114.412 12742.3 9923.26 li1535-196.memb 89.101.218.6 2 u 16 64 77 148.922 14095.7 10737.0 Edited January 21, 2018 by IanB Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 I tried running `ntpdate` manually and got the following: root@Tower:~# ntpdate 21 Jan 19:07:43 ntpdate[15518]: no servers can be used, exiting Is this expected? I checked the `ntp.conf` file and it contained a number of servers defined at the bottom: # Generated entries follow: server time.google.com iburst server pool.ntp.org iburst Quote Link to comment
Zonediver Posted January 21, 2018 Share Posted January 21, 2018 Probably the ntp service is running, that's why ntpdate can't open the socket (port 123 UDP) and connect to ntp server. Try from command line: sudo service ntp stop sudo ntpdate -s time.nist.gov sudo service ntp start Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 Thanks for the suggestion. Unfortunately, it complains about "service": root@Tower:~# sudo service ntp stop sudo: service: command not found Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 I managed to stop, update, and restart manually doing the following: root@Tower:~# /etc/rc.d/rc.ntpd stop Stopping NTP daemon... root@Tower:~# ntpdate -s time.nist.gov root@Tower:~# /etc/rc.d/rc.ntpd start Starting NTP daemon: /usr/sbin/ntpd -g -u ntp:ntp My time is now correctly in sync, but will have to now watch to see how it drifts out. Is there a way for me to validate that the NTP service is periodically updating from the remote servers? 1 Quote Link to comment
Zonediver Posted January 21, 2018 Share Posted January 21, 2018 (edited) This is also working: ntpdate -u pool.ntp.org Put this in a cron and let it run every hour. Edited January 21, 2018 by Zonediver 1 Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 Since I last ran the `ntpdate` I saw that I was already a few seconds now out of sync. I ran the command, as you suggested (the ntpd service was still running) and it updated: root@Tower:~# ntpdate -u pool.ntp.org 21 Jan 19:37:44 ntpdate[26004]: step time server 87.81.181.2 offset 26.491838 sec Does this suggest that everything appears to work when updating from a remote ntp server, but for some reason, it's not periodically happening? Quote Link to comment
Zonediver Posted January 21, 2018 Share Posted January 21, 2018 (edited) 15 minutes ago, IanB said: Since I last ran the `ntpdate` I saw that I was already a few seconds now out of sync. I ran the command, as you suggested (the ntpd service was still running) and it updated: root@Tower:~# ntpdate -u pool.ntp.org 21 Jan 19:37:44 ntpdate[26004]: step time server 87.81.181.2 offset 26.491838 sec Does this suggest that everything appears to work when updating from a remote ntp server, but for some reason, it's not periodically happening? There must be something wrong with your machine or hardware. When i run the command, i have a gap of 0.000631sec within 5min's Check the voltage of your BIOS-Battery Edited January 21, 2018 by Zonediver Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 >There must be something wrong with your machine or hardware. That's worrying The hardware is all pretty new, Asus x299 Prime Deluxe with a Intel 7980XE... Quote Link to comment
Zonediver Posted January 21, 2018 Share Posted January 21, 2018 7 minutes ago, IanB said: >There must be something wrong with your machine or hardware. That's worrying The hardware is all pretty new, Asus x299 Prime Deluxe with a Intel 7980XE... Take a multimeter and check the voltage of the batterie. Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 This actually looks pretty relevant: https://bugzilla.kernel.org/show_bug.cgi?id=197299 Quote Link to comment
Zonediver Posted January 21, 2018 Share Posted January 21, 2018 (edited) 11 minutes ago, IanB said: This actually looks pretty relevant: https://bugzilla.kernel.org/show_bug.cgi?id=197299 Also possible. Deactivate spread spectrum and see what happens. Edited January 21, 2018 by Zonediver Quote Link to comment
IanB Posted January 21, 2018 Author Share Posted January 21, 2018 >Possible, but related to Intel Skylake Yeah, it's an issue with Skylake X - like the 7980XE that i'm using. Sounds very similar... If the bug turned out to be this kernel issue, how frequently does Unraid receive updates for kernel patches, etc? I've not been running it long enough to know what their update strategy is. Thanks again for the help! Quote Link to comment
IanB Posted January 22, 2018 Author Share Posted January 22, 2018 Just thought i'd give a quick update, in case anyone else in the future hits this issue... After reading about the kernel issue, I first tried updating my BIOS as my motherboard shipped with a pretty old version. This didn't make any difference. From the description of the kernel issue, it talks about it only happening when Spread Spectrum is enabled, for which my motherboard supported being manually disabled. After I did this though, I unfortunately still found that the clock would drift. There is however a linked (dupe) issue here which suggested another workaround: https://bugzilla.kernel.org/show_bug.cgi?id=198137 I was able to follow these steps, and resolve the clock drift: /etc/rc.d/rc.ntpd stop sudo adjtimex -p ** The tick parameter shows set to 10000 sudo adjtimex -c ** Shows a correct tick value of 10400 (ie.. +4% adjustment) audo adjtimex --tick 10400 sudo adjtimex -p ** Now checking the command-line "date" to the wall clock shows no appreciable drift over a few minutes. /etc/rc.d/rc.ntpd start My server has been running the full day without any noticeable drift. I'm sure that after i reboot, it may lose the adjtimex value and if so, i'll have to look at how to add it to my /config/go. Will keep an eye out for Unraid updates which include any kernel patches for this bug, and test them. 2 Quote Link to comment
RadOD Posted May 19, 2020 Share Posted May 19, 2020 On 1/21/2018 at 1:27 PM, IanB said: I managed to stop, update, and restart manually doing the following: root@Tower:~# /etc/rc.d/rc.ntpd stop Stopping NTP daemon... root@Tower:~# ntpdate -s time.nist.gov root@Tower:~# /etc/rc.d/rc.ntpd start Starting NTP daemon: /usr/sbin/ntpd -g -u ntp:ntp My time is now correctly in sync, but will have to now watch to see how it drifts out. Is there a way for me to validate that the NTP service is periodically updating from the remote servers? Thanks. This fixed it for me. Now to see if it goes out again... Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.