Having high values of Master offset, freq and path delay
rass92 opened this issue · comments
Dear Community
I have the following testbed
Master
TQMLS1028A-IND
It just have Hardware Timestamp
Operating System,
ID="poky"
NAME="Poky (Yocto Project Reference Distro)"
VERSION="3.0.4 (zeus)"
VERSION_ID="3.0.4"
PRETTY_NAME="Poky (Yocto Project Reference Distro) 3.0.4 (zeus)"
It has ptp4l version 2.0
Slave
PC
It just have Software Timestamp
Operating System:
NAME="Ubuntu"
VERSION="20.04.1 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.1 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
I am using this command for the master
taskset -c 1 chrt 99 ptp4l -i eno1 -mq
This command for the slave
sudo taskset -c 1 chrt 99 ptp4l -S -i enp3s0f1 -mq -s
It is giving me very high values of Master offset, freq and path delay and it goes to the state s2 SLAVE on MASTER_CLOCK_SELECTED but just for a few seconds. It is giving the below error
ptp4l[7632.725]: master offset 1611850007215613701 s2 freq +100000000 path delay 17013783
ptp4l[7633.625]: master offset 1611850007110097044 s2 freq +100000000 path delay 22519248
ptp4l[7634.525]: clockcheck: clock jumped forward or running faster than expected!
ptp4l[7634.525]: master offset 1611850010370484934 s0 freq +100000000 path delay 22519248
ptp4l[7634.525]: port 1: SLAVE to UNCALIBRATED on SYNCHRONIZATION_FAULT
ptp4l[7635.425]: master offset 1611850010270476347 s0 freq +100000000 path delay 22519248
ptp4l[7636.325]: master offset 1611850010167504974 s0 freq +100000000 path delay 25493617
I have tried to configure them with the gPTP.cfg values with the command
For Master
taskset -c 1 chrt 99 ptp4l -f gPTP.cfg -i eno1 -mq
For Slave
sudo taskset -c 1 chrt 99 ptp4l -S -f gPTP.cfg -i enp3s0f1 -mq -s
But it is worst because the slave dont tries to synchronize at all
ptp4l[8748.087]: selected local clock b82a72.fffe.9ac430 as best master
ptp4l[8751.360]: selected local clock b82a72.fffe.9ac430 as best master
ptp4l[8754.814]: selected local clock b82a72.fffe.9ac430 as best master
I am not sure if the issue is because they have different timestamping or if there is any configuration file or parameters I have to give them.
Can you help on this?
Thank you in advance
Hey,
have you checked if the ntp service of your ubuntu is still running? This can cause these "clock jump" errors.
systemctl status systemd-timesyncd
In case it is running stop it:
sudo systemctl stop systemd-timesyncd
If that fixes your problem, disable systemd-timesyncd permanently:
sudo systemctl disable systemd-timesyncd
I am not sure anymore if above name is actually the correct service name, so maybe double check yourself what the service is called in ubuntu.
Cheers
J
Hey janruth,
Thank you for your answer.
I dont have the ntp service activate
$ systemctl status systemd-timesyncd
● systemd-timesyncd.service - Network Time Synchronization
Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:systemd-timesyncd.service(8)
$ timedatectl
Local time: Sa 2021-01-30 15:18:11 UTC
Universal time: Sa 2021-01-30 15:18:11 UTC
RTC time: Sa 2021-01-30 15:18:11
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
**NTP service: inactive**
RTC in local TZ: no
@janruh Do you think that the fact they have different timestamping can be the reason of it?
Hey @janruth,
It worked I have to disable the ntp service in the board and the PC with the command
timedatectl set-ntp false
then run ptp4l in the board, then after phc2sys in the board with the command
phc2sys -s eno1 -w -m
Wait until the phc2sys gets into the state "s2"
Then run ptp4l in the PC and after some seconds it get into state s2 with low Master offset values.
One thing I dont understand is why the board and the computer are setting the times into 1970-01-02,
because the hardware clock in the board is set into the correct time?
I realized that phc2sys is using the CLOCK_REALTIME, reading the POXIS documents its been said that CLOCK_REALTIME should be the system's best estimate of the current UTC or civil time. Then if the hardware clock, Local Time and Universal time are correct, where CLOCK_REALTIME is getting this old date.
Here are some outputs that maybe can help to answer this
This is the output of phc2sys in the board
root@tqmls1028a-mbls1028a-ind:~# phc2sys -s eno1 -w -m
phc2sys[161676.123]: CLOCK_REALTIME phc offset 18 s2 freq -1986 delay 480
phc2sys[161677.123]: CLOCK_REALTIME phc offset -1 s2 freq -1999 delay 480
phc2sys[161678.123]: CLOCK_REALTIME phc offset -7 s2 freq -2006 delay 480
phc2sys[161679.123]: CLOCK_REALTIME phc offset -1 s2 freq -2002 delay 480
phc2sys[161680.124]: CLOCK_REALTIME phc offset 2 s2 freq -1999 delay 480
phc2sys[161681.124]: CLOCK_REALTIME phc offset -4 s2 freq -2004 delay 480
this is an output from the board
root@tqmls1028a-mbls1028a-ind:~# timedatectl
Local time: Fri 1970-01-02 20:55:44 UTC
Universal time: Fri 1970-01-02 20:55:44 UTC
RTC time: Sun 2021-01-31 12:32:22
Time zone: n/a (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
root@tqmls1028a-mbls1028a-ind:~# hwclock -r
2021-01-31 12:32:31.856146+00:00
this is the output of the computer when synchronizing
roberto@roberto-Inspiron-7537:/etc/linuxptp$ timedatectl set-ntp false
roberto@roberto-Inspiron-7537:/etc/linuxptp$ sudo taskset -c 1 chrt 99 ptp4l -S -i enp3s0f1 -mq -s -f ptp4l_default.cfg
[sudo] password for roberto:
ptp4l[50813.921]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[50813.921]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[50814.425]: port 1: new foreign master 00d093.fffe.4a780c-1
ptp4l[50818.425]: selected best master clock 00d093.fffe.4a780c
ptp4l[50818.425]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[50820.425]: master offset 1611934596203705187 s0 freq -8417 path delay 22596
ptp4l[50821.425]: master offset 1611934596203722557 s0 freq -8417 path delay 22596
ptp4l[50822.425]: master offset 1611934596203731822 s0 freq -8417 path delay 24293
ptp4l[50823.425]: master offset 1611934596203738198 s0 freq -8417 path delay 24293
ptp4l[50824.425]: master offset 1611934596203742072 s0 freq -8417 path delay 24293
ptp4l[50825.425]: master offset 1611934596203761386 s0 freq -8417 path delay 24209
ptp4l[50826.426]: master offset 1611934596203765074 s0 freq -8417 path delay 24293
ptp4l[50827.426]: master offset 1611934596203764575 s0 freq -8417 path delay 24209
ptp4l[50828.426]: master offset 1611934596203792722 s0 freq -8417 path delay 22740
ptp4l[50829.426]: master offset 1611934596203803437 s0 freq -8417 path delay 22740
ptp4l[50830.426]: master offset 1611934596203809379 s0 freq -8417 path delay 22740
ptp4l[50831.426]: master offset 1611934596203829045 s0 freq -8417 path delay 22740
ptp4l[50832.426]: master offset 1611934596203821426 s0 freq -8417 path delay 21567
ptp4l[50833.426]: master offset 1611934596203829538 s0 freq -8417 path delay 21567
ptp4l[50834.426]: master offset 1611934596203854992 s0 freq -8417 path delay 21199
ptp4l[50835.426]: master offset 1611934596203875027 s0 freq -8417 path delay 20416
ptp4l[50836.426]: master offset 1611934596203877285 s1 freq +2339 path delay 21411
ptp4l[50837.426]: master offset 1749 s2 freq +2515 path delay 21411
ptp4l[50837.426]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[50838.426]: master offset -4228 s2 freq +1913 path delay 21411
ptp4l[50839.426]: master offset -15833 s2 freq +737 path delay 20815
ptp4l[50840.426]: master offset 8579 s2 freq +3187 path delay 20815
ptp4l[50841.426]: master offset -4928 s2 freq +1831 path delay 21411
ptp4l[50842.426]: master offset 9151 s2 freq +3248 path delay 21411
ptp4l[50843.426]: master offset -24089 s2 freq -100 path delay 22466
ptp4l[50844.427]: master offset -14329 s2 freq +862 path delay 22466
ptp4l[50845.427]: master offset 11042 s2 freq +3410 path delay 21183
ptp4l[50846.427]: master offset 11296 s2 freq +3447 path delay 21922
ptp4l[50847.427]: master offset 10632 s2 freq +3391 path delay 21922
ptp4l[50848.427]: master offset -3595 s2 freq +1964 path delay 25022
ptp4l[50849.427]: master offset 8394 s2 freq +3172 path delay 22154
ptp4l[50850.427]: master offset -18358 s2 freq +478 path delay 22154
ptp4l[50851.427]: master offset 8916 s2 freq +3215 path delay 22091
ptp4l[50852.427]: master offset -18165 s2 freq +488 path delay 22091
ptp4l[50853.427]: master offset -1101 s2 freq +2194 path delay 22091
ptp4l[50854.427]: master offset 17245 s2 freq +4045 path delay 22091
This is the time in the PC
$ timedatectl
Local time: Fr 1970-01-02 21:21:49 UTC
Universal time: Fr 1970-01-02 21:21:49 UTC
RTC time: So 2021-01-31 12:58:25
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
@janruh Is there a way for the system not to do that?
Is there a way to tell the CLOCK_REALTIME to grab the value from the hardware clock (hwclock), because it seems like the CLOCK_REALTIME is grabbing the value from somewhere that is making to have it back into the epoch of UNIX TIME
I guess actually what you are seeing is that the 1970 date and time is provided by your grandmaster clock. Therefore, if you update the time and date of your grandmaster it should be propagated down the synchronization hierarchy. The reason you see the correct time with RTC is that this is your wall clock time and its value is based on a chip equipped with a celular battery on your mainboard so it can keep track of time even if you reboot your machine. However, the RTC does not provide you with the same resolution than CLOCK_REALTIME or CLOCK_MONOTONIC would (micro or even nanoseconds).
Thank you @janruh for your answer
I would expect that, if I update my grandmaster clock (the board) will propagate to the slave (PC) meaning that if my Local Time in the board is incorrect, when I execute
hwclock -s
The Local TIme and Universal Time of the board will get corrected and then propagated, but what is happening is that having this command running, that is the syncrhonization of the system clock to a PTP hardware clock
phc2sys -s eno1 -w -m
I see that the values of Master offset get very high, as for 50 years, the difference from now to 1970. When I execute and correct the time from 1970 to 2021
So it means that always the PTP protocol will always synchronize with the epoch of UNIX TIME. It cannot synchronize with the actual real live time (2021)?
I dont know if I am doing something wrong or if that is how PTP protocol works
Please post the outputs of your ptp4l command and execute phc2sys as follows and also post the outputs of like a 60 seconds run or however long it takes to converge:
phc2sys -s eno1 -c CLOCK_REALTIME -O 0 -w -m
Also keep in mind that hwclock gives you the time of your platform, like RTC and system time. If you're NIC's clock is somewhere around 1970 and you propagate its time down the clock synchronization hierarchy and synchronize your system time to the NIC's time as well (using phc2sys) then you will see this 1970 time everywhere....
Output of timedatectl on the board (master) without running anything
root@tqmls1028a-mbls1028a-ind:~# timedatectl
Local time: Wed 2021-02-03 10:10:49 UTC
Universal time: Wed 2021-02-03 10:10:49 UTC
RTC time: Wed 2021-02-03 10:10:50
Time zone: n/a (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
Output of phc2sys in the board (Master), the rest of the values are mostly the same
root@tqmls1028a-mbls1028a-ind:~# phc2sys -s eno1 -c CLOCK_REALTIME -O 0 -w -m
phc2sys[2006.891]: CLOCK_REALTIME phc offset 1612345025660222525 s0 freq -2001 delay 480
phc2sys[2007.891]: CLOCK_REALTIME phc offset 1612345025660222532 s1 freq -1994 delay 480
phc2sys[2008.891]: CLOCK_REALTIME phc offset 4 s2 freq -1990 delay 480
phc2sys[2009.891]: CLOCK_REALTIME phc offset -21 s2 freq -2014 delay 480
phc2sys[2010.892]: CLOCK_REALTIME phc offset -12 s2 freq -2011 delay 480
phc2sys[2011.892]: CLOCK_REALTIME phc offset 19 s2 freq -1984 delay 480
phc2sys[2012.892]: CLOCK_REALTIME phc offset -7 s2 freq -2004 delay 480
phc2sys[2013.892]: CLOCK_REALTIME phc offset 13 s2 freq -1986 delay 480
phc2sys[2014.892]: CLOCK_REALTIME phc offset -21 s2 freq -2016 delay 480
phc2sys[2015.892]: CLOCK_REALTIME phc offset 11 s2 freq -1991 delay 480
phc2sys[2016.893]: CLOCK_REALTIME phc offset -9 s2 freq -2007 delay 480
phc2sys[2017.893]: CLOCK_REALTIME phc offset 9 s2 freq -1992 delay 480
phc2sys[2018.893]: CLOCK_REALTIME phc offset -19 s2 freq -2017 delay 480
Output of timedatectl on the board (Master) after running phc2sys
root@tqmls1028a-mbls1028a-ind:~# timedatectl
Local time: Thu 1970-01-01 00:35:04 UTC
Universal time: Thu 1970-01-01 00:35:04 UTC
RTC time: Wed 2021-02-03 10:12:10
Time zone: n/a (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
Output of the PC(slave) before running ptp4l
roberto@roberto-Inspiron-7537:~$ timedatectl
Local time: Mi 2021-02-03 10:19:38 UTC
Universal time: Mi 2021-02-03 10:19:38 UTC
RTC time: Mi 2021-02-03 10:19:47
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
Output of the PC of ptp4l
roberto@roberto-Inspiron-7537:~$ sudo ptp4l -S -i enp3s0f1 -mq -s -2
ptp4l[2510.089]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[2510.089]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[2510.312]: port 1: new foreign master 00d093.fffe.4a780c-1
ptp4l[2513.912]: selected best master clock 00d093.fffe.4a780c
ptp4l[2513.912]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[2515.712]: master offset 1612345047363283513 s0 freq +100000000 path delay 26433466
ptp4l[2516.612]: master offset 1612345047263285185 s0 freq +100000000 path delay 26433466
ptp4l[2517.512]: master offset 1612345047156610912 s0 freq +100000000 path delay 33103885
ptp4l[2518.412]: master offset 1612345047056610348 s0 freq +100000000 path delay 33103885
ptp4l[2519.312]: master offset 1612345046956609575 s0 freq +100000000 path delay 33103885
ptp4l[2520.212]: master offset 1612345046863271126 s0 freq +100000000 path delay 26433466
ptp4l[2521.112]: master offset 1612345046763275326 s0 freq +100000000 path delay 26433466
ptp4l[2522.012]: master offset 1612345046663280885 s0 freq +100000000 path delay 26433466
ptp4l[2522.913]: master offset 1612345046563269233 s0 freq +100000000 path delay 26433466
ptp4l[2523.813]: master offset 1612345046464058428 s0 freq +100000000 path delay 25646775
ptp4l[2524.713]: master offset 1612345046376303103 s0 freq +100000000 path delay 13398676
ptp4l[2525.613]: master offset 1612345046276303153 s0 freq +100000000 path delay 13398676
ptp4l[2526.513]: master offset 1612345046180697819 s0 freq +100000000 path delay 9007515
ptp4l[2527.413]: master offset 1612345046080690579 s0 freq +100000000 path delay 9007515
ptp4l[2528.313]: master offset 1612345045980688782 s0 freq +100000000 path delay 9007515
ptp4l[2529.213]: master offset 1612345045880682456 s0 freq +100000000 path delay 9007515
ptp4l[2530.113]: master offset 1612345045777818331 s0 freq +100000000 path delay 11876434
ptp4l[2531.013]: master offset 1612345045677782798 s0 freq +100000000 path delay 11876434
ptp4l[2531.913]: master offset 1612345045577816809 s1 freq +811303 path delay 11876434
ptp4l[2532.912]: master offset -885891 s2 freq +721828 path delay 11876434
ptp4l[2532.912]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[2533.911]: master offset 1860560 s2 freq +998333 path delay 8407140
ptp4l[2534.910]: master offset 852322 s2 freq +898362 path delay 8407140
ptp4l[2535.910]: master offset -29529 s2 freq +810147 path delay 8407140
ptp4l[2536.909]: master offset -836346 s2 freq +728629 path delay 8407140
ptp4l[2537.908]: master offset 6439041 s2 freq +1462607 path delay 411262
ptp4l[2538.907]: master offset 4974552 s2 freq +1321133 path delay 411262
ptp4l[2539.905]: master offset 3653246 s2 freq +1192655 path delay 411262
ptp4l[2540.904]: master offset 2535108 s2 freq +1083377 path delay 338048
ptp4l[2541.903]: master offset 1493624 s2 freq +980722 path delay 300152
ptp4l[2542.902]: master offset 515529 s2 freq +883428 path delay 300152
ptp4l[2543.901]: master offset -366409 s2 freq +794868 path delay 300152
ptp4l[2544.901]: master offset -1158227 s2 freq +714528 path delay 300152
ptp4l[2545.900]: master offset -1748497 s2 freq +653752 path delay 183988
ptp4l[2546.899]: master offset -2457768 s2 freq +580367 path delay 232090
ptp4l[2547.899]: master offset -2981241 s2 freq +525039 path delay 183988
ptp4l[2548.898]: master offset -3469793 s2 freq +472714 path delay 154804
ptp4l[2549.898]: master offset -3945571 s2 freq +421190 path delay 154804
ptp4l[2550.897]: master offset -4339378 s2 freq +377470 path delay 119741
ptp4l[2551.897]: master offset -4693077 s2 freq +337407 path delay 109557
ptp4l[2552.897]: master offset -5026041 s2 freq +299085 path delay 109557
ptp4l[2553.897]: master offset -5325729 s2 freq +263790 path delay 109557
ptp4l[2554.896]: master offset -5617974 s2 freq +228948 path delay 141344
^Croberto@roberto-Inspiron-7537:~$ sudo ptp4l -S -i enp3s0f1 -mq -s -2
ptp4l[2556.333]: port 1: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[2556.334]: port 0: INITIALIZING to LISTENING on INIT_COMPLETE
ptp4l[2557.896]: port 1: new foreign master 00d093.fffe.4a780c-1
ptp4l[2561.895]: selected best master clock 00d093.fffe.4a780c
ptp4l[2561.895]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[2563.895]: master offset -7590890 s0 freq +228947 path delay 74697
ptp4l[2564.895]: master offset -7818547 s0 freq +228947 path delay 76315
ptp4l[2565.894]: master offset -8044164 s0 freq +228947 path delay 76315
ptp4l[2566.894]: master offset -8286425 s0 freq +228947 path delay 91395
ptp4l[2567.894]: master offset -8521385 s0 freq +228947 path delay 102157
ptp4l[2568.894]: master offset -8757316 s0 freq +228947 path delay 112919
ptp4l[2569.894]: master offset -8982114 s0 freq +228947 path delay 113062
ptp4l[2570.893]: master offset -9214346 s0 freq +228947 path delay 113062
ptp4l[2571.893]: master offset -9438065 s0 freq +228947 path delay 112919
ptp4l[2572.893]: master offset -9664841 s0 freq +228947 path delay 112919
ptp4l[2573.893]: master offset -9882846 s0 freq +228947 path delay 102618
ptp4l[2574.893]: master offset -10107523 s0 freq +228947 path delay 102618
ptp4l[2575.892]: master offset -10327340 s0 freq +228947 path delay 102618
ptp4l[2576.892]: master offset -10561629 s0 freq +228947 path delay 102618
ptp4l[2577.892]: master offset -10797042 s0 freq +228947 path delay 113062
ptp4l[2578.892]: master offset -11013554 s0 freq +228947 path delay 102618
ptp4l[2579.892]: master offset -11240395 s1 freq +861 path delay 102618
ptp4l[2580.892]: master offset 2128 s2 freq +1076 path delay 102618
ptp4l[2580.892]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[2581.892]: master offset 15407 s2 freq +2420 path delay 91505
ptp4l[2582.892]: master offset 17291 s2 freq +2625 path delay 91505
ptp4l[2583.892]: master offset 17646 s2 freq +2678 path delay 88832
ptp4l[2584.892]: master offset 17787 s2 freq +2710 path delay 88832
ptp4l[2585.892]: master offset 40138 s2 freq +4986 path delay 64273
ptp4l[2586.892]: master offset 44671 s2 freq +5484 path delay 54018
ptp4l[2587.892]: master offset 50016 s2 freq +6068 path delay 50735
ptp4l[2588.892]: master offset 32432 s2 freq +4342 path delay 50735
ptp4l[2589.892]: master offset 53436 s2 freq +6496 path delay 50312
ptp4l[2590.892]: master offset 40581 s2 freq +5251 path delay 50312
ptp4l[2591.892]: master offset 45488 s2 freq +5787 path delay 50312
ptp4l[2592.892]: master offset 37867 s2 freq +5063 path delay 50312
ptp4l[2593.892]: master offset 32372 s2 freq +4546 path delay 50735
ptp4l[2594.892]: master offset 37564 s2 freq +5103 path delay 50594
ptp4l[2595.892]: master offset 28296 s2 freq +4204 path delay 50616
Output of the PC (Slave) after running ptp4l
~$ timedatectl
Local time: Do 1970-01-01 00:47:52 UTC
Universal time: Do 1970-01-01 00:47:52 UTC
RTC time: Mi 2021-02-03 10:25:33
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
Ok, so you can clearly see that the clock of your NIC is set to 1970. And since you pass that time to your slave and also synchronize the system time of the master to the NIC's clock you see that time everywhere. Try switching the parameters of phc2sys to synchronize the NIC to the system time.
Done, you where right I had to update the time on the NIC
first run this command to update the system clock time (CLOCK_REALTIME)
hwclock -s
Then run this command to update the time on the NIC
phc2sys -c eno1 -O 0 -w -m -s CLOCK_REALTIME
Thank you very much