PTP synchronization related query
debnathr opened this issue · comments
Hi,
I am testing the Qbv scheduling and running the PTP synchronization using the below command.
ptp4l -i eth0 -p /dev/ptp0 -m -l 7 -t 1000
But I am getting a huge delay most of the time.
So, I want to set the NXP LS1021ATSN device as a master clock and my two BeagleBone Black as slaves. The setup of my experiment is as follows.
"I have two BeagleBone Black (BBB) as two hosts connected to the ETH3 and ETH5 ports of the NXP LS1021ATSN device."
So, could you please guide me with how to set the NXP LS1021ATSN as the master clock.
Thanks in advance for your time.
Do the Beaglebone Blacks support hardware timestamping?
ethtool -T eth0
Also, ptp4l implements the best master clock selection algorithm. Which device was selected as master clock can be inferred from each device's ptp4l log.
To force the LS1021A-TSN board as master, you can use the following ptp4l option on the 2 Beaglebones (taken from the manual):
-s Enable the slaveOnly mode.
Also I am not clear which command you are running on which board. You said ptp4l -i eth0 but if you are running this command on LS1021A-TSN, it should be ptp4l -i eth2 since that is the interface that goes through the switch and towards ETH3 and ETH5.
Also, how huge are the offsets? Can you post some logs of the 3 devices?
Hi,
Thank you for the answer.
Yes, Beaglebone Blacks do support hardware timestamping. I did run the ptp4l in Beaglebone Blacks in slave mode as you mentioned.
And as suggested by you I ran the below command in NXP device.
ptp4l -i eth2 -p /dev/ptp0 -m -l 7 -t 1000
But in the ptp4l log, I got the below message in NXP device.
ptp4l[3664.230]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
This seems like the NXP device is still running as slave based on the above log.
Any suggestions will be great.
Thanks
Have you made any progress on this? Could you post the ptp4l commands and full logs on the boards?
Hi,
I am running the below command on the BeagleBone Black (BBB) to set it as a slave:
ptp4l -f /etc/ptp4l.conf -A -4 -H -m -s -i eth0
The below command on the NXP device to set it as a master:
ptp4l -i eth2 -p /dev/ptp0 -m -l 7 -t 1000
Now I think my NXP device is behaving as a master by looking at the logs, but the delay is more than what I am expecting from a Hardware clock.
Thanks in advance for your time.
I see now, you are observing high synchronization offsets for the switch, not for the Ethernet ports.
Side note: to see the sync offsets of the Ethernet ports more clearly, try to grep for "master offset" instead (as seen here).
On the LS1021A-TSN boards, the switch-to-switch synchronization is done in 2 steps. First the Ethernet port (eth0, eth1) hardware clocks are synchronized to one another (PTP slave to PTP master). Then there is another control loop which keeps the SJA1105 clock in sync with the Ethernet port clock.
Unfortunately, due to the slow access to the SPI bus that we have in the current implementation, peaks in the switch-to-switch synchronization offsets of +/- 4000ns is about what you should expect. What throws off the calculations is the high delay between calling sja1105_ptp_clk_get and the actual moment when the correction is applied (sja1105_ptp_clk_rate_set) - time by which the calculated correction might not be valid anymore. That's about the best that can be done with a userspace-only implementation I think.
From my old notes I can tell you the following:
- If the SJA1105 sync offset exceeds a max threshold, the ptp4l resets the clock. This may do more bad than good. We have found that increasing that reset threshold (
-t 1000) helps a little bit to stabilize the situation. - Here you can find a screencast where you can compare with the synchronization offsets that we got: https://asciinema.org/a/bBRN7u2ndY3nSIeQRrMy074M3 Do note that core load also affects these synchronization offsets negatively.
- Here you can find a plot of the 3 switches simultaneously (for the 3-board TSN demo). You can see the occasional spikes there as well. screenshot from 2017-07-05 11-51-21
We do have a rework of the SJA1105 driver that models the switch's 1588 clock as a proper PTP hardware clock and performs the PTP read/write operations over SPI in kernel. We haven't benchmarked it yet (as the integration with Qbv is still not there yet in the kmod) but the limitations of the old version are in theory addressed.
Another thing you may want to try is to adjust the Kp and Ki parameters of the control loop that synchronizes the SJA1105 clock to the eth0 clock. To do this, start by adding the following in /etc/ptp4l_default.cfg:
sja1105_sync_kp 0.096
sja1105_sync_ki 0.007
These are the default values (if absent from the ptp4l.cfg file, the application uses these). Perhaps they are sub-optimal. You may try to start by setting Ki to zero and increasing/decreasing Kp until you arrive at smaller (or at least more stable) offsets.
Also, important excerpt from man ptp4l:
-f config
Read configuration from the specified file. No configuration file is read by default.
So that means you need to ptp4l -f /etc/ptp4l_default.cfg for the new Kp and Ki to be read.
Thank you so much for your time and efforts. I really appreciate your detailed explanation. I will follow the guidelines provided by you. Thanks again.
We do have a rework of the SJA1105 driver that models the switch's 1588 clock as a proper PTP hardware clock and performs the PTP read/write operations over SPI in kernel. We haven't benchmarked it yet (as the integration with Qbv is still not there yet in the kmod) but the limitations of the old version are in theory addressed.
Could you please tell me if it is possible to get the rework of the SJA1105 driver that models the switch's 1588 clock as a proper PTP hardware clock and performs the PTP read/write operations over SPI in the kernel? Is it available somewhere so that I can get it?
Thanks
The kernel module is still under development, and the PTP component is not fully functional yet. I do not have an exact schedule, but it will be integrated into OpenIL when it is feature-complete.
You can track the progress of the kmod rework in this pull request nxp-archive/openil_sja1105-tool#36 which is updated as soon as new features are stabilized (PTP is not part of the pull request yet).
If you want to contribute I can point you to development branches that contain the under-work PTP feature.
Thank you so much for the help. I will track the progress of the rework. I am closing this issue.
Hi,
I am working on the NXP LS1021A TSN switch and facing issues while trying to sync the hardware clock of NXP switch with other different board. I am getting high offset values while using PTP4L commands. Can you help me out by providing the information about decreasing the offset values.
I have run the commands ptp4l -i eth0 -m on the master device and ptp4l -i eth0 -m -s on the slave device
Output log:
ptp4l[2670.323]: selected /dev/ptp0 as PTP clock
ptp4l[2670.344]: port 1: INITIALIZING to LISTENING on INITIALIZE
ptp4l[2670.344]: port 0: INITIALIZING to LISTENING on INITIALIZE
ptp4l[2671.907]: port 1: new foreign master 00049f.fffe.ef0606-1
ptp4l[2675.908]: selected best master clock 00049f.fffe.ef0606
ptp4l[2675.908]: port 1: LISTENING to UNCALIBRATED on RS_SLAVE
ptp4l[2677.908]: master offset 1814389063 s0 freq +1000000 path delay -11282235
ptp4l[2678.908]: master offset 1886559517 s1 freq +1000000 path delay -20511764
ptp4l[2680.908]: master offset 121269846 s2 freq +1000000 path delay -15896999
ptp4l[2680.908]: port 1: UNCALIBRATED to SLAVE on MASTER_CLOCK_SELECTED
ptp4l[2681.908]: master offset 184210931 s2 freq +1000000 path delay -15896999
ptp4l[2682.908]: master offset 251765244 s2 freq +1000000 path delay -20511764
ptp4l[2683.908]: master offset 314706772 s2 freq +1000000 path delay -20511764
ptp4l[2684.908]: master offset 377648525 s2 freq +1000000 path delay -20511764
ptp4l[2685.908]: master offset 440589373 s2 freq +1000000 path delay -20511764
ptp4l[2686.908]: master offset 504437035 s2 freq +1000000 path delay -21418673
ptp4l[2687.908]: master offset 567378447 s2 freq +1000000 path delay -21418673
ptp4l[2688.908]: master offset 630240205 s2 freq +1000000 path delay -21339819
ptp4l[2689.908]: master offset 693181169 s2 freq +1000000 path delay -21339819
ptp4l[2690.909]: master offset 755680662 s2 freq +1000000 path delay -20897631
ptp4l[2691.909]: master offset 818622550 s2 freq +1000000 path delay -20897631
ptp4l[2692.909]: master offset 879851362 s2 freq +1000000 path delay -19185932
ptp4l[2693.909]: master offset 944503921 s2 freq +1000000 path delay -20897631
ptp4l[2694.909]: master offset 1007954619 s2 freq +1000000 path delay -21408405
ptp4l[2695.909]: master offset 1070896194 s2 freq +1000000 path delay -21408405
ptp4l[2696.909]: master offset 1132568476 s2 freq +1000000 path delay -20138162
ptp4l[2697.909]: master offset 1194557077 s2 freq +1000000 path delay -19185932
ptp4l[2698.909]: master offset 1257321980 s2 freq +1000000 path delay -19009231
ptp4l[2699.909]: master offset 1320774158 s2 freq +1000000 path delay -19520005
ptp4l[2700.909]: master offset 1386472022 s2 freq +1000000 path delay -22276805
ptp4l[2701.909]: master offset 1451115371 s2 freq +1000000 path delay -23979093
ptp4l[2702.909]: master offset 1512354701 s2 freq +1000000 path delay -22276805
ptp4l[2703.909]: master offset 1571665092 s2 freq +1000000 path delay -18645601
ptp4l[2704.910]: master offset 1626802503 s2 freq +1000000 path delay -10840052
ptp4l[2705.910]: master offset 1689743412 s2 freq +1000000 path delay -10840052
ptp4l[2706.910]: master offset 1751953893 s2 freq +1000000 path delay -10110519
ptp4l[2707.910]: master offset 1814895418 s2 freq +1000000 path delay -10110519
The kernel version and tag are
[root@OpenIL:misc]# uname -v
#1 SMP Fri Feb 1 17:11:55 CST 2019
[root@OpenIL:misc]# uname -r
4.14.47-ipipe
[root@OpenIL:misc]#