Comparing 3D annotations to Waymo Open Lane PCD

[presmann]

hi,
As far as I understand from the documentation and discussions, the lane labels coordinate system is that of the camera. So by multiplying them by the extrinsic matrix of the camera (as appear on the file), I should get them in the vehicle coordinate system.
I've attempted to plot the point cloud (pcd) as retrieved from Waymo (cartesian representation in vehicle coordinate system)

rame_utils.parse_range_image_and_camera_projection
frame_utils.convert_range_image_to_point_cloud

What I see is:

1. Projecting pcd and labeling to image seems to fit nicely. The high-intensity lane line marking points from pcd appears to fit their respective pixels in image.

2. Plotting them in open 3d indicates that labels (in vehicle coordinates) appear slightly below the point clouds.
   --> as such, while intensities of pcd are projected in (1) to the image and seem to fit the picture, their labeling 3d representation appear below them and on the side.

---

Here are examples referring to 'segment-10770759614217273359_1465_000_1485_000_with_camera_labels/152271424044879700' in the training. The projection to the image is here

(lane lines annotations projected to the image are magenta, intensities color point clouds, and green/yellow intensities are of high intensity and seem to correspond to reflective lane lines).

On the other hand, looking at them from the side in open 3d, we see:

And also (the gray/black line is on z = 0 plane, and magenta is still the lane lines color).

Looking from BEV, we get (here we draw on x,y plane, red is the color of annotations, and pcd has higher intensity as it becomes darker. The intensities indicate lane lines, and lane annotations seem to deviate.

This seems like a discrepancy in the labeling procedure described in the paper. What am I missing here?

[YangLi]

@presmann Thanks for reporting this problem. We will try to reproduce this and see if there is anything misaligned in the data processing prodedure. Please stay tuned.

[George]

Referring to issue 146, I guess maybe some issue related to data type causes this problem.

[YangLi]

@presmann Hi,we have reproduced the problem. We show the openlane in the raw point cloud, colored lines for openlane.

In the z=0 plane, it could be found clearly, the blue lines under the point cloud, caused by some error on coordinate system. We have fixed this bug. The new lines, the red ones, abuts the points cloud.

In the bev plane, the deviate problem has been improved effectively, the red lines for new lines and blue for old ones.

Thanks a lot and the new version is coming soon.

[YangLi]

Hi, we have fixed the coordinate system bug in lastest version of openlane.