facebookresearch / pytorch3d

PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

Home Page:https://pytorch3d.org/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

How can I align rendered image to be in the center?

cmh1027 opened this issue · comments

commented

I'm following the tutorial in here, and it is so far so good except that the image is not rendered in the center.

def render(point_cloud, elev=0, azim=0, image_size=512, radius=0.003, ppp=10):
    R, T = look_at_view_transform(1, elev, azim)
    cameras = FoVOrthographicCameras(device=device, R=R, T=T, znear=0.01)
    raster_settings = PointsRasterizationSettings(
        image_size=image_size, 
        radius = radius,
        points_per_pixel = ppp
    )
    renderer = PointsRenderer(
        rasterizer=PointsRasterizer(cameras=cameras, raster_settings=raster_settings),
        compositor=AlphaCompositor()
    )
    images = renderer(point_cloud)
    plt.figure(figsize=(10, 10))
    plt.imshow(images[0, ..., :3].cpu().numpy())
    plt.axis("off")
pcd = o3d.io.read_point_cloud("pcd.ply")
verts = torch.from_numpy(np.array(pcd.points)).float().to(device)
rgb = torch.from_numpy(np.array(pcd.colors)).float().to(device)
point_cloud = Pointclouds(points=[verts], features=[rgb])
render(point_cloud, elev=2, azim=2, image_size=(253, 381), radius=0.02)

This is my code, and I generate point cloud (pcd.ply) with an image and its pseudo-depth from monodepth estimator.
image
And the rendered result it shown as follows.
image
Definitely, not only the result is zoomed in, but also it is not center-aligned (black empty space at the bottom).
Is there any post-processing step for point cloud to get correct result?
Points in pcd has the mean values of [ -0.0723, 0.2070, -14.5213].

I don't think centre-alignment or the point cloud is the problem here. I think the black part at the bottom of the image is is caused by z-clipping. Adjusting znear may fix it. In general you need to play with the camera settings to get the "zoom" you actually want.