BerkeleyAutomation / gqcnn

Python module for GQ-CNN training and deployment with ROS integration.

Home Page:https://berkeleyautomation.github.io/gqcnn

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Issue: Bug/Performance Issue [Custom Images] - Incorrect depth

ShrutheeshIR opened this issue · comments

System information

  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
  • Python version: 3.6.9
  • Installed using pip or ROS: pip
  • Camera: Realsense (Gazebo Simulated)
  • GPU model (if applicable):

Describe what you are trying to do
I'm trying to run GQCNN 2.0 for a custom image. I have a bowl on the table and I'm trying to get the correct grasp.

Describe current behavior
While the result seems to be fine, I found something slightly bizarre when I was inspecting the output in detail. The depth value of the grasp does not seem to correspond exactly with the depth of the object in the image. This is a very minor issue, but I wish to understand why this is happening.

Describe the expected behavior
I presume that I should be able to get the correct depth to grasp the object.

Describe the input images
Original capture resolution: 1240x720.
Since this is too huge for the GQCNN (I do not have a GPU so even if it were acceptable, computation time would be too much), I halved the image. It is now of size 640x360. I have altered the camera intrinsics accordingly.

Describe the physical camera setup
The object is around 0.5-0.65 meters from the camera on the table from its view
setup

Other info / logs
Screenshot from 2020-05-17 12-44-58
This image depicts the problem that I have. According to this, it says that d=0.650, however, if you look at the bottom right hand corner, the depth is actually 0.558. I checked this by placing my mouse cursor at that exact location where the red arrow is in the image. As I trace it from top to bottom along the red dotted line, the depth varies from 0.60->0.558(at the ridge of the bowl)->0.58 as we move towards the bottom.

Here is another image, that shows the grasps sampled in the 3rd iteration.
Figure_1

My camera intrinsics is : {"_cy": 180, "_cx": 319.5, "_fy": 347.997558594, "_height": 360, "_fx": 347.997558594, "_width": 640, "_skew": 0.0, "_K": 0, "_frame": "realsense_overhead"}

Here is the depth image. Please divide by 1000 to convert the metres, as required by your code.
ddddepth_000075

As you can see, my z value of my bowl, is different from the d value shown in the result of the GQCNN code.
This is a very minor issue, and I could probably offset my value by quering the location, but wanted to put it out here anyway.

Thanks